I’m trying to wrap my head around Stuart Sierra’s component architecture and how it has been implemented as part of Chestnut.
Something that’s been quite troublesome for me is coding server-side and client-side simultaneously. First, it’s not immediately clear that a separate Ring is spawned on port 10555 (while Figwheel’s own http-kit is spawned on port 3449). Second, if I run cljs-repl I lose my server context unless I connect to it via nREPL separately. Lastly, every time I make a change to the server, I need to issue a reset that basically drops everything and I lose my cljs-repl session.
I have the feeling that I’m doing something terribly wrong because Clojure development via REPL is usually a lot more fun than this. The current workflow I’ve described above has drained the joy out of iterative development.What are your thoughts?
Some good points here that could use some discussion.
Figwheel port (3449) vs Chestnut port (10555)
This split has been there since the early versions of Chestnut. IIRC later on we started using Figwheel as the HTTP server when in dev mode, which meant that either port would work. Now with the Component approach we want to start/control our own instance, which means the distinction is back. I’ve heard from a few people this has been confusing. Even though Chestnut prints that it starts on localhost:10555, people are simply used to connect to 3449.
I’ve run into this myself, and have started added a small ring handler to Figwheel that redirects to 10555. Maybe we can do something similar in Chestnut.
Changing routes requires a system restart
Any changes to the route factory functions are only picked up when restarting the system. If your environment has reloaded.repl support baked in (like CIDER) then this isn’t a big deal. I’ve gotten used to pressing C-c C-x regularly. But it’s still annoying and it trips people up.
A workaround here that I’ve used is to give the endpoint component a handler function that calls the routes factory on every request. This works quite well, main question is how to do that in a clean way so this happens in dev mode but not in prod.
Switching between CLJ and CLJS repls
Again when using CIDER I don’t really perceive this as a problem (cider-jack-in-clojurescript gives you both a CLJ and CLJS repl, so you can evaluate front- and backend forms without switching), but in other environments you end up doing a lot of (cljs-repl) and :cljs/quit to switch between contexts… Not sure what the best approach is in this case. I would probably recommend that people take Figwheel out of the system and run a lein figwheel as a separate process.
Suggestions (and pull requests) to improve any of this are very welcome!
Agree that the current state of affairs are skewed towards “use emacs or feel pain”. Cider really made all of these a non issue.
From what I can tell, possibly the largest pain with the need to restart the system after changes is due to the issue of having to exit the cljs repl.
Would it be worthwhile to at least document a way to run from the cli using two repls, one for clj, one for cljs? I’d be curious to know what the user’s expectation and setup is, and if having two distinct repls is sufficient for this.
As for the redirect, that was my bad for not having a backwards compatible path. I’d agree that a redirect would help out a ton here.
I’m also wondering if we can’t do better with directing users to the correct ports, and perhaps we can help resolve this by printing something in the repl on connection as well.
Currently the output looks like:
Figwheel: Starting server at http://0.0.0.0:3449
Figwheel: Watching build - app
Figwheel: Cleaning build - app
Compiling "resources/public/js/compiled/mychestnut.js" from ["src/cljs" "src/cljc" "dev"]...
Successfully compiled "resources/public/js/compiled/mychestnut.js" in 16.701 seconds.
Figwheel: Starting CSS Watcher for paths ["resources/public/css"]
:started
Maybe printing out something to the effect of “Started HTTP server at http://localhost:10555” would help?
I’m pretty sure Chestnut used to say “Serving on http://localhost:10555” or something like that. I didn’t realize we no longer do that, although it’s not too surprising with the refactoring to component.
Definitely would be good to have that back, although with the amount that’s being spit out to the REPL now it’ll still easily get lost. Having either the redirect, or setting up figwheel as an alternative http server (so you can use either port) is definitely still needed.
On another note, I’ve never actually managed to have cider’s built in refresh work for me, as it seems to ignore the refresh dirs, and I end up with an empty system that doesn’t actually do anything.
It works fine when running (reset) through the repl, but the moment I C-c C-x, the system seems to stop. This might be something in my setup, but I honestly can’t see any cause to it.
You mentioned that you’re used to running a cider refresh – did you have to do anything else in your setup to get clojure.tools.repl to play nicely with component/reloaded.repl?
*edit: a pure refresh also seems to clobber everything, so something seems wrong with my config. Helpful error messages point to “namespace not found” for any function in the repl.
** Edit 2: seems like I have better luck if I set the before/after refresh to reloaded.repl/stop and reloaded.repl/go rather than using the suspend/resume pairs.
Yeah, that works fine for me. Don’t think there’s anything special about my
setup. I do have this in a .dir-locals.el, but we do roughly the same in
Chestnut
What’s weird is that it only seems to be happening for a select few, as it’s working fine over there. Through the repl via reset this was why the set-refresh-dirs was called.
Wildly off topic, I suppose, as these aren’t pain points in Chestnut per-se, but something I’d like to handle. I’ll look at disable-reload! and see if I can’t find anything that plays nicely with cider-nrepl.
I got to the bottom of my issue. It has to do with the way that tools.namespace finds source files on the classpath to refresh the code. TL;DR: copied cljc files in the classpath for clojurescript is problematic with code reloading.
So the rundown is that any Chestnut application that utilizes cljc is problematic because of the fact that resources are on the classpath, specifically in a dev environment where code reloading is desired.
Every time cljs runs a compile, it copies the source files (for the sourcemaps) to the resource directory (inside the classpath). When a reload is called, it detects that these cljc files also need to be reloaded, and it ends up reloading component as well, which doesn’t really fare too well.
This was partially resolved in my original proposal for component integration in the user namespace with the (set-refresh-dirs) call. This ignores the copied cljc files in resources/public/ properly. However, Cider uses its own version of tools.namespace, which means that this call is completely ignored by cider.
A way around it would be to have an options flag in the routes to either host through a /public folder that is off the classpath (during development), or the /public classpath (when deploying/packaging to uberjar). The other non-js resources themselves are symlinked from the classpath directory to the public folder.
Adding this flag might also enable us to reuse it for having the separate routes factory that rebuilds on every change, which also tackles the original second point.
Let me know what you think – happy to start working on a PR if the concept looks alright to you.
I also would like to look into seeing how to wrap-reload in the user namespace (as before), despite the duct configuration of the routes. Something to the degree of hooking a component reset call with the wrap-reload middleware?
On further prototyping, it is possible to use wrap-reload in the same way we had been doing in the past (like so).
We’d then be able to change routes, and make edits to code outside of the system, and wrap-reload could handle that all for us.
Any changes to the system itself would still require a restart, but this would make editing your routes at least a little easier. This preserves the duct system components for endpoint/middleware/routes, but allows you to edit the routes, and be able to view changes with other code reload tools as an alternative to resetting the entire system.
Edit: I’ve managed to extend the figwheel component to add the Suspendable protocol, so cljs connections can persist across resets.
To test, start a new repl. Connect to the running repl by finding the port by cat .nrepl-port and run lein repl :connect localhost:{port} In the new repl, run (cljs-repl) and connect with the browser. Under the clj repl, you can issue (reset) commands (or suspend/resume commands, cider-refresh, etc). The cljs-repl remains connected through the reset. (eg, running (js/alert "test") will alert the browser).
(Edit: I got curious and the above also gets disconnected when using cider-jack-in-clojurescript. The clojurescript repl is unresponsive after a system restart on the clojure system.)
In the current version of Chestnut, the cljs-repl behavior above will not be able to interact with the browser after a reset again until after running :cljs/quit, (cljs-repl).
So, the last point that hasn’t really been addressed here at all is how to easily and automatically reset routes/the system, for those without a quick command such as C-c C-x in emacs.
I’ve been playing around over the past week or so with the concept of yet another file watcher component that issues a (reloaded.repl/reset) command when files are changed. After a few attempts, I’ve got a working implementation that actually seems to work pretty OK, all things considered.
(Why yes, I name all my chestnut testing projects “testnut” why do you ask?)
The component will open a repl connection at the beginning, and sends the reset command to the open repl on file changes.
On first attempt at writing this, I noticed that I was getting a very strange IllegalStateException("Can't change/establish root binding of: *ns* with set") error when running (reloaded.repl/reset) directly from a file watcher. The root of this is due to the way that tools.namespace.repl is implemented - it turns out that *ns* cannot be altered outside of a REPL environment. The workaround was to connect to the nrepl environment and issue commands there.
I’ve abstracted the way that this works, so that any command can be run in the repl, but I’m demonstrating with reloaded.repl because that’s what I had in mind when writing this.