Why is that not one project with 2 builds? Which would mean one server.
You can use shadow-cljs clj-run some.build/fn to run separate tasks one after the other which I also mentioned in my post. Again: one server, you just need to adopt the mindset of not having a command line do one thing after the other but instead calling clojure functions one after the other.
That is actually a thing I have been thinking about a lot recently. Currently there is one server per project but I’m trying to work out the details for using one server globally that each project can use. So you would start it once per machine and each project could use that one instead. It is a bit tricky since I still want to maintain the “use shadow-cljs as a library” use-case and running it from without lein or boot but I think a few things can be improved without breaking that. It will take some time to figure this out and implement.
Ideally I see shadow-cljs as a standalone tool you just have running on your machine while developing. Cuttle is the basic inspiration for this. I just haven’t gotten around to actually building the UI and working out the complicated details. Consider what we have now the “easy first steps”, it is nowhere near finished.
Enough with the off-topic shadow-cljs talk though, we can talk in another topic if anyone is interested.
There are lots of things to consider when trying to make Clojure start faster but I don’t think there is one universal answer that applies to everything. I brought up shadow-cljs as an example for things I’m doing specific to that tool. My Clojure projects have different requirements so some of the things don’t even apply or would even be bad.
Ultimately though if you move away from the command line and instead use a REPL the startup time really becomes a non-issue. “Scripting” is not a strength of Clojure but that is not a problem. lumo or planck are great options if you just want “scripts”.
It would be cool if shadow-cljs runs in background and makes all compilations smooth to start. One of my dev machine uses a “1.2 GHz Intel Core m3” processor, it would really benefit from those tricks.
For me, I’ve divided my project into many smaller projects… switching between them requires restart the compiler. I guess this habit brought from Node.js community might confuse Java people?
I’m a Lisp person rather than a Java one. My general workflow is to start a VM and leave it running for a long time. If I’m working on something that depends on several of my own libraries, I usually use lein checkouts to pick up changes to any of the underlying libs in the same session.
There are multiple conflated things here and we do a disservice to the problems by not pulling them apart. The tweet says “get faster Clojure” but talks solely about startup time. Startup time and post-startup time are mostly independent but both important (and have different problems and solutions). Even startup time is not one problem but several kinds of use cases with different audiences. I spent a chunk of time a while ago on this and kept my notes at https://dev.clojure.org/display/design/Improving+Clojure+Start+Time. In particular, I did a survey to try to tease apart the use cases.
Stressing about the raw startup time of just clojure.core itself is imo not worth the effort. I fail to see how to make Clojure an order of magnitude faster, which is what it would take to be interesting for scripting users. Those users are better served by Planck/Lumo/etc. That’s not to say that we shouldn’t be aware of it and take reasonable efforts to improve it, just that we should not delude ourselves that that will be ever be “enough”.
Tooling is another area where there are opportunities for improvement. There are a few use cases where the new command line tools will help, but generally I think there’s still room for doing a better job making tools faster to start up (by avoiding work or doing more compiling or probably other stuff).
The place that is the most interesting to me is in reducing the time per-ns and per-var to load code. If code is AOT compiled (likely for production use cases), I think there are some new interesting options if we have tools.deps.alpha which is in the loop of downloading deps. It’s not crazy to believe that we could pre-AOT-compile and cache many of our deps to avoid re-doing all of that reading and compilation. In this scenario, we would also get a lot more lift out of direct linking + lazy vars (which we have a working patch for). And there are some intriguing possibilities for leveraging invokedynamic which a few people have explored.
If code is not AOT compiled (which is where we live at dev time), then we need to look at the time to read, compile, and load the code. There might be opportunities to cache certain parts of this process (given that most of our code is identical every time we load it).
When we get into 10+ second start times, I think everyone can agree that’s too slow. It’s pretty easy to make a stack of Clojure web stuff big enough to hit this time frame (without even introducing CLJS). When that happens, typically it means you are loading 10k+ classes (and running some init code for each). I don’t think there’s any silver bullet to kill that, but there are a lot of things to look at still and I expect I’ll get another time slice on it before the next release.
We don’t AOT for production. We use clojure.main as the entry point for all our programs with -m whatever.namespace to specify which -main to run. Although we have a few Clojure processes run via cron, everything else is a server process (either a web server or some long-running background process), so startup time really is irrelevant.
For development, we start a REPL in our editor and, yeah, that takes a few seconds to load up the basic set of dependencies – but don’t do that very often, maybe once a day at most, usually only a couple of times a week. We use Boot so we can load and reload dependencies from the REPL. This means we can start multiple server projects in a single REPL easily enough (if we have conflicts, we can use pods to isolate them – but that is very rare, to be honest). Each of our projects is built with Component so we can start and stop it whenever we want – while keeping the REPL running. Starting any of our web server projects takes a negligible amount of time – once the REPL is running with the dependencies loaded.
We also run Socket Server REPLs on some of our production apps so we can easily telnet into a REPL to run quick status checks (or, occasionally, updates).
When I hear people complaining about startup time, it makes me curious about their workflow…
I was thinking about that, but there are two concerns:
I used npm deps a lot in shadow-cljs. Lumo is fine, Planck is not an option. however syntax for requiring npm modules in shadow-cljs might be slightly different from that in Lumo, I need to confirm it first.
in Lumo, dependencies are specified with -D option. Normally I use a EDN file to maintain deps. I would rely on clj -Spath to be mature firs, or Lump gets native support for that…
FWIW, I didn’t take this as negativity, but rather pointing out an issue (the elephant in the room). I think we should pay attention to this issue. I use Clojure for lots of things and while I’m very happy with server code performance, developing could definitely use speed improvements. cider-jack-in takes so long that I often go and get coffee, and if ClojureScript compilation were faster, I’d be much happier, too.
I understand there are good reasons for why it is slow, but this doesn’t mean we shouldn’t pretend it isn’t. Progress is being made and things are getting faster.