### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 12, 2017, 5:44am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/1 "2017-12-12T05:44:34Z")
Glad to hear people complaining that Clojure is too slow again:
>
What tricks do we have?
---
### Author:  [@plexus](https://clojureverse.org/u/plexus)
#### Post date: [December 12, 2017, 8:30am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/2 "2017-12-12T08:30:49Z")
> [@jiyinyiyong](#):
>
> Glad to hear people complaining that Clojure is too slow again:
It bothers me to celebrate this kind of negativity. It also bothers me that this talks about “Clojure being slow” when it’s really “Clojure startup time being slow”.
Clojure is _fast_, it’s very specifically designed to be fast while maintaining functional semantics. The JVM is _super fast_ once it’s had to time to warm up and JIT code, sometimes faster than the equivalent in lower level languages, despite being a VM.
That starting up Clojure takes time is a long known issue, but let’s get the facts straight. He says it’s a “small 15k loc project”, but later in the Twitter thread he admits that loading dependencies takes 18 out of the 20 seconds. How many LoC is this project including dependencies?
Each Clojure file needs to be compiled into java bytecode, then loaded through a ClassLoader. Both of these are costly operations. This is not unique to Clojure. People also complain about JRuby having slow startup time, but again, JRuby is generally faster than CRuby once it’s warmed up. The JVM is simply not optimized for quickly loading thousands of classes (remember that every Clojure function becomes a class).
This all said, I believe the JRuby team has put a lot of effort in over the years to improve startup time, I’m not sure if that’s as big a priority for Clojure core. Or maybe it is but there’s just less to be gained.
The only thing I’ve seen that seems to be able to work around some of this is [this article on using AppCDS and AOT](https://mjg123.github.io/2017/10/04/AppCDS-and-Clojure.html). It seems a pretty advanced approach, but maybe more tooling will start making use of it.
Apart from that the only “trick” is to avoid restarting your process. This is something the Clojure community _is_ quite invested in. You should be able to do everything you need to do from a REPL, and if you need a clean slate you use `clojure.tools.namespace` to reload your code, and whatever Component/Mount/Integrant-alike you’re using to restart your app state.
Another tool that follows this philosophy and isn’t widely known is [grenchman](https://github.com/technomancy/grenchman), it gives you a command line tool written in OCaml (starts up fast), which then connects to an nREPL process to do its work. So you can still script things from the shell, but you don’t have to wait 20 seconds for each time you’re calling into Clojure.
Or you can try making a tool like [Drip](https://github.com/ninjudd/drip) part of your process. I’ve tried that before but I find the hassle of it not worth the small gains. It’ll get you a running JVM faster, but then Clojure still needs to do its work of compiling and loading.
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 12, 2017, 9:06am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/3 "2017-12-12T09:06:24Z")
No. Just to clarify, I’m not celebrating its slowness, but glad to hear that people still care about how much time it takes to start a Clojure program. shadow-cljs still takes more than 10s to start on my laptop, I would blame that to Clojure.
---
### Author:  [@martinklepsch](https://clojureverse.org/u/martinklepsch)
#### Post date: [December 12, 2017, 12:01pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/4 "2017-12-12T12:01:01Z")
Seeing this conversation on Twitter I wondered if we could not just AOT compile & cache our Maven dependencies ([conversation on Twitter](https://twitter.com/martinklepsch/status/939641237505703939)).
Reluctance to AOT came up pretty quick and admittedly my experiences with AOT aren’t all that great either. On some projects we used [shim namespaces](https://github.com/boot-clj/boot/wiki/Circumnavigating-Watch-and-AOT-Compilation-Limitations) to avoid it all together. As far as I understand the issue with AOT is that Clojure is dynamic and loading a namespace may do all kinds of things. So you can really only safely AOT namespaces which are itself side effect free.
Such side effects may include:
- reading from the environment: `(def e (System/getenv "E"))`
- reading a config file
- top-level function calls
- etc…
As far as I understand spec’s `def` forms should probably be considered side-effecting as well? (They write to an atom right?)
I was thinking that that there could be a **community maintained list of namespaces that can be safely AOT compiled** (project + version + filepath/namespace + shasum). With such thing we could put tooling in place to cache the AOT results for these namespaces.
If spec’s `def` forms indeed are side effecting that list of namespaces is likely to shrink in the future which would be a bad outlook but maybe there are other ways to solve that particular issue with spec.
Also on Twitter, @alexmiller [mentioned that he has a partly working implementation](https://twitter.com/puredanger/status/939656357535502336) of something similar so I think maybe something will happen here eventually 🙂
---
### Author:  [@thheller](https://clojureverse.org/u/thheller)
#### Post date: [December 12, 2017, 12:55pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/5 "2017-12-12T12:55:54Z")
> [@jiyinyiyong](#):
>
> shadow-cljs still takes more than 10s to start on my laptop, I would blame that to Clojure.
As I mentioned [here](https://code.thheller.com/blog/shadow-cljs/2017/11/18/the-many-ways-to-use-shadow-cljs.html#optimization) I already tried a whole bunch to optimize this and the best thing you can do is having a `shadow-cljs server` process running in the background. Once that process is running every other command should be pretty much instant. The server only needs to be restarted if `:dependencies` changes. Instead of running `shadow-cljs server` you can also use `shadow-cljs clj-repl` which starts the server but also lets you use the process as a REPL.
Typically server restarts will be fairly infrequent and I often have the server process running for days which means that 10s is not significant at all.
I did a few other experiments which could maybe take down the startup time to 7sec-ish but really nothing even comes close to using the shared `server` process. I have a few other ideas to make it easier to manage the server process itself (eg. no restart on deps change) but that is quite a lot of work.
For Clojure projects `lein repl` is pretty much equivalent to `shadow-cljs clj-repl`, it just doesn’t re-use itself to run other `lein` commands. Since you can connect to it remotely though you can re-use it easily for everything else though.
```plaintext
time node --eval "console.log(1 + 2)"
3
real 0m0.081s
user 0m0.057s
sys 0m0.019s
```
```plaintext
time shadow-cljs clj-eval "(+ 1 2)"
shadow-cljs - config: /Users/zilence/code/shadow-cljs/shadow-cljs.edn version: 2.0.114
shadow-cljs - connected to server
3
real 0m0.197s
user 0m0.146s
sys 0m0.026s
```
Clojure is never going to start as fast as `node` but that is not a problem if you only “start” once every few days. If you have that `server` process running the results are pretty decent and you are basically just paying the startup time of the `shadow-cljs` process itself at this point. Nothing will beat actually using the REPL though.
```auto
shadow-cljs clj-repl
shadow-cljs - config: /Users/zilence/code/shadow-cljs/shadow-cljs.edn version: 2.0.114
shadow-cljs - connected to server
shadow-cljs - REPL - see (help), :repl/quit to exit
[5:0]~shadow.user=> (time (+ 1 2))
"Elapsed time: 0.053012 msecs"
3
[5:0]~shadow.user=>
```
So yeah the best thing to make Clojure fast is to not start a new JVM but instead re-use a running one. Which both `grench` and `drip` mentioned by @plexus do and `shadow-cljs` does as well.
So instead of focusing on startup time I wonder if we can instead instead improve the experience of re-using a JVM so we don’t have to start from scratch all the time. I only scratched the surface in `shadow-cljs`, there is so much more we could do.
---
### Author:  [@madbonkey](https://clojureverse.org/u/madbonkey)
#### Post date: [December 12, 2017, 1:20pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/6 "2017-12-12T13:20:14Z")
I like the general idea of reusing the VM. As far as I can tell, with my _very_ limited knowledge of these things, most processes having to to _something_ with a VM utilize a similar approach, right? I’m currently thinking of Docker, for lack of a more related example, which requires every user to run a shared “main” Docker service/process in the background.
I for one would be welcoming a mechanism like that as the default, for example when installing the new Clojure CLI (e.g. via `brew install clojure`). I’d like to see sane defaults for cleaning up loaded namespaces and so on, and a general ease-of-use approach when it comes to starting/stopping individual projects, or running them side by side.
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 12, 2017, 2:23pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/7 "2017-12-12T14:23:27Z")
On shadow-cljs… it is fast with a server started. In my case it’s a bit different from starting a server for several days. In fact I would start several servers. Take [cumulo-workflow](https://github.com/Cumulo/cumulo-workflow) for example, there are a WebSocket server and a web app, there are two servers. Besides, I need to fix bugs in another page too, which brought me a third one.
There’s also another scenario that I need to compile my app on a remote server, which has no shadow-cljs server running… And, actually shadow-cljs runs twice here, once to compile js in web page, and once to prerender index.html by reusing virtual DOM components.
Besides, I kill shadow-cljs server several times every day… because I have many tiny webpages to maintain, plus running some scripts with Node.js . I could use Lumo for the scripts part after Lumo supports `deps.edn` well though.
---
### Author:  [@thheller](https://clojureverse.org/u/thheller)
#### Post date: [December 12, 2017, 3:15pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/8 "2017-12-12T15:15:20Z")
> [@jiyinyiyong](#):
>
> Take cumulo-workflow for example, there are a WebSocket server and a web app, there are two servers.
Why is that not one project with 2 builds? Which would mean one server.
> [@jiyinyiyong](#):
>
> And, actually shadow-cljs runs twice here, once to compile js in web page, and once to prerender index.html by reusing virtual DOM components.
You can use `shadow-cljs clj-run some.build/fn` to run separate tasks one after the other which I also mentioned in my post. Again: one server, you just need to adopt the mindset of not having a command line do one thing after the other but instead calling clojure functions one after the other.
> [@jiyinyiyong](#):
>
> I have many tiny webpages to maintain, plus running some scripts with Node.js
That is actually a thing I have been thinking about a lot recently. Currently there is one server per project but I’m trying to work out the details for using one server globally that each project can use. So you would start it once per machine and each project could use that one instead. It is a bit tricky since I still want to maintain the “use shadow-cljs as a library” use-case and running it from without `lein` or `boot` but I think a few things can be improved without breaking that. It will take some time to figure this out and implement.
Ideally I see `shadow-cljs` as a standalone tool you just have running on your machine while developing. [Cuttle](https://github.com/oakmac/cuttle) is the basic inspiration for this. I just haven’t gotten around to actually building the UI and working out the complicated details. Consider what we have now the “easy first steps”, it is nowhere near finished.
Enough with the off-topic `shadow-cljs` talk though, we can talk in another topic if anyone is interested.
There are lots of things to consider when trying to make Clojure start faster but I don’t think there is one universal answer that applies to everything. I brought up `shadow-cljs` as an example for things I’m doing specific to that tool. My Clojure projects have different requirements so some of the things don’t even apply or would even be bad.
Ultimately though if you move away from the command line and instead use a REPL the startup time really becomes a non-issue. “Scripting” is not a strength of Clojure but that is not a problem. `lumo` or `planck` are great options if you just want “scripts”.
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 12, 2017, 4:05pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/9 "2017-12-12T16:05:08Z")
Branched new topic at [https://github.com/thheller/shadow-cljs/issues/147](https://github.com/thheller/shadow-cljs/issues/147)
It would be cool if shadow-cljs runs in background and makes all compilations smooth to start. One of my dev machine uses a “1.2 GHz Intel Core m3” processor, it would really benefit from those tricks.
---
### Author:  [@jackrusher](https://clojureverse.org/u/jackrusher)
#### Post date: [December 12, 2017, 4:13pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/10 "2017-12-12T16:13:32Z")
I’ve never understood complaints about startup time. Why would anyone stop and start the VM with any frequency?
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 12, 2017, 4:18pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/11 "2017-12-12T16:18:00Z")
For me, I’ve divided my project into many smaller projects… switching between them requires restart the compiler. I guess this habit brought from Node.js community might confuse Java people?
---
### Author:  [@jackrusher](https://clojureverse.org/u/jackrusher)
#### Post date: [December 12, 2017, 8:06pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/12 "2017-12-12T20:06:52Z")
I’m a Lisp person rather than a Java one. My general workflow is to start a VM and leave it running for a long time. If I’m working on something that depends on several of my own libraries, I usually use lein checkouts to pick up changes to any of the underlying libs in the same session.
---
### Author:  [@alexmiller](https://clojureverse.org/u/alexmiller)
#### Post date: [December 12, 2017, 10:16pm UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/13 "2017-12-12T22:16:59Z")
There are multiple conflated things here and we do a disservice to the problems by not pulling them apart. The tweet says “get faster Clojure” but talks solely about startup time. Startup time and post-startup time are mostly independent but both important (and have different problems and solutions). Even startup time is not one problem but several kinds of use cases with different audiences. I spent a chunk of time a while ago on this and kept my notes at [https://dev.clojure.org/display/design/Improving+Clojure+Start+Time](https://dev.clojure.org/display/design/Improving+Clojure+Start+Time). In particular, I did a survey to try to tease apart the use cases.
Stressing about the raw startup time of just clojure.core itself is imo not worth the effort. I fail to see how to make Clojure an order of magnitude faster, which is what it would take to be interesting for scripting users. Those users are better served by Planck/Lumo/etc. That’s not to say that we shouldn’t be aware of it and take reasonable efforts to improve it, just that we should not delude ourselves that that will be ever be “enough”.
Tooling is another area where there are opportunities for improvement. There are a few use cases where the new command line tools will help, but generally I think there’s still room for doing a better job making tools faster to start up (by avoiding work or doing more compiling or probably other stuff).
The place that is the most interesting to me is in reducing the time per-ns and per-var to load code. If code is AOT compiled (likely for production use cases), I think there are some new interesting options if we have tools.deps.alpha which is _in the loop_ of downloading deps. It’s not crazy to believe that we could pre-AOT-compile and cache many of our deps to avoid re-doing all of that reading and compilation. In this scenario, we would also get a lot more lift out of direct linking + lazy vars (which we have a working patch for). And there are some intriguing possibilities for leveraging invokedynamic which a few people have explored.
If code is not AOT compiled (which is where we live at dev time), then we need to look at the time to read, compile, and load the code. There might be opportunities to cache certain parts of this process (given that most of our code is identical every time we load it).
When we get into 10+ second start times, I think everyone can agree that’s too slow. It’s pretty easy to make a stack of Clojure web stuff big enough to hit this time frame (without even introducing CLJS). When that happens, typically it means you are loading 10k+ classes (and running some init code for each). I don’t think there’s any silver bullet to kill that, but there are a lot of things to look at still and I expect I’ll get another time slice on it before the next release.
---
### Author:  [@seancorfield](https://clojureverse.org/u/seancorfield)
#### Post date: [December 13, 2017, 12:22am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/14 "2017-12-13T00:22:42Z")
We don’t AOT for production. We use `clojure.main` as the entry point for all our programs with `-m whatever.namespace` to specify which `-main` to run. Although we have a few Clojure processes run via `cron`, everything else is a server process (either a web server or some long-running background process), so startup time really is irrelevant.
For development, we start a REPL in our editor and, yeah, that takes a few seconds to load up the basic set of dependencies – but don’t do that very often, maybe once a day at most, usually only a couple of times a week. We use Boot so we can load and reload dependencies from the REPL. This means we can start multiple server projects in a single REPL easily enough (if we have conflicts, we can use pods to isolate them – but that is very rare, to be honest). Each of our projects is built with Component so we can `start` and `stop` it whenever we want – while keeping the REPL running. Starting any of our web server projects takes a negligible amount of time – once the REPL is running with the dependencies loaded.
We also run Socket Server REPLs on some of our production apps so we can easily telnet into a REPL to run quick status checks (or, occasionally, updates).
When I hear people complaining about startup time, it makes me curious about their workflow…
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 13, 2017, 2:12am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/15 "2017-12-13T02:12:51Z")
In my workflow, I want trying to use Clojure like I was using Node.js … [https://github.com/thheller/shadow-cljs/issues/147](https://github.com/thheller/shadow-cljs/issues/147) switching between different projects, doing scripting…
---
### Author:  [@brain](https://clojureverse.org/u/brain)
#### Post date: [December 13, 2017, 8:09am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/16 "2017-12-13T08:09:06Z")
大佬 666
Yes,He is right,It’s slow…
It’s easy to solve this problem .
just take a cup of tea . and then go on working…
---
### Author:  [@zerg000000](https://clojureverse.org/u/zerg000000)
#### Post date: [December 14, 2017, 1:49am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/17 "2017-12-14T01:49:33Z")
I think if all you need is cljs scripting and cljs projects. **lumo/planck** should fit your usage.
- instant startup time for repl
- could compile cljs
- could be used for scripting
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 14, 2017, 2:12am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/18 "2017-12-14T02:12:25Z")
I was thinking about that, but there are two concerns:
- I used npm deps a lot in shadow-cljs. Lumo is fine, Planck is not an option. however syntax for requiring npm modules in shadow-cljs might be slightly different from that in Lumo, I need to confirm it first.
- in Lumo, dependencies are specified with `-D` option. Normally I use a EDN file to maintain deps. I would rely on `clj -Spath` to be mature firs, or Lump gets native support for that…
---
### Author:  [@zerg000000](https://clojureverse.org/u/zerg000000)
#### Post date: [December 14, 2017, 4:19am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/19 "2017-12-14T04:19:35Z")
plain old node\_modules under project root is good to go?
>
>
> First off, kudos on the great work on this project! I love Clojure, but needing …the JVM for ClojureScript has been very offputting.
>
> Is there any documentation for how to include dependencies? From looking around, it seems like Lumo supports both Node packages, as well as some lein-sourced packages?
>
> It would be nice to have this information in the readme so people can get up and running quickly.
---
### Author:  [@jiyinyiyong](https://clojureverse.org/u/jiyinyiyong)
#### Post date: [December 14, 2017, 4:34am UTC](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176/20 "2017-12-14T04:34:40Z")
I know Lumo supports that. But in shadow-cljs the syntax I used is:
```clojure
(ns main
(:require ["md5" :as md5]))
```
not sure if Lumo has support for it or not.
[Next page](https://clojureverse.org/t/tricks-to-make-clojure-startup-time-faster/1176.md?page=2)