shadow-cljs as of version 2.4.23 gained a new experimental loading mechanism which can greatly improve the initial load times of development, meaning non-optimized watch or compile :browser builds. You can test it by setting
{...
:builds
{:app {:target :browser
...
:devtools
{...
:loader-mode :eval}}}
Normal ClojureScript relies on using the Closure Library “Debug” Loader which will load files by appending a bunch of <script src="..."></script> tags to the document. This means that the browser will do one request per source file. This works fine when using a “reasonable” number of files but can get quite out of hand in projects with lots of source files or npm dependencies which quite often have a lot lot of small files.
shadow-cljs already replaced this Debug Loader with its own loading mechanism to support using <script async ...> but now adds an optional new loader which uses eval to load the files. Instead of letting the browser request each file one by one it instead generates one big file per module. This file will contain one call to SHAHDOW_ENV.evalLoad(sourceName, sourceMap, sourceCode) per source file with sourceCode being an escaped JS source string that will be passed to eval. This means instead we get one big file instead of many small ones.
Why eval?
The short answer is: Source Maps.
Since ClosureJS and ClojureScript are already fully namespaced we could easily just concatenate all the files together without eval and load the code directly but the big downside for this is generating a source map for that concatenated file. Generating the source map for these big files can take easily take a second or longer which drastically reduces the feedback times in development.
eval lets us add a //# sourceURL=... comment to the code which will tell the JS engine that the code we are evaling should be displayed as if it came from that URL. Chrome and others will then load source maps separately so we do not need to combine them.
Escaping the code to inline it as a string only adds minimal overhead and modern browsers can eval quite fast. JS tools like webpack use similar strategies and thats where I got the idea.
Since we do one request instead of potentially thousands GZIP compression also gets a whole lot better.
Performance Comparison
In my test build I have a few CLJS sources which basically do nothing. I’m only measuring how long the initial page load takes including ALL requests made.
The traditional load mechanism clocks in at
119 requests | 991 KB transferred | Finish: 709 ms
Switching to :loader-mode :eval this results in
5 requests | 913 KB transferred | Finish: 474 ms
This is already a decent improvement. The real gains however are much more visible when actually using one or more of those “crazy” npm dependencies which include a ton of files. Just for the sake of testing I added
(:require
["@material-ui/core"]
["@material-ui/icons"])
This takes the initial load up to
1561 requests | 2.4 MB transferred | Finish: 2.76 s
which is quite ridiculous and is very noticeably slow. My machine is quite high end and on slower machines or mobile devices this can easily take 10 seconds or more.
Again switching to :loader-mode :eval this results in
5 requests | 1.2 MB transferred | Finish: 836 ms
Much more reasonable. Note the substantial reduction in download size due to better GZIP compression results. The exact same code is loaded, nothing was removed.
Side Note / Rant
Please avoid using “index” packages like "@material-ui/core" and rather try to only import what you need from those packages, eg. "@material-ui/core/Button". Always audit your dependencies properly as the code quality standards regarding packaging for some JS packages are quite terrible.
Conclusion
Please help test using :loader-mode :eval in your :browser builds. There may be edge cases I haven’t covered yet and I only tested Chrome and briefly Firefox (which appeared to be even faster). If this turns out to be as viable as it currently looks I will make it the default.
Note that this really only affects initial load times and has no impact on live-reload performance whatsoever.
