I’m experimenting with a new option in shadow-cljs which can reduce the size of builds that require a lot of JS dependencies. By default I chose to use stable names, meaning that the actual name of the included file will be used as its alias in the code.
Say you include
(:require ["react" ...])
in your code. This will be resolved following the npm rules and ends up including
;; in dev mode
node_modules/react/cjs/react_development.js
;; in release mode
node_modules/react/cjs/react_production.min.js
in your build with the assigned alias
shadow$provide.module$node_modules$react$cjs$react_development = function(...) { ... };
this is quite verbose but very cache friendly since the assigned alias never changes. This only becomes a problem if you include JS libs that themselves are split into hundreds/thousands of files where this verbosity adds up quite quickly. While gzip compression takes care of this quite well it still ends up with about a 5-10% overhead when compared with that for example webpack would generate.
webpack just assigns numeric ids to each source which is obviously much shorter but doesn’t cache very well since the numbers can change when dependencies are added or removed or just happend to be re-ordered when refactoring other code. Whenever the numeric ids change the cache needs to be invalidated and the output will obviously have a different hash as well.
As of shadow-cljs@2.7.28 it is possible to also use numeric ids in shadow-cljs. This is not enabled by default yet but probably will become the default for release builds in the future. It doesn’t actually change any functionality it just potentially makes the build a bit smaller.
You can enable this currently in your build config via
{...
:your-build
{:target :browser
:release {:js-options {:minimize-require true}}}
You probably don’t want to enable it in :dev mode since that will likely lead to lots more cache invalidations which might make development slower. Note that this is only relevant for :browser builds.
I did some tests with a dummy test build that just included semantic-ui-react which ends up including 600+ JS files. I also experimented with some different alias mechanisms where I used a hash (or shortened hash) which is stable and generally shorter than the original filename but much worse with gzip compression.
;; alias mode - JS size - after gzip
;; full hash [JS: 825.62 KB] [GZIP: 205.61 KB] (md5 of the filename in hex)
;; full path [JS: 891.56 KB] [GZIP: 186.04 KB] (munged filename)
;; short hash 8 [JS: 745.50 KB] [GZIP: 184.36 KB] (md5, first 8 chars)
;; short hash 6 [JS: 738.83 KB] [GZIP: 182.84 KB] (md5, first 6 chars)
;; numeric id [JS: 719.89 KB] [GZIP: 177.36 KB]
It is pretty impressive that an additional 172KB only adds 9KB gzipped but that is still a noticeable difference in the case of the full filename. The full hash is obviously not an option since gzip really doesn’t like those. Shorter hashes increase the likelihood of conflicts which would need to be detected and invalidated so that doesn’t really seem worth it either. numeric ids work out quite well and actually end up slightly smaller than what webpack generates since we require a little less boilerplate code.
If you don’t use many JS deps or those that aren’t split into many files (eg. react, react-dom) this won’t make much of a difference but some of the larger libs this can have quite an impact. Please try it in your builds and let me know.
Best way to experiment with this would be to run
shadow-cljs run shadow.cljs.build-report your-build before.html
;; then change the config and run
shadow-cljs run shadow.cljs.build-report your-build after.html
Then open the before.html and after.html in your browser and compare the results.