I just released the shadow-cljs 3.4.1version which includes a few changes I want to highlight. On the surface not much changes, but quite a few internal changes happened.
npm-deps is now manual
:npm-deps can be specified by libraries and used to instruct shadow-cljs to automatically install those dependencies on launch. This is now reworked and needs to be triggered manually. npm install nowadays seems to be dangerous business I no longer feel safe to run it at all. I moved all my npm work into a container for this reason.
Long story short. The automatic install is gone, but you can now run the shadow.cljs.npm-deps installer directly instead. It does the same thing as before, just manual.
It can be triggered by anything that knows how to run Clojure code.
npx shadow-cljs run shadow.cljs.npm-deps
clj -X shadow.cljs.npm-deps/run
clj -M -m shadow.cljs.npm-deps
lein run -m shadow.cljs.npm-deps
You do not need to run this before every launch. Once is usually fine, or I guess when you change dependencies. :npm-deps aren’t super widely used, but libraries like re-frame use it to ensure you have react. You can of course always just install that manually. If your dependencies don’t have any npm dependencies at all you of course never need to run it.
Container Support
Although I do not recommend running shadow-cljs inside Container (e.g. Docker) in the world of AI and increasingly untrusted environments it was time to make a few adjustments. Of course everything always worked fine when running in a Container, except that file watching didn’t work at all. So no hot-reload since no changes were ever automatically detected. For this I wrote a new fswatch implementation that uses polling, meaning it just regularly scans all the files and looks for modifications. This is much more expensive, but at least it works in containers.
While at it I also added support for a new SHADOW_CLJS environment variable, that can be used to pass extra configuration since some of the defaults do not make sense in container environments.
You can look at my container setup in the shadow-cljs repo directly (i.e. Containerfile, start-dev.sh, container-start-dev.sh). And the main highlight is the start arguments in start-dev.sh.
container run \
-it \
--rm \
-c 8 \
-m 4G \
-e SHADOW_CLJS="{:fs-watch {:impl :polling} :http {:port 9601} :nrepl {:port 9602} :cache-root \"/cache/shadow-cljs\"}" \
-v "$PWD:/code/shadow-cljs" \
-v "$PWD/../shadow-grove:/code/shadow-grove" \
-v "$PWD/../shadow-css:/code/shadow-css" \
-v cache:/cache \
-p 9601:9601 \
-p 9602:9602 \
-w /code/shadow-cljs \
"${IMAGE_NAME}"
-e SHADOW_CLJSpasses the extra env var, which is just a EDN string. :fs-watch {:impl :polling} makes it use the polling file watcher. :cache-root tells shadow-cljs where to write its temporary files, which defaults to .shadow-cljs in the project folder. You can leave that as is. I’m just experimenting with moving it do a dedicated cache volume (also see my shell scripts for how I handle all other “cache” files). :http {:port 9601} overwrites the default 9630 port. nREPL normally uses a random port and writes it to the .shadow-cljs/nrepl.port file, but since we have to expose a fixed port we can do that via :nrepl {:port 9602}. Which ports you actually use doesn’t matter much. You only also need the -p 9601:9601 argument to actually expose those ports. All the extra mounted volumes are there because I use some :local/root dependencies, only the first is for the actual project.
I’m testing apple/container currently, but this should all work exactly the same in Docker or Podman.
Undertow Server replaced
For a long time shadow-cljs used the Undertow webserver (via shadow-undertow helper lib). It is a fine webserver with many nice features, but if you are on a newer JVM (24+ IIRC) you’d get these annoying little warnings on startup.
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by org.jboss.threads.JBossExecutors (file:/Users/thheller/.m2/repository/org/jboss/threads/jboss-threads/3.5.0.Final/jboss-threads-3.5.0.Final.jar)
WARNING: Please consider reporting this to the maintainers of class org.jboss.threads.JBossExecutors
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
Nothing major, and you can add a :jvm-opts flag to get rid of it, but it annoyed me so much to get rid of this dependency entirely. It was replaced by shadow-http which is a HTTP server I wrote specifically for shadow-cljs. Ideally you won’t notice this change at all. Except maybe that the warnings are gone and some dependencies are gone.
FSWatch replaced
shadow.fswatch is a new implementation of the old watcher. Basically all it does is watch the file system for changes. I mentioned above I needed a polling File Watcher for containers. The default Version still just uses the JVM WatchService.
Previously there was a special implementation for MacOS. This is now gone and replaced with a different one. The reason to replace the old one was again these fine warnings you’d get on every startup.
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::load has been called by com.sun.jna.Native in an unnamed module (file:/Users/thheller/.m2/repository/net/java/dev/jna/jna/5.16.0/jna-5.16.0.jar)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled
Again the mentioned command line flag can silence this, but it still annoyed me. I had some AI token credits leftover, so I decided to try writing an alternate implementation in c, or rather Opus did. It lives here and is a rather small utility using the MacOS API for watching files directly and just prints those changes. On the JVM side a new process is launched and that just reads the output. Rather simple, but not at all convinced this was worth doing. Overall it seems to be doing fine, but I left it as opt-in. It is not used by default until some more extensive testing.
You can start using it by setting {:fs-watch {:impl :macos}}in ~/.shadow-cljs/config.ednor in your project directly. The JVM default works just fine, but it is basically a polling implementation. So it is a bit less efficient and takes a little longer to detect changes. I haven’t noticed big differences in normal work, but I thought I try this option. No idea if it will stay this way though.