Using none-code resources in CLJS builds

There are various use cases where you’d want to use a static resource as part of the compilation of you source files. You may just want to include some longer text from a file or read a file, parse it and generate some code for it via macro.

This works fine and is done quite often all over but common implementations suffer from the problem that the compiler doesn’t know that these other resources were used and won’t invalidate caches properly or trigger re-compiles when those resources were modified.

As of shadow-cljs@2.7.29 there is now a built-in way that is fully supported by the compiler and will properly invalidate caches and trigger live-reload in watch mode.

(ns demo.app
  (:require [shadow.resource :as rc]))

(def x (rc/inline "./test.md"))
;; ends up as
(def x "the contents of test.md")
;; or in compiled JS
demo.app.x = "the contents of test.md";

This will resolve the "./test.md" file relative to the current namespace, which means it will end up including demo/test.md from somewhere on the classpath. You can of course use (rc/inline "demo/test.md") if you prefer that.

The macro used for this isn’t actually much different from what other macros of this kind would do. The only difference is that it records that it did use the resource in the analyzer data for the compiled namespace.

shadow-cljs will look for this analyzer data after compilation and properly hook up the cache invalidation and watch those extra files and trigger recompiles when modified.

This is currently just a proof of concept but I think this would be a useful thing in general and since its just metadata we could use a more generic key (eg. :cljs.resource/refs not :shadow.resource/resource-refs) so that other tools could eventually also make use of this extra data.

If you have macros like this you can either add this compiler data yourself or use shadow.resource/slurp-resource to read a resource which will also record that it was used and properly cause recompiles and so on.

Avoid using this for larger files since the results will become part of your build. Don’t include 5mb of text this way, load it at runtime instead.

17 Likes

Oh man, I was just thinking about this today, pondering whether I should bother you or not. Do you have a crystal ball???

Thanks a million times, and get a Patreon page already!

3 Likes

This is incredibly timely :smiley: We were looking at doing something like this :wink: Thanks a lot!

1 Like

What if I’m not putting the file in a folder in classpath? Say my code is:

(inline "../../../introduction.md")

and I’m sure the file does exist:

=>> l ../../../introduction.md
../../../introduction.md
=>> pwd
/Users/chen/repo/shadow-cljs/shadow-cljs.org/src/app/comp

What does the error mean?

Caused by:
Resource not found: ../introduction.md at line 32 app/comp/container.cljs

Oh, it had to be in classpath since the function io/resource

Yes, this is currently limited to the classpath since it otherwise won’t work in libraries.

I may add support for regular files at some point but I don’t think its necessary at this point.

2 Likes

Just to share a bit of joy with you. We had a ClojureScript pipeline generator that had ugly inline strings that represented bash scripts to run in the pipeline.

Nobody liked it of course because the scripts were inline and it was difficult to try them out.

Now this is what we have:

(def npm-authenticate-script (rc/inline "npm-authenticate.sh"))
(def npm-install-deps-script (rc/inline "npm-install-deps.sh"))
(def apt-install-deps-script (rc/inline "apt-install-deps.sh"))
(def assert-env-script (rc/inline "assert-env.sh"))
(def yaml-to-json-script (rc/inline "yaml-to-json.sh"))
(def upload-definitions-script (rc/inline "upload-definitions.sh"))
(def upload-dsls-script (rc/inline "upload-dsls.sh"))

And everyone is happy! :tada:

6 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.