Best practices for importing raw text files into ClojureScript projects?


#1

There are some cases I want to add large text files like mock.json content.md into ClojureScript code. When I was using Webpack, I would use raw-loader which pack my files into strings. I can keep the content I use in files.

What’s the suggested way doing this in shadow-cljs or other ClojureScript bundlers?


#2

You can use a macro from Clojure that reads the file from your resources folder an inline it:

;; in clojure file
(ns useful.macros
  (:require [clojure.java.io :as io]))

(defmacro inline-resource [resource-path]
  (slurp (clojure.java.io/resource resource-path)))


;; in cljs-file
(ns markdown.core
  (:require-macros [useful.macros :refer [inline-resource]]))

(def md-content (inline-resource "md/my-markdown-file.md"))

and md-content will be an inlined string. If you want to preprocess the markdown string during the ClojureScript compilation step, you can call any number of Clojure JVM functions inside that macro.


#3

Depending on how large the resources are you might want to stay away from inlining them via a macro. If you add a megabyte of raw text it will hurt overall performance and it would be better to load the files via XHR separately.

Its fine if you just have small snippets of text but always verify that you don’t add too much.

It also kind of sucks with caching enabled since the code is not recompiled if you change the .md files.

I can probably add a macro that sets some extra flags to inform the compiler that changes to the included file should trigger a recompile. For now you could just block the file from caching but this really sucks.

My recommendation is to just use XHR/fetch.


#4

Not that large actually. A real-world use case is https://github.com/shadow-cljs/shadow-cljs.org/blob/master/src/app/comp/container.cljs#L32 where I have tens of lines in string which is actually part of the code.

Tens of lines of text is large enough to look messy in cljs files. I do want to extract the text into a file, while it still behaves like normal cljs code(text changed, recompiled, reloaded). I think it’s a common usage. I’ve run into several projects where I need to embed Markdown or JSON files in a project.


#5

Clojure is totally fine with multi-line strings so you could just do

(comp-md-block
      "shadow-cljs provides everything you need to compile your ClojureScript code with a focus on simplicity and ease of use.
      
** Good configuration defaults so you don't have to sweat the details
* Supporting various targets :browser, :node-script, :npm-module, :react-native(exprimental)...
* Live Reload (CLJS + CSS)
* CLJS REPL
* Importing CommonJS & ES6 modules from npm or local JavaScript files
* Code splitting (via :modules)
* Fast builds, reliable caching, ...

" ...)

But I understand that including static files makes things nicer.

For now it just sucks due to the caching issue.


#6

For JSON strings it would be worse when there’s no syntax highlighting.


#7

I opened https://github.com/thheller/shadow-cljs/issues/354 and I’ll think about it.

I’ll see if I can figure out a macro-only solution that plays better with caching. No promises though.


#8

As you note, this feature of Webpack is convenient. If you use the double bundle approach (example project) you can package additional dependencies and other non-code things like strings in an additional webpack bundle, and then export them to CLJS using something like window.myData=require("./my-data.txt") in a JS file read by Webpack. See also the official docs


#9

Compile ClojureScript with shadow-cljs with :target :commonjs, that would be better than modifying window.myData.


#10

I ended up running with this piece of code in shadow-cljs:

(defmacro inline-resource [resource-path]
  (slurp resource-path))

What does clojure.java.io/resource do actually? There was an error when I was using it:

   4 | (def files
   5 |   [{:name "Component base", :content (inline-resource "snippets/component-base.tsx")}])
--------------------------------------------^-----------------------------------
IllegalArgumentException: Cannot open <nil> as a Reader.
	clojure.java.io/fn--10992 (io.clj:288)
	clojure.java.io/fn--10992 (io.clj:288)
	clojure.java.io/fn--10894/G--10870--10901 (io.clj:69)
	clojure.java.io/reader (io.clj:102)
	clojure.java.io/reader (io.clj:86)
	clojure.core/apply (core.clj:659)
	clojure.core/slurp (core.clj:6862)
	clojure.core/slurp (core.clj:6862)

#11

You might have to add snippets parent folder to the classpath.
If the resource does not exist on the classpath a nil is returned.

(io/resource "I_do_not_exist.txt")
;; => nil

#12

so it’s used combining with classpath… I would stick to slurp.


#13

io/resource is meant to find the file in the resources/ folder which will be packaged in an uberjar or be present during aot compilation. It restricts the path of the file to somewhere you know it will always be found. If you want to grab any file in your system, just slurp is a better idea.


#14

slurp sounds better in my case. Using in ClojureScript, I don’t want jar files.


#15

Just to clarify: io/resource is used to access any classpath resource. It is not limited to a resources/ folder. It can be in any of your :source-paths or any of the .jar files on the classpath.

As far as CLJS compilation is concerned its fine to use files but if you are writing a library you must use classpath resources instead as the files won’t otherwise be accessible in the library.