Is there a standard way to use npm ES6 modules in clojurescript?

With shadow-cljs,

(require '["module-name" :refer (var)])

is sufficient.
In plain clojurescript repl, the same code results in error.
I’m looking for ways to build clojurescript projects with only plain clojurescript.

In particular, I like to utilize GitHub - StardustCollective/dag4.js: Constellation Hypergraph JavaScript API in clojurescript.

The fact that shadow-cljs variant of clojurescript is different from plain clojurescript makes me want to not use clojurescript.

I honestly don’t know if this is possible in the core build tools nowadays.

shadow-cljs itself is a build tool. As such it provides many different features that the core build tools may or may not support. The language itself however is the same. The generic npm support is one of the key features of shadow-cljs and can do more than the default CLJS core tooling. One limitation of the core tools is that it currently (I believe) cannot load npm packages dynamically (eg. REPL). You may however load them as part of a regular build (eg. :target :bundle).

How come?

It is my philosophy that it is OK to add features to the build tooling without them being available in the core tools prior. A few features of shadow-cljs have made it into the code tools over time (eg. :modules) and I’m sure if someone were to work on adding “dynamic” npm support to the core tools that would get accepted.

2 Likes

I want to package node.js command line programs for gentoo linux. Gentoo linux fetches all dependencies and then execute build commands offline for each gentoo linux package. Or, it installs all dependencies as separate packages.

I don’t think shadow-cljs can be integrated with gentoo linux package system.

It seems difficult to turn JVM/nodejs programs into linux distribution packages.

@catdog you may be able to accomplish what you want with nbb.

Running any CLJS compiler as part of a package install seems like absolute madness to me but you can do it with shadow-cljs just like any other CLJS tool. If it can run java and construct a classpath it can run shadow-cljs.

I haven’t touched gentoo in decades so I don’t know what it is like nowadays. If you can point me towards a typical npm package (eg. react) and how that is integrated into gentoo I can probably tell you how to get shadow-cljs to do the same.

To me it sounds much more straightforward to just build and publish your node command line programs to npm and use it from there? Why bring gentoo into the mix?

Also note that your described use case will work with any CLJS tools given that is targetting node and also just building stuff. (require '["module-name" :refer (var)]) I assume would work fine in the default node REPL?

I was thinking about a way to turn an npm library into a gentoo package.

npm.eclass could contain bash functions that

  • install dev-node/test:3.5.1 into /usr/share/node_modules/test:3.5.1
  • install dependencies of test:3.5.1 as symlinks in /usr/share/node_modules/test:3.5.1/node_modules/
  • install a symlink to /usr/share/node_modules/test:3.5.1 in /usr/share/node_modules/anything_that_depends_on_test:3.5.1/node_modules

I used slot dependencies in my scheme because slot dependencies can cover a range of versions.

I was also thinking about using slot dependencies in jvm-jar.eclass, jvm-uberjar.eclass, clojure-jar.eclass, clojure-uberjar.eclass, shadow-cljs.eclass, …

It would be a set of eclasses that are essentially a very simple build system that replaces maven, ant, gradle, shadow-cljs, etc, … shadow-cljs likely isn’t going to work well with eclasses, but cljs.jar is more likely to integrate with eclasses.

I always forget to update programs and libraries on language-specific dependency managers. I just want everything to be updated during system update with one command. Gentoo linux already has eclasses for building Go packages and Rust packages. Haskell’s cabal integrates well with gentoo linux.

I just tried to build this with cljs.jar and succeeded.

(ns app
  (:require ["@stardust-collective/dag4" :refer (dag4)]))

(defn -main
  [& args]
  (print dag4)
  (print [1 2 3])
  (println "ok"))

(if (nil? *main-cli-fn*)
  (set! *main-cli-fn* -main))

cljs.jar can indeed import npm ES6 modules. Thus, I can potentially use shadow-cljs for development and cljs.jar for making a gentoo linux package.

The following code doesn’t work.

(ns app
  (:require ["@stardust-collective/dag4" :refer (dag4)]
            ["node-fetch" :refer (fetch)]))

(defn -main
  [& args]
  (print dag4)
  (print fetch)
  (print [1 2 3])
  (println "ok"))

(if (nil? *main-cli-fn*)
  (set! *main-cli-fn* -main))

because node-fetch is an npm ES module.

require() of ES modules is not supported.

That is a node issue and have very little to do with CLJS. Just google that error message.

Does cljs translate every dependency into require()?
require() cannot import an ES6 module according to nodejs documentation.
CLJS has to write

import fetch from "node-fetch"

or

const fetch = await import("node-fetch");

shadow-cljs does support generating ESM code as explain in this post. That however is only supported by shadow-cljs, so your desired setup using cljs.jar will not work with this attempt. The default tools cannot emit ESM import and only support require().

use v2 of node-fetch. v3 only allows the esm format.

build.edn

{:main app
 :target :nodejs
 :output-to "app.js"
 :foreign-libs [{:file "src/js"
                 :module-type :es6}]}

src/js/fetch.js

import fetch from "node-fetch"

export const func = fetch;

src/main/app.cljs

(ns app
  (:require ["@stardust-collective/dag4" :refer (dag4)]
            ["fetch"]))

(defn -main
  [& args]
  (print dag4)
  (print fetch/func)
  (print [1 2 3])
  (println "ok"))

(set! *main-cli-fn* -main)
$ java -cp "cljs.jar:src/main" cljs.main -co build.edn --target node -c app
events.js:377
      throw er; // Unhandled 'error' event
      ^

Error: Parsing file /path/to/cljs-program/node_modules/node-fetch/src/headers.js: Unexpected token, expected ( (259:12)
    at Deps.parseDeps (/path/to/cljs-program/node_modules/@cljs-oss/module-deps/index.js:483:28)
    at getDeps (/path/to/cljs-program/node_modules/@cljs-oss/module-deps/index.js:415:40)
    at /path/to/cljs-program/node_modules/@cljs-oss/module-deps/index.js:399:32
    at ConcatStream.<anonymous> (/path/to/cljs-program/node_modules/concat-stream/index.js:36:43)
    at ConcatStream.emit (events.js:412:35)
    at finishMaybe (/path/to/cljs-program/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:475:14)
    at endWritable (/path/to/cljs-program/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:485:3)
    at ConcatStream.Writable.end (/path/to/cljs-program/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:455:41)
    at DestroyableTransform.onend (/path/to/cljs-program/node_modules/readable-stream/lib/_stream_readable.js:577:10)
    at Object.onceWrapper (events.js:519:28)
Emitted 'error' event on Deps instance at:
    at Deps.parseDeps (/path/to/cljs-program/node_modules/@cljs-oss/module-deps/index.js:483:14)
    at getDeps (/path/to/cljs-program/node_modules/@cljs-oss/module-deps/index.js:415:40)
    [... lines matching original stack trace ...]
    at Object.onceWrapper (events.js:519:28)

I don’t understand why this setup requires @cljs-oss/module-deps npm module in the first place.
@cljs-oss/module-deps fails to parse node-fetch.
ClojureScript - JavaScript Modules (Alpha) doesn’t mention @cljs-oss/module-deps.

When will clojurescript support importing npm es modules?