Shadow-cljs 2.25.2 looking for testers

Update: Fixed a couple issues, try 2.25.2

I don’t usually do these release announcements, but I just released version 2.25.2 of shadow-cljs and it includes a fairly significant change, where I’d like to crowdsource some more testing. Ideally nobody will notice a thing, but history has taught me that there is a lot of weird code on npm and I probably didn’t account for all of it.

The release changes how ES Modules from node_modules are processed. If that doesn’t mean anything to you that is fine, you shouldn’t need to know how any of this works. Just verify that your build still works and report back if you get new (possibly strange) errors.

If you are interested in some background: There is some code published to npm as “modern” ESM code, meaning files that include import and export vs. the older much more common CommonJS style that uses require and module.exports or just exports. Trouble is that these two systems are fundamentally incompatible, so the code needs to be rewritten.

I opted to stay with the common approach of rewriting import/export to require/exports, i.e. rewriting ESM to CommonJS. All of this was previously handled by starting a background node process which then ran a precompiled version of the babel-worker script, which basically just called babel. Each ESM file was then sent to the node process, converted and sent back. Not perfect, but did the trick.

I always hated this approach and wanted to get rid of it since I wrote it. However, at the time investigating what babel actually did and replicating the same in my own code was substantially more work. It also was at a time where it wasn’t totally clear where this was headed and just staying with babel seemed best.

Nowadays things have settled down and things haven’t changed much. I also figured out what babel actually did over time and decided it was time to implement everything directly. Most code actually already existed in the Closure Compiler, so rewriting it to what shadow-cljs needs wasn’t all that much work.

The old babel path is now behind a flag :js-options {:use-babel true}, but the goal is to drop that entire infrastructure at some point. Please note that this was only ever really used for rewriting ESM to CJS, not running babel in some generic way.

Please upgrade to 2.25.2 and report back any breakage. Unfortunately this is a fairly low level change, so the symptoms it may surface can be rather widespread and weird. If you use a lot of npm packages you are more likely to be affected by this. I tested a few common packages which appeared to work fine, but I can’t test them all. Ideally everything still works and will probably be a lot faster too.

10 Likes

Tested on an electron app project, seems like it built correctly.

LMK if there’s anything specific to test past build.

Your log is showing

shadow-cljs - server version: 2.19.0 running at http://localhost:9630

So, you are not running 2.25.2. Looks like you are using deps.edn, in which case you need to upgrade the thheller/shadow-cljs dependency there.

Compilation is probably going to be fine, need to find out about runtime issues.

$ ls $NODE_PATH | wc -l
286

$ shadow-cljs --cli-info                                                
shadow-cljs - config: /redacted/shadow-cljs.edn
=== Version
jar:            2.25.2
cli:            2.25.2
deps:           1.3.4
config-version: 2.25.2

=== Paths
cli:     /nix/store/1vrmzm3p4kipm30bi88k875fcasbv1b0-node-dependencies-redacted-0.0.0/lib/node_modules/shadow-cljs/cli/dist.js
config:  /redacted/shadow-cljs.edn
project: /redacted
cache:   .shadow-cljs

=== Java
openjdk version "19.0.2" 2023-01-17
OpenJDK Runtime Environment (build 19.0.2+7-nixos)
OpenJDK 64-Bit Server VM (build 19.0.2+7-nixos, mixed mode, sharing)

=== Source Paths

$ shadow-cljs compile tests                                  
shadow-cljs - config: /redacted/shadow-cljs.edn
shadow-cljs - starting via "clojure"
[:tests] Compiling ...
========= Running Tests =======================

[... redacted ...]

Ran 166 tests containing 766 assertions.
0 failures, 0 errors.
===============================================
[:tests] Build completed. (364 files, 2 compiled, 1 warnings, 48.23s)

That warning is not new :wink: Hope that helps!

Can’t tell what you are building unfortunately. If the tests run in node (e.g. :target :node-test) then shadow-cljs won’t be processing npm dependencies at all and therefore won’t be affected by the changes made in 2.25.*.

Only really matters for builds where shadow-cljs actually bundles dependencies, e.g. :target :browser. If everything loads fine in the browser you are most likely fine. If you are testing with something that runs against an actual browser (e.g. puppeteer) you are likely fine too.

Ah too bad, then my result is probably not useful for you, it is indeed using :target :node-test.

Ok, updated the shadow-cljs dep in deps.edn and runtime tested the following npm libs, no issues so far.

  • react-datepicker
  • react-select
  • autosize
  • mustache
  • id128
  • three
  • sqlite3
shadow-cljs - config: /Users/zk/code/float/edie/shadow-cljs.edn
=== Version
jar:            2.25.0
cli:            2.25.0
deps:           1.3.4
config-version: 2.25.0

=== Paths
cli:     /Users/zk/code/float/edie/node_modules/shadow-cljs/cli/dist.js
config:  /Users/zk/code/float/edie/shadow-cljs.edn
project: /Users/zk/code/float/edie
cache:   .shadow-cljs

=== Java
openjdk version "20.0.1" 2023-04-18
OpenJDK Runtime Environment Homebrew (build 20.0.1)
OpenJDK 64-Bit Server VM Homebrew (build 20.0.1, mixed mode, sharing)

=== Source Paths

(base) ~/code/float/edie(main*) $ bin/dev-repl                                                                                                                                                                                       11:05:37
shadow-cljs - config: /Users/zk/code/float/edie/shadow-cljs.edn
shadow-cljs - starting via "clojure"
shadow-cljs - server version: 2.25.0 running at http://localhost:9630
shadow-cljs - nREPL server started on port 53598
shadow-cljs - watching build :browser
[:browser] Configuring build.
[:browser] Compiling ...
[:browser] Build completed. (654 files, 0 compiled, 0 warnings, 5.36s)
1 Like

I have a project here that is giving me new errors after the upgrade to 2.25.2. When I add :js-options {:use-babel true}, these errors go away again.

Shadow error output:

shadow-cljs - config: /some/path/shadow-cljs.edn
shadow-cljs - socket connect failed, server process dead?
shadow-cljs - starting via "clojure"
[:app] Compiling ...
Closure compilation failed with 38 errors
--- node_modules/react-easy-panzoom/src/PanZoom.js:79
Transpilation of 'Member references this or super' is not yet implemented.
--- node_modules/react-easy-panzoom/src/PanZoom.js:113
Transpilation of 'Member references this or super' is not yet implemented.
--- node_modules/react-easy-panzoom/src/PanZoom.js:124
Transpilation of 'Member references this or super' is not yet implemented.
--- node_modules/react-easy-panzoom/src/PanZoom.js:160
...

All the errors are only for this one file.

after some digging …

This error occurs with a github url package.json dependency. We have a fork of of https://github.com/mnogueron/react-easy-panzoom which we include in package.json like this:

        "react-easy-panzoom": "github:nette-io/react-easy-panzoom#0b44b433dac3da0ae2736a648413ef542f180dfb",

I experimented by removing that and doing the normal npm install react-easy-panzoom, which changed the package.json file to:

        "react-easy-panzoom": "^0.4.4",

With the dependency specified like this, shadow-cljs compiled the project fine, so I assume the issue is somehow related to this github URL dependency.

Hope this helps.

The errors are from the Closure Compiler and a specific feature it doesn’t support. No clue which one that is in particular. Could be that either your fork added some code that triggered this, or that 0.4.4. is just newer and fixed/transpiled that code?

I just noticed that you are including the sources in the src dir somehow? The actual package however has the “compiled” files in lib. I don’t expected the sources to work directly, since they seem to be flow js with some type references. Are you maybe missing a build step in your fork?

1 Like

Our fork changes very little and is based on 0.4.4, so I’m assuming your second hint (missing build step) might be the avenue to persue. I’m missing some nodejs-specific knowledge here about when the build step should run or how to trigger it. I assumed that the github url dependency would somehow “just work”.

How long will the use-babel flag be around for? With this set to true we don’t have this problem, which buys us a little bit of time to sort this out.

The lib doesn’t have build instructions, but by looking at the package.json I’d guess you run

npm run lib

That should build the lib dir, with the proper .js files shadow-cljs and the closure compiler will understand. Also need to revert this change so that the lib files are actually used. Looks like you also did a whole host of other changes so that shadow-cljs could build this directly, they are also probably all obsolete.

Your use of :use-babel is totally coincidental and not an intended feature, never was.

1 Like

Thank you for taking the time to look and suggest changes.

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.