Generating ES Modules (Browser, Deno, ...)

Projects shadow-cljs
9

BTW, eval loader mode made hot-reload work while I was developing a plug-in in vscode - it’s the only way to make hot reload work on the webview, so thanks for that :smile:

10

Let me see if I got it, what you are telling me is that if I want to export web components, each one distributed as a singular package in npm, even if all of them have a dependency in cljs.core and react, these dependencies would be too heavy (because they wouldn’t pass by the compiler advanced mode) and the components wouldn’t work if I define two of them in the same page (maybe I didn’t understand the “compatible with each other” very well), is that right?

But I could create like a “library of components” declaring the common dependencies in the shared module, so they would pass by the compiler advanced mode, right?

11

I’m not sure I understand your question but as a general rule of thumb the JS users should only ever include output from a single CLJS build in their projects. You may use :modules to split that build into multiple files and allow the user to include single or multiple modules.

As soon as the JS users include output from multiple CLJS builds they will have duplicated code and things become too large to be practical quickly.

2 Likes
12

Hey Thomas! I was wondering what the current state of the :esm target is with shadow-cljs but maybe also generally with the ClojureScript compiler?

Snowpack seems quite interesting and the ESM target appears to be a nice way to bridge the divide between ClojureScript and existing JS tooling. (Or at least it looks like that from my superficial level of understanding. :slight_smile:)

1 Like
13

I’m waiting for some things in the Closure Compiler to land before making any other changes.

As far as shadow-cljs is concerned the support is mostly done but I haven’t used it in a real project so there might be issues I’m not aware of. As far as CLJS proper is concerned I don’t know. I’m not aware of anyone working on it.

Eventually I’d like to write a CLJS variant that actually just outputs “modern” JS (ES2018+ or so) instead of the old and “dead” Closure JS. That would however be a completely new compiler since we really can’t retrofit that into the current one. There are drawbacks to doing this though so this is very low priority for me.

2 Likes
14

Why do you say “dead”? Still seems alive and well maintained by Google. GitHub - google/closure-compiler: A JavaScript checker and optimizer. I see a commit just 1 hour ago from this writing.

15

The Closure Compiler is alive and well, that is not what I meant by Closure JS.

Closure JS is the “old” module format that used goog.provide and goog.require to do namespacing related things. That style is dead and the Closure Library is slowly migrating to the somewhat newer goog.module style or ESM directly. It is still well supported and not going anywhere but the Closure Compiler is at a point where the newer formats are preferred. The CLJS compiler currently emits that “old” style which is pretty much the reason we sometimes have issues integrating with other JS tools since most other JS tools (besides the Closure Compiler) don’t understand that format properly.

2 Likes
16

Oh I see, I didn’t know Closure had itself slowly migrated to ESM.

17

Interesting, are you aware of any further discussions/tickets weighing the pros and cons of emitting goog.module or ESM? Briefly searched Jira but didn’t find much.

18

It would be ESM only if anything. goog.module doesn’t buy us anything over what we currently have.

Not aware of any “archived” discussions about the topic, talked briefly in #cljs-dev some time ago. If we had pure ESM output basically any JS tool out there would work out of the box. Heck you could even load the output directly in the Browser without bundling. Everything else is pretty much downside since REPL and hot-reload get much harder but it might be worth for some interesting JS-based tools so we don’t have to re-invent everything all the time (eg. storybook).

Closure Compiler is still the best option because of :advanced so I’d never want to give that up but for some tools that just not needed.

1 Like
19

Hi @thheller,

I see the two Google Closure PRs you mentioned above have been merged and you have recently pushed things to shadow (at least https://github.com/thheller/shadow-cljs/commit/8b7f3964e732bc13a9981ba390908dd90f70f2e0 and https://github.com/thheller/shadow-cljs/commit/7575a7d8533be396cd6a630e6aa36848abebbf0d) related to ESM.

Would you recommend us giving this a spin or are you aware of blocking issues still?

Thank you for your great work as always.

1 Like
20

Doing what exactly? :browser is still the best option for everything browser based.

The changes that landed mostly don’t affect what shadow-cljs does, just maybe make things a little less hacky. Overall I would still only recommend ESM for targets where no better option exists (eg. Deno). For the browser or node the other targets are still way better.

21

Doing what exactly? :browser is still the best option for everything browser based.

I’d be interested the following:

  1. using snowpack.dev for local caching. We already load deps like plotly or vega on demand using d3-require but they currently don’t work when offline.
  2. also advanced compiling modern js (typescript) deps like GitHub - codemirror/view: DOM view component for the CodeMirror code editor assuming that the emitted js is Closure compatible.
  3. enabling easy consumption of our ClojureScript code from js
  4. the isolation provided by ESM allowing us to run different versions of the same code alongside each other

After writing the above it’s beginning to dawn on me that only the last two points are really related to switching to ESM modules for our ClojureScript build, is that correct?

22

Sort of yes.

  1. You can also do this with :target :browser. The tricky part starts when the dependencies you are importing this way (eg. dynamic import()) also start importing other dependencies. Since shadow doesn’t know about these you may end up with many different versions of certain things. Not a big deal for your project but potentially blocker for others.

  2. Not related to :target :esm at all. :js-provider controls this. In theory you could put all the JS on the classpath but the odds of that working successfully are not that great. All ESM code on the classpath will the go through :advanced. Technically there could be a variant of :js-provider :shadow that tries to put some node_modules code through :advanced as well but I’d say it is way too early for that and the overall messiness of ESM on npm needs to clear up a little first. Most of it isn’t actually “standard” and relies on some new idioms the JS world invented (in particular webpack) that aren’t standard or even documented fully. My hope for ESM “fixing” things in the npm ecosystem has disappeared. Figuring all this stuff out is painful and I still don’t have a clue where this is going. Definitely waiting for things to settle before investing more energy here.

  3. That would likely improve with ESM yes.

  4. Only really possible with release builds as the watch/compile builds fake ESM and still actually live in the global scope so having two of those would break things again.

:target :esm currently makes the most sense when shadow-cljs is not bundling any JS dependencies at all and instead those are only loaded via true ESM. Currently Deno is the only runtime where this is a practical possibility since everything is ESM/TS. However snowpack might be an option too but I haven’t tried in a long time but that is about the same area as using webpack today. It might work though.

23

Thanks for all those clarifications @thheller!

Think we’ll give snowpack.dev a try, together with skypack.dev (oh javascript!) and report back. From https://www.snowpack.dev/posts/2021-01-13-snowpack-3-0 you see that it enables you to write this

// you do this:
import * as React from 'react';
// but get behavior like this:
import * as React from 'https://cdn.skypack.dev/react@17.0.1';

So as I understand you get local caching and version pinning from snowpack and ESM modules compatible with the browser and a CDN from skypack.

1 Like
24

Although my thread is no longer ClojureScript… calcit-js does explored persistent data structure combining with ES module syntax, that says, emitting code with import / export. It’s still a very young project, but in case anyone wants to try:

25

I read this post earlier. It’s mostly about Rails and their approach to JS but it also mentions a lot of the benefits in using ESM as release artifact.

With HTTP2, you no longer pay a large penalty for sending many small files instead of one big file. A single connection can multiplex all the responses you need. No more managing multiple connections, paying for multiple SSL handshakes. This means that bundling all your JavaScript into a single file loses many of its performance benefits (yes, yes, tree-shaking is still one).

When you bundle all your JavaScript modules in a single file, any change to any module will expire the entire bundle. […]

[…] you no longer need bundling for performance, you can get rid of the bundler entirely! Simply serve each module directly, as their own file, directly to the browser.

Some things I’m wondering:

  • The above sounds really nice in a lot of ways, why is it not?
    • Gzipping would be much less impactful is one thing I can think of.
  • Output files from different CLJS builds are not compatible. Does this effectively make the ESM approach unusable?

Superficially the ESM approach seems similar to how ClojureScript dynamically loads files during development.

I guess whether this is interesting completely depends on how effective this new delivery approach is compared to the bundle(s) strategy. The caching of rarely changing dependencies seems like a pretty great benefit though.

26

In my view ESM changes nothing in regard to why you’d use a bundler in the first place. All the benefits you get still apply. Many people have done experiments and shown that pure unbundled ESM is still way way slower than bundled/optimized JS. You can still gain some of the benefits by using ESM as the final target though.

I don’t believe that you’ll write any serious “web app” without a bundler anytime soon.

Yes, as a compile-to-JS language CLJS has the extra challenge of having to put the “big” cljs.core somewhere and :advanced isn’t extensible later so pre-compiled libs are a challenge. However I don’t think that bundler-less “scales” well enough for even regular JS projects.

Always works well with cherry-picked examples that are directly solved by library X but completely falls apart if you need to do “more”.

1 Like
27

The title of this post contains ES Modules (Browser, Deno). Note that with the help of @thheller I’ve been able to use this to build a CLJS scripting runtime for Node.js as well:

I was looking for a shadow-cljs Node.js target that supported code splitting, so I could split out dependencies which can be loaded on demand, to get better startup time. The shadow-cljs :esm target supported code splitting, but none of the other Node-related stuff did, so I went with this. Later it turned out that :esm was a good choice in one other aspect: it’s the only target which lets you use dynamic import. Since more and more libraries publish as ES Modules (term-size is an example) I needed the :esm target to build a runtime which allows to load these libraries.
Kudos to @thheller for building this target: I would not have figured this out using CLJS alone (moreover: vanilla CLJS doesn’t work at all with ES modules at the moment). The :esm target came at the right time for something like nbb.

2 Likes
28

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.