How to tackle a NPM dependency problem with shadow-cljs?

[witek]

Cross-post from https://stackoverflow.com/questions/58734327/how-to-resolve-npm-dependency-problem-with-shadow-cljs-using-react-swipeable-vie

I get an error from shadow-cljs when trying to use the npm package react-swipeable-views:

The required JS dependency “dom-helpers/transition/properties” is not available, it was required by “node_modules/react-swipeable-views/lib/SwipeableViews.js”

But I can not find dom-helpers/transition/properties anywhere. dom-helpers exists, but no transition in there.

I have no idea how to debug this. Any suggestions? Thanky you!

[jiyinyiyong]

any chance a wrong version of dom-helpers is installed? I remembered some npm has such problems if lock file is not working.

[witek]

This seams to be the case. I had [email protected] installed. I added [email protected] manually.

The old error is gone I got a new one

package in /p/gesetze/node_modules/dom-helpers/addClassspecified entries but they were all missing
{:tag :shadow.build.npm/missing-entries, :entries ["…/cjs/addClass.js"], :package-dir #object[java.io.File 0x2df6b5df “/p/gesetze/node_modules/dom-helpers/addClass”]}
ExceptionInfo: package in /p/gesetze/node_modules/dom-helpers/addClassspecified entries but they were all missing
shadow.build.npm/find-package-main (npm.clj:194)

Seams something else needs the new version. So I have to wait for the maintainer of react-swipeable-views to upgrade, right?

[jiyinyiyong]

…bad reputation for npm.

[witek]

I don’t think this issue is caused by NPM. The problem is in dom-helpers. It introduced breaking changes. They should have created dom-helpers-v2 for the new api. Greetings from Rich: https://www.youtube.com/watch?v=oyLBGkS5ICk

[jiyinyiyong]

if packages maintainers follow semantic version well, it won’t bring so much break changes. plus we have lock files today.

package names like dom-helpers-v2 might cause a lot more problems. We may end up with dom-helpers-v222 after lots of changes, in theory.

[witek]

How does semver help here? dom-helpers followed it by bumping major. shadow-cljs upgraded, react-swipeable-views not. My project depending on both broke. If dom-helpers would start dom-helpers-v2 nothing would break.

Having [email protected] instead of [email protected] seams perfectly fine for me if this prevents breakage.

I think bumping major (introducing breaking changes) should change the modules namespace, so the old and new can coexist.

[thheller]

I somewhat agree that creating a new package would “fix” this problem but I’m not sure this is actually a good way to fix this problem.

In Clojure or on the server-side in general we don’t care too much about “code bloat”. So including several variants of basically the same namespace isn’t a big deal. We should probably care more but usually we don’t.

In ClojureScript or in the “frontend” in general we care very much about final code size. If nothing forces us to resolve “version conflicts” then you might end up including several versions of the “similar” thing in your final output. If you ever tried deduping a bloated webpack build you know that this isn’t fun and rather tedious. webpack will silently just use multiple different versions of a package and not even tell you unless you dig into the details. Needless to say that most people do not do this and will end up with huge chunks of duplicated code. I’ve seen examples of builds that included 5 different versions of one library and so on. Since webpack also just does this silently the package authors often don’t even notice that they have a dependency conflict in their own code.

So I disagree with Rich’s approach of just creating a separate namespace/package as a viable path forward. It does prevent certain kinds of issues but if you care about code size at all you’ll be forced to upgrade code anyways to remove the “duplication”.

PS: It was my choice to not allow duplicated packages in shadow-cljs. It would be possible to support this but I never want this to happen in my builds so I never added it. The whole handling of version conflicts could certainly be improved as well.

[jiyinyiyong]

Being an npm user I already accept that APIs would probably breaking change in APIs if major version is bumped.

I’m also a maintainer of tens of small packages. Some of the packages are somehow experimental and I would break APIs anyway. And there are lots of packages like that in npm world. If there’s a problem, I would say the problem is npm ecosystem enabled too many junior programmers to share their own tiny(too tiny) packages that are not so stable.

[witek]

The community could agree on some convention. Like major can be bumped only once. From 0 to 1. Afer that no breaking changes allowed. If a package does make such changes it should be flagged so everybody can see that using this package is dangerous.

[didibus]

Interesting. I guess ideally everyone would quickly refactor their package to the latest version of their dependencies and everything would always be up to date with latest. That would minimize code bloat. Which is even more reasons for the JS ecosystem to never break APIs ever. Since a breaking change makes this move to the latest version require more work from everyone.

But, what when the ideal doesn’t happen? Now two libraries you might depend on can both require a different version of a third dependency. If the choice is not being able to release your product due to unresolvable conflicts, or having a bit of code duplication (though arguably it’s different code that behaves differently, and so not totally duplicate), which one is worse?

[witek]

I think the situation is even worse. When react-swipeable-views upgrades to the newest dom-helpers it will also break all projects which currently use react-swipeable-views and dom-helpers.

With the breaking changes (which semantic versioning implies to be ok) dom-helpers started a cascade of dependency hell for a lot of projects. And when only one library refuses to upgrade, all projects, which already upgreaded and all projects which depend on upgraded libraries (like mine) are stuck with nothing they can do.

[jiyinyiyong]

And when only one library refuses to upgrade, all projects, which already upgraded and all projects which depend on upgraded libraries (like mine) are stuck with nothing they can do.

Any thing could happen to a poorly maintained package.