A lot of libraries depend on jackson-core, which means that it is easy to get another version of jackson-core in your dependency tree while using jsonista. The problem is that if there’s a mismatch between the versions of jackson-databind and jackson-core, you will get cryptic error messages about NoClassDefFoundError.
I’m not very knowledgeable about how Maven handles dependencies, but I’m wondering: is there a way to specify the dependencies that would avoid or reduce the problem?
I’m not very knowledgeable about Maven, but I’ve had a similar issue in a Maven-based hybrid Java/Clojure project, and the solution was to play with the order of the dependencies in the pom. Maven does use the same order in which the deps are listed in the pom for building the classpath, so that could be a way to solve the issue (but of course, you get stuck with Maven)
Hum… I think this problem exists in Java as well. Depending on Jackson has always been a huge pain due to things like that. Which is why in general I prefer Gson.
Shading is a great suggestion even though I don’t think it’s the solution either. It would solve the problem but also cause some other problems at the same time. For example, I think it would break using extension modules like jackson-databind-joda with jsonista. More generally, Jsonista is a thin wrapper for Jackson, so Jackson is essentially part of its API and shading would complicate that.
Also, it’d mean that the users couldn’t upgrade Jackson without a new jsonista release. This could be a pretty big drawback for the security-conscious users because Jackson gets regular security updates.
I have a simple principle to help me get out of this kind problem. When I pick a library for json, I will ensure all the json parse/generate operations will be handled by this single libaray. The underlaying dependencies may be mess but it doesn’t matter if you don’t touch them. What only need to do, is to specify the dependency version to make this library work.
It could be best just not including Jackson at all in jsonista and just mentioning on your guide that the consumer must explicitly depend on tha latest matching Jackson libs.
They would be able to update Jackson independently (AFAICT, that’s the main idea behind the shade plugin, in this case you would have different Jackson + transitive deps at the same time, one for Jsonista, and the other for the app), though presumably they’d also need to update Jsonista, as well…
Still, this is an academic exercise at this point, I didn’t try it yet
(btw, I’m a happy Jsonista user, it did get me a ~20% wall-clock improvement in some code at work, so thanks for that)
Be aware that lein gives priority to the first occurance of a lib in the deps tree, whether explicit or implicit. Thus, a lib you specified as 5.1 may end up being imported as version 3.0 if a transient dependency on 3.0 occurs before an explicit dependency on version 5.1.
To combat the above problem, I segment :dependencies in project.clj into 3 segments:
fundamental libs (absolutely must be the versions I specify)
hi-priority libs (really want the version I choose, and unlikely to be overridden from fundamental group)
“normal” libs (probably get the version I choose, but can’t override anything from first 2 groups)
Note also that each group should be sorted alphabetically, so it is easy to see what is there.
This is so much easier with the new CLI / deps.edn since it will always prefer top-level deps declared directly in your deps.edn – order doesn’t matter.
This seems to have the same issue as my original answer. It works for the library consumers, but it’s an extra step. What @Miikka was looking for was a solution that would allow the jsonista package to work regardless of how the user sets up other Jackson deps.
That being said, other than shading, I don’t know that there’s a good solution for this issue.
Be aware that lein gives priority to the first occurance of a lib in the deps tree, whether explicit or implicit
I don’t believe this is directly true - depending on what you mean by “first occurrence”.
It prioritizes dependencies it finds “closer to the root” in the tree. If it finds the same dependency twice at the same “level” of the tree, it will choose the first (or something like that). This is the implicit, sort of scary path.
The solution to resolve this tends to be to directly declare your transitive dependency version you want at the project-root level. Then you won’t rely on the “same level, first-wins” behavior in the problematic transitive dependencies (eg. jackon-core) that can be unstable as dependencies are shifted around over time.
This seems like the same way aether works to me. The only issue I know of is resolving transitive deps at the “same level” of the tree. Which deps.edn has to have some mechanism to handle - I can’t remember what it chose.
tools.deps.alpha chooses the newer version of a transitive dependency so, given any two versions, you always know which one will get picked. You can “force” a version to be selected either by making it a top-level dependency or specifying it in :override-deps in an alias.
That’s pretty much what my intuition of “first occurrence” would mean. The first thing to depend on a lib gets to chose the version. And yes, that’s what aether and maven do as well.
tools.deps will not do that, it will pick the newest version, not just when there are conflict at the same level of the tree, but overall. So it doesn’t matter who first declared a dependency, or how close you are to the root. The only exception being direct dependencies, those take precedence always.
In my opinion, tools.deps’s resolution is almost always better, because most libs will work hard to be backwards compatible, but almost never will they try to be forward compatible. So if you get an old version, when others expect newer, it will almost never work, but if you get a newer version when others expect older, it will often work.
Yeah, I think the tools.deps resolution to newest version tends to be a better choice. If the newest version doesn’t work, it’s likely that the classpath can’t work with the older version either - so you are in a “dependency hell” situation. It’s certainly less arbitrary than the order-dependence of aether.
I’m believe this resolution strategy in aether is pluggable (I may be misremembering), but either way, I’m sure that’s not easy to tweak from a leiningen perspective.
One (ok’ish?) solution would be to add databind dependency to Cheshire. It doesn’t really need this, but if shortest path to Jackson is via Cheshire, it would nail down both versions. Jsonista doesn’t really need the latests version, mainly just security patches.
Another would be to coordinate the Jackson dependencies in various JSON libs. But if/as people don’t update their libs to use the latest all the time, the issue of version mismatch remains.
