Depending on the right versions of Jackson libraries

jsonista has an annoying problem. It depends on jackson-databind which in turns depends on jackson-core:

 [com.fasterxml.jackson.core/jackson-databind "2.10.0"]
   [com.fasterxml.jackson.core/jackson-annotations "2.10.0"]
   [com.fasterxml.jackson.core/jackson-core "2.10.0"]

A lot of libraries depend on jackson-core, which means that it is easy to get another version of jackson-core in your dependency tree while using jsonista. The problem is that if there’s a mismatch between the versions of jackson-databind and jackson-core, you will get cryptic error messages about NoClassDefFoundError.

I’m not very knowledgeable about how Maven handles dependencies, but I’m wondering: is there a way to specify the dependencies that would avoid or reduce the problem?

I’m not very knowledgeable about Maven, but I’ve had a similar issue in a Maven-based hybrid Java/Clojure project, and the solution was to play with the order of the dependencies in the pom. Maven does use the same order in which the deps are listed in the pom for building the classpath, so that could be a way to solve the issue (but of course, you get stuck with Maven)

Hm, I don’t think that playing with the order of deps is going to help me as a library author, although it can solve problems for the library users.

indeed, that’s true.

Hum… I think this problem exists in Java as well. Depending on Jackson has always been a huge pain due to things like that. Which is why in general I prefer Gson.

@Miikka, would it be an option to ship a fat jar with your Jackson deps shaded?

From here https://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html it would seem that you can ship an unberjar with only the Jackson deps included, and you can shade those to avoid the conflict.

Edit, I was thinking about transitive deps, and found this, with the example being Jackson DataBind :slight_smile: https://stackoverflow.com/questions/28458058/maven-shade-plugin-exclude-a-dependency-and-all-its-transitive-dependencies

Shading is a great suggestion even though I don’t think it’s the solution either. It would solve the problem but also cause some other problems at the same time. For example, I think it would break using extension modules like jackson-databind-joda with jsonista. More generally, Jsonista is a thin wrapper for Jackson, so Jackson is essentially part of its API and shading would complicate that.

Also, it’d mean that the users couldn’t upgrade Jackson without a new jsonista release. This could be a pretty big drawback for the security-conscious users because Jackson gets regular security updates.

I have a simple principle to help me get out of this kind problem. When I pick a library for json, I will ensure all the json parse/generate operations will be handled by this single libaray. The underlaying dependencies may be mess but it doesn’t matter if you don’t touch them. What only need to do, is to specify the dependency version to make this library work.

It could be best just not including Jackson at all in jsonista and just mentioning on your guide that the consumer must explicitly depend on tha latest matching Jackson libs.

They would be able to update Jackson independently (AFAICT, that’s the main idea behind the shade plugin, in this case you would have different Jackson + transitive deps at the same time, one for Jsonista, and the other for the app), though presumably they’d also need to update Jsonista, as well…

Still, this is an academic exercise at this point, I didn’t try it yet :slight_smile:

(btw, I’m a happy Jsonista user, it did get me a ~20% wall-clock improvement in some code at work, so thanks for that)

1 Like

I am watching @alexmiller’s talk at Conj, and this came up :smiley:

3 Likes

I have had to list all of them explicitly, like this example, near the top of the project.clj file:

  [com.fasterxml.jackson.core/jackson-core "2.10.1"]
  [com.fasterxml.jackson.core/jackson-databind "2.10.1"]
  [com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.10.1"]
  [com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.10.1"]

Be aware that lein gives priority to the first occurance of a lib in the deps tree, whether explicit or implicit. Thus, a lib you specified as 5.1 may end up being imported as version 3.0 if a transient dependency on 3.0 occurs before an explicit dependency on version 5.1.

To combat the above problem, I segment :dependencies in project.clj into 3 segments:

  • fundamental libs (absolutely must be the versions I specify)
  • hi-priority libs (really want the version I choose, and unlikely to be overridden from fundamental group)
  • “normal” libs (probably get the version I choose, but can’t override anything from first 2 groups)

Note also that each group should be sorted alphabetically, so it is easy to see what is there.

This is so much easier with the new CLI / deps.edn since it will always prefer top-level deps declared directly in your deps.edn – order doesn’t matter.

This seems to have the same issue as my original answer. It works for the library consumers, but it’s an extra step. What @Miikka was looking for was a solution that would allow the jsonista package to work regardless of how the user sets up other Jackson deps.

That being said, other than shading, I don’t know that there’s a good solution for this issue.