Should Linux distributions ship Clojure byte compiled (AOT) or not?

Sorry for the late reply, have been out of town. I don’t think Linux distributions should be recreating published jars, especially using a different build/assembly process - that has always seemed wrong and confusing to me. I would love to be doing more in the Clojure ecosystem to make published libraries verifiable via key infrastructure/provenance/signing, which should be impossible for anyone but the library creator to make (if it is to have any value).

Re AOT and the ABI compatibility etc, I do want to add some precision to this discussion. Clojure compilation is done by some version of the Clojure compiler, with access to that version of the Clojure core libs. The emitted bytecode will (for the last several Clojure versions) be Java 8 era bytecode and will contain calls into the Clojure runtime (particularly the Reflector, RT, and Util classes). This code cannot be run on JVM < version 8 (just like Java code compiled to Java 8 bytecode cannot run on Java 7).

Over time, as features are added to Clojure, there may be new methods added to the Clojure runtime called by the emitted code. We are disciplined about not removing or changing existing internal methods, so compiling with Clojure version X should produce bytecode that will run on Clojure version X+N. But an application may wish to run with an older Clojure runtime, and that may not work. Compiling libs with AOT necessarily makes a choice of minimum Clojure version for later application developers as they cannot run with an older version than was used at packaging time. There is no difference here from Java - jars compiled to a particular bytecode version cannot run on an older JVM. What is different is the ability to target a particular bytecode (and core libs now!) version from a newer Java compiler - having this support takes a lot of effort which the Java team has resources for, but we do not.

So to address the original question - the ABI is NOT “unstable”; it is “stable but evolves in additive ways”. This makes it backward-compatible (new runtime can run old binary), but not forward-compatible (old runtime cannot run new binary). This is not unusual. Java libs make these same choices when they decide what Java version to build with (both in bytecode version and in JDK library API calls). Clojure is more flexible in that it is a source-first language - precompiling all binaries takes that flexibility away.

6 Likes