I want to package clojure programs for Gentoo Linux. To do that, it seems I need to learn to compile them into uberjar or some other format suitable for system installation.
All I know for now is clojure.core/compile which produces classfiles.
What other things do I need to know?
Different Linux distributions have different approaches to software packages, but I can tell you a bit from my experience creating packages for Ubuntu (long time ago).
Most distros provide their own tools to create both source code and binary packages. Most packages are compressed archives of the binary files of the package plus some metadata about other packages that are runtime dependencies for the software to run. In the case of a Clojure-based package, you want to create a package comprised of your own code and, hopefully, you’ll be able to express that you want to depend on the Clojure Jar and some minimum version of the JDK without having to bundle their binaries in the same package, but as dependencies. This has the advantage of avoiding the download/install of multiple versions of the same JDK, getting security updates for your dependencies and so on.
The details of which tools need to be installed and how you create the medatada files that indicate your package depends on vary with each distro, for Gentoo a good starting point is this page of their wiki: https://wiki.gentoo.org/wiki/Java_Developer_Guide … you’ll want to learn about how to create a basic package for a command line Java program that prints Hello world and introduce the steps for Clojure later (eg. maybe extend the commands from the guide to introduce the step that will call uberjar, etc.).
On the other hand, you may want to reach a larger set of Linux distros by creating a large package with all the dependencies included. This would be a ~200 megabyte package, but you are in control of everything. Some names to start looking for this option are Flatpak and Ubuntu Snaps.
For my Debian/Ubuntu example of a Java package, check this repo. The build system uses Apache Ant with recipes on the build.xml file. The package metadata is derived from the files control for dependencies and rules for the installation steps.
Edit: You can see the details of the files used to create the Clojure package for Gentoo here. You’ll need to create an .ebuild file similar to this one: https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/clojure/clojure-1.10.0.ebuild?id=5ff47c259ceb0f625d61cf590a76a3f532221b75
In case of babashka, an uberjar needs to be created from babashka and its dependencies, and the uberjar needs to be compiled into a binary by GraalVM.
I am not sure that creating a clojure package is similar to creating a java package on Gentoo.
I started with the assumption that you might want to leverage on the Java packages on Gentoo, but you are right it doesn’t have to be the case. With that in mind, I’m thinking of these options:
.ebuild files for Java packages such as Clojure or Tomcat, etc.Something that I don’t know at all is how you create a repository for people to find and consume your package. In Ubuntu, Canonical provided developers the Launchpad.net website and people would create their own Personal Package Archives (PPAs) that were easy to set up as verifiable sources for new software packages to install on Ubuntu. I don’t know that the equivalent would be for Gentoo.
Gentoo has overlays which are just repositories. It has official overlay and third party overlays. I maintain a local overlay and another overlay on github. Gentoo builds from source or downloads binaries. I can choose how I build a clojure package.
I found the other thread where you mention that you want to package babashka for Gentoo.
It seems to me that you can invest a large amount of time figuring how to build everything from source (eg. depend on Clojure, figure the dependency tree for babashka, build the classpath, run babashka’s compile script) or you can just get the Linux binary from the releases page in Github and make the Gentoo package just a basic script that pulls that binary and copies it to /usr/local/bin (or equivalent).
Figuring out how to build a clojure jar or a clojure uberjar is the vast majority of the work. The dependency tree doesn’t account for much.
I was trying to figure out how to build a clojure jar manually. That would be the most flexible approach. But, it seems I should just rewrite https://github.com/borkdude/babashka/blob/master/project.clj in ant and use gentoo’s ant facility.
By the way, I already uploaded dev-lang/babashka-bin to my overlay on github. babashka-bin pulls the binary from the release page.
There are two alternatives to babashka
Joker doesn’t start up super fast, but it is useable. Janet launches super fast and is very easy to package since it is a small self-contained C program. Although janet is not clojure, it can depend on compiled dependencies.
I decided to write a minimal JVM build system for java and clojure packages on Gentoo Linux. The design can be extended to other JVM languages later. I may also create a minimal GraalVM build system for all JVM packages on Gentoo.
Did you see this https://wiki.gentoo.org/wiki/Project:Java
Seems like a similar effort.
Also, I’m a little confused. Is gentoo one of those distro where everything has to be built from source? Is that why you can’t just package the linux binary?
I read the source code. It doesn’t do what I want to do.
I already packaged the binary, but I am still not satisfied. Binary linux distributions usually build from source before distribution. A binary built from another distro may not necessarily work well on Gentoo. Every gentoo system has different binary configurations, so a gentoo binary from one system may not work on another.
Building from source allows me to apply patches and build it for various architectures for which there is no binary.
However, since joker and janet work and integrate well with gentoo’s infrastructure, I may just skip babashka and use joker and janet.
I see, that’s fair. I think what you could try is an intermediate process where you build from the uberjar, but not from source.
In a way, the uberjar is the source for the binary compilation, and that’s the compilation you are most concerned being done as part of Gentoo’s packaging, since it’s the only one that requires linking with Gentoo specific C libs.
If you were to do that, you would only need:
JVMCI-enabled JDK 8 to be present, as well as glibc-devel , zlib-devel and gcc)In that scenario, you don’t need Maven or Ant, and don’t need Lein either. Which maybe might make it easier for you to setup on Gentoo?
I still have to wait for mx to become a python package and for OpenJDK 11 to stabilize on Gentoo Linux in order to package GraalVM.
If you want a from-source build that does not depend on maven I have a set of from-source ebuilds available in my overlay and a pull request at the main gentoo repo.
When I tried to manually build clojure with ant, it needed internet connection. How did you get around internet access requirement?