Clojure requires spec.alpha and core.specs.alpha.
Spec.alpha and core.specs.alpha require clojure.
This is a cyclical dependency loop that I don’t know how to take care of, yet.
I’m trying to create a gentoo package for clojure in my own way, and I don’t know how to create a source package for clojure.
The spec.alpha and core.specs.alpha libraries were originally bootstrapped off a version of Clojure that did not yet depend on them, and then ever since they’ve been able to leverage an “existing” version of Clojure (which in turn leveraged the “previous” versions of both Spec and the core specs).
Since Spec was introduced in the prerelease phase of Clojure 1.9, you’d have to go back to an early prerelease of 1.9 and build that Clojure first, then build Spec, then core.specs.alpha, then build a later version of Clojure using those, then rebuild them with that version of Clojure (and, probably, iterate around this cycle multiple times – since all three have had many releases since then and they are becoming increasingly intertwined).
Today, if you build Clojure from (its own) source, it relies on a number of existing libraries – but you can’t easily start with everything being source code anymore. This makes satisfying the various O/S packagers’ rule about “build everything from source” very hard – but I don’t believe that things like Clojure should be packaged for the O/S: we have solid tooling for dealing with prebuilt libraries (e.g., Maven) and O/S packaging is not designed for that higher-level world.
I recently complained on Twitter about running brew install emacs because it downloaded MySQL 5.7 and built it from source as part of that… I mean, seriously?
I want to turn clojure programs, java programs, and npm programs into gentoo packages. It would be simplest for gentoo users. It is easy to turn haskell packages and janet pakcages into OS packages.
I install and upgrade all programs including hledger(a haskell programs), using gentoo package manager. It is the best experience. I really really really really really really don’t want to manage multiple package managers. I can’t manage multiple package mangers that don’t play with each other.
Also, some java programs aren’t even installable with a java package manager like maven.
Good luck. You’ve got your work cut out for you and you’re very much serving a minority market there.
Perhaps, JVM ecosystem can make it easy to turn JVM packages into OS packages.
Why should the JVM ecosystem change to satisfy a small market for a single O/S? The JVM is available on every platform. Maven and other popular JVM package managers are available on every platform – and those are how all the books and tutorials etc tell folks to work with Java. Same goes for Clojure and any number of other languages. Their market is portability across platforms and to have the same tooling work “everywhere”.
You/Gentoo aren’t the only purists in this situation. Each unique Linux seems to think that “everything” should bend to their unique way of doing things. Even the brew team have this attitude, that everything should bend to “The Brew Way™” – which is why we now have a separate brew tap managed by the Clojure team because the core brew tap ignores the sensible way to work with the JVM…
The thing is that this doesn’t really make much sense beyond the JVM.
Everything beyond the JVM itself you should think of as resources, like image files. There’s nothing to do with them except download them into the right place and call Java with the right paths.
And you don’t even need your OS package manager for this, because the JVM can do it as well, so once you have a JVM you have everything you need to install and run any Java application.
I think maybe there’s also something I’m not following, are you trying to package an application built in Clojure, or are you trying to package the libraries and tooling that you might want to use to build a Clojure application with?
The former is much easier to do, you would have Gentoo build a JVM, and then you’d just download the Uberjar for the application (an Uberjar is a zip file that contains everything the application needs inside itself, there are no executable in it, everything is a resource, source code, class file, images, fonts, etc.), along with a launcher script.
The latter is much harder, and I’m really confused, I’ve never seen any language except maybe C rely on an OS package manager for the latter. Does Gentoo contain every Haskell library in its package manager? You’re telling me every packages in here: https://hackage.haskell.org/ is mirrored inside the Gentoo package manager as well?
I’m saying JVM ecosystem could be more compatible with the general packaging pattern of POSIX-ish operating systems including MacOS, linux distributions, and BSDs.
C doesn’t have to cater to every single operating system.
Neither do janet and haskell.
No language cares enough to cater to any single opeating system.
GitHub - gentoo-haskell/gentoo-haskell: official gentoo haskell overlay contains gentoo packages for every little haskell library and haskell program ported to gentoo linux.
Gentoo linux has GitHub - gentoo-haskell/hackport: A command line tool to generate Gentoo ebuilds from Hackage packages.</t which converts cabal files or hackage packages to gentoo packages automatically.
Gentoo linux also has bash functions for packaging python, Go, and Rust.
Once we figure out how to handle npm and JVM jars, we can also create special packaging system for nodejs and JVM languages that I care about. We can also create functions for creating GraalVM binaries out of uberjars. The packaging functions will be used in gentoo packages.
In theory, we can also just install uberjar as long as it is not specific to any CPU platform. But, uberjar is not guaranteed to exist.
Gentoo people somehow packaged Gentoo Portage Overlays - dev-java/spec-alpha and Gentoo Portage Overlays - dev-java/core-specs-alpha without clojure.
They also packaged Gentoo Portage Overlays - dev-lang/clojure by combining clojure with spec.alpha and core.specs.alpha in one jar.
This looks like black magic, but I think gentoo still doesn’t have a general framwork for packaging each JVM library or program cleanly.
Currently the clj setup wants root. It wants desperately to be part of the distro.
What do you mean? clj setup wants root?
Hum, what I can say is that I have successfully built many Clojure applications using Ant, and that seems like a supported build tool by Gentoo.
I can’t share the work, but you pull the Clojure source from git or wherever it is, then you inspect the project.clj or deps.edn or whatever else build system they use to see what the package is doing to build itself.
For Clojure, it almost always is just is as simple as putting the source files in a jar.
So if Gentoo supports Ant, I’d say that’s your best bet. Rewrite the Clojure builds for various Clojure libraries and applications in an Ant build.xml and then integrate that in Gentoo.
Clojure itself includes a build.xml already, but almost no other Clojure lib or app will have one, so you’ll need to write it yourself.
Gentoo Java Packing Policy - Gentoo Wiki says ant is fully integrated with gentoo.
Gentoo can also handle plain java build without any build system.
It seems gentoo has supported java for a long time.
Gentoo Portage Overlays - dev-lang/clojure compiles clojure-1.10.3 by building clojure first and then building spec.alpha and core.specs.alpha against clojure.
The patch that lets clojure build without spec.alpha and core.specs.alpha is
From 3270c158d191f7e59540667c2dbe6c5dc7a19a59 Mon Sep 17 00:00:00 2001
From: Florian Schmaus <flo@geekplace.eu>
Date: Thu, 17 Jun 2021 14:52:01 +0200
Subject: [PATCH] Add compile-spec ant build target
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Suggested-by: Göktürk Yüksek <gokturk@gentoo.org>
---
build.xml | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/build.xml b/build.xml
index 4d29bbf9a27e..dc742113938f 100644
--- a/build.xml
+++ b/build.xml
@@ -85,6 +85,24 @@
</java>
</target>
+ <target name="compile-spec"
+ description="Compile Clojure spec sources."
+ depends="compile-java">
+ <java classname="clojure.lang.Compile"
+ classpath="${build}:${cljsrc}"
+ failonerror="true"
+ fork="true">
+ <sysproperty key="clojure.compile.path" value="${build}"/>
+ <sysproperty key="clojure.compiler.direct-linking" value="true"/>
+ <sysproperty key="clojure.spec.skip-macros" value="true"/>
+ <sysproperty key="java.awt.headless" value="true"/>
+ <arg value="clojure.spec.alpha"/>
+ <arg value="clojure.spec.gen.alpha"/>
+ <arg value="clojure.spec.test.alpha"/>
+ <arg value="clojure.core.specs.alpha"/>
+ </java>
+ </target>
+
<target name="compile-tests"
description="Compile the subset of tests that require compilation."
unless="maven.test.skip">
@@ -149,7 +167,7 @@
<target name="build"
description="Build Clojure (compilation only, no tests)."
- depends="compile-java, compile-clojure"/>
+ depends="compile-java, compile-clojure, compile-spec"/>
<target name="jar" depends="build"
description="Create clojure jar file.">
--
2.31.1
Previously, dev-lang/clojure depended on dev-java/core-specs-alpha and dev-java/core-alpha.
Gentoo people will have to figure out how to package clojure programs.
Gentoo builds dev-java/spec-alpha with this build.xml.
<?xml version="1.0" encoding="UTF-8"?>
<!-- ====================================================================== -->
<!-- Ant build file (http://ant.apache.org/) for Ant 1.6.2 or above. -->
<!-- ====================================================================== -->
<project name="spec.alpha" default="package" basedir=".">
<property name="maven.build.finalName" value="spec.alpha-SNAPSHOT"/>
<property name="maven.build.dir" value="target"/>
<property name="maven.build.outputDir" value="${maven.build.dir}/classes"/>
<property name="maven.build.srcDir.0" value="src/main/clojure"/>
<!-- clean -->
<target name="clean" description="Clean the output directory">
<delete dir="${maven.build.dir}"/>
</target>
<!-- build -->
<target name="build" description="Copy the files">
<mkdir dir="${maven.build.outputDir}"/>
<copy todir="${maven.build.outputDir}">
<fileset dir="${maven.build.srcDir.0}"/>
</copy>
</target>
<!-- package -->
<target name="package" depends="build" description="Package the application">
<jar jarfile="${maven.build.dir}/${maven.build.finalName}.jar"
compress="true"
index="false"
basedir="${maven.build.outputDir}"
excludes="**/package.html"/>
</target>
<!-- jar -->
<target name="jar" depends="package" description="Builds the jar for the application"/>
</project>
This is a black magic to me. I personally think not using ant is going to be easier…
I think dev-lang/clojure should list dev-clojure/spec-alpha and dev-clojure/core-specs-alpha as post-installation dependencies because spec.alpha and core.specs.alpha seem to be clojure’s runtime dependencies rather than build-time dependencies.
For simplicity, I think gentoo should have its own very simple JVM build system independent of ant, maven, gradle, etc, …
Ant is too complex and too cumbersome. Gradle, maven, and other JVM build tools are still complex and don’t really integrate well with gentoo.
I’m thinking of writing a very simple JVM build system written in janet programming language for building linux distribution packages. Janet is small and simple and integrates extremely well with linux distributions.
In theory, if something is a java program, it suffices to just compile java files with jar dependencies in classpath and javac. If something is a clojure program, it suffices to just compile clojure files with jar dependencies and clojure in classpath. I don’t want no bullshit complexity from ant, maven, sbt, etc, …
My very simple JVM build system should just let package build instructions take care of build steps.
Simplicity can be achieved by outsourcing build steps to package build scripts. POSIX-ish operating systems including linux, BSDs, and Mac OS X will be able to utilize my simple JVM build system to build and install JVM packages. I don’t know or care about windows.
I don’t think this is accurate. Java programs often require copying resource files into the JARs into special places and annotation processors to do code generation. There’s also commonly logic for renaming dependencies to deal with classpath collisions.
I agree that mismatches between packaging and build tools is frustrating. Trying to recreate all the necessary build logic for N packages is a huge effort and probably will result in a new build/packaging tool with its own inconsistencies for future folks to contend with. Definitely worth studying if the existing systems can be adapted or extended to suit your needs and getting an idea of the relative effort.
Ant is pretty simple to be honest, like it seems to be similar in simplicity to Ebuild. Especially if all you want is what you said, just ignore all the other more advanced Ant tasks.
This is your javac task that you want: Javac Task
Go look at the Examples section, it’s the quickest way to understand.
So for what you’re saying all you need to do is this:
<javac srcdir="${src}"
destdir="${build}"
fork="true" />
And for compiling Clojure you can use the java task like used here: clojure/build.xml at master · clojure/clojure · GitHub and explained here: Java Task
Purely out of curiosity, how do you feel about make as a way to build stuff instead of ant?
make doesn’t know how to build java source files in certain directories.
Also, package build scripts have to assume that they have to bypass in-project build systems such as make and ant.
In other words, I’m trying to write a very simple JVM package build system that can be used to build OS packages in POSIX-compatible operating systems.