I do think examples or technical details would be well received here. Clojurists tend to have a lot of industry experience and be senior engineer themselves, and those who are less experienced are still passionate about programming one way or another, you have to be to be interested in a niche language like Clojure, so I’m sure they’d be interested as well.
As for defending the JVM, yes, you might see me or others challenge and push back on your examples and technical constraints, but it isn’t defending the JVM out of loyalty, it’s more about understanding things for ourselves, and making sure we really understood so we can draw our own conclusions, which we might disagree on, but that’s fine. At least it is for me.
Java’s core semantics are very different from java, the language. The core semantics are much more minimal than a language. The core semantics are baked in bytecodes.
There is another choice. Choosing another language or another platform. By choosing another language, you don’t even have to bother with packaging your programs for linux distributions or any operating system. People will do that for you.
I don’t want to go into too much detail. I want to focus on the big picture.
What you need to understand is that JVM isn’t for everything. Raku or MoarVM isn’t for everything, either. Julia isn’t for everything. Each language and each platform have their own niches.
You have to come to terms with the fact that you have to learn multiple languages and multiple VMs.
I certainly wouldn’t create a command line program running on JVM or rely on anything that requires GraalVM. JVM just is suboptimal for command line programs or system scripts. JVM doesn’t come with “pre-determined” paths to system libraries or system applications. Perl and Raku do. The existence of pre-determined paths to system modules is why Raku is better for system applications and system scripts. That’s what you need to know at the big picture level. Details aren’t important here.
JVM is heavily optimized for runtime speed and enterprise backends. Julia is heavily optimized for math and data visualization. Python is just a toy language with great ecosystem. If you don’t look at the ecosystem, Raku is a better alternative to python.
Anything that you would use python for, you can use Raku. If you just want a simple language, you can maybe choose lua.
We definitely will have to agree to disagree. I wrote production command line applications in Clojure that I build to native static binaries to great ease and success. I run scripts using babashka to great success to manage infrastructure and CI/CD pipelines. I use a Clojure REPL for one off operational tasks quite often.
I don’t need to make my scripts and command line apps available in the public Gentoo package manager, and have them be installable through Gentoo’s package manager, so I can see how if Gentoo offers little pre-baked support to do so for Clojure apps that it would make using Clojure in that case more challenging and you might want to instead use Raku if that’s more straightforward. But that doesn’t mean that you can’t use Clojure effectively when that’s not a concern of yours, to build command line apps or write system scripts.
Babashka can’t include other clojure modules without jumping through hoops and loops. I tried. System scripts should be able to easily include other modules. If you don’t care about requiring system modules, then joker language is another alternative to babashka. At least, joker langauge can be easily built from source and installed. Even python is better than clojure in terms of system modules. You can install a python module and use it in random python scripts. Everything is streamlined. You can’t do that with clojure or babashka. Clojure doesn’t provide rich facilities because clojure aims to just be an alternative language for virtual machines.
It’s not just about gentoo. I use multiple linux distributions.
Maybe, you are just writing private corporate in-house applications. I write open-source applications for public distribution. Unless your main method of distribution is to distribute GraalVM binaries on github release page, distributing clojure programs for public consumption is going to be hard. I strongly prefer programs that were built from source by linux distribution maintainers. If a program can easily be built from source by linux distribution maintainers, then I don’t have to beg application maintainers for a GraalVM binary on my CPU architecture.
For weeks, I tried really hard to use clojure for system scripts and system applications. I ended up using joker language and then janet language which is close to clojure. Because janet language didn’t have any facility to mediate version conflicts, I ended up with raku language. Raku language has almost everything streamlined for system applications. Raku language has regular expression grammar for building complex parsers and many other facilities to help write command line applications and system applications. It is a rich language.
It’s difficult to distribute programs written on any platform that doesn’t provide built-in facilities to integrate with operating system packages. Operating system packages can handle programs written in multiple languages, and they are built together. I want to build programs written in multiple languages together so that there is no glitch between them.
As I wrote above, JVM is suboptimal outside of its main niches. I learn new languages and new platforms that better fit certain niches I want to tackle. That means moving away from Raku if it stops serving me. I also left clojure and other languages because they stopped serving my needs neatly.
What you have to realize is that language isn’t everything. Infrastructure, tooling, facilities and ecosystem around any specific language are just as important. Python has big ecosystem. Raku has superior built-in facilities and is a better language than Python and Go.
Python and Perl and shell script are used in gentoo linux infrastructure because they are better than JVM for writing high-level system applications. GNU Guix System uses guile scheme because it is also good for high-level system applications. Good system programming languages are used in operating system infrastructure.
I think you are trying to fit problems to clojure despite JVM’s suboptimal facilities instead of choosing tools that make it easy to do what you want. I am not a language evangelist. Choosing tools that make it easy also means using guile scheme instead of Raku, Perl, Python, shell script, or anything else on GNU Guix System.
Are you going to insist on using JVM clojure to interface with GNU Guix System just because you like clojure? Good luck. As I wrote, you are going to have to adapt to many environments.
I fully integrated raku’s built-in build system with gentoo linux with less than 80 lines of code. In gentoo linux packages for raku modules, I just need to specify other raku modules as dependencies.
Raku has only one built-in build system which presents a concise way to build and package modules and build C shared libraries which are included in raku modules. The built-in build system is install-dist.raku which itself is just one small script. Raku has only one built-in module system which is used by install-dist.raku and zef. Zef is a development build system like JVM maven and haskell stack. install-dist.raku is used to integrate with operating systems.
JVM has multiple convoluted module/build systems including OSGi, ant, maven, …
Ant creator apologized to people for creating a convoluted build system.
JVM doesn’t have any good built-in module/build system, and external module/build systems failed to properly compensate for inadequecy of JVM’s internal ones.
For java packages, gentoo linux had to write its own JVM build system in shell script. The build system is about 4870 lines. Gentoo linux packages for java modules have fragile complexity because JVM wasn’t designed to integrate with operating systems.
From my direct experiences, raku module system is simpler and more advanced than JVM ones.
The lesson is that if a language doesn’t have good built-in module/build systems, external module/build systems can’t compensate.
It’s not an exhaustive list, like I said, if the library depends only on pure Clojure, no Java interop, or only on Java interop for Java classes that are themselves a part of Babashka it can be required from babashka.
Another cool thing about Babashka are pods, which let you use libraries written in other programming language.
Can you explain what you mean by writing system scripts? Are you trying to write scripts that configure and install things for Linux? Like say scripts to get an OS patched or updated ?
I’m not really sure why you’d want to interface with Guix otherwise?
Anyways, I prefer Clojure as a language, I enjoy it a lot more and find myself having more fun when I can use it. So yes, I do try to use it as much as I can. It’s a niche language, so sometimes you have to fill some gaps where no one else did the work yet, generally it’s not too inconvenient. I’m a pretty experienced developer though, so that helps.
That said, Clojure is actually very close to the one language you can use for almost anything. If you’re interested in my specific thoughts I wrote in detail when I think Clojure is a good fit and when it’s not here: When is Clojure "the right tool for the job"? · GitHub
I’m sure that for whatever you’re trying to do, there wasn’t enough support for Clojure to make it worthwhile for you. Definitely if you’re goal is to write a program that you want to package and make available in Gentoo’s package system you’re going to have a bad time using Clojure, I’m not doubting that one bit. But I think you’re generalizing very quickly from this specific constraint to saying that Clojure can’t be used for writing command line programs or system scripts, it very much can be used for those things if you play ball with how Clojure works. You seem to care a lot about fitting in the Gentoo world, but not so much about fitting in the Clojure world. That’s fair, but that’s really just a personal choice.
My point is, it’s not okay to downplay something without explaining the issues, and it’s not okay to mischaracterize things either. If you want to say Clojure involved too much effort to be packaged for Gentoo, that’s fair. But saying that it’s no good for writing CLIs or for scripting that’s just not true. In many scenarios it’s totally capable of those things. Even saying Clojure makes it hard to work with Linux package managers seems wrong because you never explained in what way it did and in what way other languages made that easier. The only thing I understood is that Gentoo doesn’t have anyone who already did the work, but that doesn’t mean the work is any harder. It could be, but I failed to be convinced since you didn’t provide and data to convince me.
First off all, let’s seperate the concept of modules, build and dependency management, because those are three different things.
Let’s talk about modules, I’m assuming what you’re talking about is linking code at runtime dynamically. The JVM is great for that, it can load packages and classes at runtime and link them. It’s relatively straightforward, you tell it where the package you want to link against is, and it’ll load it and make it available. Pretty simple.
Now let’s talk about build. Build is just preparing things so you can run the program. What that involves can be anything, from downloading a 10GB dataset to compiling source, to zipping files, to running tests, etc. Clojure’s tools.build is pretty flexible here, builds are just Clojure programs. That super powerful, but maybe too much, if you’re then trying to replicate it with Gentoo’s build system, which I’m sure is way less powerful, you might struggle. That said, generally Clojure involved the most simple build step of all for most libs and programs, just copy the files from their src folders to your output folder and you’re done. That’s because Clojure compiles itself at runtime. If you depend on some Java, build is a bit more complicated, it involves a compile step.
Finally, dependency management. That’s the job of pulling down dependencies you need either to build or to run something, and knowing which version to pull down, and how to resolve version conflicts. Clojure’s is pretty good here too, tools.deps and lein are both quite good at that, handling really complex scenarios.
Normally a Linux system takes over dependency management, and building isn’t needed. In those, things would be very simple for Clojure like I said, just pull it all down into one folder.
If the linux package manager wants to work with sources though, now it also needs to know how to build. Again, that’s not that hard, just a lot of work to replicate the build of every Clojure and Java library you might need.
Like I said, at my work we have our own from source package manager. So for every Java and Clojure library we have to rewrite the build using our own, and then rewrite the version dependency using our own. It’s straightforward, but effortful.
Now, at my work we do the same for Ruby, Python, Perl, and they’re all just as effortful and involved the same steps.
That’s why I asked what’s so good about Raku, I’m curious how it makes this process less effort?
So, you set BABASHKA_CLASSPATH and manually download babashka files? I knew I could do that. I just don’t like manual dependency management.
You are saying you are okay with manual dependency management if you could use clojure.
Even Slackware linux, which enforced manual dependency management, was better than babashka in terms of dependency management.
When I say system scripts, I mean scripts that depend on system modules installed in predetermined paths to system modules. I also mean scripts that are parts of operating system infrastructure.
You are free to try using JVM languages in operating system infrastructure. I just don’t want to do it.
It can be if you accept suboptimal ways of installing clojure modules as system modules.
Stop obsessing over Gentoo. I use Artix Linux, too. I will use GNU Guix System soon. Gentoo Linux differs from Artix Linux in that it provides shared build functions for each build system. Artix Linux doesn’t provide any shared build functions. That’s why packaging JVM languages is going to be more painful in Artix Linux than in Gentoo Linux.
GNU Guix System uses guile scheme for shared build functions for each build system. Guile Scheme is more powerful than bash used by Gentoo Linux. Maybe with GNU Guix System’s advanced packaging facilities enabled by Guile Scheme, packaging java isn’t as painful.
All POSIX operating systems build packages in similar ways. Stop differentiating them. Even FreeBSD has to take the same steps used by all linux distributions when it builds and installs packages.
Bash is the right tool for some things. But, the language is too primitive for many advanced things.
I wrote less than 80 lines to fully integrate raku modules with gentoo linux packages.
With less than 80 lines for full integration, now, gentoo linux packages for raku modules just need to specify other raku module dependencies, and gentoo linux packages for raku applications just need to specify which executables should be available in /usr/bin after stating raku module dependencies.
In raku, the vast majority of the complexity in module and build is taken care of by raku itself or build modules which integrate with both zef and install-dist.raku very cleanly. For raku modules that come with built-in C shared libraries, I just need to specify a raku module dependency that builds built-in C shared libraries automatically.
In contrast, gentoo linux has about 4800 lines of code for integration with java, and the integration requires extra lines in gentoo linux packages that exist just to appease java integration code. People who package java modules for gentoo linux have to read very long java packaging policy for gentoo because the integration is incomplete and complex.
If you can’t still see any problem, you are just blinded by a language so much that you don’t care about facilities and infrastructure. Maybe, you don’t care about linux packaging systems because you haven’t created a linux package in your life?
Raku’s module and build systems are simple to use but more powerful than those of all other languages I’ve seen. I’ve written linux packages for Go, Python, Rust, Janet, C, C++, Raku, and other languages. Raku integrates the best with POSIX operating systems with no fuss. I gave up on packaging JVM languages and npm packages because they are a pain in the ass to integrate with linux packaging systems, and there are better alternatives. When there are better alternatives, I don’t have to use either JVM or javascript platforms.
I can’t stand inferior packaging systems because I personally package a lot of software modules written in multiple languages. I’m speaking from my direct experience after fully integrating raku modules with gentoo linux packaging system. It is simple, concise, clean, and powerful. It’s the current state of the art.
JVM’s packaging infrastructure is inferior. There is no debate. I have better alternatives. I don’t have to use JVM. In fact, I don’t want to hear about JVM. I don’t have a single piece of JVM on my system, and it runs everything I want perfectly. I don’t want JVM in my life.
I have never considered packaging a consideration when choosing the language for a program. My thinking is that it is the responsibility of the packaging system to adapt to work with the language.
If I was a user of gentoo or guix, I would be disappointed if it didn’t work with JVM/Python/any other language, and would see it as a reason not to use those systems. It feels wrong to build a packaging system that does not work with the applications your users would want to use.
Babashka will automatically download deps.clj, the native port of tools.deps, and then it will use it to download the clj-http-lite library from maven, and then it will require the library so that the script can run it, in my example, it is used to check the status of a get request to clojure.org.
Basically, all you need to run my script above is the babashka binary and the script will work, it will even self-download all the libraries it depends on.
You can also use an external file, bb.edn, to do the same. You would add a bb.edn file alongside your script:
That’s similar to the above, except the :deps are defined in an external file, but here too, Babashka will use deps.clj which it will self-install if not present, to pull down the dependencies needed for the script.
If you don’t want babashka itself managing pulling down the dependencies, you can defer to whatever dependency manager you want. That’s the point of the classpath. For example, if you want to use clojure’s default dependency manager which comes with clojure and is called tools.deps and whose CLI is clojure, you can do the following:
And now when you run the script, you need to specify the classpath for where it should find the clj-http-lite module. That means you can use whatever you want to pull down clj-http-lite, and you can pull it down in whatever folder you want, just specify the location folder where it was pulled down to babashka on the classpath and your script will work.
The -Sdeps command to clojure will pull down the dependencies specified (unless they were already pulled down), and the -Spath command will tell clojure to print the path to the dependencies. Effectively, this pulls down the dependencies and returns the path to them using tools.deps which we give to babashka.
You don’t have to use tools.deps, the same can be done with leiningen for example. This is also where you could use a linux package manager, so say you were on OpenSuse, you could use zipper, assuming that clj-http-lite for version 0.4.392 exists in the openSuse repos, you could install it using zipper, and then just put the path where zipper installed it to the --classpath.
Or you could manually download the jar for clj-http-lite 0.4.392 and then pass the path of where you downloaded it to classpath of babashka when running your script.
The only thing that Clojure and Babashka don’t do, is they don’t look in “default” places for dependencies when you run a program, so you need to pass in the path to where the modules that the program needs are when running the program. Most of the time people use a shell script as a launcher for that. One trick is even to put the shell script in the clojure script source such as:
Now if the clj-http-lite jar is installed in /usr/local/lib/clojure you can run your_script.clj and the shell script at the top of the script will run the script with babashka and pass it as the classpath path.
So, say you wanted to use some linux distro’s package manager, you’d just need to have it install all jars in /usr/local/lib/clojure with name of jars using this convention: <groupid>-<libname>-<version>.jar
And if someone wanted to publish a Clojure program, or script (not a lib), to that Linux package manager, they’d do what I did, they’d provide a shell script alongside their program/script which bootstraps the classpath with whatever lib-version they need for it. This would work for normal Clojure as well, which has a similar classpath feature. Here it is for normal clojure (and not babashka):
The advantage of not looking into “default” folders, is that it handles versions properly. When you have a “default” folder, if two programs need different versions you have an issue, so Clojure and Babashka and Java all force you to specify the path to the exact modules to use, that means to the folder or jar of the exact version the program needs.
That said, if the Linux package manager is one that tries to be from source, then you’d also need to figure out how to build clj-http-lite 0.4.392 into a jar, and doing all that from source is a lot more work.
So again, ignoring the challenges of building java and clojure from source, if your package manager is fine just packaging already built artifacts like jars, binaries, etc. It should be trivial to create packages for Clojure (or Babashka). A module is just a .jar file, so all you have to do in the package manager is specify where to put the jar file and what name to give the jar file. And for packaging an application, cli, or script, all you have to do is provide a shell launcher that has the path to the modules. If the package manager has a convention for where it installs modules, then that path can be hard-coded when you package your app/cli/script. Like in my example, I assume the package manager will always put clojure modules in /usr/local/lib/clojure/<group>-<libname>-<version>.jar. If that’s the case, then it is as simple as my example sh scripts above.
When you develop, I would still use tools.deps as my package manager though. That way, you can have it resolve all versions transitively to what you know works for your app/script/cli, and once you have that, you can just call its -Spath which will return the correct set of modules with their exact versions, and then I’d just replace the folders and names to map the convention of the linux package manager. Probably I’d write myself a script to do so for various different distros.
When looking at Gentoo, I feel all the complexity has to do with building the artifacts from source. The only reason for Ant would be that. I’m guessing Gentoo decided to try to integrate with Ant, so if a Java package had an Ant build file already, it could piggie back on that, but then Ant got replaced by Maven, Gradle, and for Clojure no one uses Ant, but uses Lein or tools.deps or boot instead. So now I’m guessing that Gentoo never implemented common Java build steps, like say zipping things into a Jar, or compiling java sources with javac, or running tests, etc. And I’m assuming it uses Ant for those. What that means is probably for all Java or Clojure modules, you have to re-implement the build of the module in Ant, and then setup Gentoo so it uses Ant to build it. And on top of that, I don’t know how Gentoo import the sources and all that. It would seem to me a lot of effort. It’s not complicated per say, this is actually exactly what we do at my work, but it is effortful for every module to re-implement things with Ant. For Clojure, it tends to be straightforward, since generally the build is just to zip the source files into the output jar. For Java, it is a bit more complicated, since you need to compile, and that needs to have compile time dependencies made available, and if using Ant, you now need to manage those paths with Ant.
I’m leaving this thread because people aren’t interested in learning raku or moarvm.
I’ve already spent many hours on this thread without going anywhere.
Why must I bother with discussing on JVM for hours when I can be happy with better alternatives? I don’t want to spend more time on JVM or any discussion around it.
I want life without JVM.
What I can tell you guys is that I have experiences with BSD operating systems and multiple linux distributions and multiple programing languages.
From my direct experiences with multiple operating systems and multiple programing languages, JVM module system sucks ass with regard to integration with anything outside JVM. I don’t want to spend hundreds of more hours on convincing uninterested people. It’s harder than talking with an uninterested woman.
I’m actually done here. I muted this topic so I won’t get any notification from this thread anymore.
I learnt more about packaging from this thread, and, especially about the classpath. So this discussion was pretty useful and interesting to me.
In my experience it is almost impossible to convince anyone over the Internet, so best you can do is just share your experiences and give examples, and hope to educate others.
We are going to close this thread, it seems some people have gotten value out of it, which is great, but overall I do not think it can be considered constructive.
OP seemed to have wanted to make people aware of MoarVM, there are other ways to do this than by attacking the foundation of the technology that this community is built around.
They are not the place for advocacy about what ‘ought’ to be made. If you think something ought to be made, then make it. Otherwise, respect others peoples’ right to choose what they do with their time.
That’s for good reason, this thread is a good example of what such posts lead to. Or to quote one of my favorite Why the Lucky Stiff quotes:
When you don’t create things, you become defined by your tastes rather than ability. your tastes only narrow & exclude people. so create.
We allow a broader range of discussion here than the Clojure etiquette allows for, so discussing hypotheticals is not out of the question, but we do care about conversations being friendly, constructive, respectful. Please keep that in mind when posting.