Is there a sales pitch for switching to deps.edn from Lein in 2020?

Community Center Watercooler
10

I read somewhere (as I remember it, from an ”official” source) that clj/deps is not a build tool. Which sort of have guided my assessment of the question here. But reading this thread the picture is much more nuanced than what I have realized.

Also. I totally agree about the strength of clj dependency management. I recently brought in the lein-tools-deps plugin into a Leiningen project and think that that is a very nice way to source some of that strength. (Maybe that’s abvious, but I don’t see it mentioned here.)

2 Likes
11

Yea, agreed. I think a nice future for lein is as a build tool over clj, as I feel having an official package/dependency manager now, that part of lein is the most redundant and likely to just slowly be deprecared or see itself being used less and less. So the use of lein with the lein-tools-deps plugin I think is pretty great, and I’ve used that in some projects before.

You’re also correct in that it is much more nuanced. The nuance comes partly with trying to even define what a build is, and thus what a build tool looks like.

A build is really whatever you want. Back in the day, it meant to build the binary for your application. But nowadays, what does it mean to build Clojure? Clojure doesn’t need building like in the old days. You can run a .clj file as is, your source code is also a program that you can run directly without any additional steps. But there are possibly many steps you’d want to perform prior to giving your app to others, such as generating documentation, linting, running tests, auto-formatting the source, possibly running AOT, etc.

None of these per-say are required to run your program, thus what you’re going to be doing exactly as part of building your Clojure program is kind of up to you. You might want to send yourself an email, or play a victory jingle for example. There is no strict set.

Still, you eventually realize that many of the possible tasks you’d want performed as part of your build from project to project starts to always be the same set of tasks, with only minor modifications to them. And that’s when a build tool seems like a good idea. A single program/script where you’ll implement the logic for the most common build tasks and expose a certain level of configuration to each so they can adapt to the minor changes between projects.

And with that definition in place, you’ll realize that clj/deps is not such a tool. It does not come with a set of common build tasks that you can use with some configuration to adapt them to your project, but lein does!

And yet, clj/deps can look like a build tool, because it is a package manager for Clojure programs. And so let me explain.

As you realize that for each of your Clojure project, you always want to generate documentation, run tests and perform linting. You have two follow up:

  1. Create a single tool that can do all of these, and has a unified configuration.
  2. Create a separate tool for each of these, which can be configured.

Lein is the #1 option. A monolithic build tool with a unified configuration, and some support for addition plugins that can be used to add even more tasks to its existing set which all follow its conventions and configuration format.

#2 are some of the new build tools which have been more recently developed to be used with clj/deps, such as clj-new, cljfmt-runner, clj-check, test-runner, kaocha, depstar, etc. A bunch of separate tools each targeting a subset or even a single of the possible tasks you’d want performed at build, using their own configuration and conventions.

Most of these don’t need clj/deps, though some of them
I believe do depend on having a deps.edn file. They’re all build tools, like lein, just with a smaller scope.

Previously, getting all these tools installed on your machine and running would have been painful, or you’d have used lein for it (lein is also a package manager), but if you’d have used lein for it, and since lein already comes with most build tasks built in, you’d have probably not tried to get any of those other tool and just used lein or a lein plugin to perform the same task. But now that clj/deps is here, it is super easy to get all these tools downloaded and running in your project. That’s where some people are thus saying that clj/deps can be used as a “build tool”.

Now I’m not here to fight on word definitions, the fact is indirectly, you can use clj/deps to kick-start build tasks on your project, and that alone could mean we label it a build tool, I don’t care. My goal is only to provide a deeper understanding of the differences between lein and clj/deps at a fundamental level, for those interested.

Hope I helped.
Regards!

8 Likes
12

The lein-tools-deps plugin is new to me, and has an excellent readme explaining its justification. Thanks for making me aware!

Link: https://github.com/RickMoynihan/lein-tools-deps

2 Likes
13

I like the distinction you make between monolithic solutions and the bespoke/libraries approach. Frankly, #2 seems more in line with general Clojure and maybe Linux culture. It’s probably my favorite, except that getting started can be rough: it requires a good set of templates to get rolling (something vital for me when I started with Luminus, which was just the template I needed for how an actionable Clojure web app can be laid out, built, and deployed, and with what dependencies).

3 Likes
14

I’m trying to make clj-new as good a starting point as possible so you only need to add one alias to get started:

(! 1144)-> clj -A:new app tory/webdev
Generating a project called webdev based on the 'app' template.
(! 1145)-> cat webdev/deps.edn 
{:paths ["src" "resources"]
 :deps {org.clojure/clojure {:mvn/version "1.10.1"}}
 :aliases
 {:test {:extra-paths ["test"]
         :extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}}}
  :runner
  {:extra-deps {com.cognitect/test-runner
                {:git/url "https://github.com/cognitect-labs/test-runner"
                 :sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}}
   :main-opts ["-m" "cognitect.test-runner"
               "-d" "test"]}
  :uberjar {:extra-deps {seancorfield/depstar {:mvn/version "0.5.1"}}
            :main-opts ["-m" "hf.depstar.uberjar" "webdev.jar"
                        "-C" "-m" "tory.webdev"]}}}
(! 1146)-> ls webdev/
CHANGELOG.md	README.md	doc		resources	test
LICENSE		deps.edn	pom.xml		src

At this point, you can easily test the project and build an AOT’d uberjar if you want (this is an app template). The JAR can be run with java -jar webdev.jar

I’m planning to update the lib template to include a :deploy alias (for Clojars) – it already includes a :jar alias.

Always happy to have feedback on what can make these templates easier to get started with!

6 Likes
15

In my experience, deps.edn is worth to switch.

  1. more controllable at the classpath and having less weird transitive dependencies problems
  2. start up time is faster
  3. more flexible project setup, like the project, subproject structure
  4. don’t need to publish to clojars, when I fix some upstream open source libraries for my own project.
  5. with deps, scripting is much more bearable.

but we still have many problems have to handle.

  1. don’t have a good ‘uberjar’ tool ( aot + merging resources + manifest creation + resolving libraries) could be used with deps

  2. no easy publishing flow for clojars/standard maven deployment.

  3. no way to mix with native java source files.

  4. documentation is far from perfect.

  5. If you are an advance user, you will need extensive knowledge about how maven is working.

1 Like
16

Thank you for that honest review; the pros you mention are compelling, but the problems like uberjars and java interop are nearly show-stoppers. Nearly, because it seems like I have heard people mention work-arounds.

What about Clojurescript? What’s the story with CLJS + Deps.EDN?

17

I don’t know about the mixing of java and Clojure source files, but @seancorfield’s https://github.com/seancorfield/dot-clojure/blob/master/deps.edn has a way of building uberjars. There are other ways of building uberjars as well, such as Cambada, which can just as easily be integrated into an alias. In general, everything leiningen does can be done with deps, but where with leiningen you need to use plugins, with deps you create aliases to scripts or other programs.

With regards to cljs and deps, it works pretty well. I have set up my project with shadow-cljs for cljs builds, and let deps control dependencies for both clj and cljs.

Unless you have a pretty advanced use of leiningen, my guess is that the switch to deps is not that much work.

1 Like
18

The latest clj-new does include :install and :deploy aliases in new lib and template projects.

1 Like
19

depstar builds uberjars, based on the pom.xml produced by clojure -Spom (which, admittedly, needs some manual editing), with AOT and manifest creation, and relies on the CLI/deps.edn to resolve libraries. With clj-new, the generated app projects include a fully-fleshed out pom.xml so if you use a group-id/artifact-id style name for the new project (which the docs recommend), then you can go straight to clojure -A:uberjar (via depstar) and clojure -A:deploy (via @slipset's deps-deploy`). Happy to consider additional features that you feel are missing.

1 Like
20

A data point: I tried using tools.deps recently on my reasonably large Clojure+ClojureScript application (my deps.edn was 83 lines). It turned out that my most important task (building an uberjar with AOT) is not obvious (tried depstar, cambada and uberdeps).

I carefully considered why I should invest into migrating, and it turned out that there are two advantages:

  • being “simpler”: this is true to a certain extent, but complexity rises rapidly as you add software for building uberjars,
  • access to GitHub repositories, which is a nice feature, but one I can live without.

Lein startup time (or its two JVMs) have never been a problem for me. My build time is long mostly because of AOT and because of ClojureScript compilation.

My conclusion was that I should wait until tools.deps matures, because I can’t afford the cost right now.

5 Likes
21

In case some folks reading aren’t aware, you don’t need AOT to run programs via an uberjar – AOT really only serves two purposes:

  • to make startup time of the uberjar faster.
  • to avoid the need to specify a main class at startup

If you build an uberjar without AOT, you can run it with java -cp path/to/the.jar clojure.main -m entry.point – or if there’s a manifest specifying clojure.main, you can do java -jar path/to/the.jar -m entry.point

1 Like
22

Agreed. I had some really weird errors that popped up after making an uberjar. It was recommended to me on the Slack to never use AOT (barring a real need), which took a little research because leiningen AOTs by default when making an uberjar. No errors since. Apparently lein keeps target/ in the classpath at all times, which creates some really fun interactions when you are writing code in the REPL.

1 Like
23

Definitly avoid AOT if possible. I’d say the only case I still use it is interop, and even then, if I can use other means, like Clojure’s Java API I’d use those instead. And when doing AOT for interop, it helps to be selective as well, and AOT only what’s needed, not the whole code base.

1 Like
24

This. I tried AOT / genclass for interop in some stuff I was doing in my previous job, and it was not smooth. In the end, I ended up using the Clojure Java API, and a small “factory” function in Clojure that reified what I needed.

25

For a server side application, would it be bad practice to NOT deploy an uberjar to QA or the production server, but rely instead on raw deps/git? Basically, do a git pull for the version of the server code’s .clj files and then use clj -m to run the application.

Is that nuts?

1 Like
26

That’s what we do. It’s a calculated risk since you might get a left-pad situation on your hands. Plus releases are not truly self-contained. You have the same risks when building locally though, so the best way to be sure is to run a local maven mirror…

2 Likes
27

I’ve done a variant where I create a docker container where I download all the dependencies at build time, then just run with clojure -m myapp.main at runtime.

3 Likes
28

Another legitimate use case for AOT is when you do not want your source code deployed to your production servers.

2 Likes
29

yeah, totally agree with the local maven mirror. That is what we do as well.

I guess theoretically also, you can have just a startup deps project on the production server with 1 .clj file that starts up you application from the dependencies in the deps.edn, instead of putting the entire application clj files out there. Then you just really update any version numbers in the deps and restart.