Is there a sales pitch for switching to deps.edn from Lein in 2020?

Community Center Watercooler
1

I expect the answer to be no, but the fact that deps.edn has the Cognitect seal of approval, plus @seancorfield 's, raises it in my eyes. If Lein is mostly working out for me (except for moments of complexity, particularly re: Jackson), is there any major reason to switch to deps? I assume it plays as nicely with Cider (hopefully without much setup)?

7 Likes
2

If you’re happy using lein, you might as well continue to do so.

If you’re using Emacs/CIDER, I’m pretty sure the latest version knows how to start an nREPL using the CLI/deps.edn.

As for why you might want to switch:

  • lein generally starts two JVMs: one for itself and one for your program – clj/clojure uses a JVM to compute the classpath but then caches it so subsequent uses of clj/clojure run only one JVM (with the same set of aliases/dependencies)
  • lein relies on the Maven/Aether/Pomegranate resolution algorithm for picking library versions if your dependencies bring in a conflict and that can be the source of weird problems that then drive people into lein deps :tree and “pedantic” mode; CLI/deps.edn uses a simpler algorithm that favors explicit dependencies and more recent versions, which causes fewer surprises and is more easily controlled
  • CLI/deps.edn can easily bring in dependencies from local source installations as well as from any git repo (by URL and SHA); lein can do this too but requires plugins and/or more futzing around in your project.clj file
  • lein always runs with “everything” out of the box; CLI/deps.edn runs with just the tooling you tell it to use so it’s often faster to start up (even aside from the two vs one JVMs issue above)

We originally switched from lein to boot back in 2015 because we wanted a) better programmability and b) easier support of a monorepo with lots of subprojects. boot served those needs very well but we started to run into performance issues with the fileset abstraction and bugs with the pod refresh stuff so we were happy to switch to the CLI/deps.edn in 2018 – our custom boot tasks became simple scripts that could easily be run via clojure and performance was good.

My main reason to prefer CLI/deps.edn is “simple tools”. I’m with Stu Halloway and Eric Normand and others on wanting my dev tooling to be as simple and lightweight as possible: small, composable tools, with no “magic”. That’s also why I don’t use nREPL any more, I don’t use CIDER or Compliment, I just use a plain Socket REPL. That means I can fire up any Clojure process (or even a legacy app that includes Clojure) and have it start a Socket REPL with just a JVM option, which means dev, test, and production can all expose the exact same interface: a Socket REPL that is built into Clojure itself. That in turn means that I can use the exact same dev tooling with local processes and with remote processes.

20 Likes
3

“Switch”, I don’t know. But deps.edn is poised to make the pie larger. Much larger.

When the culture eventually, inevitably comes around to embrace the notion that it’s natural to run any program straight from CVS, no need to freeze an uberjar, no need to wonder who’s really in charge over at Maven Central; that will be a momentous moment.

Until then, I like “lein test”.

4

I can see a couple of use-cases where deps.edn wins:

  1. Learner Drivers
    I’ve been working through a couple of books using Lein and Cider, having started from never using emacs before. Though I can now see why it’s a good toolset, it’s not how I’d want to introduce a complete beginner to the language and I think FP might be the best way to start children programming. I didn’t fully understand what Leningen was doing behind the scenes. I’d rather have started with clj and a simple editor window. I can see what deps.edn is doing.

  2. Old Hardware
    My ‘Laptop’ is an Eee PC that’s about 10 years old and I tried using Clojure on a Raspberry Pi 2. The startup times for a lein REPL are far too long and it stopped working on the Pi, after a Java upgrade. CLJ is much quicker on the Eee PC and I hope it might work on a Pi. They are widely used in schools in the UK. I don’t like to see people excluded from computing because they don’t have the best kit. For environmental reasons, I like to keep old hardware functioning for as long as possible. If companies are letting equipment go after 3 years, I’d like to see it re-purposed.

3 Likes
5

Beyond what @seancorfield said. I want to add that it’s not apples to oranges comparing Lein and Clj.

The new Clj tool does two things:

  1. Download appropriate dependencies.
  2. Bootstraps a Clojure program with appropriate classpath and main args and what not.

Lein does all the above, but it also:

  • Lets you perform a number of common build tasks

Clj doesn’t come with any build tasks. The best it can do is create a pom.xml which allows you to use maven for certain build related tasks. But on its own, it means that it can’t:

  • compile your project
  • put it into a jar
  • or an uberjar
  • it can’t run your tests
  • lint your code
  • clean your compilation target folder
  • deploy your library to a remote maven repo
  • compile java code
  • scaffold new Clojure projects
  • search Maven for libraries
  • etc.

The only thing Clj can do is download a Clojure program to your machine with all its required dependencies and run it. Or download all of your project’s dependencies and run it.

So if you want to do anything “build” like, you need to use something else over Clj.

But, because Clj can download and run Clojure programs. It is easy to use it to download other Clojure programs that were designed to perform “build” like tasks and run them.

You can also have it create a pom.xml like I said, and then make use of Maven for certain build tasks. Maven will be good for Java related tasks, and creating Jars, publishing to Mavn repos, etc. Not so much for Clojure specific tasks.

Finally, when you think of Clojure, the truth is, there is very little “building” required. Since Clj can download and run Clojure programs that are available as source on github, you can go very far with just git and clj.

4 Likes
6

To be fair – and put your post in context – because Clj supports a user-level deps.edn file as well as a project-level one, it’s very easy to quickly add aliases that bring in nearly all of that list of features for all your projects with just a one-time setup… My dot-clojure file contains options for many of them: JAR files, uberjar files with AOT compilation, test running, Eastwood for linting, scaffolding new Clojure projects (plus a whole bunch of options you didn’t list).

There’s at least one option out there for deploying to remote repos (I just don’t have an alias for it yet in my file).

There are several other options for JAR files (and one of those, I think, deals with Java compilation?)

Leiningen’s Maven search has never worked well for me so I’ve always gone to search.maven.org or clojars.org directly for searching :slight_smile:

p.s. It’s also worth pointing out that Boot doesn’t have a test runner or new project scaffolding out of the box either (but Adzerk’s boot-test and my boot-new are libraries that add those features – in pretty much exactly the same way the Cognitect’s test-runner and my clj-new add them to Clj).

4 Likes
7

From a user perspective, if you’re someone just looking for a “how to” with bullet points, Clj + alias and Lein might end up being an equal user experience. But I wanted to get to the more fundamental underpinning, that none of those additional build tools you’d alias are a plugin to Clj, and neither do they have anything related to it. Each one is just another tool that you need to learn, which might have its own configuration, command line arguments, etc., unrelated to that of Clj. Clj is just one way for you to download the tool and run it. You could assemble them all through Lein as well, or manually through any other means they’d offer themselves to be downloaded and run.

I like that model, I like it a lot! It is very similar to the Linux command line, you just have programs and you can pipe their input/output together. They don’t always play nice, but if you have an assortment of them that have some convention, they could. The big advantage is that none of them are coupled to Clj. The downside is there isn’t a lot of enforcement in how they should behave and interact, though hopefully that develops organically through conventions.

The biggest missing part, is there isn’t currently a way to orchestrate a build with Clj. So you need to write a script for your OS. Maybe bash, or joker, etc. Say you want to compile -> test -> uberjar. There could be another tool built that you could alias that provides this, but again, all that isn’t really related to Clj.

On that note, I think it would be nice if Clj could add the following features:

  • A way to list the available main aliases.
  • A way to add a doc-string to aliases (which would be displayed along when listing the available main aliases).

Would be cool if it could also maybe support some form of concept on man page, where inside deps.edn you could define a man, and Clj could show you the man for your main aliases.

4 Likes
8

Are you referring to deps-deploy? I’ve used it recently, and it’s pretty easy to setup.

1 Like
9

Yes, @slipset’s tool for deploying to Clojars. I’ll probably switch to that next time I cut a release of any of my libraries, instead of my mvn wrapper shell script, and then I’ll add it to my dot-clojure repo for the benefit of others!

2 Likes
10

I read somewhere (as I remember it, from an ”official” source) that clj/deps is not a build tool. Which sort of have guided my assessment of the question here. But reading this thread the picture is much more nuanced than what I have realized.

Also. I totally agree about the strength of clj dependency management. I recently brought in the lein-tools-deps plugin into a Leiningen project and think that that is a very nice way to source some of that strength. (Maybe that’s abvious, but I don’t see it mentioned here.)

2 Likes
11

Yea, agreed. I think a nice future for lein is as a build tool over clj, as I feel having an official package/dependency manager now, that part of lein is the most redundant and likely to just slowly be deprecared or see itself being used less and less. So the use of lein with the lein-tools-deps plugin I think is pretty great, and I’ve used that in some projects before.

You’re also correct in that it is much more nuanced. The nuance comes partly with trying to even define what a build is, and thus what a build tool looks like.

A build is really whatever you want. Back in the day, it meant to build the binary for your application. But nowadays, what does it mean to build Clojure? Clojure doesn’t need building like in the old days. You can run a .clj file as is, your source code is also a program that you can run directly without any additional steps. But there are possibly many steps you’d want to perform prior to giving your app to others, such as generating documentation, linting, running tests, auto-formatting the source, possibly running AOT, etc.

None of these per-say are required to run your program, thus what you’re going to be doing exactly as part of building your Clojure program is kind of up to you. You might want to send yourself an email, or play a victory jingle for example. There is no strict set.

Still, you eventually realize that many of the possible tasks you’d want performed as part of your build from project to project starts to always be the same set of tasks, with only minor modifications to them. And that’s when a build tool seems like a good idea. A single program/script where you’ll implement the logic for the most common build tasks and expose a certain level of configuration to each so they can adapt to the minor changes between projects.

And with that definition in place, you’ll realize that clj/deps is not such a tool. It does not come with a set of common build tasks that you can use with some configuration to adapt them to your project, but lein does!

And yet, clj/deps can look like a build tool, because it is a package manager for Clojure programs. And so let me explain.

As you realize that for each of your Clojure project, you always want to generate documentation, run tests and perform linting. You have two follow up:

  1. Create a single tool that can do all of these, and has a unified configuration.
  2. Create a separate tool for each of these, which can be configured.

Lein is the #1 option. A monolithic build tool with a unified configuration, and some support for addition plugins that can be used to add even more tasks to its existing set which all follow its conventions and configuration format.

#2 are some of the new build tools which have been more recently developed to be used with clj/deps, such as clj-new, cljfmt-runner, clj-check, test-runner, kaocha, depstar, etc. A bunch of separate tools each targeting a subset or even a single of the possible tasks you’d want performed at build, using their own configuration and conventions.

Most of these don’t need clj/deps, though some of them
I believe do depend on having a deps.edn file. They’re all build tools, like lein, just with a smaller scope.

Previously, getting all these tools installed on your machine and running would have been painful, or you’d have used lein for it (lein is also a package manager), but if you’d have used lein for it, and since lein already comes with most build tasks built in, you’d have probably not tried to get any of those other tool and just used lein or a lein plugin to perform the same task. But now that clj/deps is here, it is super easy to get all these tools downloaded and running in your project. That’s where some people are thus saying that clj/deps can be used as a “build tool”.

Now I’m not here to fight on word definitions, the fact is indirectly, you can use clj/deps to kick-start build tasks on your project, and that alone could mean we label it a build tool, I don’t care. My goal is only to provide a deeper understanding of the differences between lein and clj/deps at a fundamental level, for those interested.

Hope I helped.
Regards!

8 Likes
12

The lein-tools-deps plugin is new to me, and has an excellent readme explaining its justification. Thanks for making me aware!

Link: https://github.com/RickMoynihan/lein-tools-deps

2 Likes
13

I like the distinction you make between monolithic solutions and the bespoke/libraries approach. Frankly, #2 seems more in line with general Clojure and maybe Linux culture. It’s probably my favorite, except that getting started can be rough: it requires a good set of templates to get rolling (something vital for me when I started with Luminus, which was just the template I needed for how an actionable Clojure web app can be laid out, built, and deployed, and with what dependencies).

3 Likes
14

I’m trying to make clj-new as good a starting point as possible so you only need to add one alias to get started:

(! 1144)-> clj -A:new app tory/webdev
Generating a project called webdev based on the 'app' template.
(! 1145)-> cat webdev/deps.edn 
{:paths ["src" "resources"]
 :deps {org.clojure/clojure {:mvn/version "1.10.1"}}
 :aliases
 {:test {:extra-paths ["test"]
         :extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}}}
  :runner
  {:extra-deps {com.cognitect/test-runner
                {:git/url "https://github.com/cognitect-labs/test-runner"
                 :sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}}
   :main-opts ["-m" "cognitect.test-runner"
               "-d" "test"]}
  :uberjar {:extra-deps {seancorfield/depstar {:mvn/version "0.5.1"}}
            :main-opts ["-m" "hf.depstar.uberjar" "webdev.jar"
                        "-C" "-m" "tory.webdev"]}}}
(! 1146)-> ls webdev/
CHANGELOG.md	README.md	doc		resources	test
LICENSE		deps.edn	pom.xml		src

At this point, you can easily test the project and build an AOT’d uberjar if you want (this is an app template). The JAR can be run with java -jar webdev.jar

I’m planning to update the lib template to include a :deploy alias (for Clojars) – it already includes a :jar alias.

Always happy to have feedback on what can make these templates easier to get started with!

6 Likes
15

In my experience, deps.edn is worth to switch.

  1. more controllable at the classpath and having less weird transitive dependencies problems
  2. start up time is faster
  3. more flexible project setup, like the project, subproject structure
  4. don’t need to publish to clojars, when I fix some upstream open source libraries for my own project.
  5. with deps, scripting is much more bearable.

but we still have many problems have to handle.

  1. don’t have a good ‘uberjar’ tool ( aot + merging resources + manifest creation + resolving libraries) could be used with deps

  2. no easy publishing flow for clojars/standard maven deployment.

  3. no way to mix with native java source files.

  4. documentation is far from perfect.

  5. If you are an advance user, you will need extensive knowledge about how maven is working.

1 Like
16

Thank you for that honest review; the pros you mention are compelling, but the problems like uberjars and java interop are nearly show-stoppers. Nearly, because it seems like I have heard people mention work-arounds.

What about Clojurescript? What’s the story with CLJS + Deps.EDN?

17

I don’t know about the mixing of java and Clojure source files, but @seancorfield’s https://github.com/seancorfield/dot-clojure/blob/master/deps.edn has a way of building uberjars. There are other ways of building uberjars as well, such as Cambada, which can just as easily be integrated into an alias. In general, everything leiningen does can be done with deps, but where with leiningen you need to use plugins, with deps you create aliases to scripts or other programs.

With regards to cljs and deps, it works pretty well. I have set up my project with shadow-cljs for cljs builds, and let deps control dependencies for both clj and cljs.

Unless you have a pretty advanced use of leiningen, my guess is that the switch to deps is not that much work.

1 Like
18

The latest clj-new does include :install and :deploy aliases in new lib and template projects.

1 Like
19

depstar builds uberjars, based on the pom.xml produced by clojure -Spom (which, admittedly, needs some manual editing), with AOT and manifest creation, and relies on the CLI/deps.edn to resolve libraries. With clj-new, the generated app projects include a fully-fleshed out pom.xml so if you use a group-id/artifact-id style name for the new project (which the docs recommend), then you can go straight to clojure -A:uberjar (via depstar) and clojure -A:deploy (via @slipset's deps-deploy`). Happy to consider additional features that you feel are missing.

1 Like
20

A data point: I tried using tools.deps recently on my reasonably large Clojure+ClojureScript application (my deps.edn was 83 lines). It turned out that my most important task (building an uberjar with AOT) is not obvious (tried depstar, cambada and uberdeps).

I carefully considered why I should invest into migrating, and it turned out that there are two advantages:

  • being “simpler”: this is true to a certain extent, but complexity rises rapidly as you add software for building uberjars,
  • access to GitHub repositories, which is a nice feature, but one I can live without.

Lein startup time (or its two JVMs) have never been a problem for me. My build time is long mostly because of AOT and because of ClojureScript compilation.

My conclusion was that I should wait until tools.deps matures, because I can’t afford the cost right now.

5 Likes