Should I use boot or tools.deps to build command line applications?

[seancorfield]

@crocket

It took ~4 seconds to execute that simple script.

Yes, JVM startup combined with the overhead of loading and compiling clojure.core can take a few seconds. Part of the startup time with clj is building the classpath but that is cached so I suspect you’d find subsequent invocations of the script were faster (probably twice as fast).

If startup time is your biggest concern then you either need to look at something like lumo (which means ClojureScript and, I think, Node.js behind the scenes) if you’ve also ruled out GraalVM…

…but I think, realistically, you’re not going to find anything that satisfies you in the Clojure(Script) world for fast-starting scripts with everything else you also want.

[crocket]

I can live with a few seconds of startup time for certain use cases. But, I want to be able to edit dependencies with ease. Is boot script any better? I tried to write a boot script, but I couldn’t get myself to write one that compiles.

Perhaps, I should just write dependencies in shebang lines for now if I want to write scripts in clojure.

Do you know any fine programming language for command line scripting other than python or shells?

[seancorfield]

Boot doesn’t cache the classpath after construction – which clj does. Also, Boot creates a “shadow fileset” abstraction which also adds some overhead. Of Boot, Leiningen, and clj, the fastest repeated invocation is going to be clj.

As for alternative languages, I’m not sure how many of them I would consider “fine”. Things I’ve written scripts in before Clojure: lots of shell (which isn’t really so bad), some Python, Ruby, Perl… and a few obscure things that no one has heard of these days…

These days, my first choice would be shell for the simple stuff and Clojure via clj for anything more complicated.

[crocket]

Right, there are better scripting languages.

[l3nz]

@seancorfield

Yes, JVM startup combined with the overhead of loading and compiling clojure.core can take a few seconds. Part of the startup time with clj is building the classpath but that is cached so I suspect you’d find subsequent invocations of the script were faster (probably twice as fast).

You can also shave off half a second or so on each invocation by setting clojure.spec.skip-macros=true.

[l3nz]

@crocket

It took ~4 seconds to execute that simple script. Such a script cannot be used in a large loop. The loop would have to go in the script.

I am playing a bit with Planck in these holidays, and I see that repeated invocation of a script, after initial download of dependencies and compilation, is around 500 to 700ms per run. That is 100x what GraalVM takes, but very acceptable in most cases.

[crocket]

lumo is even faster to start up.

[0atman]

For the right type of command line app, Ferret might be the right tool, building tiny, native executable that have no VM overhead.

Be warned: Though it hass access to the full clojure ecosystem during compilation/macro expansion phase, the restricted environment at runtime requires a change in thinking - for instances macros that use str are fine, but functions that use str aren’t!

[ericnormand]

Hello!

I’ve been experimenting with using clojure for shebang scripting myself. I want something that is totally self-contained. That is, it doesn’t need a separate deps.edn file.

Here’s what I’ve come up with. It lets you have multi-line deps maps contained in the file. You can execute any Clojure code you want.

#!/usr/bin/env bash
#_(

cat "$0" | clojure -Sdeps '
{:deps {clj-time {:mvn/version "0.14.2"}}}
' -

exit $?
#_nil)

(println "Hello!")

(require '[clj-time.core :as t])

(prn (str (t/now)))

The biggest downside is that it is quite a hack. It uses the fact that #_ in Clojure creates a commented (ignored) expression, while # starts a comment in bash. However, for a bash script, it’s pretty good. I’ve got this one on my path, set to executable, and it runs great from the command line.

> cljtest

and

> sh ~/bin/cljtest

both work.

I’m all ears for comments.

Rock on!
Eric

[crocket]

Can you adapt it to POSIX shell?

[devn]

I messed with this about a year ago. It runs clj twice, so it’s not ideal, but I like the way it looks.

attaching clojure deps directly to the source file • ø¡–¬˚•Devin∆Walters•˚¬–¡ø

Cheers,
Devin

[ericnormand]

I’ve been tweaking my implementation. Here it is!

It now includes arguments, a place for other options, and it doesn’t depend on bash. I’ve included it below, but you can also find the most recent version in this gist.

#!/bin/sh
#_(

   #_DEPS is same format as deps.edn. Multiline is okay.
   DEPS='
   {:deps {clj-time {:mvn/version "0.14.2"}}}
   '

   #_You can put other options here
   OPTS='
   -J-Xms256m -J-Xmx256m -J-client
   '

exec clojure $OPTS -Sdeps "$DEPS" "$0" "$@"

)

(println "Hello!")

(require '[clj-time.core :as t])

(prn (str (t/now)))

(prn *command-line-args*)

(println (.. (Runtime/getRuntime)
             totalMemory))

[borkdude]

I’m using both boot and clj for command line scripts and I’ve also done some Haskell: I must say it’s a great fit for this.

This is a standalone script using clj: https://github.com/borkdude/balcony/blob/master/balcony.clj. It uses a trick comparable to @ericnormand’s script above to include the deps configuration.

I later decided to port it to Haskell for fun but then it turned into more than one file. I’m still running that on my server: https://github.com/borkdude/balcony-hs.

Here’s an example of a standalone boot script that you can call with e.g. ./foobar -t:

#!/usr/bin/env boot

(set-env! :dependencies '[[clj-time "0.14.2"]])
(require '[clj-time.core :as time])

(deftask my-task [t time bool "Show time"]
  (when (:time *opts*)
    (println (time/now))))

(defn -main [& args]
  (apply boot "my-task" args))

If you already have boot installed and you want a single file standalone script, I’d say boot works well without crazy shell hacks. The startup time of clj might be a little better due to classpath caching. Lastly, Haskell probably has much better start times for scripts .

[crocket]

That is a piece of art.

[didibus]

I wonder if @alexmiller has a plan to add “in-script” deps.edn support to tools.deps?

[didibus]

Also, to the people suggesting Haskel for scripts, how do you bring in the dependencies if you need any?

[crocket]

https://docs.haskellstack.org/en/stable/GUIDE/#script-interpreter

[eval]

My take on self-containing scripts: https://gitlab.com/eval/cljscript/blob/1d7d99723804536e6705bc808a613c7e34445dea/libexec/cljscript

It allows for scripts like:

#!/usr/bin/env cljscript
"DEPS='clj-time=RELEASE;some-other=1.2.3'"

(require '[clj-time.core :as t])

(defn -main [& args]
  (println (t/now)))

-main is invoked when present.

[eval]

The new incarnation of this script is ohmyclj.
It adds a repl and an easy way to run tests in the script.