How to run clj from an arbitrary directory

jiyinyiyong · March 1, 2018, 10:20am

I have two files:

src/
  main.clj
  lib.clj

It’s simple to run clj -m main.clj or clj -i src/main.clj in project root since src/ is default in path. How to run from an arbitrary working directory?

borkdude · March 1, 2018, 1:08pm

In case you mean: from anywhere in your filesystem you would like to execute your project using clj, I’d do it like this:

Make a script script in your project directory (or anywhere you like).

#!/bin/bash

cd /tmp/yourproject
clj -m main

and add the script to the PATH:

export PATH=$PATH:/tmp/location/of/your/script

chmod +x /location/of/your/script

Now you can call script from anywhere.

rauh · March 1, 2018, 2:03pm

I think just set :paths:

https://clojure.org/reference/deps_and_cli#_paths

$ clj -Spath -Sdeps '{:paths ["foo"]}'
foo:/home/rauh/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/rauh/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/home/rauh/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar

jiyinyiyong · March 1, 2018, 2:47pm

Far less interesting when Bash is used. I would prefer writing Bash in Clojure instead.

borkdude · March 1, 2018, 2:59pm

ok, go ahead: https://github.com/dundalek/closh

jiyinyiyong · March 1, 2018, 3:56pm

I was notified we got better a solution on Slack:

clj -Sdeps "{:deps {hello {:local/root \"/tmp/hello-world\"}}}" -m hello

https://clojure.org/guides/deps_and_cli#extra_paths

https://clojure.org/guides/deps_and_cli#_using_local_libraries

borkdude · March 1, 2018, 4:05pm

So your question was, how to include a local library? OK, that’s cool then. In which Slack channel did you get this info? Might want to join as I want to learn more about this stuff myself.

jiyinyiyong · March 1, 2018, 4:35pm

https://clojurians.slack.com/messages/clojure

In Node.js , libraries are imported via a relative path or a node_modules/ folder. It’s quite simpler to use a Node.js script as a command line tool. For Clojure, specifying the resource paths(what’s its correct name…?) is necessary and tedious. Still glad we have not-bad solution.

didibus · March 1, 2018, 7:31pm

Are you trying to do scripting with Clojure?

If so, I would say 100% go install inlein https://github.com/hypirion/inlein

Then, if you want to have local libraries you can use and reuse in your scripts, build them in any folder, add a leiningen project.clj file at the root of the library folder and then run lein install. This will install your library inside ~/.m2 which you can think of the same way as the global node_module folder. You can then declare them as a dependency inside your inlein scripts as if they were deployed to clojars.

This then allows each script you write to use all clojars and maven dependencies (each script can target the version of their choice) as well as all your local dependencies too. This include having each scripts choose which version of Clojure to use also.

You can then run your scripts with: inlein myscript.clj

Or add a shebang to your script files #!/usr/bin/env inlein and then make them executable and call them directly ./myscript.clj

Or add them to your path and call them myscript.clj

Done.

Its really great. The only downside is just the slow start-up of Clojure, though inlein by default tries its best to speed that up too.

Also, no reason there couldn’t be an inlein that used tools.deps, appart from that no one made one yet.

jiyinyiyong · March 2, 2018, 2:49am

Would love to try, the syntax look nice:

#!/usr/bin/env inlein

'{:dependencies [[org.clojure/clojure "1.8.0"]
                 [com.hypirion/primes "0.2.1"]]}

(require '[com.hypirion.primes :as p])

(when-not (first *command-line-args*)
  (println "Usage:" (System/getProperty "$0") "prime-number")
  (System/exit 1))

(-> (first *command-line-args*)
    (Long/parseLong)
    (p/get)
    println)

But I don’t trust Lein. Too slow and my machine could be even slower running Lein.

borkdude · March 2, 2018, 11:34am

Introducing cljscript:

gist.github.com

https://gist.github.com/borkdude/8ad63f256e6f834f812b86a2c0e36415

cljscript

deps=`awk '/^;; deps.edn$/{flag=1;next}/^;; end deps.edn$/{flag=0}flag' $1`
script=`awk '/^;; script$/{flag=1;next}/^;; end script$/{flag=0}flag' $1`

clj -Sdeps "$deps" -e "$script"

output.txt

$ ./script.clj
2018-03-02T10:10:47.178Z

script.clj

#!/usr/bin/env cljscript

;; deps.edn
{:deps {clj-time {:mvn/version "0.14.2"}}}
;; end deps.edn

;; script
(require '[clj-time.core :as t])
(println (str (t/now)))
;; end script

jiyinyiyong · March 2, 2018, 3:00pm

Interesting! I thought it was cljs’script for a second.

borkdude · March 2, 2018, 3:08pm

Maybe I should call this script inclj after inlein http://inlein.org/

didibus · March 2, 2018, 6:37pm

Script just prints “Hello World!” and has no dependencies, clj and inlein take equal time.

$ time lumo helloworld.clj 
Hello World!
0.76s user 0.08s system 132% cpu 0.629 total

$ time clj helloworld.clj 
Hello World!
4.30s user 0.13s system 225% cpu 1.963 total

$ time inlein helloworld.clj 
Hello World!
2.30s user 0.17s system 125% cpu 1.970 total

Script takes input argument and requires one dependency, inlein is 4 times faster then clj

$ time clj -Sdeps "{:deps {com.hypirion/primes {:mvn/version \"0.2.1\"}}}" helloworld.clj 23
89
18.06s user 0.42s system 267% cpu 6.921 total

$ time inlein helloworld.clj 23
89
2.40s user 0.13s system 134% cpu 1.886 total

And here’s the timing with the clj-time example from @borkdude

$ time inlein helloworld.clj 
2018-03-02T18:33:09.593Z
3.16s user 0.17s system 136% cpu 2.433 total

$ time clj -Sdeps "{:deps {clj-time {:mvn/version \"0.14.2\"}}}" helloworld.clj
2018-03-02T18:33:22.258Z
20.84s user 0.45s system 265% cpu 8.035 total

Still inlein is 4 times faster then clj.

Nice little script you whipped up!

didibus · March 2, 2018, 7:08pm

I guess I also wanted to add, Leiningen isn’t really slow per say, more it adds a lot of overhead because it expects to do a lot.

So when you start lein repl, you actually start two Clojure instances, which will double startup time therefore, since you’re really starting two Clojure program, the lein program, and your app. Memory wise, both programs will also be running the whole, time, but you can avoid that by using trampoline with lein.

The other part that is slow, is that lein uses nRepl as the default repl, and that repl takes longer to start. That said, the lein repl gives you tab completion and some nice features like that.

You can choose to start clojure.main instead, but the lein repl task won’t do, you could create your own lein task, or explicitly run clojure.main.

Finally, here’s a little trick you can do to make Lein start a REPL as fast as CLJ does:

$ time LEIN_FAST_TRAMPOLINE=y rlwrap -r -q '\"' -b "(){}[],^%3@\";:'" lein trampoline run -m clojure.main "$@"
2.24s user 0.15s system 119% cpu 1.994 total
$ time clj 
3.41s user 0.12s system 169% cpu 2.087 total

You can see the times are pretty much the same, but actually, Lein gives you a REPL which has dependencies loaded in it from the project.clj file. In my case, I had clj-time as a dependency, if I also want it in my clj repl, I need to use tools.deps:

$ time time clj -Sdeps "{:deps {clj-time {:mvn/version \"0.14.2\"}}}"
16.80s user 0.39s system 249% cpu 6.885 total

And as you see, it is now once again 4 times slower then Lein.

So at equal footing, where Lein and CLJ are doing the same exact thing, they are both as fast, but if you add in dependencies, Lein is faster then tools.deps by a lot.

NOTE: Special note, the only way to avoid Lein starting two Clojure process is with the LEIN_FAST_TRAMPOLINE, which will cache the process in a target folder, the first run will still be slower because the cache is missing, and every change to the project.clj will invalidate the cache, but when the cache is valid, the lein process is skipped completely, which is why you get a boot time equal to a plain clj one.

REMARK: I’m also a bit surprised tools.deps is so slow. It is supposed to cache the classpath, but it doesn’t seem to get any faster on subsequent runs. Maybe I’m not using it correctly.

@jiyinyiyong Hopefully, I convinced you that you can trust inlein and lein a little more

alexmiller · March 2, 2018, 9:23pm

Re clj being slow - the -Sdeps currently turns off classpath caching (because it has to run the Clojure program to incorporate that data into the classpath). That’s fixable though I think. I didn’t expect people to be using -Sdeps all the time when I first added it so didn’t see it as a high priority. That would probably cut the time in half.

jiyinyiyong · March 3, 2018, 1:53am

Thanks. Sure. It changed my impression about Lein a lot.

=>> time inlein a.clj
1

real	0m1.241s
user	0m1.322s
sys	0m0.121s
=>> time clj -i a.clj
1

real	0m1.258s
user	0m2.157s
sys	0m0.147s

#!/usr/bin/env inlein

'{:dependencies [[org.clojure/clojure "1.9.0"]]}

(println 1)

alexmiller · March 3, 2018, 2:37pm

A new version of clj is now available (1.9.0.358) that enables classpath caching with -Sdeps so those timings will improve after first run.

borkdude · March 3, 2018, 10:05pm

With clj 1.9.0.358:

initial usage

$ time ./script.clj
2018-03-03T21:16:06.869Z
./script.clj 10.43s user 0.55s system 280% cpu 3.911 total

notice the speedup due to classpath caching in clj 1.9.0.358

$ time ./script.clj
2018-03-03T21:16:25.889Z
./script.clj 3.77s user 0.22s system 259% cpu 1.536 total

Much faster!