Do we need a socket repl/unrepl client geared towards shell use?

I’ve been thinking for a while that it would be super great to have a socket repl/unrepl client geared towards shell and shell script usage.

My hands have been itching to get started on this, but I really have enough going on already, so I’m just going to put this up here and hope that someone takes the bait :slight_smile:

Let’s call this hypothetical command line utility sokclj. You could call it like this

sokclj -H localhost:5555 -e '(my-app.tasks/clear-cache)'

Now you might argue that you can do the same thing with nc or telnet, and indeed you can

nc localhost 5555 <<< '(my-app.tasks/clear-cache)'

The difference would be that the sokclj version hooks up STDOUT and STDERR separately, and that it exits with a non-zero exit code in case of an exception, that’s why basing it on unrepl would make sense. But also, we could pack it with a lot of useful command line flags.

# send the file to the REPL, connect to default localhost:5555
sokclj my_script.clj

# mirror Clojure's -m flag to invoke a -main function
sokclj -m my-app.tasks.do-thing arg1 arg2

# mirror Ruby's (or sed's) -n switch, read line per line from STDIN, and assign each line to $
sokclj -n '(println (clojure.string/uppercase $))'

# Same as above, but run `prn` on the result after each iteration
sokclj -p '(clojure.string/uppercase $)'

# require a namespace
sokclj -r my-app.stuff -e '(my-app.stuff/do-it)'

I can also imagine options for dealing with EDN input/output (or other formats).

It would have to boot fast, so it can’t be written in Clojure, but it could use ClojureScript and e.g. Node, or Lumo. Maybe some of these features could be added to Unravel (wdyt @pesterhazy?).

Can anyone else imagine a need for this? Calling things e.g. from Cron jobs or Ansible scripts?

2 Likes

I was looking for something like this a while ago, in my case it was for dev-tooling (integrating zprint with non-cider editors), where I would have needed sending a form to eval while passing stdin as an argument.

I think now would be a good time to revisit this meme :slight_smile: Clojure boots in under a second on modern machines. I think you could easily put together a library that does what you describe purely in Clojure. If boot time for this particular tool becomes a concern (which I doubt) then investigate AOT as we’ve done with ClojureScript. AOTed ClojureScript adds about 1.5 seconds to Clojure startup time since it’s loading about ~16KLOC of Clojure.

A second is really too long for what I have in mind, the overhead should be negligible, say < 100ms. But point taken, I really should bench this to see how suitable Clojure could be.

Last time I checked Lumo & Planck both take >200ms for trivial eval, even on similar hardware to what I’ve described above. So you’ll be dealing with at least that level of latency + compile time for whatever your lib does.

So going the Clojure library route will cost you at most half a second but now you have a normal Clojure dependency with all the benefits that implies in the post clj world.

2 Likes

Note, prepl is now included in the Clojure 1.10 alphas and we are expecting to add some special support for it in clj, specifically for the use of remote data-focused repls with appropriately bound streams.

1 Like

That sounds promising! Would you have a link for people who want to know what prepl is? (asking for a friend :wink: )

REPLs leverage the ubiquitous paradigm of stdio streams. But they have the downside of mingling evaluation output with printing output (the “Print” step). PREPL allows for (remoteable) streaming in, structured out. This is useful for many kinds of REPL tooling allowing it to respond differently to eval output vs printing output. More to come (plus some motivating uses) in the near future.

2 Likes

I think the most important thing for any CLI tool is startup speed, JVM takes too long. It should be in the range of 200-400 millisecond. What we really need is someone porting clojure to GraalVM. Which is designed for this purpose.

The JVM really doesn’t take long. Write “Hello World” in Java and you will see it takes far less than 100ms to run that. Clojure takes some time but it’s around 600-700ms on current hardware. However popular Clojure tooling with the notable exception of Maven (with classpath caching) and clj tends to add significant overheads to this baseline and from this people make inaccurate claims that this problem is somehow fundamental to the JVM or Clojure.

2 Likes

Just checked it wasnt aware that problem lies somewhere else.

I think Clojure(Script) makes it too easy to rewrite the wheel. Please don’t start fresh, extend Unravel.