Share the nitty-gritty details of your Clojure workflow!

[philomates]

Hi folks! Cool to read about everyone’s workflows.
My workflow tends to depend on whether I’m working on a new project or doing maintenance on an older project.

new project workflow

With new projects I take a more exploratory, REPL-driven approach.
I’ll open a scratch file along side the source file I’ll commit. I’ll play around sending forms from the scratch file to the REPL until I solidify something. Once a function is sort of ironed out, I’ll move it to the source file, spin up a test file, and use tests to refine the code until it feels robust enough.

existing project workflow

With existing projects, it feels harder to generate example data and mocks for function input, hence making an exploratory approach slower.
Thus, I tend to run the test suite in autorun mode, where modifying a source file will re-run all tests that depend on that code-path. I will then insert print statements (via a fancy macro like this) to get a feel for what data looks like. From there I write a new test exposing the bug or encapsulating the new behavior I’m adding, and use the feedback from the autorunning tests to direct my development

tools

• neovim: because I found vim before I found lisp
• acid.nvim: for jumping to definitions, showing docstrings, etc.
• iron.nvim: for opening an integrated repl, sending code to it, and running tests
• midje: though heavier than clojure.test, it is quite expressive. I’ve found writing tests with it requires a bit less harness code. I also help maintaning it, so if you have issues or suggestions for midje, I’d love to hear them
• ultra: REPL output highlighting plus data and exception pretty-printing

One killer feature I discovered recently is the ability modify library code on the fly. This can be done by jumping to the definition within a library, modifying it, sending it to the REPl, and then returning to your namespace. Some emacs-using colleagues of mine do this, and I was able to get it going in my nvim environment. Before I discovered this, I would to checkout the library’s project, change the code, do a local lein install, and restart my REPL!

[plexus]

What kind of REPL do you use—command-line, CIDER’s REPL buffer, or integrated with your project files? If a file, which ones?

I’m a heavy Emacs+CIDER user. Most projects I do a cider-jack-in to get started, or the project will include some custom script to start nREPL+cider middleware, so I can cider-connect.

Once it’s up and running I’ll rarely look at the REPL buffer, except to look at stack traces, println results, or to check if a command is blocking the thread (there’s a spinner in the modeline). The only things I sometimes type into the REPL are Chestnut-style workflow commands like (go) (start the system), and (cljs-repl) to switch to the Figwheel CLJS REPL.

For the most part I’m typing forms into buffers and eval’ing them, usually at the bottom of the namespace where the code will eventually live, or in the corresponding test namespace if I’m working on tests.

My workflow there is similar to @jackrusher’s, I’ll grab some concrete data (from an API or DB or a file), see what shape it has, and bit by bit build up code that works with that. I’ll use a couple of different eval commands for this, mostly eval-at-point or eval-outer-sexp, as well as eval-and-insert. I also have a variant of the latter which prefixes the result with ;;=>, which is sometimes convenient to tell code from result.

I wasn’t aware of cider-pprint-eval-last-sexp (thanks Jack!), instead I’ll eval-and-insert, then let cider format the result.

At the end most of the time I end up deleting this code. If it’s my own project and I feel it might still be useful later I might leave it in a comment block. Lately I’ve also started keeping separate “repl” files where I keep some of this exploratory code around. I like the idea of having this code “graduate” to become tests, that’s something I’m going to try to integrate more in my workflow.

How do you deal with state in the REPL, such as let bindings or other local vars?

Most of my code is pure, and all the arguments are right there in-line. If I’m debugging I might pull an inner expression out of a function, wrap it in a let block to define locals, then run it that way. It’s rare that I def things just for local state, but it happens, especially for big chunks of data that I can toy around with.

Do you save your REPL explorations—other than any “official” result like a def’d function—off to a file? Where? Does that file go into source control?

If I leave them in comment blocks then they get checked in, for my own projects I might also check in the repl files (I put them in a top leve repl/ directory, so it’s out of the class path). On client projects I tend to keep them around but not check them in.

Do you write tests directly from what you write at your REPL, or separately? From where you run those tests—the shell, CIDER, only on a CI server?

When I’m writing them I mostly run tests from the buffer where I’m writing them, either with (test-name) or with (run-tests). Before committing I’ll run a lein test (or lein eftest) to make sure they work on a clean slate.

Other things to mention

I use paredit heavily. I never got into expand-region, instead I’ll use paredit navigation (up/down/previous/next/forward/backward) plus sp-copy-sexp or sp-kill-sexp. I also rely on cider’s eldoc a lot (show signature of function at point). I started using aggressive-indent-mode so code is always correctly indented. I like rainbow-delimiters to give matching parens matching colors.

I use clj-refactor, though not as much as I used to. Nowadays I mostly use it to add+hotload project dependencies (what people also use Pomegranate for). I really like having these operations like rename, move form, extract function, but the fact that some of them don’t work for CLJS, and that the project is generally quite flaky has left a bit of a bad taste.

[vvvvalvalval]

Great thread, we need more of those!

Another aspect of the workflow worth mentioning is managing the system’s state / environment (what libraries like Component / Mount / Integrant address).

In the context of a Clojure + Datomic web backend, I have a ‘context map’ (containing e.g database connections / values, services implementations, configuration constants, etc.) that is passed to request-handling functions (a system map in Component parlance).

In my bs.dev namespace (which is the default for the REPL in my dev environment), I have a variety of functions for ‘serving’ this context map, in various configurations:

(serve-lab) ;; serves a context with test data in the database
(serve-prod-fork) ;; serves a context which is a 'fork' of the production database - possible thanks to Datomic's speculative powers
(serve-dev) ;; serves a context with the 'dev' database - a local backup of the production database 
(serve-dev-fork) ;; like the above, but operates on a fork (local writes won't be persisted and thus will only be effective the REPL session)

The configurations I use most in practice are lab, dev-fork and prod-fork.

I also have (swap-ctx f & args) & (reset-ctx new-ctx) functions to transform the context map that is currently being served, e.g

(swap-ctx assoc :mail-sender (real-mail-sender))
(reset-ctx my-ctx)

On top of that, I have a bunch of functions to change the component implementations that are currently used in the served context:

(use-conn-prod-fork)
(use-real-mail-sender)

All of this is pretty mutable and side-effecty, which is not very surprising: interactive development is all about having mutable programs (that manipulate immutable values).

Finally, thanks to Datomic, I have the ability to fork context maps (I wrote more about that here). When trying to reproduce a bug that involves writing to the database, I use this ability to ‘save and restore’ checkpoints of my state:

(def cp0 (ctx/fork (current-ctx)))
;; trying to reproduce the bug in the UI
;; failed, let's go back
(reset-ctx (ctx/fork cp0))
;; let's try again
;; failed again, let's go back again
(reset-ctx (ctx/fork cp0))
;; retrying
;; halfway there, let's save another checkpoint
(def cp1 (ctx/fork (current-ctx)))
;; let's keep trying...
;; OK, reproduced and fixed the bug, let's go back to the initial state
(reset-ctx (ctx/fork cp0))

[vvvvalvalval]

@dave.liepmann

How do you deal with state in the REPL, such as let bindings or other local vars?

Reproducing these local bindings at the REPL can get tedious indeed, which is why I made scope-capture. See this video to have an idea of the workflow it yields; In practice I use it mostly with the companion library scope-capture-nrepl in Cursive’s REPL. Comments and questions are always welcome

[brain]

I use vscode and terminal.
usually, I create project via shell cmd.
I edit or type code via vscode.
If there some complex projects I need to figure out dependency libarys, I use IDEA to do it.

I love vscode. for it’s more lightly.

[stardiviner]

I use Emacs Org-mode to do literate programming.
Usually write outlines about program sketch at first, then I start writing Code.
I will start an CIDER REPL session at a basic lein project or project directory.
Then I enable ob-clojure-literate. Execute clojure src blocks after finished a specific small function. When function is finished, then mark the task DONE. I will use Org-mode’s noweb reference to organize source code. If I have any extra tasks that can’t done in Emacs Org-mode. I will tangle src blocks. Then execute commands in Emacs Eshell.

[grierson]

I’ve only used Clojure to solve programming katas, no real world projects so don’t look to deep into my workflow.

What kind of REPL do you use—command-line, CIDER’s REPL buffer, or integrated with your project files? If a file, which ones?

I use Spacemacs.

• , ' To connect to REPL
• , e e (eval-last-sexp) so I can run expression under my cursor.
• , e b (eval-buffer) to update any code changes.

For debugging I like the process described by Stuart Halloway - REPL DEBUGGING: NO STACKTRACE REQUIRED by defining defs with the same name as parameters within functions so that I can then , e e within functions to debug them.

I don’t really use the REPL directly I just write sample code within my editor, send to REPL then delete it when I find a solution that works.

How do you deal with state in the REPL, such as let bindings or other local vars?

, s x Cider refresh. I usually run this when my program isn’t running as expected.

Do you save your REPL explorations—other than any “official” result like a def’d function—off to a file? Where? Does that file go into source control?

I keep some sample/test function calls within my files wrapped within (comment) expressions so that I can , e e on them to check the output.

Do you write tests directly from what you write at your REPL, or separately? From where you run those tests—the shell, CIDER, only on a CI server?

, T t - cider auto test mode - Runs tests each time I , e b

Hope this helps anyone starting with Clojure and Spacemacs.

[CmdrDats]

I’m a bit late to the party, but here’s mine:

I use Cursive for pretty much all my Clojure work. I used to use emacs for about 2 years, but the static analysis (aka, no live REPL required) is killer for me.

REPL I Use: 99.99% of the time, the REPL built into Cursive - connected to an external running Clojure process - I find that much simpler to manage than dealing with inline REPL sessions. Occasionally use command line or online cljs repl, just to quickly test a specific form when I don’t have my dev environment around.

State in REPL: If I want a piece of state to stick around, I usually just def it - if it needs to interact with other code in the namespace, then I’ll usually use an atom. If it’s local to the form I’m sending and don’t really care about it afterward, I’ll wrap it in a let binding.

Save REPL: Hardly ever. REPL explorations get internalised by my head and hardly ever useful for others to read, so I don’t bother. Occasionally it’s code that is useful to have around, but doesn’t warrant a formal def - those I’ll stick in a (comment ...) at the bottom of a relevant namespace

Write tests: I REPL test, or test the actual thing I’m building directly - The occasional automated tests I’ve written in the past never get run or maintained, so I just end up deleting them.

Documentation Lookup: I almost exclusively deduce functionality by first looking at the project readme/code (for new libraries) - For specific functions, I just do an argument lookup (cursive), and failing that, read the source until I understand how it works. If I can’t manage to understand how it works (or, more likely, don’t like the way it works), I find a different library.

Hope that’s helpful for someone!

[maridonkers]

@CmdrDats

I use Cursive for pretty much all my Clojure work. I used to use emacs for about 2 years, but the static analysis (aka, no live REPL required) is killer for me.

No live REPL in Emacs? I connect to a live REPL all the time in Emacs. Two connections, one for Clojure and one for ClojureScript. Jumping to function definitions, getting documentation, it all works (thanks to Cider).

e.g. to develop with Fulcro: Fulcro development with Emacs+Cider

[CmdrDats]

Yes, sorry - I meant that you don’t need a live REPL connection for Cursive’s features to work - so autocompletion + doc lookup + jumping around all works before you even have a clojure process.

It does mean that it doesn’t quite understand all macro’s that create local namespace vars (except where directly supported) - but I steer clear of those as much as possible anyhow - cleaner code at the end of the day xD

[maridonkers]

Yep, was just considering if I had misread your text. That’s an advantage to Cursive, indeed.

[madbonkey]

This is pretty much what I do as well, and I’m very happy with it. I’d like to add Dash for Documentation, I use it excessively via IntelliJ integration, since I work offline a good portion of my time.

[martinklepsch]

@madbonkey

I’d like to add Dash for Documentation, I use it excessively via IntelliJ integration, since I work offline a good portion of my time.

FYI There’s another similar discussion round upcoming specifically with questions around documentation.
I encourage everyone to save their energy until that is up

[Webdev_Tory]

Some excellent stuff here! I’ve changed my workflow as a result of some of these ideas. I work in Emacs, Cider, opening up repls; that hasn’t changed. But I no longer use the REPL buffer, instead considering each of my code files to be its own repl buffer. This in turn has changed my code-writing method; it used to be very let-heavy, but this makes interactive coding difficult. I’m looking into the referred “scope” library someone mentioned; meanwhile I’m deciding on the value of trading the let-as-documentation I’ve previously followed for the increased dynamic coding of inlining what used to be let-bound for me, aiming for the point where a function only refers to the variables that are passed in as arguments.

[krisleech]

I’m very much a beginner with Clojure. I use it in my day job, but not exclusively.

Editor

Being a long time Vim user that is where I started but couldn’t get a decent setup - I think I had conflicting key bindings. I will try again with Vim (actually NeoVim) again at some point, with a blank vimrc.

I tried Emacs but had little success due to lacking the muscle memory. I’d love to learn emacs one day. But for now I need to write Clojure. I also tried spacemacs before, but even with evil-mode there is a lot to learn.

Very recently I tried Cursive with the VimIDEA plugin, this is great. I setup some key bindings to mimic what I was used to in Vim. I really like Cursive, but I need to do work on the configuration, I still need to grab the mouse to perform certain actions.

Parinfer has been a revelation.

REPL

I’m still experimenting with different flows.

At the moment I start a REPL (within Cursive) and load the current namespace. Then I start writing the skeleton for my function. At the bottom off the ns I call the function with data pulled from the database. I can run this function in the REPL using a key binding and observe the output. I don’t often type directly in to the REPL. Once I get something working I save and commit to git. Then I create a test (either permanently in the test directory or temporarily within the ns) and start to refactor, reloading the ns and running the test as I go.

[mopedtobias]

how do you do that in spacemacs? I can not find CIDER C-u C-x C-e in evil mode:(

[dave.liepmann]

I’m not sure how evil-mode-Spacemacs does it, but C-h f on the function name (cider-eval-defun-at-point) should lead you down the right path. I think some Spacemacs configs even show keybinding suggestions when the option comes up with M-x.

[pesterhazy]

Does this help: https://github.com/syl20bnr/spacemacs/blob/master/doc/DOCUMENTATION.org#universal-argument ?