Share the nitty-gritty details of your Clojure workflow!


#12

with the results appearing transiently right where I’m working

interested in seeing a screenshot of this. In Cursive I always have a separate vertical column with the REPL history.


#13

interested in seeing a screenshot of [results appearing transiently right where [Dave is] working]. In Cursive I always have a separate vertical column with the REPL history.

Hold on a second while I disable f.lux, because it’s 3pm in Berlin…

My default method is to evaluate in place (cider-eval-last-sexp with C-x C-e, for those of you following along in emacs), which makes the result display in two places: the minibuffer (at the bottom) and in a overlay that disappears momentarily (that’s the one immediately after the with-open):


I like this command because I can evaluate inner expressions as well as the top-level form.

My “sometimes” method is to use a prefix argument (C-u C-x C-e) so that the result is spit as text directly into the buffer I’m in, so I can work with it.


Talking to folks about these nitty-gritty details reminds me to play more with similar commands like cider-eval-defun-at-point (to always evaluate the top-level form regardless of cursor position within a subexpression) and cider-pprint-eval-last-sexp.


#14

I have two main projects right now. One is an CLJS app for work, the other is an open-source library that I primary develop using CLJ (it supports CLJS, but most of my development is in Clojure). My answers are slightly different for each.

:bmo: What kind of REPL do you use—command-line, CIDER’s REPL buffer, or integrated with your project files? If a file, which ones?

CLJS-app: I start lein repl in the terminal. I then connect with cider-connect and then run

(use 'figwheel-sidecar.repl-api)
(start-figwheel!)
(cljs-repl)

My workflow might be out-of-date here, but I found that more CIDER features work when I do this - namely code navigation.

CLJ-lib: I start lein repl and use cider-connect.

In both projects, I then will create a (comment) block and start evaluating forms directly from my editor with cider-eval-last-sexp (bound to a keybinding, of course). The results show up inline in my editor.

Occasionally the results are too big too understand. I’m not too good at navigating the cider-inspect-last-result buffer, so instead I run cider-eval-last-sexp-and-replace to put the data right in my comment block. I can then use it directly in subsequent experiments, or name it with let or def.

From there, I tend to build up my expression with lots of tiny experiments, and then extract it to a function.

If I’m investigating a bug with an existing function, I tend to use some combo of:

  1. Adding println (usually prefixed with my initials like [:bhb {:x x :y y}] so I can grep for “bhb” to remove all debugging lines)
  2. Creating an example of the function call, then slowly comment out parts of the function implementation so I can see each step returned.
  3. Sometimes I’ll use “def” to define vars that mirror the local bindings, and then directly evaluate the inner sexps. I usually do this manually, but I have been trying to use scope-capture more to automate this.

I tend to run git commit -am "cp" (“cp” == “checkpoint”) often, so I can commit experiments, messy code, printlns, etc. When I have a logical piece of work to commit, I’ll clean it up, git reset <sha> to the last “real” commit, and then make a single commit of the total changes.

I use both Emacs+CIDER and Cursive: I write/edit nearly all my code in Emacs+CIDER, but occasionally open Cursive to look at warnings and also for renaming symbols and keywords.

:sunrise_over_mountains: How do you deal with state in the REPL, such as let bindings or other local vars?

This hasn’t historically been much of a problem for me. In CLJS, I can just refresh the browser if I notice a problem, which is rare. And in CLJ (perhaps because my project is a library without much state), I rarely run into bugs with stale state. In the rare cases where this occurs, my tests usually catch the issue quickly (see below)

:paperclip: Do you save your REPL explorations—other than any “official” result like a def’d function—off to a file? Where? Does that file go into source control?

I don’t normally save the REPL explorations. For experimentation outside of any specific project, I’ve been toying with a single project with a single namespace that is just a long log of REPL experiments. That project contains all the various tools I might use and I commit all my experiments, but it’s not part of my normal flow for either of my daily projects.

:books: Do you write tests directly from what you write at your REPL, or separately? From where you run those tests—the shell, CIDER, only on a CI server?

I do convert some of my REPL experiments to tests.

CLJS-app: I run tests on the terminal with karma. They re-run every time I save a file

CLJ-app: I run tests with lein test-refresh. They re-run whenever I save a file.

Although this feedback cycles isn’t as fast as I’d like (especially the CLJS tests, the CLJ tests are pretty snappy), I do like that this protects me somewhat from the occasional bugs resulting from stale state in my REPL.

I also run the full suite in CI, just as a final check (this often catches things like forgetting to commit a file :grimacing:)


#15

I forgot to mention it, but I am also a heavy user of the eval-and-insert-at-point approach.


#16

This thread is full of learnings. Thanks to every one who contributed so far


#17

I didn’t mention this, but I also use paredit. I have found that many beginners don’t realize that they will be much, much more productive once they learn paredit or parinfer. I worked in Clojure for a few years in my spare time without structural editing. I knew paredit existed, but I didn’t have a good sense of the time investment necessary vs the productivity difference.

It was only when I pair programmed with some experienced Clojurists did I realize how much time I was wasting on bugs related to parenthesis that they never experienced.

Speaking of tooling, I have been pleasantly surprised with how useful Joker is when integrated with my editor (Emacs in my case, but it can be set up with Sublime, Atom, and vim). Many typos that used to take me tens of seconds or more to identify and fix can now be fixed within a few seconds. I highly recommend it if you haven’t tried it.


#18

I blogged about REPL-based development, eg using the REPL window vs doing things in an editor window, using autoloaders vs not, reloading everything from scratch without restarting the JVM, and how REPL-based ties in with other things (eg TDD). Hopefully it’s interesting to some people: http://blogish.nomistech.com/repl-based-development/


#19

:bmo: What kind of REPL do you use
ProtoREPL in Atom. I start the REPL and keep it open for days. We use Boot so we can easily load new dependencies as needed in a running REPL. I typically have the REPL pane minimized to just three lines at the bottom of the editor and rely mostly on ProtoREPL’s inline result display (it lets you click to expand and drill into large data structures). We also use Component so we can easily start & stop processes and applications in the REPL as needed – and continue to evaluate new code into the live, running application while we work.

:sunrise_over_mountains: How do you deal with state in the REPL, such as let bindings or other local vars?
I either evaluate an s-exp (C-M-, b) or a top-level form (C-M-, B) or, occasionally, a whole file (C-M-, F). I write comments in the source files containing expressions I want to evaluate in the REPL, including defs for symbols that act as context for expressions in let bindings and function arguments, so I can C-M-, b those as needed prior to evaluating fragments of functions.

:paperclip: Do you save your REPL explorations
I’ve gotten into the habit now of using comment forms in source files for everything I would evaluate in the REPL so that my explorations are always available in amongst the source code (or the test code). Stu Halloway’s mid-2017 Chicago Clojure talk – REPL-Driven Development – talks about this: never type into the REPL itself: always type into a source file and just evaluate forms into the REPL.

:books: Do you write tests directly from what you write at your REPL, or separately?
See above. I don’t type code into the REPL at all now. I write comment forms in my source or test files and evaluate s-exp’s inside those comments into the REPL. I use Expectations for testing, with the new expectations.clojure.test namespace so standard C-M-, t (run test under cursor) and C-M-, x (run all tests in current namespace) hotkeys work (as if you were using clojure.test).

Happy to answer any questions that this approaches raises in people’s minds!


#20

I have nothing particular interesting to add regarding my workflow – besides that I am using Atom with ProtoREPL. But I’ve just been watching Tony Kay’s Fulcro 2.0 Series which includes one video about his workflow which I quite like!


#21

Hi folks! Cool to read about everyone’s workflows.
My workflow tends to depend on whether I’m working on a new project or doing maintenance on an older project.

new project workflow

With new projects I take a more exploratory, REPL-driven approach.
I’ll open a scratch file along side the source file I’ll commit. I’ll play around sending forms from the scratch file to the REPL until I solidify something. Once a function is sort of ironed out, I’ll move it to the source file, spin up a test file, and use tests to refine the code until it feels robust enough.

existing project workflow

With existing projects, it feels harder to generate example data and mocks for function input, hence making an exploratory approach slower.
Thus, I tend to run the test suite in autorun mode, where modifying a source file will re-run all tests that depend on that code-path. I will then insert print statements (via a fancy macro like this) to get a feel for what data looks like. From there I write a new test exposing the bug or encapsulating the new behavior I’m adding, and use the feedback from the autorunning tests to direct my development

tools

  • neovim: because I found vim before I found lisp
  • acid.nvim: for jumping to definitions, showing docstrings, etc.
  • iron.nvim: for opening an integrated repl, sending code to it, and running tests
  • midje: though heavier than clojure.test, it is quite expressive. I’ve found writing tests with it requires a bit less harness code. I also help maintaning it, so if you have issues or suggestions for midje, I’d love to hear them :slight_smile:
  • ultra: REPL output highlighting plus data and exception pretty-printing

One killer feature I discovered recently is the ability modify library code on the fly. This can be done by jumping to the definition within a library, modifying it, sending it to the REPl, and then returning to your namespace. Some emacs-using colleagues of mine do this, and I was able to get it going in my nvim environment. Before I discovered this, I would to checkout the library’s project, change the code, do a local lein install, and restart my REPL!


#22

:bmo: What kind of REPL do you use—command-line, CIDER’s REPL buffer, or integrated with your project files? If a file, which ones?

I’m a heavy Emacs+CIDER user. Most projects I do a cider-jack-in to get started, or the project will include some custom script to start nREPL+cider middleware, so I can cider-connect.

Once it’s up and running I’ll rarely look at the REPL buffer, except to look at stack traces, println results, or to check if a command is blocking the thread (there’s a spinner in the modeline). The only things I sometimes type into the REPL are Chestnut-style workflow commands like (go) (start the system), and (cljs-repl) to switch to the Figwheel CLJS REPL.

For the most part I’m typing forms into buffers and eval’ing them, usually at the bottom of the namespace where the code will eventually live, or in the corresponding test namespace if I’m working on tests.

My workflow there is similar to @jackrusher’s, I’ll grab some concrete data (from an API or DB or a file), see what shape it has, and bit by bit build up code that works with that. I’ll use a couple of different eval commands for this, mostly eval-at-point or eval-outer-sexp, as well as eval-and-insert. I also have a variant of the latter which prefixes the result with ;;=>, which is sometimes convenient to tell code from result.

I wasn’t aware of cider-pprint-eval-last-sexp (thanks Jack!), instead I’ll eval-and-insert, then let cider format the result.

At the end most of the time I end up deleting this code. If it’s my own project and I feel it might still be useful later I might leave it in a comment block. Lately I’ve also started keeping separate “repl” files where I keep some of this exploratory code around. I like the idea of having this code “graduate” to become tests, that’s something I’m going to try to integrate more in my workflow.

:sunrise_over_mountains: How do you deal with state in the REPL, such as let bindings or other local vars?

Most of my code is pure, and all the arguments are right there in-line. If I’m debugging I might pull an inner expression out of a function, wrap it in a let block to define locals, then run it that way. It’s rare that I def things just for local state, but it happens, especially for big chunks of data that I can toy around with.

:paperclip: Do you save your REPL explorations—other than any “official” result like a def’d function—off to a file? Where? Does that file go into source control?

If I leave them in comment blocks then they get checked in, for my own projects I might also check in the repl files (I put them in a top leve repl/ directory, so it’s out of the class path). On client projects I tend to keep them around but not check them in.

:books: Do you write tests directly from what you write at your REPL, or separately? From where you run those tests—the shell, CIDER, only on a CI server?

When I’m writing them I mostly run tests from the buffer where I’m writing them, either with (test-name) or with (run-tests). Before committing I’ll run a lein test (or lein eftest) to make sure they work on a clean slate.

Other things to mention

I use paredit heavily. I never got into expand-region, instead I’ll use paredit navigation (up/down/previous/next/forward/backward) plus sp-copy-sexp or sp-kill-sexp. I also rely on cider’s eldoc a lot (show signature of function at point). I started using aggressive-indent-mode so code is always correctly indented. I like rainbow-delimiters to give matching parens matching colors.

I use clj-refactor, though not as much as I used to. Nowadays I mostly use it to add+hotload project dependencies (what people also use Pomegranate for). I really like having these operations like rename, move form, extract function, but the fact that some of them don’t work for CLJS, and that the project is generally quite flaky has left a bit of a bad taste.


The Bare Minimum, or Making Mayonnaise with Clojure
#23

Great thread, we need more of those!

Another aspect of the workflow worth mentioning is managing the system’s state / environment (what libraries like Component / Mount / Integrant address).

In the context of a Clojure + Datomic web backend, I have a ‘context map’ (containing e.g database connections / values, services implementations, configuration constants, etc.) that is passed to request-handling functions (a system map in Component parlance).

In my bs.dev namespace (which is the default for the REPL in my dev environment), I have a variety of functions for ‘serving’ this context map, in various configurations:

(serve-lab) ;; serves a context with test data in the database
(serve-prod-fork) ;; serves a context which is a 'fork' of the production database - possible thanks to Datomic's speculative powers
(serve-dev) ;; serves a context with the 'dev' database - a local backup of the production database 
(serve-dev-fork) ;; like the above, but operates on a fork (local writes won't be persisted and thus will only be effective the REPL session)

The configurations I use most in practice are lab, dev-fork and prod-fork.

I also have (swap-ctx f & args) & (reset-ctx new-ctx) functions to transform the context map that is currently being served, e.g

(swap-ctx assoc :mail-sender (real-mail-sender))
(reset-ctx my-ctx)

On top of that, I have a bunch of functions to change the component implementations that are currently used in the served context:

(use-conn-prod-fork)
(use-real-mail-sender)

All of this is pretty mutable and side-effecty, which is not very surprising: interactive development is all about having mutable programs (that manipulate immutable values).

Finally, thanks to Datomic, I have the ability to fork context maps (I wrote more about that here). When trying to reproduce a bug that involves writing to the database, I use this ability to ‘save and restore’ checkpoints of my state:

(def cp0 (ctx/fork (current-ctx)))
;; trying to reproduce the bug in the UI
;; failed, let's go back
(reset-ctx (ctx/fork cp0))
;; let's try again
;; failed again, let's go back again
(reset-ctx (ctx/fork cp0))
;; retrying
;; halfway there, let's save another checkpoint
(def cp1 (ctx/fork (current-ctx)))
;; let's keep trying...
;; OK, reproduced and fixed the bug, let's go back to the initial state
(reset-ctx (ctx/fork cp0))

#24

Reproducing these local bindings at the REPL can get tedious indeed, which is why I made scope-capture. See this video to have an idea of the workflow it yields; In practice I use it mostly with the companion library scope-capture-nrepl in Cursive’s REPL. Comments and questions are always welcome :slight_smile:


Writing a beginner-friendly REPL guide for clojure.org, please send me inspiration
#25

I use vscode and terminal.
usually, I create project via shell cmd.
I edit or type code via vscode.
If there some complex projects I need to figure out dependency libarys, I use IDEA to do it.

I love vscode. for it’s more lightly.


#26

I use Emacs Org-mode to do literate programming.
Usually write outlines about program sketch at first, then I start writing Code.
I will start an CIDER REPL session at a basic lein project or project directory.
Then I enable ob-clojure-literate. Execute clojure src blocks after finished a specific small function. When function is finished, then mark the task DONE. I will use Org-mode’s noweb reference to organize source code. If I have any extra tasks that can’t done in Emacs Org-mode. I will tangle src blocks. Then execute commands in Emacs Eshell.


#27

I’ve only used Clojure to solve programming katas, no real world projects so don’t look to deep into my workflow.

:bmo: What kind of REPL do you use—command-line, CIDER’s REPL buffer, or integrated with your project files? If a file, which ones?

I use Spacemacs.

  • , ' To connect to REPL
  • , e e (eval-last-sexp) so I can run expression under my cursor.
  • , e b (eval-buffer) to update any code changes.

For debugging I like the process described by Stuart Halloway - REPL DEBUGGING: NO STACKTRACE REQUIRED by defining defs with the same name as parameters within functions so that I can then , e e within functions to debug them.

I don’t really use the REPL directly I just write sample code within my editor, send to REPL then delete it when I find a solution that works.

:sunrise_over_mountains: How do you deal with state in the REPL, such as let bindings or other local vars?

, s x Cider refresh. I usually run this when my program isn’t running as expected.

:paperclip: Do you save your REPL explorations—other than any “official” result like a def’d function—off to a file? Where? Does that file go into source control?

I keep some sample/test function calls within my files wrapped within (comment) expressions so that I can , e e on them to check the output.

:books: Do you write tests directly from what you write at your REPL, or separately? From where you run those tests—the shell, CIDER, only on a CI server?

, T t - cider auto test mode - Runs tests each time I , e b

Hope this helps anyone starting with Clojure and Spacemacs.


#28

I’m a bit late to the party, but here’s mine:

I use Cursive for pretty much all my Clojure work. I used to use emacs for about 2 years, but the static analysis (aka, no live REPL required) is killer for me.

:bmo: REPL I Use: 99.99% of the time, the REPL built into Cursive - connected to an external running Clojure process - I find that much simpler to manage than dealing with inline REPL sessions. Occasionally use command line or online cljs repl, just to quickly test a specific form when I don’t have my dev environment around.

:sunrise_over_mountains: State in REPL: If I want a piece of state to stick around, I usually just def it - if it needs to interact with other code in the namespace, then I’ll usually use an atom. If it’s local to the form I’m sending and don’t really care about it afterward, I’ll wrap it in a let binding.

:paperclip: Save REPL: Hardly ever. REPL explorations get internalised by my head and hardly ever useful for others to read, so I don’t bother. Occasionally it’s code that is useful to have around, but doesn’t warrant a formal def - those I’ll stick in a (comment ...) at the bottom of a relevant namespace

:books: Write tests: I REPL test, or test the actual thing I’m building directly - The occasional automated tests I’ve written in the past never get run or maintained, so I just end up deleting them.

:books: Documentation Lookup: I almost exclusively deduce functionality by first looking at the project readme/code (for new libraries) - For specific functions, I just do an argument lookup (cursive), and failing that, read the source until I understand how it works. If I can’t manage to understand how it works (or, more likely, don’t like the way it works), I find a different library.

Hope that’s helpful for someone! :slight_smile:


#29

No live REPL in Emacs? I connect to a live REPL all the time in Emacs. Two connections, one for Clojure and one for ClojureScript. Jumping to function definitions, getting documentation, it all works (thanks to Cider).

e.g. to develop with Fulcro: Fulcro development with Emacs+Cider


#30

Yes, sorry - I meant that you don’t need a live REPL connection for Cursive’s features to work - so autocompletion + doc lookup + jumping around all works before you even have a clojure process.

It does mean that it doesn’t quite understand all macro’s that create local namespace vars (except where directly supported) - but I steer clear of those as much as possible anyhow - cleaner code at the end of the day xD


#31

Yep, was just considering if I had misread your text. That’s an advantage to Cursive, indeed.