Details on how to do REPL driven development from the editor (Emacs with CIDER)

I love the quick feedback when working with Figwheel. I have my editor on the left and a browser on the right. After saving a file (which is one action which I have to do anyway) I see the result.

But when working on server code I have to do a lot more “actions” in my editor. I feel I am missing something.

Example: I have a namespace with some functions. The purpose of the namespace is to parse XML and build a map (EDN). The functions extract data, merge data and create a hierachy. To eval individual functions I need test data, so I create a comment with clojure code for laoding an XML file and calls to my functions.

Then I eval my namespace and then the functions from inside my “Rich” comments.

Now I fix a bug in my functions and would like to see the effects as soon as possible. But I have to eval the namespace (or the fixed function) and then navigate to my comments and eval the test code. Then navigate back to my function I am working on and so forth.

An alternative I tried is to put my test code to the bottom of my clojure file (not in an comment) and to eval the whole buffer after saving the file. This gets rid of jumping in code all the time, but it happened a few times, that I have commited my test code.

What workflow are you using? Is there a way to get feedback immediately after saving the file?

Hello, @witek!

Welcome to Clojureverse! Good question.

On the function level I’ve converged on wrapping my function definition and function call / test code in a do. In production, I want to end up with Rich comments, so that I don’t eval things I don’t want to. Especially when there are side effects involved.

(ns th.scratch.wrapdo-demo)

(defn zeros [n]
  (apply str (repeat n "0")))

(def digit?
  (set "0123456789"))

(defn valid? [credit-card-number]
  (and
   (string? credit-card-number)
   (every? digit? credit-card-number)
   (<= 13 (count credit-card-number) 16)))

(comment
  (for [v [nil 123 "0934" "08abc" (repeat 12 "0")
           (zeros 12) (zeros 13) (zeros 16) (zeros 17)]]
    [v (valid? v)])

  )

Whereas when I’m developing a single function, I want to re-evaluate it with one C-c C-c, and I temporarily wrap it in a do like

(do
  (defn valid? [credit-card-number]
    (and
     (string? credit-card-number)
     (every? digit? credit-card-number)
     (<= 13 (count credit-card-number) 16)))
  (for [v [nil 123]]
    [v (valid? v)]))
;; => ([nil false] [123 false])

I’m doing this for practically any pure function I’m developing.

On the system level, you’ll want to be able to reload parts of your system that you’ve changed. A simple way to do this is to take care that you can use “function redefinition” to refresh your system. Check out the HTTP-kit docs for an example. Principle: ensure that redefining the functions you’re using will actually change your system.

When you want to do this across your whole system, you might want to use something like Integrant . Integrant makes it simple to write you application as a reloadable system. Bonus: it understands which source files are in use, and is able to refresh only the files you’ve changed, and deduce which parts of the system must be reloaded. But you probably want to understand the basics by messing around a bit yourself before you come here.

Hope that helps! I have pondered the same question myself.

Teodor

3 Likes

I’m interested in this too – I like the do approach. Perhaps there might be some elisp functions you can write to do specific things that are context-specific?

When working with ClojureScript code which has no UI, I have created https://github.com/witek/bindscript to solve this problem.

I write a (def-bindscript ...) with test code directly after my functions. The idea is that for production, the macro yields nothing while in development mode it evaluates all expressions and renders the output in the browser live via Figwheel. Therefor a simple save is enough.

I would like to make it work for server code too, by serializing the output and sending it to the browser.

But there is a lot to fix in bindscript - and I hope for a more lightweigt solution.

Hum… I normally put REPL code at the bottom of the file as the last thing, and wrapped inside a #_ and a do.

Now normally I run eval buffer followed by go to end of file and followed by eval last s-exp. Even though the prior sexp is wrapped inside a #_ cider is smart enough to eval what’s inside it. From my Spacemac in Holy mode (and some of my own bindings) it goes:

M-Enter s b M-> C-Enter

This will eval the buffer, go to the end of file, eval the last s-exp, which is a big do of all my repl only code wrapped inside a #_.

I wonder if in your case, you wouldn’t prefer to just setup a test runner that runs on save?

1 Like

@witek bindscript looks great.

First I think some comment can be committed, those show case for Usage, putting in a well formatted bottom comment block of file.

Second for the testing purpose code, put them in correspond test namespace, that is foo.bar-test for foo.bar. Then CIDER have a command called cider-run-ns-tests, this will execute test code in the correspond test namespace. There’s also a command called cider-eval-last-sexp-and-replace, sometimes it’s useful for writing tests.

PS. There’s an option called clojure-toplevel-inside-comment-form, set it to t so that you can use C-c C-c to execute code in comment block.

1 Like

I tend to have Rich Comment Forms directly after each function that I feel I need to test, which avoids quite a bit of back and forth navigation. I also have a hot key that takes a let binding pair (symbol/expression) and does a global def of it, so I can more easily eval code inside a function definition directly without needing an RCF (all I need is defs for sample argument vars).

Another option, which I have not yet used but I consider from time to time is to use clojure.test's with-test function which lets you do this:

(with-test

  (defn foo [n] (* n n))

  (is (= 4 (foo 2)))
  (is (= 0 (foo 0))))

Now I can eval the top-level block as I am working, and then with the cursor on foo in the defn, I can hit a key that “runs the test under the cursor” and have the test(s) execute.

2 Likes

Or dumb enough. :smile:

At least that’s the case with Calva. Maybe @bbatsov can enlighten us if that’s so with CIDER too.

That’s similar to what I sometimes do:

(defn foo
  {:test (fn []
           (is (= 4 (foo 2)))
           (is (= 0 (foo 0))))}
  [n]
  (* n n))

Which, at least in Calva, lets me Run current test with the cursor anywhere within the defn form, so there is no moving around the cursor at all in order to test whatever change I’ve made. And when I’m done I decide if I want to leave some tests with the function. (I often decide to do that.)

1 Like

Yeah, I should really try to update my code for Chlorine to work like that – it would be a lot more convenient!

1 Like

Hi, @PEZ

Will you keep the test when commit or just use it as a quick way to evaluate code?

Most often I will leave the tests in the code. But, the full story is that I usually experiment in a comment block quite a while before my functions take any shape that I am ready to write tests for. So when I do add the tests they are most often intended to stay there.

(I often commit code with some of the comment form experiments left there as well, btw.)

I can’t add much to the discussion as while I code I typically just re-eval the forms I’m working on and test them in the REPL or trigger the ns tests with C-c C-t C-n. A cool option is also to auto-run the tests when evaluating some ns https://docs.cider.mx/cider/testing/running_tests.html#_running_tests_automatically_test_driven_development

I’ve always been quite fond of having very granular control over evaluations, though, and I never liked things happening by themselves. I just it’s a matter of what you’re used to.

Btw, auto re-loading on save would be trivial to achieve. It’s just a matter of triggering the evaluation command from the appropriate hook (e.g. after-save-hook).

1 Like

+1 on the use of do, especially if there are several forms that need to be evaluated each time. I also typically start any given function as a top level let with bindings that match what the function’s arguments will be, do all my work in that context, then change the let to a defn when I’m done.

1 Like

I want to reinforce the suggestion of @bbatsov about leveraging tests for this. That’s where my example code goes and with cider-auto-test I get my test re-tried if ever I C-c C-k evaluate the buffer. There are a couple handy helps here:

  1. C-c C-t C-p : run all project tests. Usually what I do when I get back to a project for the first time in a while. Occasionally instead use C-c C-t n to only run tests for one namespace.
  2. C-c C-t t: Run the test at cursor, once I’ve located which test I want to run from step 1.
  3. C-c C-t g: Re-run whichever of my previously run tests failed. Execute this while in my actual src namespace, not in a testing namespace, and keep at it until my test is passing.

These are the three I use most right now, and they work quite well. The advantages of this are several:

  • Tests won’t cause issues when you deploy, so you avoid the problem you mentioned with test-code getting deployed
  • They persist as documentation and regression tests; my “repl” code is no longer single-use and can give example to future-me or other developers on the project of how things should be used/tested

In other words, it gets rid of the major problems of accidents and extemporaneousness of normal in-buffer-REPL-code and, thanks to Cider, is not really slower (particularly in the long-run). As someone who has previously done a lot of re-writing of the very same repl code for debugging later in the week and month, this is great.

2 Likes

For me, the most common end point of the process: after you have written your function and verified that it does what you think it should using appropriate sample data (hopefully probing boundary conditions in the process), hoist that code out into a test. I would not start off with the verification code in a different file from the function itself (e.g. in a test namespace), as I’d rather have everything in one place (preferably on the same screen within the buffer) while working out the implementation.

What I most commonly use the comment block at the end of the file for is various interactive tools to be used from within your coding environment (e.g. emacs). Often, after things have settled down, I will factor those tools out into a group of interactive functions in a separate file/namespace called repl.

3 Likes

I like your method and agree with the principle. Because I fluidly use multiple buffers/frames and having them in separate (e.g. testing) namespaces actually works better with that, my process favors never putting them in the same buffer to start with (especially because having a testing ns and Cider’s testing facilities is actually easier for me than scrolling around the same buffer).

Does CIDER have a way to quickly switch between SUT and test namespaces? There is a feature request on Calva asking for that. Reading this thread, it sounds like a good idea.

In the event that I don’t have the test suite in my last two buffers (in which case I’d just use my version of alt+tab or C-x b to get there), the fastest way I know is to open up the most recent test result report and RET on one of the failing tests. Alternatively, emacs standard C-x ` will take you to the next error (or use cider-next-compilation-error or use an emacs register for static-speed back-and-forth: set register with C-x r SPC DIGIT and jump to it by C-x r j [digit].

1 Like

Emacs projectile package can switch between test and implementation buffers, for example in Spacemacs its SPC p a