Another topic on inline unit tests in Clojure

@plexus said something on the #clojure-europe Clojurians Slack channel which made think about inline unit tests again:

[…] Unless you drag in Clojure.test in all your namespace […]

Let me give you more context before getting to the point:

I’m writing a tool in Clojure and I use rich comments a lot to test functions and experiment “at the REPL”, like many Clojurians do I guess.
Most of the rich comments eventually end up as test cases, so I can either move them to a test namespace, or duplicate them. I don’t like any of that.

So I started using with-test and I hooked up an IDE keybindings to run my namespaces’ tests and a second one to run all the tests with eftest.
Now I just have to convert my rich comments into with-test blocks and keep’em just under the code, as in:

(defn foo [_]
  ...)

(with-test #'foo
  (testing "..."
    (is ...))
  (testing "..."
    (is ...)))

I like that setup.

I googled a bit on the subject and found this post: Inline unit tests in Clojure

Let me quote the author:

I’m a fan of inline unit testing. I firstly encountered the idea with Rust and I got hooked on it when applying it in Racket.
This post is about sharing my experiences with using inline unit testing in Clojure.

I chatted with the author via email and some things he said resonated with me. Quoting him again:

I didn’t do any Clojure for quite some time now. Both Racket and Rust, which I’ve done more recently, support tests in the same file (which was my definition of “inline unit tests” when I wrote the post), but in a separate “test” module. Here’s some Racket code I wrote: https://github.com/pyohannes/racket-symalg/blob/master/symalg/simplify.rkt#L68
[…]
I’d use inline unit tests for code that is functional and concise, because then it can actually enhance code readability.

I don’t know much about Rust but the documentation has a chapter on testing here.
Here’s a code sample:

pub fn add_two(a: i32) -> i32 {
    a + 2
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        assert_eq!(4, add_two(2));
    }
}

It then says:

The current convention is to use the tests module to hold your “unit-style” tests. Anything that just tests one small bit of functionality makes sense to go here. But what about “integration-style” tests instead? For that, we have the tests directory.

Unit testing code lives inside the modules they tests and “integration-style” tests live in a dedicated tests directory.
I like that. A lot!

I’m not the only one doing/liking inline unit tests it appears, judging by the previous discussions (all closed) on the ClojureVerse here and here.

Most of the arguments against it seem to be about readability. The author of the first post I’ve linked addresses it like so (emphasis mine):

One of the biggest concerns regarding inline unit tests is readability - the fear that productive code gets lost in test code. To address this concern, let me show you how my editor of choice vim displays the file src/inline_tests/round5.clj:

(ns inline_tests.round5
   (:require [clojure.test :refer [deftest is testing]]))`

(declare round5)

(deftest round5-test
+--  9 lines: (let [expected (fn [e] #(= e (round5 %)))]----------

(defn round5
+-- 15 lines: "Round to the closest positive multiple of 5."------

The point I want to make here is that a well configured editor can substantially enhance the readability of source files, so that the inclusion of unit tests (be it done in a sane and consistent style) has no negative effect on it.

I use Cursive so I use code folding and custom <editor-fold> sections which does the same as the author’s vim setup. I’m sure other editors can do the same.
Readability is also the reason why I like with-test better than adding the tests to the :test metadata of the function itself, but it’s a matter of taste.

I’d like to have feedback on the readability side of things or, even more, on the divide between inline unit tests and “test directories” for integration tests.
The fact that clojure tests, with clojure.test, are attached as metadata, makes them agnostic to code/repository structure and doesn’t require the usual (idiomatic?) test/my_module_test.clj pattern I see in most Clojure projects.
My gut tells me that this practice is just a habit ported from Java/Ruby/YouNameIt?

In the end I’m free to write my code however I like, but this question remains, that I don’t know the answer to and which circles back to my first quote:

What about “dragging in Clojure.test in all your namespaces”?

I’ve stumbled upon this gist (and another round here) by @dustingetz in which he says “can erase the tests at compile time depending on build target”.
The idea is interesting but I don’t see how it works in practice. Can someone help in making this bit more explicit?
Also eliding the tests during compilation is possible, but I end up on the question the previous question still:

What’s the cost of “dragging in Clojure.test in all your namespaces”?

1 Like

I talk about this in https://cljdoc.org/d/expectations/clojure-test/1.2.1/doc/getting-started#test-placement

It talks about what it takes to have different tooling find and work with inline tests, and the (fairly minimal) overhead of depending on test code in your production code. For clojure.test, you already have that baked into the Clojure JAR. For expectations.clojure.test, you’re talking about 450 lines (up from the 300 mentioned in the docs, so I should update that comment).

I have a hot key bound to “run tests in this namespace” so inline tests are quite easy for me while developing but I haven’t changed our test suite runner setup to work that way because it would need to scan a lot of (production) code that contains no test Vars so it would probably slow things down a bit (it would be checking twice as many namespaces).

If your unit tests are pretty simple, having them inline doesn’t impact readability too much, but if you have complex tests, with a lot of setup, you may end up making your source code harder to read – and you may end up bringing in a lot of additional dependencies that would otherwise not be needed in your production code. For us, it would add clojure.test.check, Expectations (as noted above), and Gary Frederick’s excellent test.chuck library.

That’s one of the few things I thoroughly disliked about Rust. To me, test code doesn’t belong in production code any more than SQL queries belong in UI code. I guess I’ve had “separation of concerns” beaten into me so much it’s just not something I can get on board with.

I personally like the doctest (or say repl-test) vs. unit/generative test separation … I generally begin with embedded tests, then migrate everything that has no documentation value outside the namespace. It’s a dynamic more than a static separation.