Recently I googled how to output of failed tests of functions that return a map with a lot of keys and I found kaocha. That really surprised me since I haven’t realized that people created another test runners. So far I always just run tests using Leiningen but kaocha has sooo much better output.
Could you please tell me are there any other test runners that I should try before I switch all my projects to kaocha?
Are there any differences in speed when running all tests using different test runners?
I tend to use the cognitect test runner a lot too which is the most basic one for deps.edn projects. For getting better output when matching data structures, check out matcher-combinators. That works with any test runner. You can implement failing fast pretty easily yourself (example). Kaocha comes with those features out of the box.
Like Michiel, I use the Cognitect test-runner – because all my projects are deps.edn now, not Leiningen – but you can use pjstadig/humane-test-output: Humane test output for clojure.test. (github.com) with that test-runner to get a lot of the same benefits, in terms of output, that Kaocha provides (although, with test-runner and deps.edn, you do need to do a little work to actually activate HTO whereas it is much easier with Leiningen).
Since you’re still using Leiningen, you might want to try HTO first, since you wouldn’t need to change much about how you run tests.
Test Runners - Practicalli Clojure covers the test runner tools I’ve used. They are all very good tools to use, so choose which one fits the needs most appropriately
After trying several Clojure test runners I decided to use lambdaisland/kaocha, as it’s easy to work with and can make for a very efficient test workflow
I use kaocha via :test/run and :test/watch aliases in the Clojure CLI user config, so it’s very simple to use with any project. A test.edn file is added to a project to tailor/optimise the test workflow where required
Overall I found kaocha useful because:
simple to use - although config is recommended it is not essential
fail fast - so I don’t get too many failed tests
watch mode - for automatically running tests on file save, which I run in a separate terminal window and occasional glance at
profiling - lists the slowest tests for each test category
highly configurable - can be very specific on which tests should be run
nice test reporter options with useful output
There is also editor integration with Emacs, which I’ve used with Doom Emacs & Spacemacs (which was quite simple to add)
I too am curious about what test runners are available. From some comments I read—albeit old ones—apparently Kaocha doesn’t work with shadow-cljs—is this true? And if so, what test runner(s) would you all recommend that does work with shadow-cljs?
Kaocha is a bit heavyweight so I resisted it for a while, but it is the easiest way to export junit output for integration into CI (eg gitlab/teamcity) so now use it for everything.
During repl/development I just use the normal run-test and/or IntelliJ integration.
I’d say the best test runner is the one you’ve built for yourself.
For me, it’s the midje inspired - code.test. Definitely not as full featured as kaocha or the clojure.test ecosystem. Testing and displaying errors on maps with lots of keys is a problem that is yet to be solved.
However, it’s super simple to both maintain and modify, with a fraction of the code of most test runners.