For the last six months or so I’ve been working on Kaocha, a new and much improved Clojure test runner.
There are a number of test tools available already, like lein-test, eftest, circleci.test, and some of them have pretty interesting features. The problem I see is that they are either bound to a specific build tool (lein, boot), or to a specific testing library (clojure.test, midje), or both. No matter which one you choose, there will be things you are missing. The current approach does not scale.
(is (= amount-of-work (* (count build-tools) (count testing-frameworks) (count desirable-features))))
Apart from that there are still features that are considered bog-standard in e.g. the Ruby world, that are not yet available anywhere in Clojure.
So I set out to create something that is feature-rich out of the box, and that is flexible and extensible. Support for other testing frameworks can be provided as an add-on (what kaocha calls a “test suite type”), and extra features can be provided as plugins.
Kaocha is both a tool and a base for tooling. It turns the imperative task of running tests into something that is data-driven, and takes care of the nitty gritty so that others can focus on the features they want to see.
So far Kaocha supports Leiningen and Clojure CLI, and can run clojure.test and (experimental/alpha) Midje tests. Features implemented so far:
- Filtering tests based on test names or metadata
- Watch mode: watch the file system for changes and re-run tests
- Randomize test order
- Detect when interrupted with ctrl-C and print report
- Fail fast mode: stop at first failure and print report
- Profiling (show slowest tests)
- Support for matcher-combinators
- Extensive documentation
After installing Kaocha you configure it create a
tests.edn in the project root with your test setup. It describes the test suites you have, with their respective types (this determines how they are loaded and run). You can set various configuration flags in there, choose your output reporter style, and configure plugins.
Kaocha is being used in production by Nextjournal, and is used to run the Cljdoc tests, and of course I’ve been dogfooding it extensively. There is still a lot of work to be done to reach my original vision, but the foundations are looking really good. Now I need your help.
You can help by trying it out on your projects. If you have an open source project feel free to ping me and I’ll create a PR for you. Try it out and report back, both the positive and the negative. If you are missing something that prevents you from switching then open an issue, this will help me prioritize.
If you have a closed source project then you can hire me to set Kaocha up for you. Especially if you have multiple test suites (unit, integration, etc) I’d love to talk to you.
Your company can also fund me to make sure Kaocha reaches its potential. Get in touch and we can work something out!