I sometimes find myself wondering if I’ve regressed in performance when working on a piece of code. I can use Tufte or Criterium to test from the repl or in a manually run and verified across time script, but it would be nice to have something to include in my automated tests that is indifferent to changes in architecture. For example, my MacBook runs differently than both my Dell running Linux and CI pipelines.
Does such a tool exist for Clojure?
I really see the need but have not seen a general clojure-solution just yet. Maybe clojure-web-server-benchmarks does something similar?
I think the main problem here is to define a “gold standard” for each architecture for each source code version, and store this result for each test-run, to be able to inform/warn/cancel the build pipeline when the measured performance changes drastically.
I realize now that some similar solution for storing test results would also be helpful when it comes to comparing HTML output, or pixel-comparisions from, for instance, websites or other UI:s.