Good CLJS library for benchmarking?

CLJ has the good criterium library, is there any equivalent thing for CLJS ? Mike fikes seems to suggest there isn’t in a recent blog post.

Really ? No wrapper or port of Benchmark.js (the lib used by jsPerf) ?

Tell me (us) what you know/use. Thanks !

Have you seen cljs.core/simple-benchmark ? https://cljs.github.io/api/cljs.core/simple-benchmark

I have done a good amount of benchmarking (check out my CLJS contributions, almost all around performance)

Benchmarking is just super tough. Benchmarking JS is much tougher than benchmarking the JVM (we have so many different JS VMs). V8 will optimize a function once it’s called 10,000 times (IIRC). You’ll hit that easily in a benchmark but in your real app lifetime you might never get that far. So maybe we should benchmark how the code performs when that optimization hasn’t been done? But then you run into the problem of getting meaningful numbers since the runtime might be too short to see a significant difference.

Having all the different JS runtimes means you have N extra dimensions to take care of.

If you benchmark in a tight loop the JS engine might inline certain funtions, which might not happen in your real JS.

Furthermore, by benchmarking just 1 function. The JS engine might see that function as monomorphic and has super efficient IC acces to class members. But then in your real app you call that function with all kinds of types and if (IIRC) there are more than 3 with a different hidden class you’ll end up with (slowish) megamorphic functions.

Another factor is that JS implementaiton move pretty fast, what today was worth it, might not matter in a few months time when a new optimization has landed in V8/JSC.

I have optimized a few functions where I was 100% convinced that my new implementation has got to be faster than the existing one, just to try it out and it was a draw… :confused:

Bottom line: It’s much tougher than what Benchmark.js makes it out.

That being said: I still use simple-benchmark. It gets the job mostly done.

3 Likes

I was searching for something better than simple-benchmark. Still good to know.

Yes I’m familiar with the problems of having many JS VMs.

What makes you think that Benchmark.js as a library is somehow limited ? Not with regard to various JS engines, and the fact they move fast, but just as a benchmarking library.

Cljsjs provides package for Benchmark.js: [cljsjs/benchmark "2.1.4-0"], the API seems simple enough to use through JS interop. Something like this (I haven’t tested the Cljsjs package, nor this code):

(def suite (js/Benchmark.Suite.))

(doto suite
  (.add "filter, first" (fn [] (first (filter #(= 1000 %) (range)))))
  (.add "some" (fn [] (some (fn [x] (if (= 1000 x) x)) (range)))))
  (.on "cycle" (fn [event] (js/console.log (str (.-target event)))))
  (.on "complate" (fn []
                    (this-as this
                      (js/console.log "Fastest is " (.. this (filter "fastest") (map "name"))))))
  (.run #js {:async true}))
3 Likes

Thank you, indeed the code should and is easy enough to work with. Good to know ! I’ll give it a try and report things here.