Avoiding recursive function replacement with-redefs-fn

Questions & Help How to?
1

I’m trying to replicate a cool tool I saw at a presentation once. This involves modifying all the functions in a namespace, for which I’m using (ns-publics foo).
I want to have a function that slows down the execution of the reffered function by x amount, so for this I decided to try and go with with-redefs-fn. Only, I’ve stumbled a bit on how to modify the functions.
Because If I use with-redefs-fn and pass a reference to the function to be modified inside the value of the map, the replacement becomes recursive! which is not intended. Not to mention I have not even started on figuring out how to do parameters properly.

(ns one)
(defn a-fun []
  (println "original!"))
(defn caller []
  (a-fun))
(ns two)
(let [ftarget (ns-resolve 'one (symbol "a-fun"))
      freplace #(do (println "replaced") (ftarget))]
  (with-redefs-fn {ftarget freplace}
    #((ns-resolve 'one (symbol "caller")))))

If anyone could give me some hints I would be most grateful.

2

This is not exactly what you asked, but it could help you get in the right direction.

First, we define a constant and a few functions we want to slow down:

(ns slow.utils)

(def hello "world")
(defn add [a b] (+ a b))
(defn times [a b] (* a b))

Now, here’s what I came up with:

(ns slow.core
  (:require [slow.utils :refer [add times]]))

(defn toggle-slow [v]
  (if-let [orig (-> v meta :orig)]
    (do
      (alter-var-root v (constantly orig))
      (alter-meta! v dissoc :orig))
    (do
      (alter-meta! v merge {:orig (var-get v)})
      (alter-var-root
       v
       (fn [f]
         (fn [& args]
           (Thread/sleep 1000)
           (apply f args)))))))

The function toggle-slow does a couple of things:

  • If the given var has an :orig key in its metadata, it assumes it contains the original function, so it restores the original var-root with the value associated to the :orig key, then removes it from the metadata.
  • Otherwise, the var points to a “clean” function, so we alter the metadata of the var to keep a pristine copy of the function, then we alter the var to wrap the original function into another function that sleeps for a second before applying the original function to the arguments passed.

Finally, we need a function that inspects a given namespace for all its functions, applying toggle-slow to it:

(defn toggle-ns [ns]
  (let [fn-vars (->> (ns-interns ns) vals (filter (comp fn? var-get)))]
    (doall
     (map toggle-slow fn-vars))))

Now we can give it a try:

;; (time (add 1 2))
;; => "Elapsed time: 0.133544 msecs"
;; 3

;; (toggle-ns 'slow.utils)
;; (time (add 1 2))
;; => "Elapsed time: 1000.335499 msecs"
;; 3

While playing with this I realized that this could be very useful too in some forms of automated testing, eg. including things like a version that instead of sleeping could throw exceptions (given a percentage limit), or writing some stats to disk (eg. for monitoring purposes)

1 Like
3

Useful indeed! That is a very elegant way of wrapping functions in the namespace. I can tweak it to suit my needs. (sorry for the late reply. I’m on central european timezone and I only do Clojure on the evenings + week-ends)
The purpose of this tool is to make a guide for where to focus on performance. By slowing everything down, then increasing the speed by one function at a time (combinations of n pick n-1) you can see where speed-ups provide the highest value in the application. (I saw this on youtube last year, but I can’t find it again!)
Plot this measurement ensemble, then you can immediately see the functions that have the most impact on your application. (why isn’t this mainstream yet?)
Would also be cool to have in a C.I. report.

4

It seems like a more complicated way of profiling then just using a regular profiler. Am i missing something?

5

I suspect it’s related to this stuff (wonderful talk, worth watching) https://www.youtube.com/watch?v=r-TLSBdHe1A (if you’re in a rush, start at minute 28 or so)

1 Like
6

It would be cool to try and integrate it with (probably the output of) an existing profiler, so you could e.g., make the slow-downs proportional to actual execution time, and so on. If you’re planning on working on this, I’d be happy to lend a hand.

7

This was the one. Thanks!

8

I need recommendations for which profiler to use/integrate into. I’ve only used Criterium for benching functions until now. When I have a working example and some tests I can upload the code to github. I’ll put a link in this thread then. When that time comes I would be more than happy to collaborate.

9

The difference compared to a profiler is that this shows where you gain the most from improving performance of a certain function. I think the video is more persuasive, and better to explain than me.

10

I’ve often used Tufte, which is quite nice, and I think it provides a nice data-based API, too (though I’ve only used it to generate reports, I haven’t played with the profiling data).

Looking forward to seeing what you come up with!

11

Ah interesting, I also found the related paper here: http://www.sigops.org/sosp/sosp15/current/2015-Monterey/printable/090-curtsinger.pdf

I understand, in a concurrent context, processes that are waiting for other things to complete can be the reason for slow downs, and that’s where this shines.

It seems there’s someone working on one for the JVM called JCoz, but a Clojure one would be cool!

I’ve only ever used JVM profilers, so I have no recommendations here.

12

The first part of the talk has some other interesting stuff about how machine code layout affects performance and so on, it’s definitely worth a watch (even though it almost certainly doesn’t apply to a JIT, so not usable for Clojure). I’ll have a look at JCoz…

13

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.