Clojure core function usage; rough frequency analysis, Jan 22

[magnus0re]

I learnt about a function that I had forgotten today. The cond->. Anyway, this inspired me to do a query
on the frequency of usage of the different functions of clojure. So I used the github api to get some repos, cloned and just ran a frequency analysis on each of the repos.

The regex was first a too simple thing, now updated with feedback from @seancorfield

(re-pattern
   (->>
    (map str (keys (ns-publics 'clojure.core)))
    (map #(str "\\(" % "\\s"))
    (map #(string/replace % "+" "\\+"))
    (map #(string/replace % "*" "\\*"))
    (map #(string/replace % "?" "\\?"))
    (map #(string/replace % "." "\\."))
    (interpose "|")
    (apply str)))

now. the results, updated:

This is the top 100 used names from clojure.core, with approx 400 repos analyzed.

Anyway, this distribution looks like a power-law thing, i.e. it follows zipf’s law.
I think this shows one side of the developer experience for Clojure, where one get a lot of power with very few functions!

This was all done in a bit more than 100 lines of code. (excluding plotting)

thanks to @seancorfield for pointing out the previosly really bad regex!

Cheers.

[seancorfield]

I think I would view this with a great deal of suspicion because:

• It has ref usage as much higher than atom and then swap! is right down the bottom – in reality, almost no one uses ref, and if you have atom without swap! what is the code doing?? (and where’s reset!?)
• I don’t see println in there at all but pr and int seem unfeasibly high on the list.
• update is a much more recent addition to Clojure than update-in and I still see a lot of code using update-in where update could now be used (update-in was in 1.0, update was added in 1.7).

I would suggest running it again and anchoring the pattern with a literal ( at the front (so you only match function calls – yes, I know this will omit higher-order uses but I think it will be much more accurate) and either $ or \s at the end so you you match exact uses of the functions such as:

(map
  (fn [x] (* x x))
  vals)

map, fn, * are the calls to core functions. vals is just a symbol name (and not clojure.core/vals).

[magnus0re]

I guess you are right, my mistake. Updating soon.