List all unique keywords in current project?

[myguidingstar]

I want to have a command that list all distinct keywords (both qualified and unqualified) in current project. The goal is to pinpoint typos and have an overview so I can improve their names for better semantics.

Some regular expressions won’t help much because of shorthand syntax like #:foo{:bar 1} or ::some-key.

Any idea?

[vlaaad]

Read files:

(let [reader (java.io.PushbackReader. (io/reader "src/clj/util/http_server.clj"))
      eof (Object.)]
  (set 
    (filter keyword? 
            (flatten 
              (loop [acc []
                     form (read reader false eof)]
                (if (identical? eof form)
                  acc
                  (recur (conj acc form) (read reader false eof))))))))
=> #{:as :require :headers :code :import :body}

[vlaaad]

Just realized that flatten isn’t enough because you need to flatten maps too…

[vlaaad]

So you’ll need something like that instead of flatten:

(defn flatten-completely [x]
  (if (seqable? x)
    (mapcat flatten-completely x)
    [x]))

Edit: And now I realized it’s not good enough either, because for (let [{:keys [a b c]} nil]) it reports #{:keys} instead of #{:a :b :c}. I guess it might be worth looking into tools.analyzer?

[teodorlu]

I was thinking that it should be possible to query (ns-interns 'your.ns) and (clojure.walk/postwalk ...) over that, but I couldn’t find any source (form) access from the namespace object. So I think we have to go through files, as @vlaaad is showing.

I would be interested in hearing thoughts on how this should be accomplished!

…

CIDER is able to autocomplete keywords within a project. Isn’t that just what we want? “Please show me all loaded keywords”? Perhaps filtered on the namespace where it was referenced?

[vlaaad]

There is a weak cache of keywords you can use to load all (?) keywords:

(let [f (.getDeclaredField clojure.lang.Keyword "table")]
  (.setAccessible f true)
  (map #(.get %) (vals (.get f nil))))

[myguidingstar]

Wow, great to learn about it
however it pulls all keywords ever loaded, many of them are from dependencies so it’s too much to be useful

[teodorlu]

Do you use namespace qualified keys to a sufficient degree that you could check just the ones qualified with your namespace?

[myguidingstar]

No, many of them are to denote data itself instead of derive from a source code file.