TL;DR Jump to “Fourth pass idea” section at the very end, where I explain a new approach I’m thinking of using to name my specs and qualify my entity map keys.
Otherwise read the whole thing if you want to follow my iteration towards the idea, and the issues I’m trying to solve with it.
First pass
When I first used Spec, I went with the approach of using :: to declare my domain model Specs. And I created a namespace to put them all in, imagine: com.org.app.data-model. So in it you’d have:
(ns com.org.app.data-model)
(s/def ::person
(s/keys :req [::name ::address]))
This turned out to be a bit of a mistake. Mostly because eventually some of those entities breached the boundary of my app, in that I persisted a ::person map to some DB, or I had client pass me one on a request, etc. And when that happened, :: became a liability. So we had someone move the data-model from one place to another, so the namespace became (ns com.org.another-app.data-model) and of course that means that all the specs and keys changed. So data generated prior had keys such as :com.org.app.data-model/name which was now no longer matching the spec of :com.org.another-app.data-model/name.
So it wasn’t the worst thing to fix, we could use the long form in the s/keys so that it would be (s/keys :req [:com.org.app.data-model/name ...]). Which is uglier, and took a little bit of time to do, but not that bad.
That said, then there was a worse problem on the other side. We were creating the ::person map in some other namespace, but also using ::.
(ns com.org.app
(:require [com.org.app.data-model :as dm]))
(defn make-person
[name address]
#::dm{:name name :address address})
And when we refactored the namespace to com.org.another-app.data-model it meant the namespace for the keys in our person map also changed. Those also got exchanged with clients and persisted.
So now we have two types of person map in prod:
{:com.org.another-app.data-model/name "..."
:com.org.another-app.data-model/address "..."}
;; and
{:com.org.app.data-model/name "..."
:com.org.app.data-model/address "..."}
Well, now what do we do with the Spec?
(s/def ::person
(s/keys :req [(or (and :com.org.app.data-model/name :com.org.app.data-model/address)
(and :com.org.another-app.data-model/name :com.org.another-app.data-model/address))]))
So we learned our lesson, and stopped using ::.
Second pass
Next time, I thought, well, using :: is prone to issues, so I won’t use it, and I will just type fully qualified namespaces everywhere. But that’s ugly, and I am lazy, so I thought, let’s not make it a really long namespace then:
(ns com.org.app.data-model)
(s/def :app/person
(s/keys :req [:app/name :app/address]))
(ns com.org.app
(:require [com.org.app.data-model]))
(defn make-person
[name address]
#:app{:name name :address address})
And this has been fine for now, no collision on app yet. But one can see how there could be in theory, so I still don’t find this ideal. Also, typing :app all the time is annoying (our real app has a long name, not a nice little 3 letter one like app)
With this approach, you can refactor rename your namespaces, move things around, and all still works, since your spec keys are unchanged, and your data keys are unchanged, thus they remain in sync always. If you ever change the name of your app though, you’re stuck on the old name.
Third pass
I also thought, do I even need my keys to be namespaced in my data? The only use case for it I can think of is if someone finds the data, and wants to know what is the authority on it, and somehow they have no idea where the data comes from or who owns it. The namespace would be self-describing in that sense, they could figure out where, and they’d know what exact spec is the schema for it. But I’m not sure how valuable this is, so I thought, screw it, going to back to unqualified. Now, when you do that, the problem I had with :: partially disappears. Since now :: is only used as the lookup for the spec in code, so you can do this again:
(ns com.org.app.data-model)
(s/def ::person
(s/keys :req-un [::name ::address]))
(ns com.org.app
(:require [com.org.app.data-model]))
(defn make-person
[name address]
{:name name :address address})
Now I could freely move my make-person function elsewhere, or refactor rename my data-model namespace, and it would all still work.
This is currently my favorite approach.
Fourth pass
But yesterday, I had an issue with the Third pass approach:
(ns com.org.app.data-model)
(s/def ::name string?)
(s/def ::person
(s/keys :req-un [::name ::address]))
;; Uh oh! I have a key conflict!
(s/def ::name #{:sony :microsoft :nintendo})
(s/def ::business
(s/keys :req-un [::name ::address]))
So the solution is to create another namespace for the business entity, but go down this path, and you get yourself a lot of little files, and my code base starts to look like Java (unless I use multiple namespaces in a single file, which I feel I shouldn’t, but that might just be an uncalled fear of mine).
(ns com.org.app.data-model.person)
(s/def ::name string?)
(s/def ::person
(s/keys :req-un [::name ::address]))
(ns com.org.app.data-model.business)
(s/def ::name #{:sony :microsoft :nintendo})
(s/def ::business
(s/keys :req-un [::name ::address]))
This would happen with the Second pass approach as well:
(ns com.org.app.data-model)
(s/def :app/name string?)
(s/def :app/person
(s/keys :req-un [:app/name :app/address]))
;; Uh oh! I have a key conflict!
(s/def :app/name #{:sony :microsoft :nintendo})
(s/def :app/business
(s/keys :req [:app/name :app/address]))
Here I can fix it by just calling it: :app.business/name to distinguish it, which comes with its own drawbacks, like making it weird to use #:app{:app.business/name "" :address ""}. You could choose to always append the entity so you’d have :app.business/address as well, but now the typing gets longer and longer.
Fourth pass idea
So I thought a little about all of this, and I came up with this:
(ns cool-def-key-lib)
(def key-ns
(atom {}))
(defn defkey
[alias keyns]
(swap! key-ns assoc (str alias) (str keyns)))
;; This would go in `data_reader.clj` ideally
(set! *default-data-reader-fn*
(fn[tag-sym value]
(when (.startsWith (str tag-sym) "key")
(if (map? value)
(let [tag-key-str (second (re-find #"key:(.*)" (name tag-sym)))]
(if-let [tag-key (@key-ns tag-key-str)]
(reduce (fn[acc [k v]]
(if (qualified-keyword? k)
(assoc acc k v)
(assoc acc (keyword tag-key (name k)) v)))
{} value)
(throw (ex-info (str "No keyword namespace defined for " value) {}))))
(let [ns (namespace value)
na (name value)]
(if-let [k (@key-ns (str (or ns na)))]
(keyword (name k) na)
(throw (ex-info (str "No keyword namespace defined for " value) {}))))))))
This gives you a way to defkey, which creates a keyword alias in some separate alias to keyword namespace registry. And then a #key and #key:foo {:bar "baz"} tagged literal where you can use the keyword namespace alias to give you a fully qualified keyword that is decoupled from the current namespace or other code namespaces.
With this I can now do:
(ns com.org.app.data-model
(:require [cool-def-key-lib :refer [defkey]]
[clojure.spec.alpha :as s]))
(defkey 'person 'com.org.app.data-model.person)
(s/def #key person/name
string?)
(s/def #key person
(s/keys :req [#key person/name
#key person/address]))
(defkey 'business 'com.org.app.data-model.business)
(s/def #key business/name
#{:sony :microsoft :nintendo})
(s/def #key business
(s/keys :req [#key business/name
#key business/address]))
(ns com.org.app
(:require [com.org.app.data-model]
[cool-def-key-lib :refer [defkey]]))
(defn make-person
[name address]
#key:person {:name name :address address})
To help you understand:
(defkey 'foo 'com.my.long.namespace.foo)
#key foo
;=> :com.my.long.namespace.foo/foo
#key foo/bar
;=> :com.my.long.namespace.foo/bar
#key:foo {:bar "baz" :biz "fuzz" :some/other "ns"}
;=> {:com.my.long.namespace.foo/bar "baz"
:com.my.long.namespace.foo/biz "fuzz"
:some/other "ns"}
I don’t like using something that is pseudo some new syntax and non standard, but I also like this approach quite a bit. It means my keys can be fully qualified and guaranteed globally unique, making them self-describing and also letting you know exactly what spec in the whole world specifies their value.
It solves my issue where I can move things around, refactor, and it still all works, because the code namespaces and the keyword namespaces are separate.
And it creates a convention for naming entities and their keys, where the entity is uri.entity-name/entity-name and the keys are uri.entity-name/key-name
So I’m curious how others deal with this, if there are strategies I missed, what you think of these approaches, and especially your thoughts on that Fourth pass one.
P.S.: @alexmiller told me on the slack:
just fyi, I am currently working on a solution to this (lightweight alias) for Clojure with Rich (probably 1.11)
No idea if this will resemble at all my Fourth pass or not, but something to look out for.
, but I think others might find my process through this interesting, so I’ll leave all the above posts in place.
