How do I namespace keywords in event data?

Questions & Help General Questions
1

I’m wrapping an existing api, and exposing a user interface that’s inspired by Sente’s. Users send messages that contain an event-id (which could be thought of as the function/command) and a map of named arguments, and asynchronously receive responses.

For example, the user would send the following message to the connection object:

[:my-lib.request/market-data {:stock {:symbol "AAPL"} :live? true :request-id 42}]

or

[:my-lib.request/historical-data {:stock {:symbol "AAPL" } 
                                  :start #inst "2018-03-07T04:20:02.318-00:00"
                                  :days 30 :request-id 52}]

I’m just wondering which keywords should be namespaced here?

All these events and arguments will be fully spec’ed.

I feel it’s somewhat easy to decide that the first item in the vector, the event-id, should be a namespaced keyword, because it’s sort of like calling a function but since it might go over the wire it needs to be referred to in a global context (and also because I’m just ripping off this requirement from sente). But what about the rest, the named “arguments” map? Should those be namespaced?

There are a large number of types of functions besides the two shown with a wide range of arguments. Some of the arguments are common among multiple of the calls, like :stock here. Overall there are a ton of things to be named by keywords.

Does the fact that they’re wrapped in a vector which describes the intent of the command with the event-id provide enough context to not require namespaced keys?

If I do namespace the keys, what namespace do I use? Do I use “nest” them under the event-id like so:
:my-lib.request.market-data/stock,:my-lib.request.market-data/live?
But then shouldn’t the common arguments across the api be denoted by the same keyword if they refer to the same type of thing (ie, the same spec)? The stock “type” appears across many calls, so should I just always denote it by :my-lib/stock but then fall back on the :my-lib.request.market-data/live? for non recurring arg names?

Do I chop off the first few segments and just go with market-data/contract and :market-data/live?

Are any of these questions themselves a possible hint that I’m designing this api all wrong?

I’ve been agonizing over these decisions for a few days and not getting anywhere. Would love to hear other people’s thoughts!

1 Like
2

I find it hard to give advice on this since design means juggling constraints and keeping a lot of context in mind, which I don’t have, but I have some general thoughts.

I would mainly try to make the meaning of symbol globally unambiguous. The philosophy that’s being promoted alongside clojure.spec is that people can have maps containing various different pieces of information with different sources/destinations, and that these should not clash, and be self-explanatory.

:live? can mean different things in different contexts, buy :my-lib/live? could have precise semantics, and be specced as such.

I would use a single namespace for all keywords in your library if possible, that way people (and you yourself) can easily use a shorthand, for creating and destructuring maps

#:my.lib{:live? false :stock {:my.lib/symbol "AAPL"}}

(let [{:my.lib/keys [live? stock?]} m]
  ,,)

Only if a symbol has multiple meanings in your library would you have a reason to use multiple namespaces, but in that case you could also try to avoid that by making the symbols more explanatory.

:my.lib.historical-data/days
:my.lib.market-data/days

# or

:my.lib/historical-days
:my.lib/market-days
1 Like
3

Thanks for the response!

Something still bothers me a little about this as a default (and it’s possible I’m way overthinking all of this). I could be wrong but to me it feels like some of the intent of keyword namespacing is to establish a “hierarchy of names” similar to how namespaces themselves are named. Putting them all in one library wide namespace feels to me like putting all functions of a library in one namespace, when clear logical lines to separating them. And indeed in my library namespaces separate the functions/symbols along these lines, something seems off about then lumping all the keyword names together.

To me this feels like an example of choosing “easy” over “simple”. On the other hand, I don’t immediately foresee any actual real life disadvantages to grouping almost all keywords in the same namespace for a library.

On the other hand longer namespaces used everywhere get pretty ugly pretty quick and I don’t really see any libraries that do this when they use a large amount of keywords. I believe there will probably be more aliasing capabilities for keywords in a future version of clojure. But it’s not clear how this will look or when it will be available.

I guess another way to pose this question is:

What if Ring were redesigned today in this post-spec world. How would the request/response specification look?

Would all keywords be given the ring namespace, ie :ring/query-string and :ring/status?
Would there be a ring.request and ring.response namespace: ring.request/headers, ring.response/headers
Or would keys be kept unnamespaced as they are now?

4

I wouldn’t use ring/request but http/request. The semantics matters more than the origin. I would also avoid using the namespace where the key is defined as the keyword namespace but again try to find the proper semantic. What’s the entity? What’s the concept being manipulated/transmitted? Namespace in keyword should convey semantic information, not technical information. It’s almost “place-oriented” programming.

That’s my understanding of spec, hope this helps.

2 Likes
5

Plenty of libraries have all their public functions in one namespace. Some will even import all the functions they expose from their original namespaces into a single namespace for easy consumption. Arguably some multi-namespace libraries would be better off with a single namespace.

Obviously this isn’t true for all libraries, I guess it’s up to you to decide if you have enough API surface area to warrant splitting it up.

Namespaces should be globally unique, especially when you’re going to use them with spec. What if another library also defines a http/request spec? Who “owns” the http namespace?

To really be a good citizen maybe it should even be com.weavejester.ring/request.

This discussion would also be different if we had an easy way to define keyword namespace aliases without having to load the namespace with the same name, however it seems we’re nowhere near to any solution (see CLJ-2123 and CLJ-2030), so you really have two options if you don’t want consuming your library to be a major PITA.

  • use short single segment namespaces for keywords a la ring/request, so you don’t need an alias ({:ring/keys [request response]}), at the risk of clashing with other libs
  • use namespaces for your keywords that people will likely be requiring in the same place where they will be accessing said keywords
6

Thanks for both your responses. This definitely helped guide my thinking over the past couple days on this.

I guess the confusion was I was thinking namespaces as a means to organize names. As in “these are all the names in my library, let’s group them logically”. This is how namespaces are used with symbols pretty much. But with keywords it’s different. I do know that there’s a preference to the word “qualified” when it comes to keywords vs namespaced. But the differences between these two concepts aren’t quite clear to me.

I feel like there is some complection going on with clojure namespacing. Is there a way to separate avoidance of clashes in names from the organization of names within a library? Are there legitimate reasons to want to organize names in a program for a reason other than name clashes?

Yes, I’m coming to see namespaced keywords in maps as being sort of a ultra lightweight protocols. Your map can have a :person/name and :person/age and :programmer/favorite-language and to an extent these indicate that this “entity” “implements” those “types”. You’re describing what type of thing it is by the attributes it has.

Along these lines, I have actually decided not to use namespaced keywords for the argument maps in the original problem I described.

[:my-lib.request/historical-data {:stock {:symbol "AAPL" } 
                                  :start #inst "2018-03-07T04:20:02.318-00:00"
                                  :days  30 :request-id 52}]

The first item in the vector here is analogous to a function call. It should be namespaced because I want to keep the ability to call specific functions in a global environment.

Now the second argument, what is the “entity of that”? It’s just named arguments. Arguments themselves generally aren’t to be thought of as “entities”. And just because these arguments are a map in a vector instead of traditional clojure function arguments, I don’t think they suddenly become an entity.

The arguments don’t make any sense outside of this fake function call vector, any more than any other arguments make sense when you strip them out of the calling function. They are intrinsically tied to this fake function call. So instead of gaining context from namespaced keywords, they gain context from being in the vector pair. They don’t need to be namespaced for the same reason normal function parameters aren’t namespaced, even when they are named keyword arguments: They don’t really need to have a globally unique representation.

My originally question probably didn’t give enough context or explained things clearly enough, that these were “fake function arguments”.

Feedback welcome on this decision!

As for namespaced keywords in general, I think it is an area where strong idioms have yet to emerge. It will definitely be helpful to see what comes of the JIRA tickets mentioned above in clojure 1.10.

1 Like
7