What terminology to use in a keyword hierarchy for messages in a client/server api

[jjttjj]

I’m wrapping a java API, specifically Interactive Brokers’ with clojure. You establish a connection to server running on your computer. Then you send messages to the server, and receive messages from it. I’m close to releasing a library that I think improves a lot on the java api.

One thing I’m really struggling with is how to name concepts in my library.
I’ve adapted the Sente style of messages for both incoming messages and outgoing messages.

[<ev-id> <?ev-data>], e.g. [:my-app/some-req {:data "data"}]

Sente calls them events for messages in both directions. This doesn’t feel right in my context. For one, there are incoming messages with the same name as outgoing messages, but which are fundamentally different, for instance there’s a :mylib.request/historical-data and a mylib.incoming-msg/historical-data so I need some sort of namespace segment to seperate them and these are what I’m struggling to name.

:mylib.request/x and :mylib.response/x doens’t totally feel right because the incoming messages are not necessarily responses, sometimes they come in without a request, such as when the internet connection is lost.
So I’ve been using :mylib.request/x and :mylib.event/x but this feels a little awkard too, because there are some times in the code i want to refer internally to both the requests and events, and then what do I call these, what is the term in the hierarchy above these?

Another options I’ve been considering is calling everything a message and using :mylib.msg.in/x and :mylib.msg.out/x but in practice I find this awkard, I keep forgetting/having to think for a minute about if “in” refers to coming in to me or going in to the server.

So right now I’m leaning towards :mylib.msg.to/x and :mylib.msg.from/x
I kind of like how these feel but I haven’t really seen to/from used like this to name things like this, does this seem awkward to anyone?

One important note is that users of the library will not really be exposed to these names in regular usage. Unqualified keywords are used in the api and then qualified internally by the library and used to spec/translate values at boundaries. For example the usage api looks like

(request connection [:historical-data {:arg1 x :arg2 y}])

internally becomes

[:mylib.request/historical-data {:mylib.request.historical-data/arg1 x :mylib.request.historical-data/arg1 y}]

and then these are validated and translated from clojure to the java value the server understands.

Of course I’m overthinking this but I’d prefer to get this stuff to a point I’m happy with before working on documentation, any thoughts or opinions would be appreciated, thanks!

[teodorlu]

;; Hello!
;;
;; Hope you don't mind this comment in code.
;;
;; So, I feel that you're touching a train of thought that I've been to myself.
;; In my words:
;;
;;  - Namespaced keywords are organized in a hierarchy
;;  - For what you're doing, you might want to consider normalized data.
;;
;; But what do I mean by that? You're suggesting this:

[:mylib.msg.to/x {:x 3 :y 5}]
[:mylib.msg.from/x {:total 5}]

;; You could consider that messages might have a target as well as their
;; payload. That somewhat resembles HTTP having header metadata.

[:mylib.msg/x {:target :server} {:x 3 :y 5}]
[:mylib.msg/x {:target :client} {:total 5}]

;; Another option is to use Clojure's metadata facilities.

(let [message [:mylib.msg/x ^{:target :server} {:x 3 :y 5}]]
  [message (meta (second message))])
;; => [[:mylib.msg/x {:x 3, :y 5}] {:target :server}]

(let [message ^{:target :server} [:mylib.msg/x {:x 3 :y 5}]]
  [message (meta message)])
;; => [[:mylib.msg/x {:x 3, :y 5}] {:target :server}]

So what’s the right thing to do? I think namespaced keywords should be considered as identifiers. Thus, you should have one identifier per thing. If the message moving in and out should be considered the same, I’d keep the keyword the same. On the other hand, if that’s not the case, split.

For instance, what behaviour would you like if you slapped all the messages sent into a list and sorted by keyword? Would that make sense?

I’m not sure whether using keywords or using metadata of some sort is right in this case. But I think normalized data is a proper alternative to using keywords, and that namespace qualified keywords should be considered identifiers; the names of an identity.

Teodor

[jjttjj]

Thanks for your input!

That’s interesting too about separating the target as a separate thing entirely (either as metadata or the map). But I would say the messages are definitely different even if they share the same “name” ie final keyword segment, and I agree that because of this they should have unique identifiers.

In thinking about it a bit more I guess my issue might just boil down to wanting to use “request” for a “to server” message but then having no good name for “from server” messages since they aren’t technically always “responses” which is what would feel natural.

Again I’m definitely over thinking this.

[teodorlu]

I just realized that Duct’s database migrations address this. Why? It treats migrations as one thing (Duct migration <–> your message). But it differentiates between the code for migrating up and down.

Exerpt from a Duct project I’m working on now:

;; from resources.pd.api/config.edn
,,,
  :duct.migrator/ragtime
  {:migrations [
                #ig/ref :pd.api.migration/concept
                ]}
,,,
  ;;; Concept migrations
  [:duct.migrator.ragtime/sql :pd.api.migration/concept]
  {:up   [#duct/resource "migrations/concept/concept-up.sql"]
   :down [#duct/resource "migrations/concept/concept-down.sql"]}
,,,

1. One keyword to a “split” map. Up and down are static properties on a migrations, identified by :up and :down.
2. With the files, we split into one identifier (filename) for the up code and one identifier for the down code.