I’d really appreciate some help on some design decisions for an API I’m designing for an internal system, however I think the general problem is very common in many database-backed systems.
We want have a core domain/model API that enforces business rules, e.g. given a user, when asked for a list of widgets, show me widgets this particular user has access to. In our specific case, we do that by constructing a particular query against Mongo.
On top of that, we also have some presentation rules — extra filtering (e.g. to support search), sorting, pagination etc. Or perhaps we want to fetch and join some relationships as well. This happens by taking the core query and adding more clauses, or perhaps going to some more low-level Mongo functionality.
I’d like to keep the core API surface quite agnostic to these presentation rules, that is, I don’t want the core functions to take 7 different arguments to cover all the different presentation rules (or a map of options, it’s kind of the same).
What I think I want to do is have the core API return something that can be further refined by calling other functions, but also eventually realised into a collection of items (or a count, etc). So perhaps it’d look like this:
(def square-widgets
(-> (q/get-widgets db user)
(q/filter-by {:type "square"})
(q/sort-by :name)
(q/paginate {:limit 20 :skip 3})))
Now, at this point, square-widgets could be further changed (perhaps add another filter clause, by using another q/filter-by) or actually realised into a collection of results – or just a count.
I could imagine this going two ways:
- The result of all these functions is just a hashmap. Add a
q/executeandq/countand other “top level” operations that know how to interpret this to give you back aseq(potentially lazy) or a number. - The result is actually a
Queryrecord that implementsSeqable,Countableand perhaps other Clojure protocols. Then the result of these functions can be passed as-is around the system, and only when something needs to actually iterate over it (e.g. for serialisation purposes) is the query run against the DB.
It also would be nice to be able to cache these results, so if you have executed the query once, you don’t need to execute again (or if you have fetched all the results already, you don’t need to re-execute the count DB operation).
It kind of feels that the record/protocols approach is too magicky, since reading the code might be confusing, but I can’t think of any other drawbacks. Consumers of the API can just call vec if they want to execute the query and get back non-lazy results. I’d also need to implement a bunch of protocols if I want to support things like reverse/nth and so on (though there could be ways around it, I think).
It also feels though that keeping the record approach allows for more flexibility to perhaps implement other useful protocols, or realise the results while keeping the original query around so that could be used for logging slow queries at some other layer.
I’d be very interested to hear your thoughts about this! Thanks!
