How do you store entities on the client?

Hello everyone :slight_smile:

When writing frontend apps we maintain some state on the client and often this state can be described as entities, e.g. a user or a message etc. The most fundamental things we want to do with these entities can be described in the following ways:

  • Get a single entity by ID
  • Iterate over a list of entities (filtering/sorting/rendering etc.)
  • Update one or more entities with new information

Now the data structures we have in Clojure for this are maps and lists and I think both have their shortcomings for the use cases above which is why I am wondering if any of you have other approaches that you’d be willing to share (I’ll touch on one of my own later).

{"my-id-1" {:some :data}
 "my-id-2" {:more :data}}

Maps make it very easy to avoid duplicating entities and also make it easy to find/update single entities due to the simple key based access. The problem with maps is that when iterating we’ll need to always call vals or iterate over a sequence of pairs which can be a little annoying at times. When using the vals approach you’d also potentially need to duplicate the identifier into the values if it is needed for e.g. rendering etc.

[{:id "my-id-1" :some :data}
 {:id "my-id-2" :more :data}]

Sequences are nice because each entity is self contained and iterating is as straightforward as it gets. Updating and finding single records is a lot more complex compared when using maps. Potentially even impractical if the collections get too big.


Now a few years ago I wrote this library custom-comparator-set that is a bit of a hybrid between the two. Essentially it is a set but instead of using the identity of the values you can provide a function (e.g. :id) that will be used to determine if two entities are the same. This is backed by a regular map and you can use get with a comparator value (e.g. an "my-id-1") to retrieve individual values.

I haven’t really kicked the tires much with this library and it’s been more of a learning experiment but I’d love to hear other approaches to this problem? I’d assume that tools like Datascript solve this but I’m also wondering if there are other, less buy-in-heavy approaches out there.

:v:

3 Likes

I mostly just started using entity-db because that’s the default with keechma, but is does make storing and consistently updating client side data pretty easy:

It is an independent library so you can use it without keechma.

1 Like

That’s neat and especially this concept of naming individual entities can be very handy to keep track of stuff like :current — which I haven’t considered before. Thanks for sharing!

EDIT: For those who are curious here’s the link to the source: https://github.com/keechma/entitydb

I think it’s interesting to view the client as just another database peer. If we remove performance considerations, then we never* have to store state on the client (no optimistic updates). In which case, your data is typically modeled in terms of our persistent storage layer (postgres, datomic, etc…). If we compare the semantics of these tools to the query capabilities of the data structures in clojure, there is a wide gap. depending on your background it’s fair to argue that storing data in Clojure structures and using Clojure to query that data, is sufficient. You could model your data as a collection of facts and then filter over that collection, this would be the most flexible, but also it means every search scans the entire set. I dont know of any clojure libs that do this.Or you can nest your data in hashmaps, which means you have the same properties as a document store. re-frame has you model your data this way. Document stores dont have semantics for cross root joins. e.g

{:dogs {:names ["buddy"] :people {:names ["drew"]}}

now requires you loop over the keys to get all the names. This is why you see frameworks like Fulcro use a Graph database.

Rete Networks (i’m still learning about these) seem to invert graph databases and make it easier to insert new data by removing logic order. (someone please chime in with a better description)

Each of these ways of modeling data comes with tradeoffs.

The truth is, the story for modeling data on the front end is as diverse as it is on the backend. In fact, it’s the same story, but the front end literature seems improvised comparatively because as far as I can tell, we’re still figuring things out at every level. Performance considerations and optimizations tend to always lower the abstraction level and its our job as engineers to balance the two.

I will never get tired of recommending “Designing data-intensive applications

For introducing me to a lot of these ideas. It’s also important to always be humble about these topics, i know every time i start thinking i “know better” i learn how very very wrong i was all along :slight_smile:

3 Likes