In my re-frame+datascript app, data loaded from the server goes into the datascript DB. I have re-frame subscriptions that query that database. Some of those queries can be slow for large data sets. The queries are typically not just
d/pull or similar, but more involved, sometimes involving walking
d/pull’s return value and doing additional work w/ the
db for sub-entities.
The particular issue I’m seeing is that, because those queries depend on the datascript DB, whenever that DB is updated, the queries rerun, even if nothing changed that is relevant to the query at hand. For instance, my app has a UI where referenced entities can be opened in a “drawer” to edit the referenced entity. This loads the data for the referenced entity from the server into datascript, which causes all the subscriptions for the background UI to rerun, which causes sluggishness for the newly rendered UI.
I’ve tried breaking up the re-frame subs into multiple smaller subs, but that approach doesn’t help when the datascript DB is needed throughout the subscriptions.
I’ve tried optimizing the query logic, but I’m not sure how much more I can squeeze out. I’m using
clojure.walk, which isn’t particularly fast in my experience, so perhaps I should use a custom function to walk the data?
Another option I’m considering is to maintain multiple datascript DBs, and only load data relevant for particular parts of the UI into each of them.
Have you dealt with this problem and come up with a good solution?