Ive been working on a Clojure application that is becoming increasingly complex, and Im finding it challenging to manage state effectively; While I have been using atoms and refs, I am starting to feel like my approach might not be scaling well as the application grows. I have been through these resources/articles State management for the server?what is splunk but I want to learn more from community members.
I wanted to reach out to this community to see what best practices or patterns others are using to manage state in larger, more complex Clojure applications. Are there any libraries or techniques that you recommend? How do you handle things like state synchronization across different components or subsystems?
Any advice or resources would be greatly appreciated!
Hi, Communicating Sequential Processes (CSP) might help which you can implement using core.async. This talk explains everything in detail:
I had a hard time wrapping my head around these ideas. Helpful for me was to imagining your system as a factory. You have a bunch of machines (core.async go-loops) and conveyor belts between them (core.async channels).
What kind of application is this? And how large–a few thousand lines, 50k lines? If it’s a web server, I’d say the first thing is to start using a regular database like Postgres instead of atoms and refs. If you’re doing a desktop app like the OP in the other thread, I might suggest sticking with a single atom and no refs.
What kind of state? How is the app implemented server-side (scalability and redundancy), is it distributed across multiple servers?
If your app is distributed, your only option is to keep most state in the database, which will hopefully be distributed as well (I use RethinkDB, migrating to FoundationDB right now, because I want correctness and a fully distributed database). If it is not, things are considerably simpler, but I would suspect that the problem is not with atoms/refs, but with data structures — if I were developing an app from scratch now, I would plan for scalability/redundancy in the future, so I would probably use data structures that could be moved into the database later on (and these days I would only use a database with strict serializability consistency guarantees).
You may wish to check out my library EntityGraph for some inspiration. It focuses on this exact problem in the context of web/mobile apps. You may find some ideas applicable to server state management as well. EntityGraph works in Clojure and ClojureScript.
In this talk I discuss the design of the library:
If you choose to use it or something like it, it’s not a full solution of course, but I think you may need a this type of a component in your solution (i.e. a database-like thing for managing app state).
