I’m very new to web development. I’ve built a small app at work with a back end database managed in Clojure, and a front end in Clojurescript. It’s by and large an event monitoring and logging application.
Given how easy it is to deal with data in CLJS, I’ve ended up doing most of the work on the front-end. That is, the db is built on the server, events are monitored on the server, but then the whole raw db is sent to the front end and every calculation (cleaning data, filtering, grouping, etc) is done in the front end.
From my little experience of GUI and MVC programming this feels wrong - I would have expected the server to do all the computations and the front end to just make requests and visualisations. What are the pitfalls of my current process? When does it break? At the moment the db could fit in ~50MB CSV file but that will grow.
I had the same approach as you for my latest project. But the client gets only a fraction of data on page load, and the rest of data gets loaded on demand.
Given that Clojure allows for platform independent code via cljc files, you can have the logic present both on client and server. Same for markup. So the client is basically a server with only a part of the data.
TL;DR Your app’s “brain” can be anywhere enough data fits (with sufficiently low usability latency for your domain), barring security concerns.
I’d take the learning opportunity to question that feeling of “wrongness” and see if you can justify it from observation and logic within your actual use. Maybe it’s just dogma!
When faced with a decision like this, I try to find a way to avoid making it. In Clojure land, you can get pretty far with fns on data, including from libs, in pure CLJC (platform-independent) territory. Then, you’re free to make platform-dependent decisions where there’s a specific, valid reason to (like security, as @Webdev_Tory mentioned, which should still be specific to the sensitivity of data you’re sending, relative to your trust level of the browser’s memory or durable storage). As to storage, memory, bandwidth, or processing limitations, that again depends entirely on the realities of your use case. The specifics of what you’re building and what you observe trump generic wisdom.
Clojure-ville gives you the flexibility to make decisions granularly, and specifically for your use case. Don’t artificially constrain yourself early. Researching online can lead you astray that way. It’s easier to write about generic, framework/platform questions than problem-specific ones, so that’s what an overwhelming majority of online material covers. Conversely, it’s easier (and wiser) to make programming decisions based on the specifics of what you’re doing (and where you’re going).
By assuming less about the environment while building, you’ll end up with smaller, simpler, decoupled pieces that can be flexibly re-composed as needed. These sorts of small, stable, specific, yet composable pieces are some of the most valuable bits of software we build.
Thank you all for your thoughtful answers. This is very helpful, and it’s reassuring to see there are several school of thoughts!
The security concern is extremely valid - in my case it’s an enterprise solution in an intranet, that is largely replacing an Excel sheet whose password everyone knew (!..) - so I’m chilled on that.
I think I need to balance points 2 and 3 raised by @Webdev_Tory. Namely, many of my users are running on laptops or iPads with limited processing power, while most of the time the app is used on a (cable or enterprise WiFi) network connection so downloading 50MB is a breeze. So I feel that computation time is more “expensive” than download time, but on the other hand the request / handshake process still takes time and makes the application less responsive. Maybe I should do all the calculations on the server and then just send a 100MB database in one go with all the possible displays pre-computed.
Related - my JSON database is big, largely because I have long key names that are repeated throughout. If I just zip the JSON file I can divide the size by 20x. Does Ring / the browser zip the requests automatically? Or would there be a benefit in renaming the keys before sending them to the client?
Hi, regarding your question ‘Does Ring / the browser zip the requests automatically?’ The browser will send a Accept-Encoding: gzip header in the request and the server will respond with a Content-Encoding: gzip response header, if it has gzipped the response body (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding). If this happens automatically depends on the web server, if I remember correctly Jetty does support gzip out of the box and httpkit does not. In the latter case you can use a Ring middleware like this one:
In the beginning of our SaaS service Storrito.com I’ve also sent the whole db for the current user to the browser, it was a shortcut and as expected we ran into performance problems as our user base growed. I have written a blog post series about this and what we are using now: