I can’t find a good place for this, so I’ll turn it into a question. How do you deploy your code nowadays? It’s been a while.
After looking at the new contender HOP I keep wondering:
What are we doing while Github Actions run
There’s certainly enough time for a proper office chair fight https://xkcd.com/303/
Can we do better?
We have a dynamic runtime, we should be able to update our running environments for most of the changes without tearing it down first, committing the code, building 99.9% of the same things we’ve built before, running tests you already ran locally (if you are good citizen) and then updating and starting containers…
And all I did was re-def a few fns.
In short: the amount of changes that need to be done to my test and prod system should be minimal.
Bazel (et al) embrace this on the build-side of the equation but I’ve yet to see something that understands change management all the way into the runtime.
We keep layering tons of stuff on top of the actual JVM or node process that runs our code with not even an option to restart them, when all I wanted was to re-def a few fns. (i.e. REPL into all prod instances, send new code)
Yes, I might have brought in a new library and even then I should be able to load it in. Only if that fails or is not possible, the process should restart - in a consistent way.
It feels like we are re-architecting and re-building the house with every change, when all we did was change a light-bulb.
One of the (many[1]) possible reasons is that you don’t have a single instance where you run your Clojure application, but a bunch[2] of them that you want to scale horizontally (“scale out”). Even dynamically up & down. Which is harder with your approach.
Another one is that “It runs on my machine” is a well-known problem. Even if you use conainers for development. Even if you run tests locally. You never know what local hidden state is there to get you! That is why (again, among other reasons) we run CI in separate machines. Ideally in a reproducible way (easier said than done!).
Finally, even if we hate to admit it in the Clojure/LISP world, changing a program while it’s running is not a failure-proof thing. Again application state, that you didn’t remember or didn’t even think about, may be waiting there for you and bite you in your lower back.
In short, because there are several hard problems to solve in each of those phases (mostly related to local state), and “turning it off and on again” is an easy way to avoid them[3][4]. Assuming that you can afford that downtime, that is. Which is why many people go the “scale out” way that I mentioned in my first reason.
[1] Each company, project and use case can have others than the listed here.
[2] A cluster, a fleet, or whatever fancy term is used nowadays.
[3] Given that not every company can have only the smartest, brightest developers and devops, keeping things as easy as possible is (almost) always a good idea.
[4] Starting things from scratch gives us a consistent “local state” every single time
In the context of a side project/early stage business, I find deploying via repl works quite well, and I’ve done the majority of my development this way over the past year. I’ve made a bb prod-dev command for Biff projects that watches the local filesystem for changes. Whenever I save a file, it gets rsynced onto the server (a vm, not a container) and evaluated. The prod app is ran with clj rather than building a jar, so if the app restarts it’ll keep the new changes.
I’ve structured Biff to use late-binding as much as possible, so most changes don’t require an application restart/refresh. When possible new dependencies are added automatically with tools.deps.alpha, though these do require app restarts sometimes.
I usually am running on a single server, but I do have one app that has a web server and a separate worker. For that project I’ve modified the commands so I can run e.g. bb web prod-dev or bb worker prod-dev to connect to a specific machine. If I had multiple web servers/workers it wouldn’t be hard to have the commands deploy to multiple machines simultaneously.
I’ve rarely run into problems developing like this. The main thing to look out for is functions that have been deleted/moved but are still referenced somewhere. To double check for these, after I’ve finished working on something, I usually start up the app locally to make sure there aren’t any compilation errors.
But yeah–while this works for the easy case of solo development, I probably wouldn’t enjoy using this workflow on a team .
Even the smartest and brightest benefit from keeping it easy (simple?). Less cognitive load means more spare processing power for the inherently difficult stuff. I like to keep things simple for the benefit of the brightest. That the junior devs can then catch up quicker is just a bonus.
I am deploying full-stack ClojureScript with Piku. When I want to deploy I run make build which builds the server and client files in a deployable state into the build folder. Then I commit the change to this folder and git push piku which does the actual deployment to the server.
If you are looking for an interesting devops tool for setting up servers check out Spire.
