Making a Clojure / ClojureScript Project (Oct 2024 Edition)

Community Center Beginners
1

It’s that time again.

That time when I come back to my Clojure project, to try to make some progress with it. And again, fail, humiliated, as I spend 3 days masochistically battering my head against the desk, failing to get the build scripts working correctly.

As before, this is a Clojure + ClojureScript application.

It has figwheel for working on the cljs front-end. Clojure at the back-end.

I need to be able to run a :dev alias where figwheel hot reloads the page. But also where I work on the back-end.

I need to have a :prod alias to compile the whole system into an uberjar for distribution. This :prod build should compile both the cljs and the clj back-end. And bundle everything into the uberjar.

I prefer not to use shadow.js. AFAICT you can compile cljs without it, using the build tools.

I need to import some dependencies from github. (Otherwise I would totally just use Lein)

And yet again, I can’t figure out how to get this build setup working properly.

I originally created my setup with Zack Oakes’s LightMod. Which was fine. Until it broke when I needed to import dependencies from github. Some time ago Sean Corfield helpfully diagnosed that this was because the build that LightMod created was actually still using lein behind the scenes. So I’ve been trying to migrate to only using cli and build tools.

But I simply cannot massage what I have into something that works.

I confess, I have taken advice from ChatGPT. And this may have doubled my problems. (Although it did initially seem to give some useful hints and solve some problems.)

So I am once again asking : does anyone know of, or have, a simple template or minimal hello world type project that has all these properties?

  • Clojurescript front end talking to Clojure backend
  • figwheel for hot reloading
  • uberjar for distribution
  • uses build.tools
  • can import dependencies from github
  • ideally a way to avoid repeating a long list of dependencies for the :dev vs :prod builds / :aliases (although maybe this is impossible)

At this point, I’m happy to start from a working template or minimal example and move all my other code back into it.

But every time I start with something with less than all the features I listed above, and try to add some of those back in, I fail. Ignominiously.

2

It would help us to help you if the actual state and the symptoms were stated. A list of libraries/approaches and a vague “doesn’t work” are not useful by themselves.

However, I can give some generic recommendations:

  • Never combine frontend and backend dependencies - keep aliases for those separate. There’s never a need to have something like reagent in backend dependencies (well, this one might be useful for tests) and next.jdbc in frontend dependencies
  • If you have trouble with uberjars, then… don’t build them - uberjars are not the only way to distribute an app. Or learn how uberjars work, in and out - otherwise, it’s very hard to figure out what an issue could be on your own
  • If you have trouble with CLJS, IMO you should most definitely use shadow-cljs. You say “you can compile cljs without it, using the build tools”, but you’re already using Figwheel, and shadow-cljs is just a replacement that I have personally found to be much more flexible and with fantastic docs
  • Take things step by step - it’s much easier to get a single thing working than to create a Frankenstein monster from non-working parts and then debug it. Your post suggests that you’ve tried it already, but again - that can only be helped if there’s an MRE. Nobody can help if the only thing they hear is “I fail”.
4 Likes
3

If you can put what you have so far up on GitHub, I’d be happy to take a look and see if I can help you get it up and running.

I don’t have much experience with ClojureScript (yet!) and I do still have a lingering preference for Figwheel, even tho’ it is very much a minority tool in the cljs world these days. Last year’s State of Clojure showed 80% of all cljs devs use Shadow (vs 25% using Figwheel – it was multiple choice).

Part of the problem here is that Figwheel hasn’t been updated for two years at this point, and tools.build is only three years old, so I’m pretty certain no one has created a CLI-based front/back-end template based on deps.edn / build.clj that use Figwheel.

That all said, I’m happy to spend some time helping you try.

6 Likes
4

I’d challenge this recommendation for beginners. I have projects that use no aliases whatsoever and just have everything in the top level :deps map. This is absolutely fine. It doesn’t matter to me that the uberjar would be 3 times the size it needs to be. I do not uberjar anyway, but just having a few extra unused dependencies really doesn’t hurt. Aliases only really become necessary when playing classpath shenanigans with conflicting dependencies and such.

Yes, it is best practice to split them. No, it is not required to get started.

5

Personally, even as a beginner in something I prefer to know the right set of approaches immediately instead of relying on some beginner-friendly convenience that ends up being competent-hostile.

And in the case of having a single alias for frontend and backend - I used to do that, it ended up saving me probably half an hour initially and costing me a couple of days eventually. It started from needing to update a backend library X that forced me to update some frontend libraries, and that forced me to update frontend code. And it ended with a few cases of straight up incompatible dependencies and me spending hours trying to figure a solution while not wanting to split things up.

The OP already has a problem that doesn’t seem to be trivial. I don’t think there’s a need to stick to footguns here that only promise not to shoot at beginners.

2 Likes
6

I wrote a post that maybe can shed some light on problems you might have. Seems to me like you are just starting out too complicated, instead of adding piece by piece.

It isn’t entirely what you asked for, given that it heavily relies on shadow-cljs, but I’m sure its possible to adapt this to work with figwheel too.

5 Likes
7

@interstar try skipping cljs altogether and just using Clojure. Since we have macros you can recreate the Clojurescript experience on the backend with SimpleUI. Its really easy to get started if you have Clojure tools installed, just follow the instructions at simpleui | JS Free Single Page Applications

8

Thanks to everyone who has responded.

Right now, I like the look of the Fullstack Workflow tutorial from @thheller, so I’m going through that. I’ll keep you posted if I manage to get this working for me, and if I manage to migrate my project to using shadow-cljs.

Particular thanks to @seancorfield for offering to take a look directly.

I WILL put this up on GitHub very soon, but I actually need to roll it back to a sane earlier version first, because it’s in such a mess right now (after I let GPT mess with it) that I wouldn’t want to waste anyone else’s time looking at it. :slight_smile:

Thanks again

1 Like
9

So @thheller’s tutorial is useful.

I’m making some progress, but I have a few questions.

  1. I don’t really understand these paragraphs

One essential missing piece for this workflow is of course how changes make it into the running system. Given that the REPL is the driver here, making a change to the handler fn (e.g. changing the :body string) and saving the file does not immediately load it. You can either load-file the entire file over the REPL and just eval the defn form and the change should be visible if you repeat the HTTP request.

Cursive has the handy option to “Save + Sync all modified files” when pressing the repl/go keybind. Or just a regular “Sync all modified files” in the REPL, since often the stop/start cycle isn’t necessary. This is super handy and all I have for this. Another option may to use the new-ish clj-reload to do things on file save. I have not tried this, but it looks promising.

I’m not using Cursive so haven’t set up hot-keys.

But what is it I need to type into the repl to restart when I’ve made a change to either the Clojure or the ClojureScript?

I’ve tried just (repl/go) but that doesn’t seem to reload / recompile the server-side code. It’s only when I literally stop and restart the repl that this seems to happen.

  1. You didn’t discuss building the Uberjar in your example, and I’m not quite sure how this would work.

In my existing setup I have a resources directory in which I have my graphql schema (gql_schema.edn). And the code is loading it using

(io/resource file-name)

The public directory where the index.html, main.css etc. are - and where the compiled js is placed - is also a subdirectory of resources.

I presume it needs to be this way to work correctly when put into the uberjar. As the distributed application will need to find all these files in the uberjar itself.

However, right now :

a) your example doesn’t have the public directory inside a resources directory.

So how is the path to the public directory configured in shadow? Can I put it inside the resources directory? I assume I’ll need to tell the shadow cljs compiler to write the output js file there. And the server to get the files from there.

b) When running in dev mode, the io/resource doesn’t seem to be able to find my gql_schema.edn, despite it being in a resources directory at the same level as in my original project. So I assume that this also needs to be configured somewhere. Any idea where that is?

cheers

Phil

10

You really shouldn’t be typing anything into the REPL. Whatever editor you use almost certainly has support for eval’ing things directly from your editor. Things to look for is either “Load File in REPL”, which loads the entire open file into the REPL. Or eval’ing individual forms. Both should be possible with your editor.

Often (repl/go) will not even be necessary when doing the above.

If you’d rather have the code loaded automatically when you save the file, I’d suggest looking at the clj-reload lib I linked. I have never used it because Cursive has this handy sync feature built-in.

resources nothing else than a folder that is configured to be included in the classpath. So, either you just add this to the :paths ["src/main" "src/dev" "resources"] in your deps.edn, or you move the files to src/main for example. I also wrote a post about the classpath a while ago, maybe that helps.

It is common for people to have something like a resources/public folder, but I do not like that, so there is the top level public instead. Any easy change if you prefer to use resources.

As far as shadow-cljs is concerned the default output directory is "public/js". If you want to change that you can use the :output-dir option in the build config, e.g.

{:deps true
 :builds
 {:frontend
  {:target :browser
   :output-dir "resources/public/js"
   :modules {:main {:init-fn acme.frontend.app/init}}
   }}}

I don’t use uberjars anymore, but if you want you can follow the guide for tools.build. They cover that.

1 Like
11

Thanks @thheller

You really shouldn’t be typing anything into the REPL.

Wait! What?!?!?

So it’s not that the REPL watches for files changed on the disk? It’s that we need an editor to explicitly inject new versions of the code into the REPL in order to be recompiled?

That feels very weird to me. I mean, should the development process be so tightly coupled to the editor? In practice I use Emacs. Which I’m by no means an expert on. I just managed to get it set up with paredit and a couple of other Clojure specific modes. I’ve not really got to grips with Cider, but I guess it could be used for this?

I’ll check out the clj-reload lib.

I also wrote a post about the classpath a while ago, maybe that helps.

That looks very helpful. Thanks.

I don’t use uberjars anymore, but if you want you can follow the guide for tools.build. They cover that.

OK. I’ve looked at that a couple of times but ended up confused. What I’ve tended to come up against is that

clj -T:build jar

doesn’t seem to include all the dependencies I need to actually build the code. AFAICT this is because the -T option is for tools and so doesn’t bring in the top level :deps from the deps.edn

What I understand in principle is that the line

(def basis (delay (b/create-basis {:project "deps.edn"})))

is meant to be a programmatic way of telling the build process which libraries to use in the build, but in practice I’ve never found the right way of expressing it to include my dependencies.

Can anyone throw any light on this?

cheers

Phil

12

REPL is Read-Eval-Print-Loop. It reads what you send to it, evals it, and prints it. It watches nothing. It doesn’t even care if there are any files at all.

If you want automatic-on-save-reloading for .clj use clj-reload. For CLJS this is built into shadow-cljs.

Emacs+Cider can do all the REPL related things. Just don’t ask me what the commands/keybinds are.

clj -T:build jar is building a jar, not the uberjar. The regular jar does not include dependencies. uberjar is covered later in the guide. Yes, it gets the needed dependencies from deps.edn, at which point it makes sense to start using aliases to separate out the development only dependencies (e.g. shadow-cljs), so they do not end up in the uberjar.

Have you considered not building uberjars? I don’t even build regular jars anymore for my actual apps.Just way less to worry about.

1 Like
13

Thanks.

OK. I understand the point about the REPL. I’ll try to figure out the Emacs / Cider thing.

clj -T:build jar is building a jar, not the uberjar.

Yes I copied the wrong line from that page, but I think it’s the same issue, even the uberjar example uses the same line

(def basis (delay (b/create-basis {:project "deps.edn"})))

which in the past hasn’t seemed to find my dependencies. But I’ll add this to the new project and see if it’s working. If not I’ll post another question about it here, for anyone who might be more familiar with uberjar building.

As to why build one, it’s very convenient to be able to distribute my software as just a jar file with a couple of shell / batch scripts to run it, so that the only dependency the user needs is Java. Despite having a web UI, it’s still personal software to run on your local machine. I don’t know a better way to distribute it.

A separate issue as I’m starting to add my dependencies to this example :

Here’s my current deps.edn

{:paths ["src/main" "src/dev"]
 :deps {thheller/shadow-cljs {:mvn/version "2.28.18"}
        ring/ring-jetty-adapter {:mvn/version "1.12.2"}
        hiccup/hiccup {:mvn/version "2.0.0-RC3"}
        instaparse/instaparse {:mvn/version "1.5.0"}
        markdown-clj/markdown-clj {:mvn/version "1.10.1"}

        org.clojure/core.logic {:mvn/version "0.8.11"}
        io.replikativ/hasch {:mvn/version "0.3.7"}
        
        com.walmartlabs/lacinia {:mvn/version "0.36.0"}
        cljstache/cljstache {:mvn/version "2.0.6"}
        com.alchemyislands/patterning
        {:git/url "https://github.com/interstar/Patterning-Core.git"
         :sha "04075f800b768aedd6c445625515d943160aa6e2"}

        remus/remus {:mvn/version "0.1.0"}
        clj-rss/clj-rss {:mvn/version "0.2.5"}
        org.clojure/core.memoize {:mvn/version "1.0.236"}
        http-kit/http-kit {:mvn/version "2.4.0-alpha6"}
        org.babashka/sci {:mvn/version "0.9.44"}

        reagent/reagent {:mvn/version "0.8.0-alpha2"}
        cljsjs/react {:mvn/version "15.6.2-0"}
        cljsjs/react-dom {:mvn/version "15.6.2-0"}
        }}

And I just added some of these to the app.cljs

(ns xyz.app
 (:require
   [reagent.core :as r]
   [reagent.dom :as dom]
   [clojure.string :refer [lower-case trim replace]]
   [clojure.string :as string]
   [cljs.core.async :refer [<! timeout]]
   [cljs.core :refer [js->clj]]
   [cljs.reader :refer [read-string]]
   [cljs.pprint :refer [pprint]]

   [sci.core :as sci]
   [markdown.core :as md]

   )
  (:import goog.net.XhrIo)

  (:require-macros [cljs.core.async.macros :refer [go]])

)

Now when I try to run the dev mode I get

shadow-cljs - nREPL server started on port 36403
[:frontend] Configuring build.
[:frontend] Compiling ...
[:frontend] Build failure:
The required namespace "react" is not available, it was required by "reagent/core.cljs".
14

Well, react is a npm package that you need to install. My blog post mentioned that. cljsjs/* is unsupported, unused and can be removed from your deps.edn file.

Make sense to use an uberjar if you are distributing an application. I assumed you are building a web server that is going to be hosted somewhere. In that case you want to switch to using ring to serve files from the classpath instead of the filesystem. That would be the ring.middleware.resource instead of ring.middleware.file.

1 Like
15

Ok. Thanks for the ring middleware tip.

So the cljsjs namespace is deprecated?

And are you saying there’s no way to include react without npm?

16

Kind of. As I outlined in my post you do not actually need npm installed, although react being an npm package you need some way to get this on your computer in some way.

To be extra clear though this is only needed during development and only needs to be done once. Once the project is built (via shadow-cljs release frontend) the produced outputs are standalone an no longer need access to the npm package files, i.e. they do not need to be in the uberjar or anything.