I cannot speak for what other people would like in a framework, just what I would like to see myself. Regardless, I apologise for derailing this otherwise very interesting topic with a discussion of frameworks and how to build one.
If anyone wants to take up building a framework, go for it; and make one that pleases yourself, not some imaginary user base.
Man… That was a bit of a cop out. I wasn’t looking to be rhetorical - I was actually really curious as to how you’d design a framework. I’m sorry that it seemed boring to you.
There wasn’t really a compelling argument made for existing clojure architectures, nor why people should choose clojure over something Next.js. So one looking at this discussion can come to the conclusion that Next.js is pretty good choice. It’s got a great user base, definitely not imaginary. And it’s already there, just another iteration on top of a very mature ecosystem.
It didn’t seem boring at all, it’s a very good question to which I’m afraid I don’t have a good answer at the moment. I have been thinking some about whether I want a framework or not over the last couple of weeks, and how I would go about it. The boring conclusion is that I am undecided at the moment. What I can say though is that I think HTMX for the front end is very promising, as is XTDB for the data storage and query layer. Cross compiling the routes from the back end to a service worker for offline capability is something I want to explore at some point. Most frameworks do more than I need in any one app, which is a good thing, but maybe building one is better in small bites.
I can see how it felt like I was cutting the discussion short, apologies for that. I actually think this is a very interesting topic.
awesome. looking forward to the fruits of your research.
In my above gist, I indeed forgot to list RMDB as an alternative pattern to the Biff+Macro pattern. The reason is that I did not really find out what the acronym RMDB stand for (related to RDBMS which Wikipedia has?). I found a 2019 YC discussion between @linpengcheng and James Reeves (aka weavejester) and did not get how the relation of RMDB to Clojure is.
What Clojure adds to the table (for newbies and everyone) are immutable data structures by default (also in the browser). I think those can make the world a better place and should be used more often than done nowadays.
If anyone wanted to play around with what some of these ideas might look like if they were added to Biff, you could copy the code from this namespace into a gist and rewrite it: platypub/sites.clj at master · jacobobryant/platypub · GitHub
(That namespace handles CRUD for the user’s websites, and it also handles previews + deployment to Netlify)
I got that acronym from his posts. It’s probably easier to just call it sql. I meant applications developed like this:
https://developer.oracle.com/dsl/odb-browser-apps-js-rest.html
It doesn’t really have anything to do with clojure.
@linpengcheng had a series of articles pointing out clojure can be mapped to a relational model and why it’s the best thing ever. It’s quite fascinating. His argument tended towards ‘relational is the best’ - which I found strange at the time but definitely rings much more true now.
I don’t care about static or dynamic types, nor about FP, LP, or OO. For me, they are overly complex, unreliable, and unscientific. I think they are very bad and upset me.
The production methods and business management ideas of large industries are also more mature than FP&OO. I have used them as programming ideas.
I think that RMDB is the simplest and most reliable in theory and practice, and it is the most rigorous, long-term, high-stress test in critical situations.
Before using clojure, I was a Foxpro programmer. I used clojure as a super Foxpro, and I also used it successfully in the field of WebApp and R language mixed programming. I will continue to apply this routine to the AI field in the future.
The main development goal of clojure is to write the database. The development idea is actually from the database, not the FP.
The Clojure system can be treated as a RMDB using the Lisp language, RMDB Schema (Clojure Spec) is a static type, and SQL (LISP) is a dynamic type.
Anything besides the ‘because it’s immutable’ argument? There’s lots of ways to achieve immutability - ie. generators.
I think the most common style in Clojure is the slightly more explicit. You would explicitly have to require some namespaces to bring in the common framework functions you need. And then you would have config for hooks.
Granted, I think the most Rails/Pheonix thing missing in Clojure is probably an ORM, like ActiveRecord or Ecto.
But I too am kind of curious, can we discuss framework design pattern here? What other options are there?
What pattern does Next.JS make use of for its framework?
I’m still focused on web, so the target is to run it in a web browser, but that would include wherever a web browser is commonly used, tablet, smartphone, desktop computer, tv os, etc. I would therefore exclude native Windows, Mac, Linux, Android or iOS/FireTV et all apps, even those that would just wrap a web browser and expose additional, OS specific APIs that are not W3C web standard.
I think FoxPro and maybe Visual FoxPro is probably closest to this. I’ve never used FoxPro, but know a bit about it, it’s a programming language that is also a SQL Database, like two in one. The language is thus designed as meant that you would store all state in the database that is inherently a part of FoxPro, and all features are just changes to the DB with possible IO side-effects happening out of that.
Edit: Actually, I’m not sure if its SQL based, might be its own hierarchical like DB.
In a way, this is a common style in Clojure, with say Datascript, or even how Datomic lets you put Clojure code inside the DB itself.
I’m not sure what a framework based on that style would end up looking like though, I’m also not sure how it would design itself as a framework, and not just a set of libraries one can use in the RMDB style.
Joy, fun, pleasure, adventure, and excitement 
Jokes aside, are there frameworks for those in JS, can you speak to their strategy to expose it a as framework?
Here is their website - https://nextjs.org - it’s pretty detailed. I’m not sure what you mean by ‘pattern’. They use more than just a single ‘pattern’ - a lot of it is directly influenced by rails as well as react.
Definitely right about Datomic and DB functions. I remember reading a comment about going back to the 60s when Datomic first came out. That’s what sparked my interest in learning about how the old heads did things.
I wrote a library called adi which stood for a datomic interface in the early days of datomic to try and simplify querying. Currently hyperfiddle is that on steroids - connecting a data model straight to the web. I’m not sure not permissions is managed but I’m sure the team has that down pretty good. You can probably call that RMDB style.
In the js, in terms of data access, there used to be a library called meteorjs that first did that before react become absolutely dominant. The most batteries included solution that I can think of right now is http://gun.eco → which is a distributed Datomic without the time travel serving straight to the web along with an encrypted permissions model.
@jacob Biff is a really nice library. how are you defining data access models? are you also generating the front end as well in platypub?
I’m asking like how do devs using it go about adding/customizing behavior for it, or is it more like a set of utils/libs that you can use in your own project?
You write jsx for frontend ui, the frame will do server side rendering to generate pages based on your routes. Data access is pluggable. Most projects these days use graphql.
I’ve been using a pattern for building our application that I think is very “framework-like”, that I also think fits in well with Clojure’s unique philosophy.
It is based on the domain representation described here.
It works like a data-driven framework, as creating an application using the framework mostly consists of specifying domain entities using plain clojure data.
All the core elements of the application is described as domain entities, and behaviour is implemented mostly using multimethods. For example, a route would be specified as
{ :dm.entity-type/name = :task-feed.media-management/image
:modules.server.core/route-handler-type = :task-feed.media-management/image
:modules.server.core/methods = { :get { :parameters { :query [ :map [ :src-url :string ] [ :image-args :string ] ] } } }
:modules.server.core/route-name = :task-feed.media-management/image
:modules.server.core/path = "/task-feed.media-management/image"}
A data migration might be specified as:
{ :dm.entity-type/name = :task-feed.content/migrate-crop-data
:data.entity.migration/date = 2022-05-11
:data.entity.migration/type = :task-feed.content/migrate-crop-data}
A datomic attribute that stores large html content in a separate storage system is specified as:
{ :dm.attribute/ref-typed? = true
:form-field/data-load-type = :task-feed.content/text-content
:field/form = { :form/type :form/html-editor, :form/label "Review short" }
:modules.decisions.core/attr-type.save-type = :task-feed.content/text-content
:modules.decisions.core/attr-type.type-key = :task-feed.content/text-content
:dm.attribute/name = :reading.book/review-short
:dm.attribute.ref-typed/type = { :dm.entity-type/name :content/Content }
:modules.decisions.core/txdata-hooks = [ :task-feed.content/html-statistics ]
:dm.attribute.ref-typed/many? = false
:dm.schema/doc = ""
:task-feed.content/content.mime-type = "text/html"}
A long running background process is specified as:
{ :dm.entity-type/name = :task-feed.pocket/pocket-articles-download
:processing.task/type = :task-feed.pocket/pocket-articles-download
:task-feed.processing/skip-on-dev? = true
}
Or a websocket:
{ :dm.entity-type/name = :remote-workers/worker-websocket
:modules.server.core/route-handler-type = :modules.server.core/websocket-handler
:modules.server.core/methods = { :get {}, :post {} }
:modules.server.core/route-name = :remote-workers/worker-websocket
:modules.server.core/path = "/chsk/remote-workers/worker-websocket"
:modules.server.core/websocket? = true
:modules.server.core/websocket-handler-type = :remote-workers/worker-websocket
}
The framework part then comes in when the application is run. It uses all of these data specifications to define the behaviour of the application.
For example, it would wire up all the route domain entities. Run all the migration entities. Set up the datomic fields. Render the edit fields for the html content. Ensure the background processes are running.
I’ve found this pattern to be very useful, as it allows separation of the components of the application from the actual running of the application.
Because the whole application definition is stored inside a datascript db, it is also easy to introspect and change. It also works very well with a repl-driven workflow, as everything can be reloaded.
Ok, I dug a bit into Next.JS. It seems it’s a client-side first framework, so your logic would all live in the client as an SPA, and the server-side part is pretty basic, meant to run as serverless functions in the cloud that would internally call some other APIs doing the real backend work, or the DB. But it handles nothing of backend data modeling and all that it seems. And it also is explicitly designed for pre-rendering first page load for faster initial loading and SEO.
They list:
- An intuitive page-based routing system
(with support for dynamic routes)
They seem to let you create new “pages” by watching files in a folder.
Each page is associated with a route based on its file name
I guess they’re going the code-gen route here. Some command probably runs to “build” the app at the end, checks this folder, for every file, generates code and uses the file-name as the route name implicitly.
I guess this is an alternate code-gen approach, more like a post-processor, you don’t run it initially to scaffold, so instead of running a command to add a “page” where it creates the file in the folder for you, you create the file in the folder and after you run a command that will use this file to create the real page/route out of.
Interestingly, they use the filename as code, like you can use a DSL as your filename to create some code.
For example, if you create a file called
pages/posts/[id].js, then it will be accessible atposts/1,posts/2, etc.
So they transform the filename here into a dynamic route. Making the name of files in a project into code is also something I would ask myself: madness or genius?
/pages/[…slug].tsx
Is this even a valid filename on all OS ?
For the rest, it looks like you have to explicitly require whatever you need to use from inside the page. And to “plugin” to the framework, so it can call you, it is done by convention it looks like, so you need to have functions of specific names that the framework will call.
It also seems like it inspects what of those “by convention” functions you have used, and depending if it sees certain of them, that customizes the behavior. So if you implemented getServerSideProps it’ll automatically assume you want to do per-page load SSR instead of build time SSR. That’s also an interesting trick.
It also looks like it uses the filesystem to override some of the framework behavior itself. Like if you want to replace how it loads a page, you create a _app.js file in some specific place, and it will use that module instead of the default one instead. Similarly, it just expects that you meet by convention the interface for it, so have the right functions that other things are meant to call.
Something else they do, which might be a JS thing, is that some functions you expose (using the convention), which it will call, those functions are passed as input a set of helper functions. So instead of requiring helper functions, you are provided them as input. Can you do that in Clojure? Can you pass a namespace to a function and use it?
And it seems that config is also handled by convention, like in the pages themselves, if you have a constant var with some config map, it uses that to configure certain aspect of itself.
Alright, so I think the interesting patterns here are:
Post-processing, like your code is not the complete app, you have to run a command and from it, it will generate the final result, and that’s where some of the customization and override/config logic gets applied.
File based extension points and overrides. You don’t override things or extend thing “in-language”, it’s all based around files in the project, almost like source code modules were config files with a set of rules to look them up and choose which one to use.
Ya, this is more similar to what I’ve normally seen in Clojure. The system is defined and hooked up with config.
But I wonder if this is easy enough, though it might be simple.
It seems it still forces people to put together things and know how to do so, but instead of combining things in Clojure code, you’re doing it instead using some configuration mini-DSL.
Didn’t Rails start the motto: Convention over configuration?
Some framework will still use config, but they’ll provide a CLI that scaffolds it all, so you don’t need to setup the config yourself. If you say cli add baz --route "/foo/baz" and it’ll setup everything for you.
But if everything is set up as data, then it becomes easy to provide functions to create these as part of the framework.
For example, if I want to add an html field to an entity in our application, I add the following code to the list of domain entities for the application:
(content/dm-html-attr :review-webpage/article {})
This just expands to an entity stored in datascript as pure data. When the application is run, the framework can now:
All of these can be done and managed by the framework, and the “user” of the framework just needs to understand how to add the building blocks.
What I’m getting at is that maybe for Clojure the motto should be: Data over configuration
I have been working with Elixir in the last year or so, and while the environment has some lessons to teach us, I’m not sure this is one of those. Working with Phoenix you get a number of places where “magic just happens” and as often the case with macros, when they work they are great, but when they don’t are awful.
The main issue that Elixir has, IMHO, is that if you have a living process that wraps state (the GenServer and all its siblings) what you want is OO-inheritance, Java style. I know it’s out of fashion, but that’s it. Macros writing plenty of code for extension/specialization is just bad.
This said, when appropriate, Elixir+Phoenix just rocks. When you try it, Javascript-free client programming is such a blessing that you’d put up with anything - even Visual Basic - just to have it. 
I think that could work, but you would need these functions or accompanying CLI from the get go I think, to make it easy from the start.
Biff is a really nice library. how are you defining data access models? are you also generating the front end as well in platypub?
Thanks!
Data access is plain XTDB. Biff inserts a db value into incoming ring requests, so you can call xtdb.api/q etc on it. Biff does add schema with malli. Instead of writing data with xtdb.api/submit-tx, you use com.biffweb/submit-tx, and that’ll check your documents against some malli schemas you define before submitting the transaction.
Platypub is basically a static site generator bundled with a CMS. It’ll generate the front end for websites you make with it in the sense that it’ll spit out a bunch of static files and upload them to netlify for you.
Going back to a much earlier point in this discussion …
I think you’re spot on to call this out. I think this is exactly why many newbie Clojurists I meet find Clojure “hard” even though they are often experienced devs in other languages (to reference another interesting current Clojureverse topic!) It’s because they lack the abstractions that either frameworks or static typing or some layering approach usually gives them. Sure frameworks are ultimately inflexible but those abstractions do allow folks to forget what’s on the other side and either use them, or implement them by “filling in the blanks”, which is critical to aid understanding. It made me look at Polylith recently which has its own style of implementing component “interfaces” through vaguely similar means to your Elixir example I think, with less macro magic.
But Clojure itself does not discourage abstractions right? It has two core language features introduced explicitly to support them - protocols and multimethods. It surprises me that there are not more Clojurists writing about using these features to “abstract away” - say to access external state via http vs s3 and allow easy testing or switchover in future, or create other boundaries inside apps. I would love to try and write a clean architecture implementation of the Realworld backend in Clojure using these plainer core Clojure abstraction features to achieve the dependency inversion that “modern” software architectures espouse (clean, onion, ports & adapters, hexagonal.)
I only discovered the Realworld site while looking into this and wonder what a “canonical” Clojure implementation might look like? Would it bother with any internal abstractions? If not would that make it easier to read and test as a newbie Clojurist on your team, or harder? The only Clojure implemention on there currently uses Polylith. I notice there are some Elixir/Phoenix implementations on there for comparison too 