For ultra-dynamic web applications, Electric Clojure is a reactive and network-aware Clojure/Script DSL that fully abstracts over client/server state sync at the programming language layer, in order to achieve strong composition across the frontend/backend boundary. You don’t need a web framework, you need a web language.
Leo, Geoffrey, Peter and I are thrilled to finally release this! After two years of hardcore dogfooding and 8 months of private beta, we’re comfortable that brave Clojurists will be successful and have an acceptable development experience.
Many FAQs are addressed in the Hacker news discussions, or just ask here.
Really cool stuff. congrats on the release. It’s a really interesting way of solving a nasty problem.
I’m wondering why you guys have done it this way as it goes against the whole idea of ‘separation of concerns’ and actually embraces the opposite of that.
Also, a couple more questions:
Is this framework orthogonal to datomic? ie. it’s plug and play with any db.
how theme-able is it?
is the intermediary code clojure? and is it viewable?
How do you organize your projects given that there may be bits of server and client code scattered in a lot of places?
I’m wondering why you guys have done it this way as it goes against the whole idea of ‘separation of concerns’ and actually embraces the opposite of that.
The 2 types of complexity are
having 2 things when you should have 1
having 1 thing when you should have 2
The separation of frontend and backend creates a boundary you now have to manage. Why not let the computer take care of it?
Think of the GC as well - are you bothered the JVM or JS engine manages memory for you? We just decided to manage the network too.
Is this framework orthogonal to datomic? ie. it’s plug and play with any db.
Yes, the db is orthogonal. We have a Postgres example in the Slack channel.
how theme-able is it?
You can theme it any way you want, you can use standard tools like tailwind etc.
is the intermediary code clojure? and is it viewable?
Electric clojure is clojure and clojurescript code with reactive semantics.
How do you organize your projects given that there may be bits of server and client code scattered in a lot of places?
The same way you organize your cljc projects today
Firstly, what you guys have done is fantastic. The network layer being generated is great - it’s something that I’m doing as well - though our solution is not as automated/packaged as electric. One question I have for doing the full pipeline is it’s explictness - ie. how would an external ui team consume the server endpoints exposed by the codesplitting?
I meant more in terms of application architecture. I feels like there needs to be a skeleton somewhere otherwise the code’s going to end up really messy really fast. Ie, would you still do the db/system/network/ui boundaries, or is it more split as db/app in the case of electric?
Larger scale idioms yet to be established. Let us know what you end up doing if you try it! I like to organize my code by business concern, not technology/platform concern – so I like to keep all together my queries/views/css etc for a given business concern, ideally even in the same file so that when I change a view I don’t have to go searching for the css and query stuff that it is coupled to.
Electric is additive, you don’t lose anything from clojure! If you need to expose your backend business logic to an external service you’ll write that part in standard clojure code and e.g. expose it through HTTP
Here’s a test showing a baseline Electric program, and then the approximately same as a low-level Missionary program. The actual compiler output is more complex as it instruments the missionary DAG with network operations and debugging/observability metadata. Sure it’s viewable, it’s just macros so you can perform the macroexpansion, but it won’t be helpful to look at, you should think of it as bytecode really, for a distributed virtual machine (like JVM or BEAM).
And I’m trying to “reason locally” about it; and the try part is confusing: it looks like if the server doesn’t respond right away—which is expected—an exception of type Pending is raised. Now, I guess I’m supposed to know that when the server does respond, that the tr will be inserted?, and I’m supposed to know that the "loading" will be removed from dom/props?
Any clues as to how to understand this snippet would be appreciated.
it looks like if the server doesn’t respond right away—which is expected
the client/server coordinate in advance like a sports team; so this isn’t RPC. The server client does not need to ask the client server for anything at all [ed: fixed errors]; the information simply shows up. Therefore, there may not even be a Pending exception at all, because maybe the server information arrived before the client was ready to render it! For more info about the network planner see UIs are streaming DAGs.
Now, I guess I’m supposed to know that when the server does respond information is available, that the tr will be inserted?, and I’m supposed to know that the "loading" will be removed from dom/props ?
Here with try , the catch body mounts when Pending is present and unmounts when Pending “goes away.” (dom/div ...), (dom/props ...) represent point writes to the DOM. This resource management of the DOM is performed by a functional effect system (Missionary) which guarantees that all the resources are constructed (mounted) and destroyed (unmounted) in the right order, per the DAG. Because all the interesting work is done by Electric & Missionary, electric-dom is only 300 LOC. Note it’s not just try; this is the case for all control flow operators - if, e/for, try/catch etc.
See demo-2-toggle — clone the repo and run the demos it takes 20 seconds — you’ll see the functional effect system maintain the DOM in response to the atom changes. Quiz to check your understanding: what is dom/text ?
[ed: retracting this, upon review we feel it is too inaccurate and not helpful for understanding]
To understand what’s really happening in terms of dom effects, it may help to think of Missionary as a FRP monad and Electric as “do-notation for Missionary”. The comparable Haskell monad comprehension would be really nasty; luckily we have metaprogramming!
Sweet. that’s what I was looking for. The network planner is really interesting stuff. I’m not the biggest fan of reactive programming but this is different. You guys are literally drawing the DAG (the network call descriptions) in code. I can see how that is really beneficial in a data analysis/back office environment where requirements are fast moving and things can be really arbitrary.
A couple more questions:
How is caching/offline mode handled?
Can these components be tested non-visually?
Why do you say Graphql goes away and what would replace it in an app?
This was the sort of thing that was really mind-bending when I first started playing with Missionary…
Do you think that people need to at least try Missionary first before they try Electric? The implicit cycles of execution can be pretty hard to even spot, since we’re so used to straight-line code…
missionary is an implementation detail of electric, it is neither required nor recommended to learn it before getting started. You may need it at intermediate level, depending on your needs, e.g to craft your own effects for use in an electric program.
Also the semantics are slightly different - missionary’s extended syntax with backtracking operators aims to be compatible with clojure regarding ordering of effects, while electric clojure has more relaxed constraints.
One metaphor that can help is thinking of electric code as a circuit. An if is a switch with a true and false branch. It stays up until the whole electric function is mounted and mounts/unmounts (electrifies ) the true/false code branches for you.
Now imagine try/catch the same way - it’s a switch that mounts the try branch. If it throws a catch handler mounts. The try branch can stop throwing, at which point the catch handler is unmounted again.
It is mind-bending because it is new. It turns out to be worth getting used to!
What cache are we talking about? Offline mode we haven’t tackled yet.
Can these components be tested non-visually?
You mean DOM components? What kind of tests do you have in mind?
We have some tests of our UI controls, e.g. a select picker, that run in CI in a headless browser. The tests were written through a cljs repl, interacting with a browser.
Why do you say Graphql goes away and what would replace it in an app?
We’re moving away from a model of “transform a graph into a big tree” to small, reactive queries that run on demand. This is a simpler, more granular and natural model. You don’t need to prefetch the tree if you can drop an (e/server ..) anywhere! The impedance mismatch between your UI and your DB/query language disappears.
Testing the data-flow logic only without the presentation layer. Would it be possible to strip out the ui/* stuff to compile it as an input/output block? It just seems hard to test otherwise.
Saving the newest version of the data on the client side and only calling remote updates when necessary. Hasura does this for graphql subscriptions through polling.
I’m still a bit skeptical of electric in its current iteration (though I’m fascinated to see where you guys take it). I’ll explain why:
This is code for a reactive type with react integration here that reruns the query when it’s arguments change with the ability to set remote and a local cached query (via sqlite). The implementation is pretty dumb and the calling code needs to be really explicit because there’s around half a dozen different ways that data can be updated. I’m getting really sick of coding in this way and having a reactive language would save a lot of mental stress. but having said that there are also benefits to coding this way as well, the primary advantage is that there is good separation between the data and ui and pathways can be tested independently, as do the components - which do not need to know about the network logic to work.
Also, data that is being pulled in by the app might be displayed in 10 different places spread across different components - so the tight coupling in electric may not be ideal in many of the use cases I’m familiar with. Having multiple components react to changes on a ‘view’ that can be updated via a number of methods is something I’m more comfortable with.
I think that having a schema on both the backend and the frontend is really important in reconstructing data graphs. Very interested in your thoughts on this.
Testing the data-flow logic only without the presentation layer. […] data and ui and pathways can be tested independently, as do the components - which do not need to know about the network logic to work.
I think you fear mixing client and server code breaks testability. I feared this initially as well, but it turns out to be a non-issue. You can use the same techniques for making your code testable as you’re used to. For example, you can separate a server-side action into an electric function and pass it in as an argument. Now you can substitute that behavior. You can even use a function in your test that doesn’t transfer to the server, so you’re not tied to the network logic in any way!
Hyperfiddle has put a lot of effort into testability. Anecdotally I have yet to find a case where mixing client and server code introduces untestable code. We also have a lot of tests in our repo and have an in-house testing library that is great at async tests.
If you could provide a piece of code you find problematic to test I’d be glad to take a look!
Saving the newest version of the data on the client side and only calling remote updates when necessary. Hasura does this for graphql subscriptions through polling.
Electric is database agnostic. If you want to use e.g. Hasura you can do so.
data that is being pulled in by the app might be displayed in 10 different places spread across different components - so the tight coupling in electric may not be ideal in many of the use cases I’m familiar with.
A typical way to solve this is to request the data once and pass the data down to your components. You can do this in electric much the same way.
If you’re interested you should play around with electric a bit to get a feel of it. I think your fears would quickly dissolve. We’re answering user question in the hyperfiddle slack channel and also provide onboarding zoom calls for those who want a jumpstart!
I think that having a schema on both the backend and the frontend is really important in reconstructing data graphs. Very interested in your thoughts on this.
What schema are we talking about? Your data in your DB is probably tied to a schema. You can also use malli/spec/schema to enforce type constraints on both peers. If you meant something else let me know!
Why is this a non issue? I did have a look at the repo and the examples it seems like the browser tests are more like integration tests/examples than anything. I’m talking about isolating tests on the network call - or do you just trust the compiler at this stage?
And it’s fine if you do trust the compiler. It makes sense. But I just need a straight answer, or some examples at least. There’s no point linking to a test library and saying - ‘we’re focused on testing’ then hand waving the question away. Maybe I’m not your target audience but from my point of view, testing is hard to do. How do you test the Teeshirt-orders on the front of the repo?
Lastly I only see 10 tests overall in the electric repo (which are integration/examples tests) covering a library of about 25 source files. And I don’t see any tests at all for the compiler, unless you count the comments. Datascript’s an interesting choice btw… first time I’ve seen it done.
Can you point to an example?
I’m not advocating for Hasura, I was clarifying what I meant by caching. So I guess the answer is no?
I’m talking about a relation schema in both the back and front ends.
The full Electric Clojure test suite passes in browser, node, and jvm.
We will increase our test coverage further in the next quarter as we begin working seriously on performance.
Our priority to date has been correct language semantics and validating the approach by dogfooding the Hyperfiddle sister project.
Before reaching this point, we did not wish to have too many tests weighing down refactors, which is why only 1/3 of our LOC is tests, you’re right that it should probably be 50%+.
As Peter said, we specifically developed an asynchronous testing library for the purpose of making Electric easier to test (and pleasant!).
We also have 12+ demos, including stress tests.
We run toxiproxy while we develop. Toxiproxy is a testing tool that simulates network and system conditions for chaos and resiliency testing.
One of our customer projects uses Electric to manage a server-paginated grid over a database query resultset with 100k records. (So each flick of the mouse wheel is potentially streaming thousands of records through Electric)
) the true/false code branches for you.