Pls. explain the goog.module part of 4th Nov Clojurescript news

Questions & Help How to?
1

In the lastest Clojurescript News it says:

“Like ES modules, the goog.module format is simply incompatible with REPL driven development.”

Does “incompatible” mean here that one could not re-evaluate (e.g. in Emacs/Cider) the expression (ns my-namespace (:require [first.namespace :as f] [second.namespace :as s])) on top of a cljs file and thus not be able to interactively load the second.namespace?

Can someone pinpoint some (other) minimal example of the REPL workflow that would become impossible with goog.module?

3 Likes
2

goog.module as well as common ES Modules each operate in their own isolated scope. Meaning that if you let a = 1; in two files they won’t conflict. It also however means that you can’t easily get a unless the module explicitely exports it.

Contrast this to the way ClojureScript and the older goog.provide mechanism works where everything executes in the same globally shared scope. If you did the let thing again they’d conflict and as such everything must be namespaced. This is already how CLJS works so it fits just fine. However writing this kind of namespaced JS by hand is kinda tedious, so it is understandable that the Closure folks move to something a little more developer friendly.

Overall you really don’t need to worry about this at all when using CLJS. Consider it an implementation detail you’ll only encounter when using code from the Closure Library directly. CLJS itself is not affected and didn’t get any new constraints.

Regarding the REPL driven development comment I disagree on the ES modules part. With some adjustments to the CLJS compiler that could work fine. goog.module however is pointless from the CLJS perspective since it gives us only constraints with no benefit.

3 Likes
3

@thheller thank you for the explanations. My post was of the kind “I do know everything is fine, but what if … when …”. And people probably say “why bother”. I’d like to explain my motivation for wanting to understand.

The CLJS-news post also contains the following sentence:
“The complexities and tradeoffs for interactive development are readily apparent when comparing typical JavaScript “hot-reloading” workflows and the development experience available to Clojure developers.”

I never got my head around when to use REPL workflow and when to use figwheel/shadow hot reloading. I find myself using hot-reload most of the time. And I think I am missing some point by doing this. And my hope was to understand the use of the CLJS-REPL better by understanding the “complexities and tradeoffs” compared to hot-reload. Your explanation helped me anlog this way, but still it will take some time to fully grasp the advantage of REPL over hot-reload.

4

Let me expand on that a little since that is the gist of everything.

In CLJS if you do (def a 1) in a namespace. Lets say (ns foo.bar). It’ll become foo.bar.a = 1 in pure JS. When loaded in the globally shared scope we can

a) access it from anywhere easily via just foo.bar.a
b) easily replace it later, just assign foo.bar.a = 2 and everything accessing foo.bar.a will get that new value easily.

This is how the REPL works. Manipulate one globally shared scope where everything can access everything. Hot-reload just re-assigns the things and the rest of the code sees it immediately.

In general with JS modules (of any kind) each file has its own scope. So you can’t just access everything without prior importing it and you can’t easily replace it after since each place that has imported the thing needs to be updated to “refresh” their local reference.

6 Likes
5

@thheller I think, in this moment, you gave the answer to the question “what does REPL mean”. Lots of questions are popping up in my head, but I need to ponder over this before I can formulate something meaningful. Thanks again.

6

What I understood is that each module is a closure, so basically if it uses a function or variable from somewhere else, or if you want to change something inside it, you need to reload the entire module and all modules that depend on it.

Normally in Clojure you can hot-swap individual functions and globals without having to reload the entire namespace and the namespaces that depend on it. But with modules you can’t do that, because you can’t mutate their definitions after they are loaded, so all you can do is reload the entire module a new and reload all modules that closed over it so they too now close over the newly changed module.

4 Likes
7

Thank you @didibus. I understand from your answer that REPL driven development in ClojureScript is possible, because it was a design decision that in this language, as opposed to JS, one always can

After pondering and understanding this, now the next question bugs me. Which other design decisions were made for ClojureScript so that REPL driven development became possible?

I ask this, because I try to find an answer to the question “what does REPL driven development using Emacs/Cider really mean”. And maybe I never found an answer because the question is wrongly posed. The real question maybe indeed is “Which set of design decisions of ClojureScript made REPL possible”.

The first necessary design decision is now found with your answer: All functions etc. share a global common scope.
The second design decision I can come up with by myself is: Lisp syntax, so one can put the cursor on every expression and evaluate, even when it is deeply buried inside a function.

Were there other than those two design decisions necessary to make REPL possible?

8

Ya those two are definitely part of it.

Another big one is on the compiler side, you need to be able to compile one form at a time and independently of the other forms.

A lot of compilers work at the file level, but for REPL you need form level.

One consequence of this is the strict order of functions and variables, the compiler doesn’t know what forms you’re going to send next, because it doesn’t know what comes next in the file. So even if you declare the function afterwards it doesn’t know that, and will say the function doesn’t exist.

That’s why you need to declare things ahead of time to tell the compiler, by the way, those are coming I’ll make sure of it so just allow them for now even if they are missing.

This is also why you don’t have a module scope, if you look at a namespace it doesn’t have an open and close bracket or parenthesis, that means it doesn’t surround the functions and vars it will contain. That’s why we have *ns* as an implicit state. So namespaces aren’t tied to files except for some tools that make that assumption as a convention. When you evaluate a form, you tell the REPL evaluate this in the context of the current value of *ns*. If you think of other languages, generally you’d say something like here is the full module starts here and ends there. That’s even where modules themselves in Clojure are mutable, you can add more Vars to an existing namespace after it’s been created, or remove them, etc.

Something else which is needed for REPL, is that the compiler must be available at runtime. This is actually where ClojureScript gets tricky compared to Clojure, because when you’re not in self-hosted mode, the compiler isn’t available at runtime. That’s where you need something complicated like first start a Clojure REPL which will have the ClojureScript compiler in it, and when you send forms to it, it will expand macros and compile to JS, and then you have a ClojureScript REPL inside that.

I can’t think of any more for now. But there’s a lot of small design decisions I think that stem from optimizing for REPL driven development. Sometimes the nature of Java and JS makes it a bit trickier, and that’s where Common Lispers will complain that Clojure REPL is more finicky, but Clojure has gone as far as it can to still make a good REPL even though it is hosted on JS and Java platforms.

Some more ergonomic choices, a bit similar to how the Lisp syntax helps with defining clearly scoped blocks to send to the REPL and compile/run. I think some of the immutability and the standard library also is pretty conductive to REPL. Running state is one of the hard thing to manage when at the REPL, so the less state you have, the more pure your logic, similar to how easier it becomes to test, it also becomes easier to use at the REPL. So I think that’s another aspect. Similarly all the EDN syntax makes defining maps, vectors, sets at the REPL much nicer then always having to use a constructor function to do so. Having a syntax that ignores whitespace is also great for REPL, where you don’t always have good multiline support.

I feel whenever Clojure is working in adding new features, the people who are adding them are also constantly using them at the REPL and adding them from a REPL, so you can imagine in that way how everything leads to small choices that probably worked well at the REPL itself.

3 Likes
9

@didibus Your explanations are so valuable. The main message of Clojure is that it is a tool that gets out of the way when making stuff. In trying not to shift away from this central message, designers are often deliberately not spending too many words about all their daily painstaking efforts to

This is a testimony to the pragmatic modesty all good engineers have shared ever since. After all, explanations risk triggering a myriad of new questions. For example, it will take me a week to bring the following sentences into one coherent picture:

But given your clear exposition of the context, I’ll eventually be able to understand REPL more and more. I hope that others profit too, so that the time you invested multiplies.

2 Likes
10

Yes I guess it seems like a paradox at first, but it makes sense when you dive into the details of what is mutable and what is immutable.

Your application logic and data is immutable all throughout, as much as it can be, and your boundary is mutable. Your programming environment is mutable as well.

That’s the context like you said which matters here.

A function reference being mutable allows you to hot-swap all references to it from one function to now use another inside of a running program without needing to restart or lose your runtime state. That’s really great when you want to iterate interactively at the REPL on the implementation.

Being able to add new functions into an existing module (aka namespace in Clojure) that is already compiled and loaded at runtime is also great when you want to interactively and iteratively expand a program capabilities as it is running.

The ability to load new modules at runtime is generally called dynamic linking, as opposed to static linking. Dynamism is the property of a system to change over time.

The concept of a dynamic programming language was born out of this idea of a program that can change itself as it is running. Lisp was the first attempt (that I know of) at creating a dynamic programming language.

While we tend to think of a dynamic language as referring to dynamic types vs static types, a real dynamic language is really one with the ability to change itself as it is running. What that means is if you have such a language, you can extend a program as it is running, you can start the creation of any program by first starting an empty program, like a blank page, you run this empty program and as it is running you add more functionality to it so it changes and becomes something more, maybe a text editor, a backend service that processes crypto trades, a command line for searching through files, etc.

Even the REPL is not truly what matters, what matters is that Clojure, like prior Lisps, is trying to be a dynamic programming language, that means one that can change itself as it is running.

Even the types being dynamic in nature is a consequence of this, how can you type check a program that can change as it runs?

The REPL is just the user interface to the dynamism of your running program. It is what you use to make changes to it as it is running, it allows you to sculpt.

Of course, with great power comes great responsibility. Changing the very design and nature of a program as it runs is great when you’re developing your program, it lets you iterate ideas quickly, see immediately the effects of your change, it lets you explore and poke around, etc., giving you that live interactive development that is so fun, like how a band jam session is more fun than a serious composition of music on sheet paper. But it is very dangerous once the program is to be used by your users, once people rely on its behavior for production use cases they depend on, they want a stable program, not one that changes as they use it. So Lispers know not to make changes to it when it is being used by users, unless absolutely necessary.

But this is about changing the program code and structure at runtime, and its benefit is when developing a program for the developer.

When it comes to modeling the business data and business logic, if you have various pieces of logic all changing the same data, it gets very easy to get the time and place wrong, and now you have bugs. So making those immutable and pure brings better correctness.

Now also at development, when you change the program, add/remove/modify the variables and functions inside namespaces and the namespaces themselves, how do you mentally keep track of the shared data between them that they’d be using if they were impure and mutable? As you change the program they might each leave corrupted state behind that your new functions don’t expect as they possibly would not create that same state. So even there it helps a lot that the application data and logic remains pure and immutable.

So in conclusion, this is where sometimes you’ll see people say that Java has a REPL too, that Python has a REPL too, that Haskell has a REPL too. But they’re missing the point, it’s not because you have a command line interface to send a chunk of code and have it compiled and the result printed back that you also have a program which can change itself as it is running. What the “REPL” really is referring too is a fully dynamic program, one whose functionality can change as it runs. In theory you should be able to start a Clojure Tetris game and slowly modify the entire code and memory of it, all as it is running, so that you end up with a calculator.

I hope that helped, I got a bit carried away there, but I think it’ll be an interesting read.

6 Likes
11

The Clojure explanation for JAVA people typically is that you go from

a) (1) Statically Typed and (2) Object Oriented
to
b) (1) Dynamically Typed and (2) Functional
or
b’) (1) REPL and (2) Immutability

To me, this b1/b2 pair was always blurry and I never fully understood the explanations given.

I learned from your explanations that another slogan could be that you go from

a) (1) immutable programming environment and (2) mutable data structures
to
b) (1) mutable programming environment and (2) immutable data structures

In this way, it is clear which two orthogonal steps one really has to master when moving from JAVA to Clojure.
And after getting explained (and making ones mind up about) this new slogan, words like “dynamic” and “functional” or “REPL” and “immutability” can be assigned their meaning.

For me now it is clear which two different things b1/b2 Clojure is pairing together in a revolutionary way. Again I hope that your efforts helped others too.

1 Like
12

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.