ClojureScript Compiler Internals with Mike Fikes

Learning Resources Talks
1

Hello all :nerd_face:

Yesterday Mike Fikes (@mfikes) gave a 3.5 hour (!) guest lecture on how the ClojureScript compiler works to the initial cohort of Professional ClojureScript. Mike takes his time to explain the major steps of the compiler (reader, macros, analyzer, emitter) and shows examples in a REPL and in the compiler source code.

The lecture is a great deep dive into this topic and I thought it would be an excellent learning resource for the community:

Professional ClojureScript is a 3 month bootcamp-style course to learn ClojureScript. More details can be found at https://cljs.pro or this introduction post.

15 Likes
2

Thanks for putting this up Chris, and thanks also to Mike. I’ve only seen the first hour but I’ve already learned a few things about the reader which I didn’t already know. The compiler is an extremely interesting aspect of ClojureScript, and it’s always good to learn more about it.

Regarding the word “form” in Lisp which came up in the video, this is something that bugged me for a while (as does “s-expression”). I eventually decided, as I think you all did in the video, that you can use “form” and “expression” interchangeably, they mean exactly the same thing. So anything which can be evaluated is a form, whether in a list or by itself. A list is a collection of zero or more forms, and is a form itself, and so on.

3

Thank you for posting! Looking forward to viewing and learning.
Alan

1 Like
4

Haven’t watched the video yet but plan too, but I’ve used form to mean anything that can be evaluated, which would be synonymous with just expression, but s-expression implies a list of expressions (or forms), and is itself an expression/form.

;; A form, but not an s-expression
hello

;; Also a form, but not an s-expression
123

;; A form and an s-expression
(+ 1 1)
1 Like
5

@didibus That’s very confusing to me, as special forms like (def) are only treated as special forms when inside parentheses. def can be used as any normal symbol outside of that case.

6

def isn’t special everywhere, so it won’t be special in a string for example:

"def"

Similarly it won’t be special as a symbol, unless used in first position of an s-expression.

But arguably this part is a bit confusing, that’s why it is special hehe.

Note: That’s just the taxonomy I settled on by the way. I’m not sure what Rich Hickey actually thought each should be called or not. That said I suspect he simply chose to use Common Lisps taxonomy, and mine is consistent with it, read more about it here:

1 Like
7

Using “form” for special forms is rather confusing in the sense that it muddies what “form” actually means. Special “functions” would really have been more accurate.

Also worth mentioning that def (and all other special forms) are also special when you try to evaluate it in any context other than at the start of an s-expression, for example:

 (def x def)

def

(apply def [,,,])

all produce an error.

That you can’t pass a special form to apply makes perfect sense since the evaluation rules of each special form are unique to that form, and can’t be known to apply (an ordinary function).

But, and this is very useful to know, you can rebind special forms in both let blocks and receive them as function parameters. Whether or not this is good practice or you want to is another matter, but there are times when this can be useful.

(let [def :something-else]
   def)

(fn [def]
,,,
)
8

Are you talking about Clojure or ClojureScript? I believe in ClojureScript special forms are only special as the first element to an s-expression.

For example one can do:

(def def 10)

def
;;=> 10

In ClojureScript.

So in ClojureScript the symbols for special forms are just normal in all cases except when used as the first element in an s-expression.

In Clojure though I believe you are right.

I disagree, the first element of an s-expression is not necessarily a function, it can be a macro for example, or a special form. A special function would be less accurate because they are not functions, which is also why you can’t pass them to apply, don’t implement IFn, and don’t have function semantics.

The way I see it, a form is any unit of evaluation, a special form is just one that has special evaluation semantics in some circumstances. (Also reserved to the language and cannot be user defined)

Edit: It makes me think, when macros treat some symbols specially, I wonder if it be a good name to call them special macro forms? As opposed to Clojure special forms which would refer to the standard Clojure ones, but like they are basically the same thing but in the context of a macro.

9

Thanks! Those references are very helpful.

10

Both - I tried your code and my code in both REPLs and get the same results in both. (BTW - if anyone tries @didbus’ re-def of def, call (ns-unmap 'your.namespace 'def) to undefine it.)

When we talk about “special”, we mean that they can only be evaluated in the first position of an s-expression and nowhere else (attempting to evaluate anywhere else will result in a syntax error or an undeclared var warning in Clojure and ClojureScript respectively) , and that in that situation, arguments are not necessarily evaluated in the same way as they are normal functions.

Ok, I have no problem with that. Perhaps “special action” would be the most general term, with both functions and macros being actions. But it’s a moot point as they’re called special forms, and we just have to accept that.

By default I treat all macros as special, since more often than not they will have some “special” treatment of forms that you pass them. In fact I would argue that in most cases if they don’t provide some special treatment then you could probably get away with just writing a function.

11

Interesting, works for me in the self-hosted ClojureScript REPL. Maybe it doesn’t in a non self-hosted REPL.

True, but now you’re using the word special as its general english definition. To my understanding, the “form” is anything that can be evaluated. Now in the context of an s-expression, which are themselves forms, there as some that have function evaluation semantics and there are some that have macro evaluation semantics. Those are both “normal” evaluation semantics in the context of Clojure. Like the semantics are well defined, and they fit within the language model so the user can even define custom ones. But there are a few which have neither function or macro evaluation semantics, and those don’t have consistent evaluation semantics amongst themselves, thus they are each a “special case” and are called “special forms”, because of that.

Now the question is, what is the form which is “special”? Is it the s-expression that begins with a special symbol like:

(def a 10)

Or is it the symbol itself which is special:

def

And if it’s the symbol itself, is it special in all situations, or some scenarios doesn’t make it special? I think you pointed out it’s not special in a let binding, so it seems it isn’t special everywhere.

I’m not sure of the answer, I think in Clojure the symbol is special when auto-dereferenced by Clojure. So wherever it is used unquoted.

In ClojureScript I’m no longer sure, it seems it depends if using self-hosted ClojureScript or not.

12

We’re in agreement that a form is an expression. A form will have evaluation rules based on what it is.

A symbol is generally bound to something else, e.g. a value or a function, so evaluating just the symbol in any context will result in what it is bound to.

Keywords, booleans, numbers, strings evaluate to themselves. Vectors and hashmaps also evaluate to themselves.

Now if we try to evaluate a list, the first item is assumed to be a callable, hence its value is looked up and that is used to call the rest of the arguments. If the callable is an ordinary function, the remaining arguments are evaluated before being passed to the callable. If the callable is a macro, the arguments are passed in exactly as they are without being evaluated, and something is then returned by the macro, usually new code.

Now there is a small handful of “special” symbols, such as def, if, loop, quote and so on which do not fit this evaluation model, hence they are called special forms. My observation was that calling them “special forms” can be confusing, especially to beginners. The main things to be aware of about these special forms is that each special form has its own custom evaluation model with respect to the arguments you pass into them. You cannot write these using ordinary functions, as their functionality happens during compile time, whereas functions evaluate at runtime.

The way I see it, macros allow you to write custom functionality which is a lot closer to special forms than to functions.

In terms of evaluating the symbols which refer to special forms, they can only be evaluated as the first item of an s-expression, and nowhere else. Which is exactly the same case with macros.

Not quite, I said you could rebind a special form symbol within a let binding or a function binding, and you then pointed out you could also do this at a global level using def, at least within a namespace. So you can rebind a special form symbol, but you can’t evaluate it anywhere else except as the first item of an s-expression.

13

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.