My latest blog post, Array programming for Clojurians, grew out of a discussion on the Clojurians Zulip about lessons APL and related languages have for Clojure. It was fun to take a brief tour of their world and find some the analogies between the two.
9 Likes
Fantastic write up. The historical context adds such clarity. I wish more programming concepts were taught this way.
The exploration of array languages (as opposed to Common Lisp or a stack-based language like Forth) made me think of this quote:
I will, in fact, claim that the difference between a bad programmer and a good one is whether he considers his code or his data structures more important. Bad programmers worry about the code. Good programmers worry about data structures and their relationships.
~ Linus Torvalds, Re: Licensing and the library version of git [LWN.net]
I found it striking that Blazeski reached for macros and you avoided them completely (as a Clojure programmer often does). It made me wonder if this cultural difference is born, in part, from the fact that Clojureās collection offerings (vector, list, map, set) are both robust and syntactically clear out of the box.
From Blazeskiās own conclusion:
Learning anything of APL, J or q makes programmers aware of opportunities opened by thinking at the higher level of container abstraction.
It has been many years since Iāve spent significant time in CL, but Clojureās vectors just seem so clearly differentiated. It makes it easy for me to think about what abstraction to reach for.
Furthermore, you use polymorphism where Blazeski uses a macro. Could it be the weight of CLOS that again inspires CLās use of macros over a polymorphic solution?
If we presume that matching the data structure to the data itself is of preeminent importance, having clearly defined data abstractions with idiosyncratic operators can lead to clear thinking and clear code. The decision between developing a domain specific language vs. using a Lispās core features is debatable. But trying different data abstractions/operators (like array-based languages) can lead to new ways of thinking. Pretty cool to see Blazeski hedge towards a DSL and you work with core language features.
3 Likes
I actually expected to need macros on first skim of the article. But then for each task it quickly became clear that there just was no need.
Iād like to see a Common Lisper chime in on a potential CLOS approach.
I think youāre absolutely right about Clojureās collection API making macros less necessary. You can see this fairly clearly in this 2002 common lisp discussion where someone suggests precisely that approach of abstract data types with underlying implementations that switch according to size of the data. Another contributing factor appears to be that CL devs were simply operating at a different level of abstraction in those days: they want to implement that layer themselves, just the way they talk about implementing their own hash functions or how they just love doing everything from scratch with cons cells. Most developers today donāt do that in their day-to-day.
So far as I can see, the original article only calls out the argument hoisting macro f as a macro? I imagine he thought to do it that way because he didnāt want the arguments to be evaluatedābut who can know the heart of another programmer?
As regards the rest, the Clojure apple has not fallen far from the Common Lisp tree in regards to generic function dispatch:
(defgeneric j+ (x y)
(:method (x y)
(error "Cannot add those types!")))
(defmethod j+ ((x number) (y number))
(+ x y))
(defmethod j+ ((x vector) (y vector))
(map 'vector #'+ x y))
(j+ 1 1)
;; => 2
(j+ #(1 2 3) #(2 3 4))
;;=> #(3 5 7)
(Not also that CLās vectors are distinct from lists and have their own reader form.)
1 Like
Calling Gerry Sussman into this conversation to re-enforce your argument:
The Structures and Interpretation of Computer Programs (SICP) curriculum no longer prepared engineers for what engineering is like today. Sussman said that in the 80s and 90s, engineers built complex systems by combining simple and well-understood parts.
Today, this is no longer the case. Sussman pointed out that engineers now routinely write code for complicated hardware that they donāt fully understand (and often canāt understand because of trade secrecy.) The same is true at the software level, since programming environments consist of gigantic libraries with enormous functionality. According to Sussman, his students spend most of their time reading manuals for these libraries to figure out how to stitch them together to get a job done. He said that programming today is āMore like science. You grab this piece of library and you poke at it. You write programs that poke it and see what it does. And you say, āCan I tweak it to do the thing I want?'ā. The āanalysis-by-synthesisā view of SICP ā where you build a larger system out of smaller, simple parts ā became irrelevant. Nowadays, we do programming by poking.
The example of generic function dispatch in CL is appreciated. So similar!
I was particularly interested in Blazeskiās speculation a little later:
Writing a macro that understands forks and hooks is an exercise for a beginner ā if the operators covered are strictly monadic and/or dyadic as are those in J. However in Lisp all those optional auxiliary and keyword arguments immensely complicate matters. In order to make it operational within a month or so of effort a tacit macro could be implemented to work just on a subset of Lisp operators.
My only point of comparison are the macros in core.matrix. I havenāt programmed in J, but core.matrix ends up feeling very Clojure-y. It also looks to use macros in edge cases.
Blazeskiās article ends by speculating on what it would take to build something that felt more like J in CL.
Much has been written about how language shapes our thinking. Without opening an argument on the power of Common Lisp vs. Clojure (the value of reader macros, lisp-1 vs. lisp-2, etcā¦), it is striking that the thinking is so different with such a similar language.
Clojure comes out of a post-Windows world that thinks about APIs. Common Lisp comes from a pre-Windows world that thinks about DSLs.
1 Like
Ah, so far as I can tell heās gesturing toward combinators here (possibly without realizing it).
Iām having trouble figuring out what you mean. One of the first APIs I learned pre-dates Windows by a fair few years, and it seems to me that all Lisps are interested in both APIs and DSLsā¦?
CL is essentially silly puttyāyou can make anything with it in any shape you like. Another CLer might well have written approximately the same thing that appears in the Clojure version from the blog post. (That one can produce anything from GOTO-laden spaghetti to functional pearls in CL is both a strength and weakness of the language.)
Iāve always had mixed feelings about that snippet. Of course Sussman knows what heās talking about, and as you say itās even part of my point. But though itās definitely less common to drop into the internals of things nowadays, itās still absolutely necessary sometimes. And of course even when itās not necessary to develop something from raw materials, itās still usually helpful to understand. Programming by poking only goes so far; some number of developers need/want an understanding (in at least one domain) perhaps not from electromagnetics, but at least from hardware all the way up to product.
2 Likes
Fair enough. Iāve always considered the Clojure programmerās perspective on writing macros a departure from the Lisp tradition. Perhaps this is unfair.
As a point of clarification, I was suggesting that language may not have so much to do with how approaches to programming have changed.
Microsoft built a closed-source software API that drove 90% of all hardware for over a decade - an unprecedented event. I believe this had a tremendous impact on how a generation of programmers approach the craft of writing software. As much as any advance in computer science or language specification.
Iād love to write a book on the impact of the Windows API on both computer science and computer programming. Sort of in the spirit of MITās Platform Studies. But thatās for another lifetime.
2 Likes
Is it me, or does this feel like a backhanded criticism to the entire field ?
I had a coworker tell me the other day, at our company, we build robust systems out of twigs and haystacks.
Its somewhat true, but also that was very much a complaint of his to show that weāre just totally dropping the ball and that there doesnāt seem to be any motivation for any kind of careful consideration in building real robust systems. But there was also some hopelessness in his tone, and some abandonment to this faith, as out of his control and inevitable.
Iām feeling a similar tone to that Sussman quote, am I the only one? Is this me projecting my own feelings to it?
1 Like
It certainly reads like a criticism to me.
But I donāt suppose there are many code-slinging veterans who would suggest there was a golden era where engineering quality was prized in the face of various market pressures.
Itās important that the original quote includes Sussmanās view on intellectual property. How can you do anything but poke at something that youāre not even allowed to understand?
Every programmer is eventually forced to work with systems they donāt understand. That hasnāt changed. But do you think the techniques for engaging the unknown have improved? Gotten worse? About the same?
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.