Worried about types + Clojure Elitism


#1

Rich Hickey spoke again recently, doing the keynote at Clojure/conj. While not quite up there with classics like “The value of values” or “the language of the system”, it was a really interesting talk, providing some welcome context about Clojure’s design. If you haven’t watched it yet here’s the link:

For me there were two things that really stood out. One was Rich’s assessment about static typing and his reasoning against it. Lots of people have jumped on this aspect of the talk, either in agreement or disagreement. I think Eric Normand of Purefunctional.tv has really provided the best write up. Having worked both as a professional Haskell and Clojure dev he’s well placed to comment on this.

But the other thing that struck me was a more off-hand thing. Rich asked how many people had only been programming for less than $x years (5 maybe? I’d have to rewatch the video). You don’t see the show of hands but from his reaction it seems there weren’t many, and he jokingly concludes that Clojure is a language for “grumpy old programmers”.

The idea seems to be that Clojure only makes sense to people who have felt the pain of other languages, who have matured as programmers, and have gone through the enlightenment that led them to Clojure. I’m sure this is true for many people, but there’s another aspect to this. If after ten years there are almost no beginners at your main conference, then maybe there’s a problem with your outreach.

This goes hand in hand with a certain “elitism” I notice in some Clojure circles. A limited amount of patience, a limited amount of willing to indulge young people who are trying to learn. A “be quiet now the adults are talking” kind of vibe.

Luckily there are great counterexamples. ClojureBridge and everything around it comes to mind. I also think this is a little less true in ClojureScript, because it naturally draws a younger crowd with affinity to JavaScript.

What do you think of this? Have you experienced or witnessed this elitism? Or am I imagining things?

Looking forward to your thoughts!


#2

I’m not sure “elitism” is exactly the right term for it, but I do worry (increasingly so as time passes) that Clojure might not be as immune to the “LISP Curse” as many of us thought (hoped?). This most recent keynote from Rich only deepens that worry for me.

I am only, myself, an amateur observer of programming languages and programming language communities, but my understanding of the LISP Curse has been, in short, that LISPs are so powerful and so elegant, that any individual developer using a LISP can code circles around any other developer using some other language. Unfortunately, this also leads developers to prefer coding as individuals, forgoing the organization of teams that other languages often encourage.

Quick: name a Clojure library with more than 3 core contributors.

I think this is Rich’s biggest oversight in his keynote. Nothing he says in wrong, per se. No, static types don’t fix logic bugs. Yes, types can cause unnecessary coupling. But here’s the thing about coupling. Developers would say that coupling, where changing one portion of a codebase affects a distantly related other portion of the codebase, is a bad thing. I wonder if Programming Managers would say the same? Coupling does slow down individual developers, but it also forces individuals on a team of developers to communicate with each other.

In a perfect world, we wouldn’t need a flaw in language design to force us to talk to the other developers on our team, but we don’t live in a perfect world. Until we can come up with some other, better way to encourage collaboration and communication within a team, types and coupling seems to be what we’re stuck with. I have high hopes that clojure.spec might just be that “better way”, but this remains to be seen…


#3

This is interesting. I really hope clojure doesn’t fall victim to the lisp curse. Maybe we should purposefully introduce some coupling and complexity in popular clojure libraries :smile:


#4

https://github.com/clj-time/clj-time/graphs/contributors (80)

https://github.com/aboekhoff/congomongo/graphs/contributors (28)

https://github.com/clojure-expectations/expectations/graphs/contributors (13)

https://github.com/clojure/java.jdbc/graphs/contributors (12 – but that doesn’t include many patches submitted by the community, under the Contributor’s Agreement)

https://github.com/clojure/tools.cli/graphs/contributors (8 – and, again, there are probably patches not included)

https://github.com/boot-clj/boot-new/graphs/contributors (7)

That’s just the first half dozen I could think of – that I maintain myself. Now, how many of those would you consider core contributors? More than three for at least some of those, so I think it’s pretty easy to meet that challenge.


#5

Maybe it’s not surprising that I put a lot of blame for this on tooling. Of course another big chunk of blame goes to outreach, and another to the dynamics of the broader programming world (JAVASCRIPT IS EATING EVERYTHING), but it’s legitimately difficult to get started with Clojure tooling. I’m happy people are working on this in a bunch of different directions: the clj command line tool, Parinfer (even though it’s not for me), maria.cloud.


#6

I think a lot of the early adopters were used to rough-edged tooling. That’s why Emacs and Leiningen held sway for so long. “Cranky old programmers” will put up with bad tooling for a great language, and have very little incentive to improve the tooling. Look at how terrible Leiningen was on Windows for so long! Look at how terrible Boot was on Windows at first.

It takes a critical mass before tooling improves, and it takes a critical mass of younger, less tolerant programmers to move that needle.

I actually think Boot is amazingly beginner-friendly (since it can be used from the command line without writing any arcane files). We have a great Clojure experience in Cursive/IntelliJ now. I think ProtoREPL/Atom is also a great experience (for Clojure anyway – ClojureScript still seems horribly fussy).

As you say, we have clj coming along nicely – but having that from day one (or at least day one thousand) would probably have helped adoption.

I actually think the JavaScript ecosystem is worse: the tooling is so fragmented and constantly changing, library version changes cause huge cascades of failures and cost a fortune in dev time figuring out compatibility issues. At work my team writes all our server-side code in Clojure and we have a front end team using React.js etc – and in our daily standup I am often horrified at the problems they run into with tooling and ecosystem issues on a regular basis.


#7

I think a lot of the early adopters were used to rough-edged tooling. That’s why Emacs and Leiningen held sway for so long. “Cranky old programmers” will put up with bad tooling for a great language, and have very little incentive to improve the tooling.

100%! This is certainly true for me. I have my method, it works, and frankly I’m just not that interested in fiddling with it. But I only achieved it through a long iterative process of mentoring when I first came on board.

I actually think the JavaScript ecosystem is worse: the tooling is so fragmented and constantly changing, library version changes cause huge cascades of failures and cost a fortune in dev time figuring out compatibility issues. At work my team writes all our server-side code in Clojure and we have a front end team using React.js etc – and in our daily standup I am often horrified at the problems they run into with tooling and ecosystem issues on a regular basis.

You’re right. I didn’t make my reasoning clear :slight_smile: I have the same experience working alongside React folks. But I think it will take time for the dev-experience gap between Clojure-land and JS-land to make a difference. One reason is that the ClojureScript tooling story only matured within the past couple years, even though the building blocks have been around for a while longer. (Thus, David Nolen’s recent call to evangelize of CLJS within the broader JS community.) Another reason that I think we’re not seeing more adoption than we are is that plain JS will still be the tool new devs reach for. There is a degree to which devs have to feel the pain to see the benefit of a better way that has less mindshare and might require a little more setup and an additional layer between oneself and the problem.


#8

same goes for me. it’s amazing how many of us, although adopting new languages, are very rooted in our ways :slight_smile:


#9

I’m very set in my ways, but those ways were extremely open to change when I took up Clojure. It’s like John Pither said:

The best time to learn a new editor is when you’re learning a new language.

There’s something to be said for leveraging that precious, short period where everything is new and difficult. Of course for some people or some circumstances the opposite is true: “only one new thing at a time”. But it’s helpful to understand the dynamics of workflow changes.


#10

If I may relay my own experience as a newcomer to CLJS, and to compare it with Elixir, which I learned and used almost a year ago to this day.

I started dabbling in Clojure with the objective to use ClojureScript to replace all uses of Javascript in my projects. It was CLJS, Reason or Dart, and CLJS won out. I’m not yet interested in server-side Clojure.

What I liked when I tried out ClojureScript a few weeks ago were the Leiningen templates - this helped me bootstrap a new project to get my feet wet without having to copy-paste boilerplate. This should be promoted more in the official webpages. You can choose to skip Leiningen or Boot later.

Just by using the re-frame template, I could get a fantastic environment out of the box, and with a few tweaks to Spacemacs, I could get light years ahead of the equivalent Javascript or any other web development tools.

This is quite similar to what Elixir with Phoenix offers: a set of generators that give you access to a “killer app”. In the case of Elixir, the killer app is Phoenix, as it “unlocks” functionality that is quite difficult to attain using other dynamic languages.

I think ClojureScript’s tooling is already a killer app. Re-agent and re-frame are already mature tools with very good documentation and adoption, so I’m sure you’ll see more people joining the community for those alone.

I had evaluated Clojure a few years ago, and the syntax, plus the immutable data structures scared me away. This time though, I learned the immutable data structures from Erlang, and the syntax isn’t that scary any more :slight_smile:


#11

Agreed. The reason my company has a React.js front end, built by JS devs, is that we tried ClojureScript for an internal-facing tool a few years back and, while we got a proof of concept working – first in Om, then in Reagent – it was clear to us that we couldn’t use the tools and frameworks available then to build a complete SPA for our member-facing apps.

I think our JS team are interested in ClojureScript, but I doubt we’ll get a chance to rewrite our SPA in it. We may revisit the choice of front end tools when we replace our internal-facing admin console (maybe next year), but I suspect we may just go with server-side generation of HTML from Clojure.


#12

This thread hasn’t been talking much about types and elitism on “the other side”. I see some people on social media that are smarter than me who are moving into the other direction: Clojure -> Haskell, Elm, PureScript etc. This makes me wonder and even worry, am I missing something by having Clojure as my main language? There are moments where I feel “cowed by the proof people”, like Rich says towards the end. In this talk they go so far as to say that not relying on a type system is unethical. Really? I’m starting to feel guilty already for writing Clojure.

I really liked that Rich emphasized in this talk the goal of software is not to satisfy some mathematical properties (a type system), but to have an effect in the world. You write software to help people or to replace people (to let them do more interesting stuff). Next time I feel “cowed”, I’ll try to remember that. Am I serving other people with the energy I’m putting into my software, or am I solving interesting puzzles?


#13

Thanks for the list! I was being slightly facetious, but I’m happy to have been so easily proved wrong.

When I wrote that I was just coming from having looked at options for including an Etag header on resources served via Ring. What I discovered was that the semi-official middleware hasn’t been updated in 7 years and is still at version 0.1.0-SNAPSHOT. Now, I totally understand the “it’s finished” mentality for libraries, and it’s not like Etags are rocket science. It’s mildly annoying that the version remains 0.1.0-SNAPSHOT. I’d prefer, if a library is “done” that the author at least remove the “-SNAPSHOT” if not label it v1.0.0, but even that’s not that bad.

What really irked me is that ring.middleware.etag only generates Etags for string and File objects, while Compojure’s route/resources returns static assets as stream objects if your app has been packaged as a JAR (e.g. by running lein uberjar). This is trivially fixable, if you know about multi-methods, by providing your own implementation for ring.middleware.etag/calculate-etag, and indeed from looking around it seems that most everyone has either done this or simply implemented Etags on their own.

I’ll be the first to admit that I’m as much to blame as the next Clojurian, as I didn’t bother to send a pull request, fork ring.middleware.etag, write a blog post, or anything else. If I were to offer an excuse, it’s that, as an experienced Clojure programmer, the amount of effort required to fix this on my own is an order of magnitude less than what would be required to fix it in such a way that contributes back to the community. That said, for a beginner this could potentially be a major road-block, especially considering that ring.middleware.etag would appear to work in development (since ring.util.response/resource-response will return a File when serving from a local filesystem, as in lein ring server) but break only in production (if deploying as a JAR, since resource-response will then switch to providing a stream).

The one bit of hope I take from this experience is that this clearly seems to be the sort of thing that clojure.core.spec is designed to catch (though you’d have to be running checks in production, or at least some sort of staging environment). On the flip-side, in a language like Haskell, where resource-response would have to return a product type covering File and InputStream and calculate-etag would only have implementations covering string and File, the compiler would immediately be able to identify the issue.


#14

Indeed. We all need to be better FOSS citizens.


#15

I know what you mean. Some Clojure developers I used to look up to totally jumped ship because of types. When people whose opinion you respect disparage your favorite language it can cause a mini existential crisis. Like “am I living in blissful ignorance?” and then a video like this comes out which is comforting to know you are not alone in your enjoyment of the language. Then people post negative responses to it because he dare say anything about type systems. Then you have positive responses to the video to which some people respond with things like “oh these people just blindly follow their dear leader” You hear words thrown around like “cargo cult” and “cult of personality”.

But then you remember how much fun you have programming in Clojure and you accept that if dynamic typing is wrong, then you don’t want to be right. You’re happy, your customers are happy, and if you ever find one those things not to be true then you’ll reevaluate your choice of a main language.

I, for one, am an unapologetic Clojure developer.

I love learning knew languages but I also love learning knew paradigms and different ways to solve problems. Clojure gives me that.


#16

This struck me too. It prompted me to ask Rich Hickey about it:

It might be self-serving to believe that this is wisdom (since I am a Clojure booster). Intellectual honesty, however, forces me to acknowledge that I might be influenced by nostalgia. Nostalgia for previous Lisps. I’m also inclined to agree with Rich Hickey on this point, because I agree so violently with him on so many other points.

I watched Snively & Laucher’s Strange Loop talk linked later in the thread. Seems like they are guilty of elitism too, slamming Ruby programmers, claiming programming with neither static typechecking nor TDD constitutes malpractice. I didn’t hear any actual support for either of those assertions in the talk.

That being said, I work on a big-ish Java system in my day job. Just yesterday I had to do a refactoring converting some horrendous constructors with up to 20 (positional) parameters, to static (factory) methods. Some needed their parameters re-ordered.

In the old days before IntelliJ refactorings, I might have done this w/ Emacs macros and regexps. What now takes 30 seconds would have taken me a morning.

The point is well taken, that you simply shouldn’t have positional parameter lists longer than say 3. But I think so many people are used to big messy Java codebases, I think it’s hard for them to imagine life without this sort of refactoring.

And this is an example of a larger problem: that until you’ve implemented a real system the “Clojure Way”, it’s hard to really have a well-founded position on Clojure’s approach to types vs your favorite other approach.

It’s Paul Graham’s “Blub” problem all over again. If you haven’t done it, it’s really hard to envision it.


#17

Hi folks, I write The REPL which has a weekly section titled “People are worried about types”. I’m not sure if that’s what’s being referred to in the title of this thread? Here’s last weeks letter as an example. I have an explanation I link to on the “?” going to Why are people worried about types?, which explains the title of the section.

I use it as a joke, and somewhere to put spec/type related Clojure things. I hadn’t considered it before but I can see how you could take it as being dismissive of types. That’s not my intention at all, I think types are neat and useful.


#18

The thing is - there’s a multitude of people, projects, environments and combinations of thereof. In some circumstances dynamically typed languages work great. In others - not so much.

If you have a team which:

  • feels comfortable using advanced statically typed languages
  • knows their way about building powerful abstractions without getting bogged down in type/category theoretic ivory towers or language-imposed minutiae
  • knows their end result in terms of business value

let them use Haskell/Idris/Scalaz. I strongly believe the systems they deliver will contain a lower number of defects than similar systems produced in dynamically typed languages. I think it’s definitely worth the time to learn some Haskell, get comfortable with modelling through types and understand how the “statically-typed” people think even if you program and love programming in Clojure. It only adds to your toolkit, never takes away.

Personally, I’ve been doing Clojure for a few years. It really shines when you’re in a small team working on data pipeline-like tasks. Which are most of them in the enterprises if we’re being honest :slight_smile: I love many things about Clojure. I love doing UI with Hoplon, not having to think about all the crazy componentDidMount lifecycle hooks and Form 1/2/3 component types. I love using Mount to structure application state on the backend. I love building asynchronous pipelines with Manifold so much I’ve ported it to Clojurescript. I love being able to read the source code for any library while not having my head explode due to the layers of abstraction (OK, this doesn’t apply to core.async, but that’s why I use Manifold :)). I love using the same language to automate build and deployment processes with Boot. I love not having to worry if the next Clojure version upgrade will break my code. I also love the fact that many people hate all of the above and use Lein, Re-frame, Core.Async and Component instead.

However, recently I had to do more numerically intensive code which naturally assumes algebraic structures (semigroups and the like). This sort of thing benefits immensely from type-driven development and a Haskell-like pseudocode is the first thing I reach for. Once the project grows, no matter how well you have divided and subdivided the pieces, the confidence that you get from the compiler telling you everything is fine is way higher than the test suite can provide, even if you’re doing extensive generative testing. “What if we’ve missed something?” - at least that’s how I feel. It’s true that there’s a huge hurdle of all the monad transformers, profunctors, yoneda lemmas and other scary “patterns” to learn in order to be fully productive. If that’s what it takes to make less defects when the cost/quality/scope allows - I think it can be worth the pain.

Anyway, I think that Clojure is the best thing there is on the JVM, for now. Hopefully Eta-lang will catch up and become a production-grade competitor as I’ve lost my hopes for Scala ever becoming pleasant to use.


#19

I think there’s a lot of benefit to being a polyglot. When I first discovered Frege, it rekindled a desire to re-learn Haskell again (I’ve lost count of the number of times I’ve “learned Haskell” over the last two and a half decades!), and the first thing I did was write a Leiningen plugin to compile and run Frege code so I could experiment with mixed language Clojure/Frege code.

I’m currently trying to learn Kotlin (“learn a new language every year”) and, again, the first thing I did was write a Boot task to compile and run Kotlin code so I could experiment with mixed language Clojure/Kotlin code (and partly so I can easily compile my Kotlin examples and fire up a Clojure REPL and try them out – boot kotlinc repl :slight_smile: )

Over the last decade or so I’ve learned Groovy, Scala, and Clojure and taken them to production in mixed language environments. I’ve learned several other languages that have been just for fun projects.

Despite all the languages I’ve learned over the years – and the large subset of those languages that I’ve used in production – I still gravitate to dynamically typed languages but I have good friends who’ve always gravitated to statically typed languages. I know I’m not missing out – my friends and I just think differently about problem spaces. That’s why programmers have so many different tools.

(and I’m a mathematician by education and did category theory etc at university – so it’s not like I haven’t gone deep on it at some point past)


#20

I disagree. I’ve been making stuff for Clojure beginners for nearly 5 years now. I’ll be the first to admit that my projects aren’t very widely used – the “market” for novice Clojure tools is incredibly tiny. That never bothered me (i make things for my own edification), but i don’t think it has to do with a lack of outreach.

Think back to when you were a beginner. What made you choose your first language? Chances are, you didn’t really choose it at all. For me, my first was C++, because i took a class in high school. In my free time, i started learning PHP, because the LAMP stack was so ubiquitous i thought it was the web.

If a young person today wants to make games, they’ll probably learn C# (unity) or GML (gamemaker). Beginners aren’t equipped to compare languages. They will come across some framework or tool that decides their language for them. Outreach is great but to get numbers you really need a killer app.

That may happen for Clojure at some point but it hasn’t happened yet. For now it’s the kind of thing you learn to gain a new skill (FP) which means mostly intermediate/advanced devs. I don’t think that spells doom – a lang can be healthy that way. Just keep making cool stuff and who knows, it may end up being our killer app, and i’ll be there reaping the benefits! :sunglasses: