Why Clojure over Python?

clojure

#1

I would like some cogent discussion on the following question that I am faced with regularly:

Python has many advantages; why use Clojure?

Python advantages (the ones most often facing me)

  • Familiarity: used by many colleagues and in the community (academic + datascience)
  • Ecosystem: well-established libraries like nltk and scipy
  • Relative easiness: a language emphasizing readability

Clojure advantages

  • Functional (writing & expressivity advantage)
  • Performance (handles multi-threading better, JVM beats non-C Python)
    • Not often relevant in this field, though
  • spec generative testing – useful
  • Lisp (structural editing, full REPL)
  • Ecosystem: Java + Javascript at your disposal

What would you add as selling points of Clojure in this debate?


#2

I always heard as an argument, that both (python and R) are not well perceived “in production”.
Not sure, how far this is true.


#3

This is not a very technical argument, but for me, programming in Python feels like programming with a straightjacket on. For the same “kind” of language, I’d rather use Ruby.

Otherwise, I think once the lispy-ness clicks, and if you’ve done functional programming in the past, then Clojure just is a better fit (at least for my mental model of computation).


#4

(Better “performance” means Clojure does not need “extensions” in C.)


#5

Allen Downey’s wonderful book, “Think Python”, suggests making a copy of a list if you need to be sure it won’t change. How backward! In Clojure, everything is safe. No “safe copies” needed.


#6

Does someone here use Django? How does it stack up next to Rich Hickey’s “Simple made easy”?


#7

For me the main selling point (aside of the thought-through design of the language) is interactive development, with REPL connected to the live, running app and developing it while it is running, getting immediate feedback. Even if your app restarts quickly upon code change, you likely loose state, so it doesn’t give you the same interactivity as Clojure REPL.


#8

So, let’s see what can even differentiate two language which are both Turing complete?

  • Performance
  • Scalability
  • Memory use
  • Correctness
  • Speed of delivery
  • Speed of change
  • Supported platforms
  • Fun

Now let’s compare against Python:

  • Clojure is more performant, but will start slower.
  • Clojure will scale quite a lot more, both in cores on a single machine, or distributed.
  • I’m not sure, but I think Python might actually better utilise memory. Though I might be wrong, I’ll say this is a tie possibly.
  • I would argue Clojure tends to more correct code, due to tbe nature of FP, immutability and the REPL. But this is debatable and hard to demonstrate one way or another.
  • I think both Python and Clojure are quite productive languages, with lots of existing high quality libraries, so I’ll say this is a tie.
  • I’d argue Clojure code bases are quicker to evolve and change over time, again, mostly due to the nature of FP, immutability, the REPL, and the strong emphasis on simple designs. That said, this is also hard to demonstrate, so let’s call it a tie.
  • I think Clojure maybe wins in supported platforms, because it can run in the browser as well with ClojureScript. But, that’s also arguable, so let’s make it a tie, since Python runs on quite a lot as well.
  • Well, Clojure is just way more fun than Python. I mean, is there any language more fun to work with than Clojure?

There we go, Clojure beats or ties with Python on all aspects, so I guess… It’s overall a better Turing complete language.


Beginning FP knowing OOP and more
#9

I think that Clojure provides a number of advantages both in terms of language design and ecosystem, especially in the domain of services and web applications.

In terms of ecosystem I would say that having Clojure and ClojureScript is a clear advantage. You can use the same language on both the server and the client. You get shared code, shared tooling, and shared data structures. All the devs on the team are now also able to work with the entire code base, and you can easily move logic between frontend and backend.

Since Clojure doesn’t have popular server-side frameworks, it’s a common pattern to manage the UI with client-side frameworks like re-frame and implement the server side as a stateless service. On the other hand, Python frameworks tend to be focused more around server-side rendering. Treating the server as a stateless API makes it easier to add other clients such as mobile apps, and to scale horizontally.

Another piece of tooling that makes Clojure development more productive is the editor integrated REPL. Any code you write, you evaluate in the REPL straight from the editor. The REPL has the full application state, so you have access to things like database connections, queues, etc. You can even connect to the REPL in production. So, say you’re writing a function to get some data from the database, write the code, run it to see exactly the shape of the data it produces. Then you might write a function to transform it, and so on. At each step you know exactly what your data is and what the code is doing.

Having a live development environment makes natural to do exploration. If you’re solving a problem and you don’t know the best approach, it’s easy to experiment by running the code as you’re writing it. You can quickly try a few different approaches and settle on one that works best. This also affects refactoring, as you can immediately test what you did. The REPL is also very helpful when you’re looking at new code you’re not familiar with. By running things as you read them you can confirm that you understand the code correctly.

I think that dependency management story in Clojure is much better. And packaging applications as uberjar results in a single artifact that only requires the JVM to be run. It’s basically like having all your apps running in containers for free.

Clojure also has a number of advantages as a language. I would say the main feature is defaulting to immutability making it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don’t need to track types across your entire application, and you’re able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components.

Next, immutability makes it possible to work with threads in a sane fashion. Clojure provides excellent parallelism and concurrency primitives out of the box. Python provides nothing comparable in this area.

Thanks to immutability Clojure APIs tend to be data driven. API functions take data as arguments, and return data as a result. You don’t have to worry about any hidden behaviors like you do with objects because data is inert.

This kind of isolation also tackles a lot of the same problems as microsevices. One of the main motivations behind microservices is to reduce coupling. With Clojure you can think of each namespace as its own microservice. You have a set of functions that you call, each function takes an input and produces an output. So you have similar benefits during development time, but you don’t have to pay the cost of a orchestrating many services in production.

I recently saw a Cognitect presentation where they used Pedestal as an example application. One slide shows that over 90% of the code is pure. All the state and IO accounts for less than 10% of the application. This is a completely normal situation in Clojure. [1]

Meta-programming in Clojure is strictly superior to what’s available in Python thanks to s-expressions. You can take any piece of code and transform it like you would any other data structure using the same language. Being able to treat code as data also means that you can serialize and load code for free.

Immutability and s-expressions combined lend themselves well to refactoring. You just select the expression you want to factor out, and pull it out into a function that takes the same arguments.

Finally, I think that s-expressions provide superior editor experience thanks to structural editing. When you’re manipulating Clojure code you’re thinking in terms of blocks of logic. When I use other languages I tend to think more in terms of lines of text.

[1] https://www.youtube.com/watch?v=0if71HOyVjY


Server side as a stateless service vs server-side rendering and REPL help developing a project
#10

Excellent answer, Dmitri. Thank you. It seems like a fair summary that Clojure lends itself to better-engineered software, better deployment story, and better development story.


#11

Mentioned in passing above, but worth emphasizing: There are Java libraries for nearly everything. In many cases it’s utterly trivial to use them from Clojure. Even if you don’t use Java libraries often, when you need one and it’s available and it’s easy to use, that makes a big difference. Build tools (I use Leiningen, but I would guess that others are similar) make it very easy to use Java libraries–even ones available only as source.

Also mentioned above and worth emphasizing: fun. I discovered that on my own, and then found others mentioning it independently. One of the reasons Clojure is fun is the thoughtful design of many functions that makes it so convenient to use them together: What makes Clojure fun is what makes it more productive to code in Clojure. Also, a lot of the thoughtful design of functions is intimately tied to the FP-orientation of Clojure. So FP is part of the fun and productivity.


#12

I’ve been using Python professionally for about 6 years now. It’s a decent language and it’s cool seeing how it progresses with so many people behind it. That said I’m trying to move our Django app into smaller Clojure services somewhere between micro-services and a monolith.

With Python, and most OOP based languages, you are stuck having to describe each system in play and its relationship to other systems. This commonly happens with classes, and you’re going to end up with a lot of them. The problem is most of these classes you write often just end up wrapping a list or a dictionary. However, they are not compatible with any function or method that typically works on lists or dictionaries so you have to write more classes, more methods, and create more vocabulary to make two classes talk to each other. It’s also way too easy to over-engineer in Python. Often you’ll find discussions about what MIGHT happen, how a system MIGHT change, how people MIGHT use it. That is very difficult to know so you end up making your systems more complicated in preparation that they may have to do more later. Lastly, at the end of a project you end up with all this code that is only relevant to the specific framework or how one person thinks about the software. It’s unprincipled and decisions are often made by what makes the most sense at the time to whomever is writing it or what a team agreed upon.

In Clojure and functional programming you model most of your logic as pipelines receiving data and transforming it over time to a destination such as a database, server response, API request, etc… What feels very satisfying about this approach is you’re usually writing the most minimal, simplest code to accomplish those goals directly. You focus on solving the problem at hand with code best suited for that task. When you need more functionality you can build up complex functionality by combining smaller units of work. For instance if you have a function and you compose it with another function you get a new function that can do both those tasks.

Say for instance a web server. If using Django you have to work with Django’s Request and Response classes. You will spend a few hours studying the docs to get started, then you have to consult them every time you want to make a change because you have to use some method for setting a property on the request object.

In Clojure you can write a web server using a library like Ring. What’s great is that each request handler is a function that takes a request map and returns a response map. You rarely need to consult the docs once the general concept clicks. There are like 100s of Clojure core functions that operate on maps, changes are trivial to build up your request\response pipelines. You can easily plug in another person’s ring handler and since they are just functions as opposed to code written against some Django version’s Request class or View implementation.

Often you may hear that Clojure has very terse docs, that is true and there is room for improvement but at the same time it hasn’t been addressed because it isn’t as big of a problem as it might seem. In Python, you NEED docs, lots of docs to learn how anything works. Want to see how a Django view works under the hood? Good luck. Enjoy lots of side-effects, mutable state, and glue code to make all the classes involved talk to each other. Docs are absolutely required in languages like Python to get anything done. In Clojure, you can often just look at the source and find out the functionality is implemented in less than 20 lines of code. Typically it’s good practice to document what a function takes and what it returns which may not cover every nuance, but is usually enough to get started with it.

Functional programming is pretty universal. I had spent years learning Django, and felt that I definitely understood web development. However, as soon as I tried a different language I realized I barely had any idea what I was doing. All I had done was learned one tool, the concepts don’t translate to similar projects, each solution uses an entirely different vocabulary. Instead, functional programming is principled relying on pure functions, immutability, and reducing side effects. Whether you use F#, OCaml, ReasonML, Clojure, or Haskell those principles will still be in play. Transitioning is far easier because you’re mostly learning new syntax rather than a vocabulary to express problems within a specific domain.


#13

@eccentric_j, that is one of the best descriptions of advantages of dynamically typed FP (and Clojure in particular) over OOP that I have seen anywhere.

(I think that some of the points don’t necessarily apply to a statically typed language like OCaml/ReasonML, which has different costs and benefits. You get type safety, but not the same convenient flexibility. I like both approaches.)


#14

Thanks @mars0i, with a bit of refinement and a more empathetic touch this could be my next article.

That said I’m very curious which aspects don’t apply to static typed languages. Could you please elaborate?


#15

Well, it could be that I was reading in more than you intended.

All of this could apply to OCaml (or ReasonML or F#), but what I was thinking was that often there would be a bit more work involved, because you have to get the types right. That’s where I think I might have been reading in more than you said.

In OCaml you can’t just pass an array to function that expects a collection and return a lazy list (which is unpopular and rare anyway) to another collection function, and you can’t just throw together defrecords and convert them into or from maps, and on top of that you can’t mix floats and ints in your arithmetic or in your sequences for that matter. And if you actually want some kind of map, you have to decide whether it’s a hash map or a some kind of tree or something else, and use completely different libraries for each, and then maybe jump through awkward hoops to specify the types of the elements in your map.

So a bit of care would be needed to build that pipeline in OCaml, and to me it seems like some of the upfront work that you described for Python would be needed in OCaml. I still suspect that OCaml is easier in that regard than Python–though probably harder in other ways. In addition once the OCaml program compiles, you know that you will never have a type error at run time; it just can’t happen, unless you are doing funny things like calling C. What I like, also, about OCaml, is that you can usually do all of the above without any type annotations (compare that with Java), because OCaml will figure it out–though a lot of OCaml programmers specify types for every function in a separate interface file.

If I was sounding down on OCaml, I’m not; it’s a great language, and type safety and efficiency are nice things. But it’s a very different experience, imo, from writing Clojure, which I love for different reasons.

(I would guess that Haskell programming is similar to what I described for OCaml, but I haven’t used Haskell in a very long time, and I never knew it well.)

(About the fact that you can’t mix ints and floats in OCaml, there is a project in development–“modular implicits”–that will allow you to mix them, and do other cool things, but as I understand it that’s a big project that might not be done for a while.)


#16

Yup, with static types, you trade simplicity for correctness guarantees to varying degrees.

Your programs aren’t necessarily more correct, but at least you do know that they are correct in some ways, and to some people, that gives them the piece of mind they need.


#17

In order from most to least important: Immutable data structures, improved performance, better concurrency, dialects for both client and server, more sophisticated repl.

My top gripe about Python: Python’s dictionaries only permit immutable data as keys (which is smart), but Python has very limited ways of constructing immutable data (which is incredibly constraining). This severely restricts the utility of dictionaries and the ability to nest data.

The main things I miss about Python:

  1. Highly readable syntax
  2. Great interop with some high-quality C-based numerics packages

Overall, though, I’m far more productive in Clojure. It’s especially superior for any use case that one might describe as “lightweight data modeling” requiring a “data-oriented language”.


#18

Thanks for explaining @mars0i. I’m not as familiar with static FP languages as I am with dynamic ones, so it’s great to learn more about the differences :slight_smile:


#19

As someone who would like to use Clojure more often, it pains me to say it’s often hard to make the case.

In particular Python’s performance for most use cases is very good - if you use numpy you’re effectively doing computations in C. You can parallelize a lot of problems simply with the multiprocessing module. Worse, sometimes a naive Clojure implementation can be slower than pure Python.

Also compared to Python - the I/O part is still painful. Wes McKinney, the creator of pandas, himself said that a large part of the package’s popularity is the ease of use of the read_csv / to_csv functions. I’m sure that’s a barrier to entry in Clojure for many.

I think Clojure’s advantage is in live environments where you receive a lot of data asynchronously (tracking a production line, trading financial markets) - the multithreading abilities and the safety of immutable data structures then shines.


#20

It sounds you’re specifically speaking about data science experimentation use cases?

I think that would fit inside the “already available mature libraries” criteria. Which I count as contributing to faster “speed of delivery” and “speed of change”. Which I feel, depending on the problem at hand, sometimes Clojure would win, other times Python would.

That said, you.are 100% correct. Python has way more available and mature libs for data science. I don’t think you should even make a case for non production data science to use Clojure over Python, unless your team is very strong in the software engineering side and willing to pioneer the tech on its own.