Hey all.
Some years ago Zenrobotics of Finland posted Clojure jobs. I am to remember that it was one of the largest Clojure codebases (according to the post).
They now have one Software Generalist job opening that states “Our main languages for new code are Python, C++, and Javascript.”
Is anyone familiar with the story? I’d love to know more about why the shift. I would guess that the rapid rate of Python ML libraries is influencing it some.
Hi! I’m familiar with the story! I worked for ZenRobotics (ZR) as a software developer 2013-2015.
For some context, the main product of ZenRobotics is the ZenRobotics Recycler (ZRR), a robot system for sorting waste. If you’re curious, check out this marketing video. For more details, these slides have some architecture diagrams.
How did ZR pick Clojure in the first place? The initial prototype software for ZRR was implemented in Java, but to allow quicker experimenting, a more dynamic language was needed. For interoperability with the existing code, JVM support was a must. I believe that they first tried Jython, but it wasn’t very robust back then, so they settled on Clojure.
By the time I joined in 2013, ZR had one of the biggest Clojure codebases around. It was somewhere between 150k and 200k lines of Clojure, covering all the big components including the motion control and the machine learning. It was always a polyglot project, though: there was some Java and C++ for performance-critical components, some MATLAB/Octave for the scientists, some Python and JavaScript for the browser-based user interface, and a lot of Makefiles to put this all together. There was some PLC, too, for the low-level control of the industrial robots and for integration with the waste processing plants.
ZR was my first Clojure job; before that I had only used it on a course at the university. There were good things like how Clojure handles mutable state and multithreading but there were problems as well. In motion control, we had hard time dealing with the real-time constraints when running on JVM on Windows. We even had our own real-time DSL implemented with macros on top of Clojure! I never worked on the machine vision component, but it relied on OpenCV and due to limitations of OpenCV and/or JVM, the OpenCV resources couldn’t be managed correctly by the garbage collector and had to be handled manually instead (the details are hazy, but I don’t believe this is a problem anymore). This made the programming experience less than great.
Eventually it came time to build the second generation of the system and the motion control had to be rewritten from scratch. The initial prototype was in C++, mostly because C++ was the strongest language of the person who did it. It turned out very well and it was decided that C++ would be used for the production version as well. It was a successful choice, as far as I could tell. I’m not a C++ expert, but I never had to debug that component, and I had to debug almost all the components in the system, so that says something.
It might sound implausible on this forum, but Clojure wasn’t actually anyone’s favorite language! There were C++ fans, Haskell fans (incl. me back then), Common LISP fans, MATLAB fans, etc. At least the C++ fans got their way, the rest of us didn’t.
Since the motion control was replaced, a big chunk of Clojure code was deleted from the codebase. At the same time the ROS (Robot Operating System) message passing framework was adopted. This made Clojure a second-class citizen, because the ROS Java client library was unusable. It badly leaked memory, if I recall correctly. Many small components ended up being written in Python.
Finally, as you say, the Python ML libraries were becoming really good. The scientists and the machine-learning engineers were using them heavily for experimental stuff. I don’t think anything was put into production by the time I left in 2015, but I assume that Python has basically replaced Clojure for machine learning in the product as well.
To summarize: Clojure probably wasn’t the right or the best tool for the job and it was replaced.
This is kinda sad 
Anyway I guess that was stuff really close to the system level, otherwise I keep seeing very performant stuff spinning on the JVM (Spark, Kafka, Storm, Onyx…).
Miikka, thanks for taking the time to share the story.
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.