Misconceptions about REPL-driven development

I’ll keep arguing that REPL-Aided Development is a much better term, worth the re-branding :slight_smile:

Repl-Driven Development creates natural polarization because it feels exclusive to many people:

  • TDD aficionados will be afraid that tests are no longer in the driver’s seat
  • likewise for BDD aficionados
  • likewise for DDD aficionados

The reality is that the benefits of the REPL are orthogonal to all these approaches.

Granted, one might argue that this misconception about “driving” is their fault, but framing the problem that way does not help us fix it.

I’m not a native English speaker, and so I’m not sure why developers are so intent on naming things with a “-driven” suffix - it’s always felt weird to me. I suspect that, more or less consciously, it’s just because it sounds cool and fashionable. But even if you care about coolness, think about the acronym. RAD. RAD. Isn’t that much cooler than RDD?

6 Likes

@mjmeintjes also, FWIW, the last chapter of the REPL guide on clojure.org arguably addresses some of the concerns you mentioned.

I trace this habit back to my C64 BASIC programming in 1984. You only need to lose your work once before you get in the habbit of typing SAVE"PRG38",8 and then scribble “38” on a piece of scrap paper, to remember which version you’re on.

As newer machines came out, and all you needed to do was hit ctrl-s, it was like heaven!

1 Like

Your advice to NEVER type into the REPL, along with your screencast https://youtu.be/UFY2rd05W2g really helped me unlock the concept of REPL driven development, so thank you for that!

5 Likes

I’d really credit Stu Halloway with that advice, but I think it needs to said more often and spread more widely :slight_smile:

2 Likes

I have personally become convinced that REPL-driven development is a terrible name for what we do, precisely because of the reasons outlined above. People associate “REPL” with a specific place where you put code, but it’s not about the REPL as a place, that is entirely secondary, it’s about the interactivity that is possible when communicating with a live process.

We don’t need to invent new terms for this, we just need to go back to the roots. This has been known since the 70’s as interactive development. Here’s a paper from 1978 that digs into what that means. (hat tip to @jackrusher for that).

15 Likes

Interesting paper. Worth a read, I guess?

The recording from my London Clojurians “REPL-Driven Development: Clojure’s Superpower” talk is available at https://www.youtube.com/watch?v=gIoadGfm5T8

7 Likes

We don’t need to invent new terms for this, we just need to go back to the roots. This has been known since the 70’s as interactive development.

This!
Writing a practical guide to newcomers (and the rest of us) that incrementally and iteratively introduces this to a developer who is used to the “Java/C++/C#/etc. way” is on my to-do list, and I hope that I’ll be able to start this year. (BTW that’s why my book series title is “Interactive Programming for Artificial Intelligence”. BBTW suggestions welcome)

6 Likes

It’s also why Calva has ”Clojure & ClojureScript Interactive Programming” directly in its title in its VS Code marketplace entry. :smiley:

In fact, when I titled it like that, I was all new to Clojure and had only seen the style of coding called interactive programming. I came across the term REPL Driven Development later.

How close is Clojure getting to this guy’s definition of RDD? I don’t think breakloops are possible because of the JVM, is that the only thing missing?

Pretty sure I’ve seen at least one conference talk that showed a library that essentially provided that functionality in Clojure… You could certainly have a function that starts a new REPL, lets you interact for a while, and then “quit” the REPL and have it actually throw an exception or continue or whatever.

You can’t do it with “native” error handling (i.e., JVM exceptions) without wrapping all interop etc, but you can still start a new REPL in a catch I guess.

As I was scanning through the discussion above, I remembered Rich Hickey’s expression PLace Oriented Programming, PLOP. Some of the comments refer to the REPL as though it’s a place we go to, rather than the process we follow when we get ‘there’, or to one of the other ‘theres’, where we then loop through Reading, Evaluating and Printing code-as-data and other data. Adding ‘a REPL place’ to other languages doesn’t achieve the same because they aren’t a Lisp.

It’s more like a ‘REPL-enabled’ development process than ‘REPL-driven’. Surely we are driving, even if some of us are still learners.

I was definitely confused by the huge and constantly changing array of tooling choices. It’s hard for beginners’ books to explain well, without frightening people away.

I use scope-capture extensively, and it provides something similar. I find it indispensable to set up the context when testing or developing a function.

This has been known since the 70’s as interactive development.

That’s a good point. That said, I wouldn’t want people to believe that REPLs are the only approach to making development interactive. For example, auto-reloading UIs and test suites contribute to interactive development as well.

Not per the Lisp definition, though. The Lisp definition of interactive is to build the image in memory by compiling it piece by piece. A REPL is really the only viable way of doing that.

UIs kind of do it, but usually require a full reload at some point. TDD, which I use instead of a REPL, restarts everything each iteration. They have shortened feedback loops, but I wouldn’t lump them in with what Lisp calls interactive.

1 Like

Ya I agree, that’s why I’ve likened the term: Dynamic Development

Which is also how Clojure officially describes its development approach: https://clojure.org/about/dynamic

And there’s a whole Wikipedia page about it: https://en.wikipedia.org/wiki/Dynamic_programming_language

So ya, I’d vote for the Clojure Community using the word: Dynamic Development personally.

Dynamic Development isn’t bad. To me, it sounds similar to dynamic programming so is a bit confusing. I think RDD is fine.

I could see not referring to it as interactive, though, since Clojure doesn’t live up to the full interactive experience. That’s mostly due to the lack of restarts, but the Common Lisp REPL really is a different experience than Clojure’s. Not trying to sound like one of “those” CL people, especially since I don’t really care for either REPL, but having a live stack that can be modified when an exception is thrown is nice. Comparatively, the Clojure REPL feels more like a hot loading debugger.

1 Like

I think RDD is fine

The thing is I don’t think RDD is the same as Dynamic Development. You could conceive an editor integrated REPL in Java for example, but Dynamic Development implies certain language features, the REPL and integrating it into the editor is just one thing that Dynamic Development allows.

Fundamentally, it’s about what the Clojure page on it says:

Clojure is dynamic. That means that a Clojure program is not just something you compile and run, but something with which you can interact. Clojure is not a language abstraction, but an environment, where almost all of the language constructs are reified, and thus can be examined and changed.

And like the page says, this allows you to:

grow your program, with data loaded, adding features, fixing bugs, testing, in an unbroken stream.

And like Wikipedia says, some of the features you need for it are:

  • Runtime eval
  • Type runtime alterations
  • In-place code replacement (swapping one function with another)
  • Full evaluation and modification of a program’s code as data
  • Macros that provide access to the inner workings of the compiler, and full access to the interpreter, virtual machine, or runtime, allowing the definition of language-like constructs which can optimize code or modify the syntax or grammar of the language
  • Additions/Update/Removal of definitions

These features are why Clojure doesn’t “feel like a hot loading debugger”. I know what you mean here, but even Visual Studio’s debugger for C#, which is probably the most advanced debugger for languages without Dynamic Development features still has limitations when it comes to what I listed above. I know, because I came from using that debugger daily prior to Clojure. You got to be careful what you can replace and re-evaluate when in the C# hot-loading debugger, because the language just doesn’t have the dynamism needed for full on Dynamic Development.

Ya, the word “Dynamic” is a bit overloaded. Dynamic programming and Dynamic typing could both easily be confused with Dynamic Development. Though I still think it’s the right adjective for “development”:

characterized by constant change, activity, or progress

Like this is a perfect definition for what it means to develop in Clojure.

Ya, Clojure has to live with its host in harmony, and I’m quite surprised how much it was able to achieve on the “dynamic” front while running on Java. CL still has it beat in that aspect. But I also don’t think that’s the be all end all, like CL is able to leverage dynamic development to this feature because of its choice of how to handle errors, you could do it in Clojure as well, the language is dynamic enough to do it, but doing so would be fighting the interop with Java, and I think the trade off here is worth it; of being able to leverage Java and the performance gains of not adding an additional exception layer.

For example, Emacs lisp similarly didn’t bother adding a conditional restart with debugger default handling, but the language is 100% dynamic, you can redefine everything and reify everything at run-time.

I haven’t used condition/restarts in large enough systems to know if I wish Clojure went with it by default or not. But Cider’s debugger, isn’t actually a debugger, it actually just dynamically reify your code to actually stop where the exception is first encountered and allows you to modify everything that follows, which gets pretty close, except you don’t have the ability to “restart”, unless you actually add that in yourself like by using a library for it like https://github.com/clojureman/special

I meant relative to CL. Compared to Java or Python that’s not the case.

It’s the one feature compiler writers seem to always say they miss. I’m pretty sure Hickey mentioned that in one of his talks. Having used them more, though, I’d probably vote against them. They make the debugger nice, but not in regular code. It’s like having 3D spaghetti code. Your code can not only jump around the current function, but up and down the stack at runtime. If it’s not working right, it’s a nightmare to fix.