Misconceptions about REPL-driven development

I have personally become convinced that REPL-driven development is a terrible name for what we do, precisely because of the reasons outlined above. People associate “REPL” with a specific place where you put code, but it’s not about the REPL as a place, that is entirely secondary, it’s about the interactivity that is possible when communicating with a live process.

We don’t need to invent new terms for this, we just need to go back to the roots. This has been known since the 70’s as interactive development. Here’s a paper from 1978 that digs into what that means. (hat tip to @jackrusher for that).

15 Likes

Interesting paper. Worth a read, I guess?

The recording from my London Clojurians “REPL-Driven Development: Clojure’s Superpower” talk is available at https://www.youtube.com/watch?v=gIoadGfm5T8

7 Likes

We don’t need to invent new terms for this, we just need to go back to the roots. This has been known since the 70’s as interactive development.

This!
Writing a practical guide to newcomers (and the rest of us) that incrementally and iteratively introduces this to a developer who is used to the “Java/C++/C#/etc. way” is on my to-do list, and I hope that I’ll be able to start this year. (BTW that’s why my book series title is “Interactive Programming for Artificial Intelligence”. BBTW suggestions welcome)

6 Likes

It’s also why Calva has ”Clojure & ClojureScript Interactive Programming” directly in its title in its VS Code marketplace entry. :smiley:

In fact, when I titled it like that, I was all new to Clojure and had only seen the style of coding called interactive programming. I came across the term REPL Driven Development later.

How close is Clojure getting to this guy’s definition of RDD? I don’t think breakloops are possible because of the JVM, is that the only thing missing?

Pretty sure I’ve seen at least one conference talk that showed a library that essentially provided that functionality in Clojure… You could certainly have a function that starts a new REPL, lets you interact for a while, and then “quit” the REPL and have it actually throw an exception or continue or whatever.

You can’t do it with “native” error handling (i.e., JVM exceptions) without wrapping all interop etc, but you can still start a new REPL in a catch I guess.

As I was scanning through the discussion above, I remembered Rich Hickey’s expression PLace Oriented Programming, PLOP. Some of the comments refer to the REPL as though it’s a place we go to, rather than the process we follow when we get ‘there’, or to one of the other ‘theres’, where we then loop through Reading, Evaluating and Printing code-as-data and other data. Adding ‘a REPL place’ to other languages doesn’t achieve the same because they aren’t a Lisp.

It’s more like a ‘REPL-enabled’ development process than ‘REPL-driven’. Surely we are driving, even if some of us are still learners.

I was definitely confused by the huge and constantly changing array of tooling choices. It’s hard for beginners’ books to explain well, without frightening people away.

I use scope-capture extensively, and it provides something similar. I find it indispensable to set up the context when testing or developing a function.

This has been known since the 70’s as interactive development.

That’s a good point. That said, I wouldn’t want people to believe that REPLs are the only approach to making development interactive. For example, auto-reloading UIs and test suites contribute to interactive development as well.

Not per the Lisp definition, though. The Lisp definition of interactive is to build the image in memory by compiling it piece by piece. A REPL is really the only viable way of doing that.

UIs kind of do it, but usually require a full reload at some point. TDD, which I use instead of a REPL, restarts everything each iteration. They have shortened feedback loops, but I wouldn’t lump them in with what Lisp calls interactive.

1 Like

Ya I agree, that’s why I’ve likened the term: Dynamic Development

Which is also how Clojure officially describes its development approach: https://clojure.org/about/dynamic

And there’s a whole Wikipedia page about it: https://en.wikipedia.org/wiki/Dynamic_programming_language

So ya, I’d vote for the Clojure Community using the word: Dynamic Development personally.

Dynamic Development isn’t bad. To me, it sounds similar to dynamic programming so is a bit confusing. I think RDD is fine.

I could see not referring to it as interactive, though, since Clojure doesn’t live up to the full interactive experience. That’s mostly due to the lack of restarts, but the Common Lisp REPL really is a different experience than Clojure’s. Not trying to sound like one of “those” CL people, especially since I don’t really care for either REPL, but having a live stack that can be modified when an exception is thrown is nice. Comparatively, the Clojure REPL feels more like a hot loading debugger.

1 Like

I think RDD is fine

The thing is I don’t think RDD is the same as Dynamic Development. You could conceive an editor integrated REPL in Java for example, but Dynamic Development implies certain language features, the REPL and integrating it into the editor is just one thing that Dynamic Development allows.

Fundamentally, it’s about what the Clojure page on it says:

Clojure is dynamic. That means that a Clojure program is not just something you compile and run, but something with which you can interact. Clojure is not a language abstraction, but an environment, where almost all of the language constructs are reified, and thus can be examined and changed.

And like the page says, this allows you to:

grow your program, with data loaded, adding features, fixing bugs, testing, in an unbroken stream.

And like Wikipedia says, some of the features you need for it are:

  • Runtime eval
  • Type runtime alterations
  • In-place code replacement (swapping one function with another)
  • Full evaluation and modification of a program’s code as data
  • Macros that provide access to the inner workings of the compiler, and full access to the interpreter, virtual machine, or runtime, allowing the definition of language-like constructs which can optimize code or modify the syntax or grammar of the language
  • Additions/Update/Removal of definitions

These features are why Clojure doesn’t “feel like a hot loading debugger”. I know what you mean here, but even Visual Studio’s debugger for C#, which is probably the most advanced debugger for languages without Dynamic Development features still has limitations when it comes to what I listed above. I know, because I came from using that debugger daily prior to Clojure. You got to be careful what you can replace and re-evaluate when in the C# hot-loading debugger, because the language just doesn’t have the dynamism needed for full on Dynamic Development.

Ya, the word “Dynamic” is a bit overloaded. Dynamic programming and Dynamic typing could both easily be confused with Dynamic Development. Though I still think it’s the right adjective for “development”:

characterized by constant change, activity, or progress

Like this is a perfect definition for what it means to develop in Clojure.

Ya, Clojure has to live with its host in harmony, and I’m quite surprised how much it was able to achieve on the “dynamic” front while running on Java. CL still has it beat in that aspect. But I also don’t think that’s the be all end all, like CL is able to leverage dynamic development to this feature because of its choice of how to handle errors, you could do it in Clojure as well, the language is dynamic enough to do it, but doing so would be fighting the interop with Java, and I think the trade off here is worth it; of being able to leverage Java and the performance gains of not adding an additional exception layer.

For example, Emacs lisp similarly didn’t bother adding a conditional restart with debugger default handling, but the language is 100% dynamic, you can redefine everything and reify everything at run-time.

I haven’t used condition/restarts in large enough systems to know if I wish Clojure went with it by default or not. But Cider’s debugger, isn’t actually a debugger, it actually just dynamically reify your code to actually stop where the exception is first encountered and allows you to modify everything that follows, which gets pretty close, except you don’t have the ability to “restart”, unless you actually add that in yourself like by using a library for it like GitHub - clojureman/special: Special (Conditions). A condition system for Clojure

I meant relative to CL. Compared to Java or Python that’s not the case.

It’s the one feature compiler writers seem to always say they miss. I’m pretty sure Hickey mentioned that in one of his talks. Having used them more, though, I’d probably vote against them. They make the debugger nice, but not in regular code. It’s like having 3D spaghetti code. Your code can not only jump around the current function, but up and down the stack at runtime. If it’s not working right, it’s a nightmare to fix.

I’ve only toyed with it, never used on the job on anything big enough, but ya, as you say, my instinct is I don’t know what I’d do with it in a production environment with a live service. I feel Erlangs supervision model is much simpler and in a way better because of that. Like instead of going more granular and allowing code up the stack to choose the handling of the error down the stack, which seems really confusing to get right, instead just restart the process at a much coarser level, like just redo the entire request or something like that seems much simpler to get right.

I question the usefulness of that definition (heretical, I know), as it’s so much more about the specifics of one solution than about the essence of the general problem. To me it sounds like saying “the Airbus definition of flying is piloting a fixed-wing aircraft propelled by a jet engine. Therefore, paragliders and balloons don’t qualify.” :slight_smile:

I don’t think we should let Lisp’s definition of interactive pre-empt the semantic space of that word at the exclusion of other embodiments. Arguably all the more so because that definition is not mainstream.

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.