Misconceptions about REPL-driven development

[mjmeintjes]

After reading some of the Hacker News comments about On Repl-Driven Programming, it seems that a lot of people have trouble seeing the benefits of Repl-Driven development. However, for me it is one of the biggest benefits of Clojure.

The most common issue is that people have misconceptions a Repl is, based on their own experience with other languages like Python or Ruby.

It seems common that people expect Repl-driven development to be about editing code directly in a Repl prompt, instead of using an editor and sending code to the Repl. See here, here, and here

Other concerns about Repl-driven development:

• Repl-driven development is about developing an “image” without source files, making it difficult to know what code is actually running.
• Concerns about documentation: Repl-driven leads to code that works without knowing why it works, because it makes it easy to explore undocumented functionality.
• Repl-driven development leads to bad design because the lack of planning. Also here.
• Repl-driven development is similar to debugger-driven development, where you edit in-memory values to see the effects. See also here and here.

I wonder whether it is worth “rebranding” the term Repl-driven development to prevent these types of misconceptions. What do you think?

[seancorfield]

So Clojurians should stop saying “REPL-Driven Development” because non-Clojurians don’t know what we mean by a REPL?

A lot of Clojurians don’t really know what REPL-Driven Development is either.

You’re not going to “fix” Hacker News by choosing a different name for what many of us strive to do with our Clojure REPL. As seen on that page, a lot of non-Clojure/Lisp/Smalltalk/… folks think they also have a “REPL” and therefore they fundamentally can’t understand what we’re talking about (it’s the Blub Paradox).

We’ll just keep trying to educate folks about RDD. Mostly, we need to continue to educate Clojurians about what we mean by it. It really doesn’t matter what those other folks think because they’re not writing Clojure code!

[didibus]

Rebranding just isn’t realistic, but probably we can do a better job at demonstrating what it is, maybe more gifs, screenshots, short videos.

I’ve also noticed some people who bring up some languages having something similar (for example you can setup Emacs to show a python command prompt that you can then from the editor send the buffer content or the marked content to the command prompt), fail to miss the part that the language semantics of Clojure also makes this workflow much nicer. For example, Vars have double indirection to allow this scenario:

(defn a []
  (b))

(defn b []
  (c))

;; Change c and re-evaluate c, now call `(a)` and it will make use of the latest version of `c`, even though you didn't re-evaluate `a` and `b`.

Similarly, and I’ve found even compared to say Emacs Lisp, the immutability of Clojure also helps a lot with managing REPL state, which is very helpful in REPL-driven-development.

[seancorfield]

One of the key distinctions I try to draw is that Clojure’s REPL can be started up in any process and that you can connect to such a REPL running inside an existing process, without needing any code-level additions and in a full-on production application. I don’t think anything (outside the Smalltalk/Lisp families) has that functionality?

[simongray]

I think there’s two issues with trying to explain what a REPL is:

• Users of languages like Python or Ruby who think they what they have is comparable to a Clojure REPL. Also, as Sean mentions, many Clojure users also don’t really understand that the REPL can be more than a CLI shell where you input one-liners. It took me a couple of years to really start using it properly.
• Users of Common Lisp who feel the need to chime in and tell others that Clojure isn’t a real Lisp or that the Clojure REPL isn’t a real REPL compared to the powerful Common Lisp one… Then it’s no longer about explaining REPL-driven development, but instead just another boring round of “Old Lisp v New Lisp”. I might just be biased against Common Lisp users, though.

[mvarela]

That seems like an accurate description of how these discussions usually go…

[dave.liepmann]

@mjmeintjes

I wonder whether it is worth “rebranding” the term Repl-driven development to prevent these types of misconceptions. What do you think?

I think you’ve correctly identified the misconceptions, but I don’t think a rebranding is a good solution. We should keep calling it REPL-driven development, because it is. When people express wrong things about it we should correct them. Chop wood, carry water.

If I were to get involved in the HN discussion, which I hope I don’t, I would liberally post links to Arne Brasseur’s The Bare Minimum, because it explicitly lays out how the way I and he and Sean interact with our REPLs is different from what they think REPL-driven development is. Links to livecoding sessions or screencasts or GIFs of CIDER eval-in-place or whatever are good too.

[PEZ]

Rebranding is probably not worth it, but I agree it would be a way to stay clear from the particular misconceptions that are brought by previous experiences with other REPLs. The rest would remain though.

I think that often you can turn a weakness into a strength so here maybe we can get good at using any previous REPL experiences as a bridgehead to selling the Clojure one. And I do think this selling is important. We Clojurians can’t afford to suck at PR. The REPL is a huge USP, that we need to get masters in using as that. It is impossible to convince anyone else about anything, though, so the key is to find ways for helping people to convince themselves.

Also:

Repl-driven development is about developing an “image” without source files, making it difficult to know what code is actually running.

I fall into this trap all the time. Wishing I had tools that could help me keep track of the diff between my running application and my files. I anyone is aware of such tools, please let me know.

[Richard_Heller]

It doesn’t need rebranding, everybody’s minds don’t work the same. For some people, RDD is awesome. For others, it’s not. Doesn’t mean there’s something wrong with them, or they just don’t get it, or whatever.

If you look at the issues people have with RDD, the root cause is the same. There’s a loose coupling between what’s running in the REPL and what’s in the source files. They’re not guaranteed to be the same. It’s up to the developer to make sure they stay in sync.

For me, I’ve tried following along with videos of people demonstrating RDD, I did a few of the AoC problems forcing myself to use RDD, and I still prefer not using the REPL. It can be useful for debugging some things, but not for active development.

Again, that’s just me. I’m not trying to talk anybody out of using the REPL or anything like that. For me, personally, using the REPL requires more typing, which makes my hands hurt, and generally feels like I’m jogging rather than running. It slows me down.

I don’t think the message about the REPL is somehow wrong and needs to change, I think people just need to accept that not everybody does things the same way.

[seancorfield]

@PEZ

Wishing I had tools that could help me keep track of the diff between my running application and my files. I anyone is aware of such tools, please let me know.

I think just getting into the habit of always eval’ing each top-level form as you edit it goes a long way here. Evaluate every change immediately. Also, never type into the REPL: use it purely as a way to evaluate code that you are editing. Use comment blocks liberally (as a way to keep scratch code and explorations around for “future you”, so you can see how a solution evolved).

[PEZ]

@seancorfield

I think just getting into the habit of always eval’ing each top-level form as you edit it goes a long way here. Evaluate every change immediately. Also, never type into the REPL: use it purely as a way to evaluate code that you are editing. Use comment blocks liberally (as a way to keep scratch code and explorations around for “future you”, so you can see how a solution evolved).

That’s a pretty good description of what I do, actually. I have been listening to you a lot about these things. (Even if I am still where I sometimes need to restart my REPL because I’ve lost control over what is defined where, but I am improving).

What happens to me is that at the end of a work session I can have 10+ unsaved files, because I haven’t saved anything for some hours, my app is in a new state (hopefully better) and I feel unsure wether I can just save my files and that will be the same as what is running in the phone simulator. It most often is, but it would be nice to know somehow. Probably just need a bit more discipline, but sometimes there are tools allowing me to be my sloppy self.

[seancorfield]

It’s funny, when I did my talk/demo to Clojure Provo, I noticed that my files were auto-saving whenever I ran a task to eval a block (and send it to Reveal). After the talk, I dug around and noticed that VS Code defaults to saving files before running tasks, so I turned that off! Now I can edit/eval as I go without needing files to be saved – and then I can save them (and immediately “load file” them) when I want instead… so I stay “in control”.

[simongray]

@PEZ

What happens to me is that at the end of a work session I can have 10+ unsaved files, because I haven’t saved anything for some hours, my app is in a new state (hopefully better) and I feel unsure wether I can just save my files and that will be the same as what is running in the phone simulator. It most often is, but it would be nice to know somehow. Probably just need a bit more discipline, but sometimes there are tools allowing me to be my sloppy self.

I got sweaty palms reading that paragraph

I am basically in the habit - to the point of it being OCD - of clicking ⌘-s every time I’ve typed something and there’s a slight pause. It happens with every kind of editing, Word documents, writing code, taking notes in Preview, etc. IntelliJ actually saves everything automatically, but I still can’t help doing it. Somebody send help.

[mvarela]

Same here, my “save-buffer” shortcut is so ingrained, that I rarely spend more than a couple of minutes without saving. Often much less.

[PEZ]

VS Code has this feature where I can see and act on diffs between the unsaved editor document and the saved file. And my git extension of choice enhances this. I use this to branch out in experiments. VS Code also keeps the state of unsaved documents, undo stack and all, between reloads of VS Code, so I don’t really need to worry about my changes getting lost. The only problem is that I have picked up a habit that requires a brain slightly larger than mine is.

[vvvvalvalval]

I’ll keep arguing that REPL-Aided Development is a much better term, worth the re-branding

Repl-Driven Development creates natural polarization because it feels exclusive to many people:

• TDD aficionados will be afraid that tests are no longer in the driver’s seat
• likewise for BDD aficionados
• likewise for DDD aficionados

The reality is that the benefits of the REPL are orthogonal to all these approaches.

Granted, one might argue that this misconception about “driving” is their fault, but framing the problem that way does not help us fix it.

I’m not a native English speaker, and so I’m not sure why developers are so intent on naming things with a “-driven” suffix - it’s always felt weird to me. I suspect that, more or less consciously, it’s just because it sounds cool and fashionable. But even if you care about coolness, think about the acronym. RAD. RAD. Isn’t that much cooler than RDD?

[vvvvalvalval]

@mjmeintjes also, FWIW, the last chapter of the REPL guide on clojure.org arguably addresses some of the concerns you mentioned.

[chepprey]

@simongray

to the point of it being OCD - of clicking ⌘-s every time I’ve typed something and there’s a slight pause. It happens with every kind of editing, Word documents, writing code, taking notes […]

I trace this habit back to my C64 BASIC programming in 1984. You only need to lose your work once before you get in the habbit of typing SAVE"PRG38",8 and then scribble “38” on a piece of scrap paper, to remember which version you’re on.

As newer machines came out, and all you needed to do was hit ctrl-s, it was like heaven!

[Jim_Megas]

Your advice to NEVER type into the REPL, along with your screencast https://youtu.be/UFY2rd05W2g really helped me unlock the concept of REPL driven development, so thank you for that!

[seancorfield]

I’d really credit Stu Halloway with that advice, but I think it needs to said more often and spread more widely