What are the alternatives to using remove-ns to clean up definitions

PEZ · April 9, 2019, 12:17pm

In a Slack thread there was a question about how to clear out definitions/bindings in a file so that when you reevaluated it it would complain if something that was previously defined by the code in the file, now wasn’t defined, but referenced.

I mentioned remove-ns, and it seemed to solve the need for the guy who asked about it. However, one of the (two) examples on clojuredocs, clearly discourages the use of this, ”unless you know what you are doing”, which I think does not apply to the guy who asked, and certainly doesn’t apply to me.

The example suggested using ”libraries like tools.namespace”, but left it at that.

So, how do you people go about cleaning up mess you might have created while REPLing your code into existence?

teodorlu · April 9, 2019, 2:05pm

Hello!

I run refresh from CIDER (Emacs) with M-x cider-refresh. In Spacemacs, cider-refresh is bound to , s x, so executing it is simple. Poking around in tools namespace, I think clojure.tools.namespace.repl/refresh and clojure.tools.namespace.repl/refresh-all are the functions we’re looking for.

I’ll see if I can get a test or two off!

T

PEZ · April 9, 2019, 2:19pm

Yeah, I use those all the time in my work project. We have embedded them in some other start/stop cleanup that our project needs, so have forgotten about what actually is going in. I must investigate…

From my super-quick check, I don’t have the refresh function in my user namespace in projects where tools.namespace is not among the dependencies. I wonder of CIDER gets hold of it… (I want to add this to Calva, of course.)

teodorlu · April 9, 2019, 2:27pm

So! This took a bit longer than I thought.

First I tried to get this working in my playground project. Unfortunately, I couldn’t refresh, because I had code that didn’t work dangling in other namespaces. Ironically the problem that tools namespace tries to solve …

Then I tried running tools namespace from within a source file. That gave me an error when reloading the file, “No such namespace: ns-repl”. (aliased tools.namespace.repl to ns-repl, then that got unloaded, which got me in trouble.

So I should probably do this in a normal REPL, no shenanigans. Setup:

;; In th/code.clj
(ns th.code)

;; Some old code we plan to prevent from dangling
(def old-val "old")

;; Some new code we plan to keep
(def new-val "new")

… and let’s test this from a “normal” repl (one that’s not tied to the th.core namespace).

;; In a REPL
user> (require 'th.code)
nil
user> (ns-publics 'th.code)
{new-val #'th.code/new-val, old-val #'th.code/old-val}

Yup, that got loaded. Now, uncomment the old val

;; in th/code.clj
;; (def old-val "old")

… and let’s see if we can remove old-val from th.code.

;; in a REPL
user> (require 'clojure.tools.namespace.repl)
nil
user> (clojure.tools.namespace.repl/refresh)
:reloading (th.code)
:ok
user> (ns-publics 'th.code)
{new-val #'th.code/new-val}

Worked!

I think CIDER injects the tools.namespace dependency when it starts Clojure. Found this in my *Messages* buffer, but the full command is cut off:

[nREPL] Starting server via /usr/local/bin/clojure -Sdeps ‘{:deps {nrepl {:mvn/version “0.6.0”} refactor-nrepl {:mvn/version “2.4.0”} cider/cider-nrepl {:mvn/version “0.22.0-SNAPSHOT”}}}’ -m nrepl.cmdline --middleware ‘[“refactor-nrepl.middleware/wrap-refactor”, “cider.nrepl/cider-middleware”]’…

PEZ · April 9, 2019, 2:33pm

Thanks a lot for helping me investigate this! I’ll go right ahead and try to get this refresh command into Calva. Feels awful that users have to resort to remove-ns…

Since Calva is distilled from CIDER, it has also stolen the injection dependencies from there (this exists only in dev releases of Calva) and at the time of the theft there were no tools.namespace injected that I noticed. But maybe the command was cut of for me as well.

teodorlu · April 9, 2019, 2:40pm

refresh.clj in cider-nrepl looks relevant.

PEZ · April 9, 2019, 2:49pm

OMG. I already have these ops prepared in the Calva code. Just need to make them available as commands.

teodorlu · April 9, 2019, 3:01pm

That feeling when you get more traction with code than you expected, and not the other way

PEZ · April 9, 2019, 4:56pm

Now my poor beta testers can test refresh and refresh-all. It only refreshes the clj repl right now. Do you know if CIDER supports refresh for cljs repls?

teodorlu · April 9, 2019, 9:53pm

Nice!

As far as I know, the default Duct template makes use of refresh for browser REPLs, and refreshing from the editor then reloads cljs code as well.

seancorfield · April 10, 2019, 7:25am

FWIW, I’ve never needed any sort of “refresh all” in my workflow. I have a REPL running with all the dependencies of our monorepo across a dozen or so subprojects, and it runs for days and days and days. I probably restart it once a week at most…

PEZ · April 10, 2019, 7:58am

But you still use refresh? Just checking.

Webdev_Tory · April 10, 2019, 2:52pm

I can’t believe I never knew about cider-refresh… wow!

seancorfield · April 10, 2019, 5:09pm

No, no refresh, no refresh all. I just don’t seem to need them.

PEZ · April 10, 2019, 6:27pm

That’s interesting. Have you written about this somewhere? Even if your style of coding is not for everyone, all the time, it seems I could learn a lot from getting to know more about it.

seancorfield · April 10, 2019, 7:37pm

I honestly don’t think I’m doing anything different – I just don’t seem to need a “refresh”. I wonder if folks have rushed to adopt it without waiting to see whether they experience pain without it?

No refresh is mentioned anywhere in the clojure.org https://clojure.org/guides/repl/introduction pages (I just checked). Pretty sure I’ve never heard Stu Halloway mention/show it in his various REPL usage talks? (just searched the https://github.com/matthiasn/talk-transcripts/tree/master/Halloway_Stuart transcripts repo and the only refresh is in one of David Nolen’s talks in a completely different context).

teodorlu · April 10, 2019, 9:07pm

Interesting to hear that you’re not using it. There is Component, though, and the related component.repl/reset, which does something similar. I’m left with a few loose threads, though,

Do you use a component system like Component or Integrant? If not, how do you manage stateful components, and the need to reload / refresh those?
Do you ever run into problems with multimethods?
Do you ever run into ordering problems in namespaces? I’ve had instances where I just define and redefine things in the REPL, and suddenly when I come back after a REPL restart, I realize that I’ve used some var before defining it.

James Reeves (Ring, Component) seems to be influenced by reloads / refreshes in his work. See Advancing Duct.

To be clear, I’m curious, not trying to argue.

didibus · April 11, 2019, 1:29am

I also mostly don’t use refresh. Though sometimes it is useful. So occasionally I do need it.

I think with regards to components, what component really need to be refreshed? That’s mostly necessary if you change the config for it. So normally, I don’t need to do that.

For example, my database doesn’t really need refreshing. Create the connections on start, close them on stop. But once I have it, I can just modify my query code and call it on the connection.

So I guess I just never had components that constantly need to be restarted.

For multi-methods, well, first, I don’t use them a lot to begin with. Also, only the defmulti has to be refreshed. The defmethods don’t need to be (from my memory). So I guess, it is less often I need to change the defmulti and more often I need to change the defmethods.

For ordering, I think this is my biggest use of it. Or actually, more when I have changed things in multiple namespace and don’t remember the require order. That said, you can also often just trust yourself and manually re-eval the namespaces.

seancorfield · April 11, 2019, 1:38am

Yes, we use Component very heavily. No, we don’t use its reset function.

No, I don’t run into problems with multimethods: you can use (def multi-fn nil) to remove a multimethod definition so it can be predictably redefined (putting that above your defmulti should be enough).

No, I don’t run into problems with ordering in namespaces. I suspect that’s a combination of 8+ years of Clojure experience, being careful to check for first use of functions when I do move things around, and never typing directly into the REPL (I always eval code from a source file, so I use “Rich Comment Forms” for non-production/exploratory code in my source files).

Also, we have a stub test namespace for every single source file so when we run clj -A:test:runner we get a failure if there’s an ordering problem – and we run our automated tests a lot, locally! We have a script that generates a stub test namespace for any source file that doesn’t already have one: it requires the source namespace and refers all symbols as a baseline for “can this source file be loaded” successfully.

Perhaps another aspect of my workflow that helps with all this: whenever I change any code at all, I always eval the top-level form that it’s in – so the REPL always has the up-to-date version of my code. That’s just habit, but it’s also part of a good REPL-Driven Development workflow: make small changes and always test every change in the REPL as you go. That means that instead of making a slew of changes across multiple namespaces and hoping you got it all correct, you work from the bottom up, testing each changed file as you go.

HolyJak · April 11, 2019, 1:08pm

Thank you! The trick with (def multi-fn nil) is neat.

How do you avoid the problem of renaming/removing a function and forgetting old references to it? I guess it is the same as with ordering and the stub tests would catch it?