Reloading core.async without derailing legacy threads/goroutines running on the legacy threadpool

[joinr]

Say I have this ns:

(ns blah
  (:require [clojure.core.async :as async]))
   
(def log-chan (async/chan (async/sliding-buffer  1000)))
(def logger
  (async/go-loop []
    (when-let [msg (async/<! log-chan)]
      (do (println msg)
          (recur)))))

(defn log
  "Logs messages asynchronously, prints synchronously.
   This allows logging to occur from multiple writers
   asynchronously, while retaining serialized printing
   for readability."
  [msg]
  (clojure.core.async/put! log-chan msg))

So we have a go routine parked waiting for messages on a global log chan, with a little log api for async logging. I go off to do some work and test, and as is my want, I reload another ns that depends on 'blah using :reload-all doing some iterative dev and testing.

(ns blee 
   (:require [blah]))

;;does lots of fascinating stuff

Sadly, I get some complaints:

user>(require 'blee :reload-all)

Execution error (IllegalArgumentException) at clojure.core.async.impl.protocols/eval105333$fn$G (protocols.clj:43).
No implementation of method: :exec of protocol: #'clojure.core.async.impl.protocols/Executor found for class: clojure.core.async.impl.exec.threadpool$thread_pool_executor$reify__9246

exec protocol function is now technically different (after reload), the executor is defined by protocol reification, but the original threadpool is defonce’d, so all future dispatches are applying the (new/current) exec protocol fn to the original defonced’ threadpool reified object here.

So the “new” logger goroutine is trying to be run (ultimately via the redefined clojure.core.async.impl.protocols/exec fn), which isn’t technically implemented by the retained clojure.core.async.impl.dispatch/executor object, and we get the protocol complaint.

My current way to sidestep this problem is to hack the require functionality in clojure.core and allow namespaces to exclude on :reload-all. This is clunky but workable for now (I am pretty much manually/intermittently reloading via reload-all during dev and interspersing test suite runs).

Is there a more elegant way to respect the implicit singleton threadpool + brittle protocol relationship in core.async? I had thought about hacking the threadpool var manually to work around defonce (maybe shutting down gracefully as well), via some other namespace that loads earlier. That would let the stock require machinery handle everything.

[p-himik]

It might sound disappointing but the solution is not to use :reload-all.

There are two common approaches to REPL development: the “regular” RDD and the so-called “reloaded” workflow.

The former is a more manual approach where you evaluate the forms that you’ve changed in the context of their namespace and keep track of the dependencies - either manually via reloading all that must use updated references or by sprinkling #' to introduce an indirection.

The latter is a more automatic approach. It’s much more similar to (require 'blee :reload-all), but you use a different function, usually (clojure.tools.namespace.repl/refresh ...). It keeps track of the things that have changed and of the things that must be reloaded because of those changes. It will not be reloading clojure.core.async namespace at all because it doesn’t depend on the code that you wrote.

[joinr]

Thanks. Might check out cider-refresh to see if it fits the bill, but uncertain if the codebase conforms to the assumptions c.t.n. requires (haven’t needed for ~13 years, and only now because of the core.async dev-time issue).

[p-himik]

I don’t think there are any assumptions apart from namespaces corresponding to the file structure and being created and loaded only by the (ns ...) form so it can build a proper dependency graph (so no (load ...) or (require ...)).

[joinr]

warnings and potential problems

found tonsky’s clj-reload which I overlooked.