FWIW I did have a crack at a similar problem in the past and ended up relying chiefly on core.async in a manner where I had a sliding buffer with “work slots” (just a bunch of numeric indices). The dispatch mechanism would then wait for a slot (or timeout), and spawn a “worker”: a thunk accepting a control-channel and a cleanup fn (“put that worker index back on the worker chan”).
It worked, but I wasn’t very happy with it. Seemed frightfully convoluted for something that seems like a fairly mundane problem. I also never felt in control. “Workers” were futures which I could only supervise through control-chans.
As an aside: I find that the wealth of options wrt async/parallel tooling (from Java’s executors, through core.async, manifold, promesa, missionary…) is overwhelming to a point of paralysis.
I think core.async is relatively low-level and can benefit from building convenience layers on top (e.g. process control, supervision, etc.). There was an attempt to bring some of erlang/OTP over to core.async via the otp-like library (I never used it, but I respect OTP).
Yeah that’s it. Sometimes things are just good enough or in a stable state. It’s also EPL, so trivially forked if someone feels like using it and expanding on it.
There was also an earlier library called Pulsar that borrowed a lot from erlang as well. Pretty ambitious. Author went on to work on project Loom at oracle (since he already implemented fibers on the jvm for his stuff).