I’m wondering about the usefulness of core.async in the world of backend applications. What advantages does it have over regular concurrency primitives like futures, pmap or libraries such as claypoole? Our applications do a lot of IO (RabbitMQ consumers and HTTP requests) and I don’t seem see how core.async would be useful to us. That said: I admit I haven’t used core.async and the closest I got to using a similar functionality was in the early days of Golang.
I’m sure I’m missing something here so I’d love to know more.
One thing I’ve noticed that core.async seems to be popular in Clojurescript world, I assume it’s because it helps with the callback-full nature of frontend environment.
I write JavaScript apps. core.async is popular in ClojureScript because cljs does not have native support for async/await. Most async functions in web app can be described via async/await + Promise.all, which is not fully supported in cljs. core.async(with some helper functions) is at least not a bad option for handling async functions.
We make extensive use of core.async in our backend ETL processes, with great success.
The joy comes from combining transducers with core.async to spread computationally expensive work over all the cores available to the JVM, and being able to use sequence in the repl while playing with your functions. We do all of our IO at the edges of the pipelines and keep the pipelines themselves free from side-effects.
Comparing it to futures & pmap isn’t quite accurate, they serve different purposes and both of them spawn real threads. core.async by default doesn’t and you need to use specific core.async functions to get real threads when you need them (generally only for IO).
Consuming messages from AMQP or SQS could be a good fit for core.async, but it really depends. We use core.async here to pull messages from SQS and manage the message visibility while the “worker function” churns away at the job. AMQP gives you the ability to ack the message at the end, so you don’t quite need this behaviour.
Like most things I would say “it depends” really, I wouldn’t use it for everything. It can be a very sharp tool and a real pain some of time during development/testing. When it works though, it works pretty well.
so I’d love to know more.