Introducing Proletarian, a durable job queuing and worker system for Clojure backed by PostgreSQL. It provides at least once processing guarantees with customizable retry strategies.
Use Proletarian for asynchronously executing tasks in the background. It’s useful for offloading long-running tasks from the request thread in user-facing web applications.The initial public release is 1.0.32-alpha. It’s in alpha since public scrutiny and adoption might necessitate API changes, but I consider Proletarian mostly done and ready for production use.
Please give it a spin! I’m happy to discuss and answer questions here or over at the #proletarian Slack channel.
D’oh, thanks for spotting! I’ve updated the links. Examples B and C have no documentation, but they follow the same pattern as Example A, so you should be able to run them in the same fashion.
Some interesting feature that’s needed is some observability / logging. Eg I’d like to see the number of workers, job completion time, number of items in the queue, a dead letter queue perhaps? But all those seem possible to add externally, I think.
For logging there’s the :log option, which is just a function that Proletarian calls whenever something interesting happens. It certainly needs documenting, and maybe an example or two to show how to do the things you mentioned. The default implementation is just a println.
I don’t think the Proletarian library should prescribe use of any particular observability or logging library, which is why the :log option exists. I’ll be happy to include any information that’s needed in the log messages. I’ll look into your suggestions for what should be available.