Netty HTTP + Websockets wrapped in core.async: jdf/talk

Hi everyone,

I’ve written a wrapper for Netty to try to make websockets easy to use from Clojure without bringing in lots of dependencies. It also provides some conveniences for managing HTTP uploads, taking advantage of Netty’s MixedFileUpload functionality to stream to disk after a certain size. It’s intended for use on JDK ≥11.

It converts Netty’s channelActive and channelRead events into clojure records and puts them on a core.async channel for the next layer up to handle (this includes handling most HTTP semantics). Arbitrary channel-specific metadata can be stored using clojure map-like wrappers of Netty’s ChannelGroup and ChannelHandlerContext's AttributeMap.

I’ve been pretty careful to ensure backpressure is maintained throughout the system, so that incoming Netty channel messages are not read until the previous incoming message was successfully put! to the incoming core.async channel, and outgoing messages (for a given Netty channel) are not take!n from the outgoing core.async channel until the previous outgoing message was successfully written to the Netty channel.

HTTP uploads need to be “approved” by the next layer up before being read (HTTP status 102 is used to indicate this and is not sent to client). This prevents abusive uploads, although unfortunately browser UI support for cancelled uploads is poor.

I’m working on the “next layer up” too in the form of jdf/foundation, but it’s not ready for primetime yet.