Any syntax to match async/await in creating web servers?

Community Center Beginners
1

I browsed http://koajs.com/ today. The last time I read about Koa, it was Koa 1(or something) with yield syntax. Now in Koa 2 it’s look a lot better with async/await support:

const Koa = require('koa');
const app = new Koa();

// x-response-time

app.use(async (ctx, next) => {
  const start = Date.now();
  await next();
  const ms = Date.now() - start;
  ctx.set('X-Response-Time', `${ms}ms`);
});

// logger

app.use(async (ctx, next) => {
  const start = Date.now();
  await next();
  const ms = Date.now() - start;
  console.log(`${ctx.method} ${ctx.url} - ${ms}`);
});

// response

app.use(async ctx => {
  ctx.body = 'Hello World';
});

app.listen(3000);

I knew that we have core.async for async code. Now there’s a scenario, I want to create a very simple server with ClojureScript to mock requests with simple JSON. Any quick solution for such cases? (besides JS interop…)

1 Like
2

I was thinking about whether or not creating async/await for cljs would be a useful project to learn about CPS transforms and along the way I discovered that core.async already does everything async/await does and a whole lot more. The syntax is very different and I think that’s what stopped me from seeing the relation sooner.

In javascript, async/await is based around promises. An async function always returns a promise and any promise can be awaited inside an async function.

In core.async, one uses channels instead. But with a pair of macros we can create the same syntax as javascript:

(defmacro async [& body]
  `(let [out# (chan)]
     (go
       (if-let [res# (do ~@body)]
         (>! out# res#)
         (close! out#)))
     out#))


(defmacro await [bindings & body]
  {:pre [(even? (count bindings))]}
  `(let [~@(mapcat (fn [[bind# val#]]
                     (list bind# (list '<! val#)))
                   (partition 2 bindings))]
     ~@body))

The async macro takes a body, wraps it in a go block and returns a channel which will eventually yield the return value of the body (or close if body returns nil).

await works just like let, except that the values on the right hand side are channels. It would work like this:

(defn slow-inc [x]
  (async
    (<! (timeout 1000))
    (inc x)))

(defn multi-inc [x y]
  (async
    (await [i (slow-inc x)
            j (slow-inc y)
            k (async (* i j))]
      (println [i j k])
      [i j k])))

(multi-inc 4 5)
; => returns a channel
; => prints [5 6 30] 2 seconds later

You can see that the syntax is getting closer to javascript. In addition, you could implement promises using core.async (there already is a promise-chan) and use those instead of channels. In this case that might be more clear.

If you look at the await macro again, you’ll see that it waits for the async values one at a time. We could rewrite it to grab them in parallel as well. If we wanted to be really fancy, we could topologically sort the bindings and parallelise what we can while still allowing dependencies between bindings.

So in answer to your question, no there’s no ready to go solution to async/await in clojure(script) (not that I’m aware of in any case), but core.async give you the building blocks to build whatever solution you need.

5 Likes
3

These macros are cool! Really inspiring.

There’s another direction I was thinking about. Now ClojureScript interops with JavaScript, in which async/await is already supported. If we can create functions the utilize async and await, there’s a chance maybe we reuse modules from Koa2… It maybe a easier way to connect to the whole Node.js ecosystem.

4

I’m no expert on the clojurescript compiler, but from what I can tell, it doesn’t emit the async or await keywords right now, and I don’t think it’s possible to refer to a javascript keyword from clojurescript code. So I don’t think what you suggest is possible, though I wouldn’t mind being proven wrong.

1 Like
5

I wish that there could be a restricted subset of async/await syntax in cljs without all machinery generated by core.async :thinking:

1 Like
6

JavaScript async/await needs to be compiled into state machines too, it’s not like async/await is broadly available. Try using Google Closure directly on JS source with function* & yield.

Side note, compilation approach that core.async takes was informed by the C# implementation of async/await.

2 Likes
7

async/await is not support that broadly, but available in most modern browsers currently https://caniuse.com/#feat=async-functions And in Node.js it was supported in 7.6, now I’m using 8.9 in my laptop. There are many occasions we can use them directly now, especially in Node.

I feel a bit of embarrassment when we are supporting npm modules but async/await still prevents us from it.

8

I’ve been trying to do some things with Lumo and the Slack API, and realized quickly that I needed some kind of sugar to make the async stuff more palatable. I consider core.async but it seemed overkill for what I was doing. So inspired by @thheller’s approach I came up with some similar macros, but based on js/Promise.

They got a bit hairy, especially the await one, and they really need more testing, but so far I really like them already.

Usage

;; Pretty basic syntactic sugar macro for creating a promise. Good for when you
;; have some API that takes callbacks, and want to transform it into promises.
(def p
  (promise [resolve reject]
    (callback-based-api (fn [result error]
                          (if error
                            (reject error)
                            (resolve result))))))

;; Async macro which just returns a promise that resolves to the result of the
;; block. Note that due to how promises work you can also return a promise from
;; the block, or if you throw in the block it turns into a rejected promise.
(def num (async
           (inc 41)))

(def num2 (async
            (async (dec 24))))

;; The fun part: wait for multiple promises to resolve, and bind their results
;; to local variables.
(await [n1 num
        n2 num2]
  (println (+ n1 n2)))


;; To handle the rejection of a promise, use `(catch ...)` directly inside the
;; await block. Note also that this is chaining resulting promises. fetch
;; returns a promise, and (.text page) returns a promise based on the result of
;; fetch.
(await [page (fetch "https://clojure.org/")
        text (.text page)]
  (println (re-find #"clj-header-message\">.*<" text))
  (catch :default e
    (println "Failed to access the network.")))

I’ll need to dogfood this some more to see what the rough edges are, but so far I feel this could turn into a pretty useful little library.

My first version of await used js/Promise.all, which was pretty straightforward, but when I started using fetch and ended up with nested await blocks, I figured it should really be more like let in that it lets you chain results.

;; before
(await [page (fetch "https://clojure.org/")]
  (await [text (.text page)]
    ,,,))

;; after
(await [page (fetch "https://clojure.org/")
        text (.text page)]
  
  ,,,)
1 Like
9

I think you meant @tgetgood not me :wink:

10

your avatars look the same…

11

Oh my I didn’t even realize… Sorry @tgetgood! An avatar for either of you might not be a bad idea :wink:

12

How about Koa then? Any chance we use this macros for Koa?

13

No idea, I know nothing about Koa. If it uses promises then sure, I imagine they could be helpful.

14

I think Koa may support Promise, based on old snippets:

app.use((ctx, next) => new Promise((resolve, reject) => {
  ldap.search('dn=users', {filter: 'cn=waldo'}, (err, result) => {
    if (err) return reject(err)
    result.on('searchEntry', entry => {
      ctx.body = entry.object
      resolve()
    })
    result.on('error', reject)
  })
})

might be interesting to explore…

15

one more piece found on Google https://funcool.github.io/promesa/latest/#code-async-code-macro-general-purpose

(def p (async
         (dotimes [i 3]
           (p/await (p/delay 100))
           (println "i=" i))
         10))

@p
;; i=0
;; i=1
;; i=2
;; => 10
16

async/await is just syntax sugar for Promise.

If you look closer at how the Closure Compiler translates async/await to ES3 it looks pretty similar to what core.async/go does, except that core.async uses channels under the hood which are much more flexible and powerful.

Modern JS engines don’t need to do the translation in code but instead to the same at the runtime level. Either way its just syntax sugar for callbacks, no other magic involved.

I don’t recommend just looking at the “happy path” and trying to write a macro to make everything look like async/await. What happens if something goes wrong? Things fail in real apps. Try adding in some error handling and timeouts. Suddenly core.async looks soo much better (IMHO).

Say you want to only wait 1sec for something and otherwise timeout and do something else?

core.async looks nice and can easily select one of many results.

(alt!
  some-chan
  ([x]
    (when (some? x)
      (process-some-result x)))

  other-chan
  ([x]
    (when (some? x)
      (process-other-result x)))

  (async/timeout 1000)
  ([_]
    (do-something-else))

How does this look in async/await?

2 Likes
17

Thanks for the vision of async/await in transpiler. :clap:

However we need to confirm that how JavaScript engines implement async/await today, since V8 has native support for that already. Not sure if it has its own solution(possibly like a transpiler though).

My purpose was to figure out a way I can use Koa with ClojureScript. If possible, I want to try:

(.use app
  (async-fn []
    (await (get-whatever))
    (println "done")))

Or maybe we have to pick modules from Koa ecosystem by ourselves and create a new library calling it koa-cljs or something.

Somehow scripting Koa is really simple in js/CoffeeScript:

Koa = require 'koa'
app = new Koa

app.use (ctx) ->
  ctx.body = 'Hello World'
  await return

app.listen 3000
18

Promise is part of the spec for async/await so every engine is doing it that way.

Sure you can use koa with ClojureScript. There is no magic involved.

(ns demo.server
  (:require ["koa" :as koa]
            [clojure.string :as str]))

(defonce app-ref (atom nil))

(defn handler [ctx next]
  (set! (. ctx -body) "hello world")
  (next))

(defn delay-by-two-seconds [ctx next]
  (-> (js/Promise.
        (fn [resolve reject]
          (js/setTimeout resolve 2000)))
      (.then next)))

(defn upcase-async [ctx next]
  (-> (next)
      (.then
        (fn []
          (set! (. ctx -body) (str/upper-case (. ctx -body)))
          ))))

(defn main []
  (let [app (koa.)]
    (reset! app-ref app)

    (doto app
      (.use delay-by-two-seconds)
      (.use handler)
      (.use upcase-async)
      (.listen 3000))))

At least thats what I can gather from glancing over the docs. It is all built on top of the usual “middleware” stack. So each “use” is supposed to call “next” while making any adjustments to “ctx” it wants (mutable object). It is build on top of Promise.resolve so a handler may either return a Promise or any other value.

You can hook this up with core.async by using a little utility that created a Promise that resolves once a go block finishes.

I might be wrong, I literally looked at the docs for 30 seconds. It is just based on Promise nothing else. async/await is just syntax sugar for that and not required at all.

2 Likes
19

Agreed that since we are sure that Koa takes promise, script Koa with js interop is not big problem anymore. My brain ran really slowly. I may need more syntax sugar to make writing such code in a easy way.

20