Async/Generator functions in CLJS - (requesting feedback!)

[thheller]

@didibus

But, I have an objection before we even go there, which is with adding async/await semantics to Clojure.

They are already in Clojure. You can easily do (future (some-async-work)) or promise and deref that later or use any other kind of JVM concurrency util like ExecutorService. The issue that you can easily do a blocking deref without your whole program deadlocking but JS needs native syntax to emulate this. It still isn’t anywhere to close to actually being able to block but the code does look much more “friendly” overall.

Besides that generators are just a more generic version of the IOC code rewriting the core.async go macro does. In fact you could make cljs.core.async significantly simpler by using generators. js-csp works like core.async but completely without macros. This would make core.async a lot easier to debug and it would generate way less code as well. Closure also has no issue with translating the code to ES3 if needed.

[didibus]

@thheller

They are already in Clojure. You can easily do (future (some-async-work)) or promise and deref that later or use any other kind of JVM concurrency util like ExecutorService .

Hum, maybe we have different terminology. I’m talking about async semantics in the sense of events really. So the Clojure future would be synchronous to me in that work starts right away (its not queued), and you must poll for the result, which could block your thread, ie. long poll.

I’m not talking about concurrent/parallel.

And then on top of those possible async semantics, I’m specifically talking about adding the async/await stackless coroutines semantics like in JS or C#.

So like the link I posted suggests, it would mean adding callbacks to Clojure’s promise and future for example. And then adding the async/await syntax sugar to it. That would be one way. Adding continuations or generators are another.

[leonoel]

The impossibility to try/catch a promise await in an async block is due to a flaw in promesa implementation. This is not an inherent limitation of the macro-based ioc design.

[jmlsf]

@leonoel

This is not an inherent limitation of the macro-based ioc design.

Agree that it is possible to do this in user space, as I mentioned above. It’s also clear that it is hard to get right and tooling is always going to be subpar. Part of the subtext here is that this community would be better served with more users and more contributors so that there is more energy on polishing existing software. For historical reasons, this has always meant catering to the java->clojure->clojurescript pathway first and foremost. I am advocating for giving some attention to the javascript->clojurescript pathway too.

I just don’t understand the macho attitude of: “Real programmers don’t use well-understood and well-supported solutions with good tooling; they reimplement everything themselves and spend a decade hammering out hideous bugs while inspecting obscure stack traces.” I’ll never understand that. This is the Lisp Curse in action right here.

@didibus

So really the debate is with adding async/await syntax to Clojure. And to be honest, when it comes to that, I think its Rich Hickey’s way or the highway.

I know this is heresy, but has anyone considered that this is a broken and toxic way of doing things? The goal of keeping parity between the two languages is admirable, but not when you harm one of the two languages. Should we take out all thread-related functionality from Clojure because ClojureScript doesn’t have that stuff? Of course not.

@didibus

But I think the choice to add a async/await like syntax and semantic to the language must be discussed first.

Isn’t that what we’re doing?

[dnolen]

General point on your tone - I think your arguments can be made just as convincingly without all of the unnecessary rhetoric.

@jmlsf

I am advocating for giving some attention to the javascript->clojurescript pathway too.

We of course spend enormous amounts of time on this. But our decisions and analysis of tradeoffs are informed by the both the needs of the existing user base and data collected from the annual surveys. From what I’ve seen over the past seven years there’s scant evidence that the majority of users care that much about generator/async/await support or that they believe such work should prioritized over more impactful JavaScript ecosystem integration work.

While it’s great that people spend time putting together and discussing these kinds of proposals, in the end I have to see evidence that it’s going to be meaningful for the amount of effort that will be expended. So far I don’t see this proposal or any similar ones getting prioritized any time soon. There is a simply too long a line of bigger fish to fry for the foreseeable future

[jmlsf]

@dnolen

General point on your tone - I think your arguments can be made just as convincingly without all of the unnecessary rhetoric.

Actually I’m failing to convince anyone, regardless of my rhetoric. But your point is taken.

[didibus]

@jmlsf

Isn’t that what we’re doing?

Well, I feel the proposal is about interop. The claim is certain JS libs expose APIs that all return Promises, and the APIs are hard to use from ClojureScript without the convenience of async/await. For this, I feel a user space library is good enough. You don’t even necessarily need async/await, just anything that helps you work with Promises would help in this front.

I’ve yet to see anyone discuss the merits, downsides and alternatives to async/await for asynchronous programming. The Clojure team already chose to go with CSP style concurrency for asynchronous modeling. Even then, they preferred to make it a user level lib. Every programming language is currently in a debate about if they should add generators, coroutines, continuations, CSP, actors, fibers, async/await etc. to their language. So maybe we should discuss the same.

You already have core.async, what’s wrong with it? Where would async/await be better? Where is it worse? Apart from the interop inconveniences? Why not add continuations instead? Or full coroutines? Should they be stackless or stackful? Maybe generators is a better avenue, etc.

Or is it really just the interop inconvenience people are having?

P.S: There’s a quote I found form Timothy Baldridge where he mentions core.async started with a more async/await model, and ended up with CSP from Redirecting to Google Groups

It’s interesting to note that core.async started as something that looked a lot like C#'s Async/Await, but that was dropped in favor of CSP pretty quickly. So there’s reasons why the language isn’t optimized for this sort of programming style.

So I feel they must have had a lot of thoughts around async/await vs CSP

P.S.2: I also found a proposal for Release 1.6 to add async/await, which never seemed to have happened: Home - Clojure Design - Clojure Development for the interested.

P.S.3: And a similar thread debating async/await vs csp on the ClojureScript mailing list 3 years ago Redirecting to Google Groups

[arichiardi]

On this topic, a thin layer over promises can also be found here: https://github.com/jamesmacaulay/cljs-promises

[leonoel]

To follow up on this idea, I just released a proof-of-concept for a generic coroutine macro. As you can see from the examples, it can be easily leveraged to implement various coroutine patterns, including generators and async/await.

[thheller]

This just popped up in my twitter feed and seemed relevant:

https://mathiasbynens.be/notes/async-stack-traces

[thheller]

Seems like this will be getting more relevant in the future as async/await will get better performance than other approaches.

https://v8.dev/blog/fast-async

[didibus]

@thheller

Seems like this will be getting more relevant in the future as async/await will get better performance than other approaches.

Maybe I’m reading the post wrong, but what they say is they’ve improved the performance of Promises. And the charts actually even show that Promises are even faster then async/await.

[thheller]

I said “in the future”. Right now Promise code performs better. The coming improvements are mentioned later on in the post.

async / await outperforms hand-written promise code now . The key takeaway here is that we significantly reduced the overhead of async functions — not just in V8, but across all JavaScript engines, by patching the spec

And we also have some nice performance advice for JavaScript developers:

• favor async functions and await over hand-written promise code, and
• stick to the native promise implementation offered by the JavaScript engine to benefit from the shortcuts, i.e. avoiding two microticks for await .

[didibus]

Interesting. I’m not sure what makes it faster then hand written Promises. Unless they mean non optimal hand written Promises.

[jmlsf]

The article @thheller posted in August explains one reason why. The gist of it is:

The fundamental difference between await and vanilla promises is that await X() suspends execution of the current function, while promise.then(X) continues execution of the current function after adding the X call to the callback chain. In the context of stack traces, this difference is pretty significant.

[jiyinyiyong]

Do we still have chance to have async/await in ClojureScript?

[thheller]

Only @dnolen can answer that. His last answer was

basically I’ve already rejected this at least 2 times over after past 3 years

and I haven’t seen anything happen in the JavaScript world that makes me think it’s delivering enough value to take it on

and I doubt anyone will start work on it unless that has changed. It also would take quite a bit of work with lots of internal compiler changes so it might just not happen for that reason alone.

I’m still not sure it is a good idea either but given that the Browser support is now better for async/await then manual Promise code it might be time.

[jiacai2050]

Although async/await is just syntax sugar over Promise, it has two advantages:

1. emancipate programmers from chaining then or deep-nested callbacks.
2. make async error handling easy. we can have a big try catch block at the beginning of our app, errors thrown from async/await functions will go up to the upper catch.

core.async solve first advantage as well, but I struggle to handle error in thrown in different functions.
After some search, I found async-error solve error handle issue with two macros go-try, <?. eg:

function fetchImg() {
  return Promise.reject("img");
}

function fetchCss() {
  return Promise.reject("css");
}

async function init()  {
  let img = await fetchImg();
  let css = await fetchCss();
  return img + css;
}

(async function() {
  try {
    let ret = await init();
    console.log("ok = " + ret);
  } catch (e) {
    console.log("e = " + e);
  }
}());

// e = img

(ns myapp.core
  (:require [async-error.core :refer-macros [go-try <?]]
            [cljs.core.async.interop :refer-macros [<p!]]))

(defn <fetch-img []
  (go-try
   (<p! (.reject js/Promise "img"))))

(defn <fetch-css []
  (go-try
   (<p! (.reject js/Promise "css"))))

(defn init []
  (go-try
    (try
      (let [img (<? (<fetch-img))
            css (<? (<fetch-css))]
        (println "ok = " (+ img css)))
      (catch js/Error e
        (println "err = " e)))))

(init)

err =  #error {:message Promise error, :data {:error :promise-error}, :cause img}