Have everyone here actually looked at promesa’s async/await macro? I honestly can’t see why it doesn’t fully solve the problem?
Checkout this blog post for a simple example:
The only downside it you need to use bluebird promises, the benefit is bluebird promises are much faster then native JS.
I must say, it is a bit frustrating that I keep pointing out the two main disadvantages of user-space solutions and then folks follow up with responses that ignore these points and reiterate the same points about user-space solutions. Feel free to disagree with my points, but ignoring them is super annoying. To respond that it is “possible” to code something that will work completely misses my two main points:
For example:
(p/alet [x (when true (await (p/delay 100 "foo")))]
(println x))
;; =>
;; WARNING: Can't take value of macro cljs.core/when at line 7 /Users/justinlee/test/cljs-qs/src/tst/core.cljs
;; WARNING: Use of undeclared Var tst.core/await at line 7 /Users/justinlee/test/cljs-qs/src/tst/core.cljs
(try
(throw (new js/TypeError "unexpected"))
(catch ExceptionInfo e
(println "outside async: ExceptionInfo catch"))
(catch :default e
(println "outside async: default catch")))
;; =>
;; "outside async: default catch"
(async
(await (p/delay
(try
(throw (new js/TypeError "unexpected"))
(catch ExceptionInfo e
(println "inside async: ExceptionInfo catch"))
(catch :default e
(println "inside async: default catch"))))))
;; =>
;; silence of the lambs
I’m not complaining that these libraries have bugs. I couldn’t fix them myself, but I suspect people will be finding difficult corner cases until the end of time. I just think that if the native solution could be made to work, then we get to use the power of a billion eyeballs on the javascript implementations and save this smaller community’s resources to solve other problems. (For the record, I, personally, ran into these issues myself pretty much immediately upon trying to use them to port a serious piece of javascript to clojurescript.)
Also note @shaunlebron’s point about cancellable generators.
Finally, I get that there is justifiable angst about the javascript churn, but async/await is a syntax that came from F# in 2007, then went to C#, and only then came to Javascript. It’s essentially the same syntax as go and <!. I understand it’s in python, kotlin, and is coming to other languages like rust. This is a tried and true piece of syntax that is obviously a solid building block for event-based i/o.
I strongly favor conservatism in my language and convenience is very subjective. I’m still for adding async/await in general but the power of Lisp lets us do this in different places. Doing it at the compiler level requires much more thought than doing it in a library. I don’t know what the correct place is, just trying to discuss the options.
Async functions are contagious, once you go async everything has to go async. They are not the correct solution for every problem. Timothy Baldridge gave this excellent talk (already linked above) about the lessons learned with core.async. Those are the exact same lessons in async/await that the JS world has not learned yet. async function should be used just as sparingly as go. Doing async correctly is really really hard and just adding some syntax to make the easy case look easier does not help anyone. They are the “hot new thing” and are probably going to stick with us for a while though. Whatever happened to FRP (e.g. RxJS)? That used to be the answer to all of this?
Nobody is ignoring you. You are correct that the situation is not great. I do think that those are fixable issues however. At least in the core.async case, I have never used (and don’t like) promesa so I can’t speak for that. Yes, letting the JS world deal with the problems and not having to deal with them ourselves is certainly the easier path. That does not make it the correct path though. As with everything someone just has to go and fix it.
I think the Erlang crowd will disagree wildly with you here. The go (lang not macro) folks would probably disagree as well.
if all you have is a hammer, everything looks like a nail
Don’t dismiss other options just because JS has dictated that they are going that route. Yes, we should absolutely have an easy to use/understand interop option but that does not mean we have to use the exact same implementation.
I’d love to learn what you are using for working in promises in a non-convoluted way. I have used nothing so far, and I’d like to hear some opinions 
Most of my own code uses core.async. Most the node interop is still callback based. I never had a case for await since my own code is otherwise already setup with channels in mind, so whenever I get a promise (e.g. js/fetch) the result is just put onto a channel which already exists. I find it much easier to think in terms of channels these days.
(-> (some-js/asyncFn 1 2 3)
(.then one-thing)
(.then another-thing))
is also completely acceptable to me.
I’m a bit late to the party, but looking at the debate from a distance makes it pretty clear that two problems are tackled at the same time (which is not the lisp way)
My main concern with javascript’s solution (hence with this proposal) is that it endorses promises at the syntax level, and promises (especially in JS flavor) are a very opinionated solution to a very hard problem. They promote imperative programming because they’re eager, they don’t support cancellation, and implicit flattening is just nonsensical. core.async has this problem as well because it forces ubiquitous usage of channels, which are opinionated as well and have their own drawbacks. Implementing error handling or cancellation on top of channels has overhead and feels clunky. go blocks are eager as well, making it inherently imperative. Clearly, asynchronous programming is not a solved problem and that’s why many developers have their own taste on which strategy is the best, making discussions emotive and subjective.
On the other hand, the inversion of control problem looks much less far from being solved and it would be really nice to see any effort to make it more generic and more solid in clojure. It makes sequential logic arguably more expressive than what monadic binding provides. The IOC mechanism used in the implementation of core.async go blocks is used in several other libraries, which is a good sign that the idea is good on its own. I have spent a fair amount of time myself digging into the internals to implement missionary’s sequential processes, and the amount of work required to decouple it from channel semantics is very reasonable. As a starting point, we could have it as a separate library and think harder about how to cover each situation (for instance, to have a less hacky way to throw an exception from an asynchronous parking), and fix some annoying corner cases such as nontrivial exception handling or self-hosted clojurescript support. Maybe in a second time we could think about improvements in the compilation stage to support host platform features such as javascript’s new syntax. This would be the lisp way.
Thanks. That syntax is also quite acceptable to me, but you do have to define one-thing and another-thing as functions. If the code is already written with this in mind, that’s fine — sometimes something like alet has its uses.
Do you use channels for Node callbacks as well? Essentially just using #(put! % c) as the callback?
I hadn’t really understood clearly your points, now I do. I’ll address them.
I think the major objection I have is at a higher level. You point out an implementation choice, like, how would ClojureScript implement async/await, and you say it could do so by leveraging ES8 (aka ES2017)'s implementation. I don’t disagree with that, if its possible to do so. If its a reliable implementation, you’re right, could make stack traces better and maybe its robust and already covers all edge cases.
But, I have an objection before we even go there, which is with adding async/await semantics to Clojure. Remember that ClojureScript is a Clojure to JavaScript compiler. Thus I feel any discussion to modify fundamental language semantics are a Clojure level discussion. So we should discuss that.
That’s because async/await is not a question of interop in my mind, since the actual things to interop with are Promises, and that’s already fully supported. Async/await are language level syntax. So really the debate is with adding async/await syntax to Clojure. And to be honest, when it comes to that, I think its Rich Hickey’s way or the highway. But I’m all for having an open conversation about it none the less.
So, if it was decided that Clojure would add async/await semantics, then maybe the ClojureScript implementation could leverage native ES2017 async/await. Though it would still need to transpile back to older ES targets, not sure if Google Closure can handle that.
In that perspective, I thus am not in favor of the proposal. This interop syntax will just end up being used like they are native Clojure syntax features I’m afraid. For now, without the use of libraries, Clojure doesn’t have any native mechanism for async modeling.
There’s an old proposal by Stuart Sierra to extend that: Home - Clojure Design - Clojure Development
It suggests adding callbacks to promises. This would then allow promises to be implemented in ClojureScript as well. Currently, it can’t, because they only work with multiple threads. If they were extended to support callbacks, you could use them in single threaded environments like ClojureScript. After a call to deliver a value to a promise, you just execute all registered callbacks in the order they were registered with the promise.
Now, back when the proposal was made, deref (aka @) was suggested to just throw an exception in ClojureScript if the promise is not yet delivered. But, given ES2017 async/await machinery, it might be possible to have deref implemented using await.
So, I’m not saying there’s no pros in implemented async/await with the native ES2017 solution. There could also be downsides to that as well, I haven’t really explored it. But I think the choice to add a async/await like syntax and semantic to the language must be discussed first.
In the meantime, since I think its a very long road to any changes in regards to Clojure’s native support for async/await, promesa can be used in equivalent ways. Other libraries could be written to mimic JavaScript’s behavior even more closely probably. So for those interested in available right now solutions…
Here’s the puppeteer example using promesa:
(ns cljs-playground.core
(:require [cljs.nodejs :as nodejs]
[promesa.core :as p]
[promesa.async-cljs :refer-macros [async]]
[puppeteer]))
(defn ->bb
"Coerce native JS Thenables to bluebird ones, so they
can be used with promesa."
[es6-thenable]
(js/Promise.resolve es6-thenable))
(defn -main
[& args]
(async
(let [browser (await (-> puppeteer .launch ->bb))
page (await (-> browser .newPage ->bb))]
(await (-> page (.goto "https://example.com") ->bb))
(await (-> page (.screenshot #js {:path "example.png"}) ->bb))
(await (-> browser .close ->bb)))))
(set! *main-cli-fn* -main)
Make sure you have a dependency on promesa and core.async and that you’ve got puppeteer available for consumption from npm obviously.
And here is an example for error handling:
(async
(-> (p/promise (fn[] (throw (ex-info "unexpected" {}))))
(p/catch ExceptionInfo (fn[error] "recovered from ex-info"))
(await)
(println))
(-> (p/promise (fn[] (throw (ex-info "unexpected" {}))))
(p/catch (fn[error] "recovered from any exceptions"))
(await)
(println))
(-> (p/promise (fn[] (throw (ex-info "unexpected" {}))))
(p/catch (fn[error] "recovered from finally example"))
(p/finally (fn[] (println "run this if we catch or not.")))
(await)
(println)))
This is unlike JS async/await. If a promise throws an exception, it can’t be caught by a try/catch, you have to add a p/catch and p/finally to them.
They are already in Clojure. You can easily do (future (some-async-work)) or promise and deref that later or use any other kind of JVM concurrency util like ExecutorService. The issue that you can easily do a blocking deref without your whole program deadlocking but JS needs native syntax to emulate this. It still isn’t anywhere to close to actually being able to block but the code does look much more “friendly” overall.
Besides that generators are just a more generic version of the IOC code rewriting the core.async go macro does. In fact you could make cljs.core.async significantly simpler by using generators. js-csp works like core.async but completely without macros. This would make core.async a lot easier to debug and it would generate way less code as well. Closure also has no issue with translating the code to ES3 if needed.
Hum, maybe we have different terminology. I’m talking about async semantics in the sense of events really. So the Clojure future would be synchronous to me in that work starts right away (its not queued), and you must poll for the result, which could block your thread, ie. long poll.
I’m not talking about concurrent/parallel.
And then on top of those possible async semantics, I’m specifically talking about adding the async/await stackless coroutines semantics like in JS or C#.
So like the link I posted suggests, it would mean adding callbacks to Clojure’s promise and future for example. And then adding the async/await syntax sugar to it. That would be one way. Adding continuations or generators are another.
The impossibility to try/catch a promise await in an async block is due to a flaw in promesa implementation. This is not an inherent limitation of the macro-based ioc design.
Agree that it is possible to do this in user space, as I mentioned above. It’s also clear that it is hard to get right and tooling is always going to be subpar. Part of the subtext here is that this community would be better served with more users and more contributors so that there is more energy on polishing existing software. For historical reasons, this has always meant catering to the java->clojure->clojurescript pathway first and foremost. I am advocating for giving some attention to the javascript->clojurescript pathway too.
I just don’t understand the macho attitude of: “Real programmers don’t use well-understood and well-supported solutions with good tooling; they reimplement everything themselves and spend a decade hammering out hideous bugs while inspecting obscure stack traces.” I’ll never understand that. This is the Lisp Curse in action right here.
I know this is heresy, but has anyone considered that this is a broken and toxic way of doing things? The goal of keeping parity between the two languages is admirable, but not when you harm one of the two languages. Should we take out all thread-related functionality from Clojure because ClojureScript doesn’t have that stuff? Of course not.
Isn’t that what we’re doing?
General point on your tone - I think your arguments can be made just as convincingly without all of the unnecessary rhetoric.
We of course spend enormous amounts of time on this. But our decisions and analysis of tradeoffs are informed by the both the needs of the existing user base and data collected from the annual surveys. From what I’ve seen over the past seven years there’s scant evidence that the majority of users care that much about generator/async/await support or that they believe such work should prioritized over more impactful JavaScript ecosystem integration work.
While it’s great that people spend time putting together and discussing these kinds of proposals, in the end I have to see evidence that it’s going to be meaningful for the amount of effort that will be expended. So far I don’t see this proposal or any similar ones getting prioritized any time soon. There is a simply too long a line of bigger fish to fry for the foreseeable future
Actually I’m failing to convince anyone, regardless of my rhetoric. But your point is taken.
Well, I feel the proposal is about interop. The claim is certain JS libs expose APIs that all return Promises, and the APIs are hard to use from ClojureScript without the convenience of async/await. For this, I feel a user space library is good enough. You don’t even necessarily need async/await, just anything that helps you work with Promises would help in this front.
I’ve yet to see anyone discuss the merits, downsides and alternatives to async/await for asynchronous programming. The Clojure team already chose to go with CSP style concurrency for asynchronous modeling. Even then, they preferred to make it a user level lib. Every programming language is currently in a debate about if they should add generators, coroutines, continuations, CSP, actors, fibers, async/await etc. to their language. So maybe we should discuss the same.
You already have core.async, what’s wrong with it? Where would async/await be better? Where is it worse? Apart from the interop inconveniences? Why not add continuations instead? Or full coroutines? Should they be stackless or stackful? Maybe generators is a better avenue, etc.
Or is it really just the interop inconvenience people are having?
P.S: There’s a quote I found form Timothy Baldridge where he mentions core.async started with a more async/await model, and ended up with CSP from Redirecting to Google Groups
It’s interesting to note that core.async started as something that looked a lot like C#'s Async/Await, but that was dropped in favor of CSP pretty quickly. So there’s reasons why the language isn’t optimized for this sort of programming style.
So I feel they must have had a lot of thoughts around async/await vs CSP
P.S.2: I also found a proposal for Release 1.6 to add async/await, which never seemed to have happened: Home - Clojure Design - Clojure Development for the interested.
P.S.3: And a similar thread debating async/await vs csp on the ClojureScript mailing list 3 years ago Redirecting to Google Groups
On this topic, a thin layer over promises can also be found here: https://github.com/jamesmacaulay/cljs-promises
To follow up on this idea, I just released a proof-of-concept for a generic coroutine macro. As you can see from the examples, it can be easily leveraged to implement various coroutine patterns, including generators and async/await.
This just popped up in my twitter feed and seemed relevant:
Seems like this will be getting more relevant in the future as async/await will get better performance than other approaches.