I want to learn about running ClojureScript on AWS Lambda, but the ecosystem is hard to understand. How do people deploy to Lambda? How do you test your code locally? I’d be grateful for experience reports, recommendations and other links that can point me in the right direction.
I have been using serverless-cljs-plugin for lamdbas for very long time. Serverless Framework is easy to get started with and makes it easy to accomplish big tasks like setting up an API Gateway.
Now, my personal choice was to use the lumo compiler (which I contributed) plus inf-clojure but you can easily use cljs + lein - it actually has been developed with that in mind.
Serverless is also so pluggable that it is very easy to write custom tasks for many different purposes, when you cannot find a plug-in that does it already.
So when does it break? In my experience the multi cloud promise has not really been delivered. Also, if you plan to use other AWS resources you might need to write a lot of Cloud formation. You can use Serverless for it, but I don’t like Cloud formation. Serverless does not play very nicely with CI also, because it stores locally the AWS state of the deployed resources.
Therefore, recently I moved to compiling the zip and using Terraform for deployment. Terraform IMHO is the right tool for the job when you deal with properly sized stacks.
Too many words, let me know if you have any specific question
Not related to Cljs, but Martin Fowler wrote a very detailed overview about the whole serverless thing https://martinfowler.com/articles/serverless.html
It includes many references to AWS Lambda functions, you might find it useful.
Good stuff, keep em coming! Quick summary of what I’ve learned so far.
Re: the article on martinfowler.com, very useful, I always appreciate longform explanations.
Re: deployment tools, I found that even just getting a simple “echo” Lambda/Gateway endpoint off the ground requires creating a whole litany of resources (function, gateway, resource, method, permissions, policies). Setting things up manually through imperative CLI commands is possible but tedious, especially when updating an existing stack.
So almost everyone uses a declarative description of the resources. CloudFormation does that but it’s so verbose that it gets unwieldy fast. The main options seem to be:
- AWS SAM is an extension of CloudFormation’s yaml syntax. It includes templates that expand out to CF resource descriptions.
- serverless is a project that targets multiple cloud providers and provides a simplified CLI for creating APIs using lambda functions
- terraform is a general-purpose, declarative cloud provisioning tool which also includes a module for Lambda/API Gateway
Quick update. SAM has a lot of rough edges. You can make it work but you need to dive into CloudFormation and be prepared to search for snippets on Github/StackOverflow.
Do you have a requirement to run ClojureScript instead of Clojure? If you can run Clojure instead, Ions appears to be a relatively easy way to use lambdas:
Even if Ions aren’t a good fit, it may be worth exploring using Clojure instead of ClojureScript. There is a good analysis of the tradeoffs, including performance and startup time, here:
(Of course, if you have a hard requirement to use CLJS - say, to interop with some existing CLJS or JS code - then this won’t apply )
@bbrinck, good point. Clojure on Lambda is an option that we’re exploring as well. The JVM is still the better platform for writing servers compared to Node.
FWIW, from my experiments the CLJS part of the proces is pretty straightforward. All you need to do is to compile to a bundle that does the equivalent of this line, e.g.:
(goog.object/set js/module.exports "handler" (fn my-handler [event context callback] ...))
The trickier bit by far has been dealing with general lambda bureaucracy, i.e. setting up all the necessary resources and permissions on AWS.
Ah, I see. In that case, definitely take a look at Ions. My understanding is that they solve exactly this problem - they provide sensible lambda configuration out of the box and also allow rapid deployment of code changes via AWS codedeploy.
I’ve used both Clojure and ClojureScript for AWS Lambdas, although both in personal projects (read: not a lot of traffic, no major constraints, etc.). Some notes from my experience:
- I originally did a lot via the AWS CLI and NPM scripts. I’ve also done some simple Lambda deployment using Terraform (this is nice and easy if your Lambda does NOT use API Gateway, that gets more tedious in Terraform). This worked and I learned a lot, but is very manual, and now…
- I’d highly recommend using Serverless framework to deploy. Depending on your needs, you can also use this to setup a lot of other AWS infra, e.g. I use it for DynamoDB, S3, CloudWatch alerts, etc. Further, it will make your life easier when using AWS profiles (if you have multiple AWS accounts, etc.).
- the serverless cljs plugin works well for that, or you can do your own and use Serverless hooks to build.
- My take on Clojure vs. ClojureScript is: it depends I prefer Clojure, but on a low volume service where you will frequently deal with cold starts, the JS/ClojureScript Lambdas absolutely run faster (it’s night and day). That said, once a Lambda is warm, the Clojure ones run faster. There are some ways (including a Serverless plugin) to keep your Lambda warm, but this is kind of a lie. It only keeps a single one warm. If you have minor traffic, then that may actually work, as unless you have a lot of concurrent traffic (even occasional simultaneous requests will likely not trigger Amazon to fire up another instance), it can work.
- Also, I’ve found the libraries for working with some AWS services like Dynamo and SES, to be more to my liking (and simply work out of the box) on Clojure. I’ve had a lot of issues getting some of the ClojureScript ones to work (or even have my compile process succeed)). So right now, I’m favoring Clojure, but do have some written in ClojureScript where I knew the use was going to be super sparse, the function is simple, and I didn’t want to keep it warm, etc.
- I build Clojure Lambdas with Lein, and use the serverless-plugin-scripts plugin to add custom commands that get run as part of my build (these become just regular “serverless” commands (e.g. you can run “serverless my-command”), but then combined with the “hooks” feature, you can have that triggered at various stages of your build/deploy, so I use this to build the Clojure code, but also I have it run smoke tests on the then deployed service after the deploy.
- All that said, honestly, if you prefer ClojureScript, go with it. I am guessing if you do run into any library trouble, you can likely use an NPM module directly instead (maybe not quite as nice in the code, but if you’re used to the Node ecosystem, then you’ll likely find it fine).
- Anywhere I cannot use Serverless to orchestrate the infrastructure, I use Terraform, but you could use whatever you like of course.
Hope that helps, have fun!
I have a few Alexa Skills running CLJS on Lambda. I did a walkthrough here: https://www.youtube.com/watch?v=JXlmubcb5xo
I started with this library : https://github.com/nervous-systems/cljs-lambda
From there I just replaced their code with my own. It was pretty simple. Here is a gist: https://gist.github.com/jreighley/b6d394d0efa743292bf32d776be97682
The primary advantage to CLJS over CLJ is the startup time.
A heads up, if cold start time is a concern, Lambda functions in a VPC take 10s (in our tests, sometimes 20s) to start up. As it happens, the only way to access RDS (Amazon’s Database-as-a-Service offering) is when your function part of a VPC.
CLJ starts up more slowly than CLJS but should be under 2s. So if your function makes database connections the choice of language might not matter as much, comparatively speaking.