Can you do anything useful with it?
At CloudNative, we took that as a challenge. Could we, using AWS Lambda, create a multi-user, dynamic web application without using EC2 or resorting to hacks? I mean no instances, no ELBs, not even a security group. And especially no hard-coded AWS credentials.
We did, and it's called Lambda Chat.
Code on GitHub
Lambda Chat is a simple chat application using AWS Lambda, SNS, DynamoDB and S3.
Remember, one of our goals was no hacks. It turns out AWS has a solution for this: Web Identity Federation. This allows users to sign in using some existing account (Amazon, Google and Facebook (they just added Twitter too)). Signing in issues a token to the user, which can then be used to call AWS's Security Token Service (STS) to issue temporary AWS credentials. Those credentials only last an hour, and are used to publish messages to SNS. (Note: Check out the AWS Web Identity Federation Playground to learn more)
Challenge 2) How do we read the data back out?
We have temporary AWS credentials -- we could read the message out of DynamoDB directly. Hmmm, that means giving the front end more permissions. Not hard, but can it be avoided?
Why not use DynamoDB Streams to call another Lambda function. This function can read the data out of DynamoDB once, and write it to a JSON file stored on S3. DynamoDB is fast, but not for scans. By doing it this way, we can store the last 20 messages in S3, which acts as a DynamoDB cache, and it means we can keep the permissions down to just SNS publishing.
The resulting flow becomes:
◎ ◎ ◡ │(1) │ ┏━━━━━━━━━━━━━━━━━━┓ │ ┃ ┃ │ (2) ┃ Google OAuth2 ┃ (4) │ ┌─────────────▶┃ API ┃◀────────────┐ │ │ ┃ ┃ │ │ │ ┗━━━━━━━━━━━━━━━━━━┛ │ ▼ ▼ ┏━━━━━━━━━━━━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ (3) ┃ AWS STS ┃ ┌─▶┃ Website ┃◀──────────────────────────▶┃ AWS Web ID Fed ┃ │ ┃ ┃ ┃ ┃ │ ┗━━━━━━━━━━━━━━━━━━┛ ┗━━━━━━━━━━━━━━━━━━┛ │ │ │ │ ┏━━━━━━━━━━━━━━━━━━┓ │ │ ┃ ┃ │ └───────▶┃ SNS Topic ┃ │ (5) ┃ ┃ │ ┗━━━━━━━━━━━━━━━━━━┛ │ │ │ │ ┏━━━━━━━━━━━━━━━━━━┓ │ │ ┃ ┃ │ └──────▶┃ Lambda fn1 ┃ │ (6) ┃ ┃ │ ┗━━━━━━━━━━━━━━━━━━┛ │ │ │ │ ┏━━━━━━━━━━━━━━━━━━┓ │ │ ┃ ┃ │ └─────▶┃ DynamoDB Table ┃ │ (7) ┃ ┃ │ ┗━━━━━━━━━━━━━━━━━━┛ │ │ │ ┏━━━━━━━━━━━━━━━━━━┓ │ │ ┃ ┃ │ │ ┃ Lambda fn2 ┃◀─────────┘ │ ┃ ┃ (8) │ ┗━━━━━━━━━━━━━━━━━━┛ │ │ │ ┏━━━━━━━━━━━━━━━━━━┓ │ │ ┃ ┃ │ └────────────┃ S3 Object ┃◀─────────────┘ (10) ┃ ┃ (9) ┗━━━━━━━━━━━━━━━━━━┛
- The user opens their browser and goes to the website which is hosted entirely on S3
- The user signs in with their Google account and gets back an
id_tokenis sent to get temporary AWS credentials from STS.
- STS verifies the token with Google
- The users types in a message, hits ENTER, and the website publishes the message to an SNS Topic.
- A Lambda function is triggered by the SNS message, which has the contents of the message, and...
- Stores the message in a DynamoDB table
- The process of adding a new chat message to the DynamoDB table triggers another Lambda function. This requires the currently-in-preview DynamoDB Streams feature. This second Lambda function reads the last 20 messages from DynamoDB, and...
- Writes them to an S3 object in JSON format
- The website polls the S3 object every second, and updates the chat box with any new messages it finds.
As mentioned above, we have open sourced this, and made it kinda-sorta easy to get up and running in your own account if you wish.
- Our CloudFormation script (using troposphere) brings up all the AWS resources
- An update script is used to push the website code to S3
- Kappa packages and deploys both Lambda functions
- Easy clean up process
Now a note on the latency: we purposely didn't fake the display of the message you wrote when you hit ENTER. We wanted you to see what the latency is. I have seen it as high at 13 seconds, and as low as 2. Interestingly, it gets faster the more people use it.
And now for the cost: This thing costs just a few dollars per month, and in theory scales near infinitely. We would need to increase the read/write capacity on the DynamoDB table, but not until a few 100 people are using it concurrently. Chat rooms tend to be self-limiting anyway, as humans can only read so many messages in a given time.
Lambda Chat is no Slack replacement. It is however, a well documented and open source example of how to do something useful using AWS Lambda. Thanks for reading.
-- Ready to have reliable infrastructure? Sign up for CloudNative today. --