AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. AWS Lambda is chosen for its flexibility, the ease of integration with other AWS Services, and reducing the amount of infrastructure you and your team own. But over time, when the number of clients and requests start to increase, and you start caring about latency, you may discover that there is no free lunch. Clients complain about latency, things you've taken for granted when running your software on EC2 or Fargate no longer apply, and costs start to ramp up. In this talk, I'm going to describe some of the lessons learned from working on multiple services backed by AWS Lambda: what are and how to reduce the cold starts, how the JVM makes them even more problematic, when AWS Lambda is more expensive than the less abstract platform, how to use provisioned concurrency and why one of the biggest problems in Computer Science (caching) is even bigger on Lambdas.

1. How to avoid common mistakes
and misconceptions when working
with Java on AWS Lambda
Andrzej Dębski

2. Technicalities
• Disclaimer
• https://github.com/Adebski/LambdaCommonMisconception
sExamples

3. About me
1. A decade of experience with JVM languages, mostly Java
and Scala
2. Working (mostly) on backend services
3. Pratical experience with AWS Lambda since
February 2021
• Maintenance and development of 2 existing services
• Development of additional 2 services/products on
AWS Lambda from scratch

4. Agenda
1. Cold starts (generic and those specific to JVM)
2. AWS Lambda Pricing
3. In-memory caching on Lambda

5. Cold starts

6. Lambda instance lifecycle

10. Cold start type 1 (container recycle)
Time
Duration
(ms)
Init Duration
(ms)
18:16:32 85.24 1821.08
18:18:50 1.95 N/A
18:24:05 1.74 N/A
18:30:40 72.91 1870.40

12. Cold start type 2 (JVM JIT and lazy load)
Code path # Duration (ms) Notes
Code path 1 731.32 First request, cold
start + JVM JIT
Code path 1 19.86 Code is already
loaded by JVM, no
cold start
Code path 1 4.31 Further optimizations
Code path 2 1261.40 JVM executes new
branch for the first
time
Code path 2 2.12 No JVM overhead

13. Reducing cold starts
1. Provisioned concurrency (best but costly)
• Keeps a set of instances ready to respond to requests
2. "Background" traffic (best effort)
3. Use a different language
like Go or use AOT compilation through GraalVM
• https://shinesolutions.com/2021/08/30/improving-cold-
start-times-of-java-aws-lambda-functions-using-
graalvm-and-native-images/

15. Cold start for provisioned concurrency
Code path # Duration (ms) Notes
Code path 1 33.12 Small overhead, first
request to Lambda
Code path 1 2.09
Code path 1 2.34
Code path 2 2.03 New request type
handled for the first
time
Code path 2 2.4

16. Provisioned concurrency best practices
1. Make sure the function is invoked using the alias.
2. Monitor (and alarm) on metrics
• ProvisionedConcurrencySpilloverInvocations
• ProvisionedConcurrencyUtilization
3. Pre-warm the code paths in the constructor
4. Use code deploy policies to gradually deploy new
function revisions.

17. Recap
1. Cold starts are real and they affect the
JVM Lambdas even more.
2. Either accept them
or pay for provisioned concurrency (and use it well).
3. Monitor and instrument your functions
to understand the bottlenecks in your code.

18. $$$

19. Lambda pricing
1. Lambda price (https://aws.amazon.com/lambda/pricing/):
• On demand: billed for every GB-second (sum(duration ) *
allocated memory) + number of invocations
• Provisioned concurrency is more complicated. We pay for
keeping the containers "warm"
2. Free-tier

20. How to calculate AWS costs
1. AWS pricing calculator
• https://calculator.aws/#/
2. Cost estimates for the following examples:
• https://tinyurl.com/yk7v3wek

22. Assumptions
1. "Average" request time: 200 ms
2. Monthly costs
3. Region: us-east-1
4. Focus on compute cost, ignore everything else

23. Scenarios considered
1. "Hello world": on-demand vs provisioned capacity
2. Smallest Fargate vs Lambda
3. StackExchange on Lambda vs Fargate

24. "Hello world", 1 request per second
256 MB RAM
On demand Provisioned
capacity of 1
0.33 $ 4.55 $

25. Fargate, 1 request per second
1769 MB RAM 1vCPU
2GB RAM
On demand Provisioned
capacity of 1
Fargate
8.8 $ 28.28 $ 36.04 $

27. Simulating Stack Exchange
1. https://stackexchange.com/performance
• 1.3* 10^9 monthly page views means ~495 requests per
second
• For Fargate we assume 9 tasks to protect from AZ outage, 82.5
RPS per task.
• For Lambda provisioned concurrency try to handle every
request with prov capacity.
• 495 (RPS) / 5 (requests per second per container) = 99

28. SE, 495 requests per second
1769 MB RAM 1vCPU
2GB RAM
On demand Provisioned
capacity of 99
Fargate,
9 tasks
7,744.27 $ 6,502.63 $ 324.36 $

30. Recap/tips&tricks
1. Free-tier
2. Use the AWS Pricing Calculator
3. With more traffic Lambda gets really costly
4. Take advantage of the "free" compute during init phase
for on-demand lambdas
5. For consistent workloads provisioned concurrency may
be cheaper
6. Always consider factors other than $$$ when evaluating
technologies

31. https://hichaelmart.medium.com/shave-99-93-off-your-
lambda-bill-with-this-one-weird-trick-33c0acebb2ea

32. Caching

36. Caching in Lambda - challenges
1. No hard guarantees when the container will be recycled
2. Best effort async updates
3. No control over the routing algorithm
(no sticky sessions)

37. Caching in Lambda - approaches
1. (Mostly) rely on out of process cache
2. Use different compute platform if L1 cache is critical
3. Use Lambda extensions
• That's how AWS AppConfig does it

38. Caching in Lambda - extensions
1. Code that can execute alongside your Lambda
2. Extension continues to execute AFTER the Lambda
handler returned a response.
3. Implementing cache in extension
• Save the data on Lambda FS
• Expose local HTTP server and serve the data

39. Lambda extension overview

40. Extensions for caching
1. (+) "Asynchronous" cache updates
2. (-) Overhead (in ms):
• Avg (no ext/ext): 1.88 ms/14.1 ms
• Tm99.9 (no ext/ext): 1.88 ms/14.2 ms
3. (-) Lambda instance can't handle new requests until
extension is done
4. (-) Extension instances are not shared between
containers

41. Recap
1. In memory caching in Lambda is not straightforward
• Small window of time where the cache is "warm"
• Asynchronous updates are best effort only
2. Extensions improve the situation but have their own
downsides
• Runtime overhead
• Complicate the flow
3. L2 cache or different compute platform

42. How to avoid common mistakes
and misconceptions when working
with Java on AWS Lambda
Andrzej Dębski