Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Adopting Java for the Serverless world at Serverless Meetup New York and Boston
1. Adopting Java for the
Serverless world
from the perspective of the AWS developer
Vadym Kazulkin, ip.labs, Serverless New York and Boston Meetups, 11 January 2021
2. Contact
Vadym Kazulkin
ip.labs Bonn, Germany
Co-Organizer: Java User Group Bonn and
Serverless Bonn Meetup
v.kazulkin@gmail.com
@VKazulkin
@ServerlessBonn (Meetup)
https://www.linkedin.com/in/vadymkazulkin/
https://www.iplabs.de/
11. AWS Java Versions Support
• Java 8
• With extended long-term support
• Java 11 (since 2019)
• Only Long Term Support (LTS) by AWS
• Next LTS Java version is Java 17 in 2021
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: https://aws.amazon.com/de/corretto/
12. Java ist very fast
and mature
programming
language…
Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea Christian Bannes and Vadym Kazulkin @VKazulkin , ip.labs GmbH
… but
Serverless
adoption of Java
looks like this
13. 2020 AWS Lambda Benchmark Report for Developers, DevOps, and Decision Makers
https://newrelic.com/resources/ebooks/serverless-benchmark-report-aws-lambda-2020
Vadym Kazulkin @VKazulkin , ip.labs GmbH
14. Developers love Java and will be happy
to use it for Serverless
But what are the challenges
Vadym Kazulkin @VKazulkin , ip.labs GmbH
22. Challenge Number 1 with Java is a
big cold-start
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: https://www.serverless.com/blog/keep-your-lambdas-warm
23. Cold Start
:
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
24. • Start Firecracker VM
• AWS Lambda starts the JVM
• Java runtime loads and initializes
handler class
• Static initializer block of the handler class is
executed
• Init-phase has full CPU access up to 10 seconds
for free for the managed execution environments
• Lambda calls the handler method
• Full CPU access only approx. at 1.8 GB
“assigned” memory to the function
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
Michael Hart: „Shave 99.93% off your Lambda bill with this one weird trick“
https://hichaelmart.medium.com/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
25. Function lifecycle- a full cold start
:
Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go
Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
26. AWS Lambda cold start duration
per programming language
Source: Mikhail Shilkov: „AWS Lambda: Cold Start Duration per Language. 2020 edition” https://mikhail.io/serverless/coldstarts/aws/languages/
27. Cold start duration with Java
• Below 1 second is best-case cold start duration for very simple
Lambda like HelloWorld with no dependencies
• It goes up significantly with more complex scenarios
• Instantiation outside of the handler method (static initializer block) to
communicate with other (AWS) services (e.g. DynamoDB, SNS, SQS, 3rd party)
• Artifact size
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
28. AWS Lambda cold starts by memory size, runtime and artifact size
Source: Mike Roberts "Analyzing Cold Start latency of AWS Lambda" https://blog.symphonia.io/posts/2020-06-30_analyzing_cold_start_latency_of_aws_lambda
Artifact Size:
• Small zip (1KB)
• Large zip (48MB)
• Large uberjar (53MB)
29. Cold start duration with Java
• Below 1 second is best-case cold start duration for very simple
Lambda like HelloWorld with no dependencies
• It goes up significantly with more complex scenarios
• Instantiation outside of the handler method (static instantiation) to communicate
with other (AWS) services (i.e. DynamoDB, SNS, SQS, 3rd party)
• Artifact size
• To minimize the cold start time apply best practices from this talk
• Worst-case cold starts can be higher than 10 and even 20 seconds
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
30. 2020 AWS Lambda Benchmark Report for Developers, DevOps, and Decision Makers
https://newrelic.com/resources/ebooks/serverless-benchmark-report-aws-lambda-2020
Vadym Kazulkin @VKazulkin , ip.labs GmbH
32. Jeremy Daly: “Mixing VPC and Non-VPC Lambda Functions for Higher Performing Microservices”
https://www.jeremydaly.com/mixing-vpc-and-non-vpc-lambda-functions-for-higher-performing-microservices/ Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda behind the
Virtual Private
Cloud (VPC)
33. Lambda in VPC Improvements:
• The network interface creation happens
when Lambda function is created or its
VPC settings are updated.
• The network interfaces are shared across
execution environments
• Only a handful of network interfaces are
required per function
• Reduced additional cold start from approx.
10 seconds to below 1 second
Chris Munns: "Announcing improved VPC networking for AWS Lambda functions”
https://aws.amazon.com/de/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/
34. Improvements 1/4
• Switch to the AWS SDK 2.0 for Java
• Lower footprint and more modular
• Allows to configure HTTP Client of your choice (i.e. Java own Basic HTTP Client
or newly introduced AWS Common Runtime async HTTP Client)
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Zoe Wang: „Introducing AWS Common Runtime HTTP Client in the AWS SDK for Java 2.x”
https://aws.amazon.com/de/blogs/developer/introducing-aws-common-runtime-http-client-in-the-aws-sdk-for-java-2-x/
S3AsyncClient.builder()
.httpClientBuilder(AwsCrtAsyncHttpClient.builder()
.maxConcurrency(50))
.build();
35. Improvements 1/4
Provide all known values (for building clients i.e. DynamoDB client)
to avoid auto-discovery
• credential provider, region, endpoint
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withRegion(Regions.US_WEST_2)
.withCredentials(new ProfileCredentialsProvider("myProfile"))
.build();
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
36. Improvements 1/4
• Initialize and prime dependencies during initialization phase
• Use static initialization in the handler class, instead of in the handler method (e.g.
handleRequest) to take the advantage of the access to the full CPU core for max 10 seconds
• „Fake“ the calls to pre-initalize „things“, e.g. Jackson Marshallers in case of DynamoDB client
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()...build();
DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable(„mytable");
Item item = table.getItem("Id", 210);
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
getItem() call forces Jackson Marshallers to initialize
37. Improvements 1/4
• Less (dependencies, classes) is more
• Include only required dependencies (e.g. not the whole AWS SDK 2.0 for Java, but the
dependencies to the clients to be used in Lambda)
• Exclude dependencies, which you don‘t need at runtime e.g. test frameworks like Junit
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
<version>2.10.86</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.10.86</version>
<type>pom</type>
<scope>import</scope>
</dependency>
38. Improvements 1/4
Further Improvements:
• Consider using JDK Class-Data Archive and Application Class-Data Sharing
available since Java 10 and improved in Java 13
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Snicolai Parlow : "Application Class-Data Sharing In A Nutshell„ https://nipafx.dev/java-application-class-data-sharing/
39. Improvements 2/4
Avoid:
• reflection
• runtime byte code generation
• runtime generated proxies
• dynamic class loading
Use DI Frameworks like Micronaut, Quarkus or Dagger which aren‘t
reflection-based
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/
44. Tooling for AWS Lambda in Java
Vadym Kazulkin @VKazulkin , ip.labs GmbH
45. Lambda Power Tuning 1/2
• Executes different
settings in parallel
• Outputs the optimal
setting
Image: https://github.com/alexcasalboni/aws-lambda-power-tuning Vadym Kazulkin @VKazulkin , ip.labs GmbH
46. Lambda Power Tuning 2/2
• Executes different
settings in parallel
• Outputs the optimal
setting
Image: https://github.com/alexcasalboni/aws-lambda-power-tuning
Alex Casalboni: “Deep dive: finding the optimal resources allocation for your Lambda functions“
https://dev.to/aws/deep-dive-finding-the-optimal-resources-allocation-for-your-lambda-functions-35a6
Vadym Kazulkin @VKazulkin , ip.labs GmbH
47. Optimizing AWS Lambda cost
and performance using AWS
Compute Optimizer
Source: Chad Schmutzer „Optimizing AWS Lambda cost and performance using AWS Compute Optimizer”
https://aws.amazon.com/de/blogs/compute/optimizing-aws-lambda-cost-and-performance-using-aws-compute-optimizer/
49. AWS Lambda
Powertools Java 2/3
https://aws.amazon.com/de/blogs/opensource/simplifying-serverless-best-practices-with-aws-lambda-powertools-java/
When using different Java
annotations provided by the
library, configure the aspectj-
maven-plugin to compile-time
weave the aws-lambda-
powertools-java aspects into
your project.
51. Testing AWS Lambda functions written in
Java with aws-lambda-java-tests project
«Testing AWS Lambda functions written in Java” https://github.com/aws/aws-lambda-java-libs/tree/master/aws-lambda-java-tests
https://aws.amazon.com/de/blogs/opensource/testing-aws-lambda-functions-written-in-java/
53. Cost optimization
• Java is well optimized for long running server applications
• High startup times
• High memory utilization
Vadym Kazulkin @VKazulkin , ip.labs GmbH
And both memory and execution time are cost dimensions,
when using Serverless in the cloud
55. GraalVM
Goals:
Low footprint ahead-of-time mode for JVM-based languages
High performance for all languages
Convenient language interoperability and polyglot tooling
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
56. GraalVM
Architecture
Sources: Practical Partial Evaluation for High-Performance Dynamic Language Runtimes http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-truffle.pdf
„The LLVM Compiler Infrastructure“ https://llvm.org/
57. SubstrateVM
Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript”
https://www.youtube.com/watch?v=a-XEZobXspo
58. GraalVM on SubstrateVM
A game changer for Java & Serverless?
Java Function compiled into a native executable using
GraalVM on SubstrateVM reduces
• “cold start” times
• memory footprint
by order of magnitude compared to running on JVM.
And both memory and execution time are cost dimensions,
when using Serverless in the cloud
59. GraalVM on SubstrateVM
A game changer for Java & Serverless?
Current challenges with native executable using GraalVM :
• Most Cloud Providers (AWS) doesn’t provide GraalVM as Java
Runtime out of the box, only Open JDK (i.e. AWS provides Corretto)
• Some Cloud Providers (e.g. AWS) provide Custom Runtime Option
62. GraalVM Complitation Modes
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
63. AOT vs JIT
Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
64. Support of GraalVM native images in Frameworks
Spring Framework: working toward GraalVM native image support
without requiring additional configuration or workaround is one of the
themes of upcoming Spring Framework 5.3
Spring Boot: Ongoing work on experimental Spring Graal Native
project. Probably ready for the 2.4 release
Quarkus: a Kubernetes Native Java framework developed by Red Hat
tailored for GraalVM and HotSpot, crafted from best-of-breed Java
libraries and standards.
Micronaut: a modern, JVM-based, full-stack framework for building
modular, easily testable microservice and serverless applications.
Source: „GraalVM native image support“ https://github.com/spring-projects/spring-framework/wiki/GraalVM-native-image-support
65. Common principles for all Frameworks
• Rely on as little reflection as possible
• Avoid runtime byte code generation, runtime generated proxies and
dynamic class loading as much as possible
• Process annotations at compile time
• The common goals:
• increase developer productivity
• speed up start up times and decrease memory usage for Microservice and
Serverless Java applications
• with and without usage of GraalVM and native image
Vadym Kazulkin @VKazulkin , ip.labs GmbH
73. Quarkus Additional Features
• Website for creating the App
• AWS API Gateway integration
• Funqy for multi cloud solutions
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8
79. Framework Comparison 1/2
• Project Initializer
• Programming Model
• Database Support
• Test Support
• Standards Support (i.e. MicroProfile)
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: „Battle Of The Microservice Frameworks: Micronaut versus Quarkus edition! by Michel Schudel“ https://www.youtube.com/watch?v=hnEXOqcNXPs
„Micronaut 2.0 vs Quarkus 1.3.1 vs Spring Boot 2.3 Performance on JDK 14“ https://www.youtube.com/watch?v=rJFgdFIs_k8
„Java EE, Jakarta EE, MicroProfile, or Maybe All of Them” https://www.eclipse.org/community/eclipse_newsletter/2019/february/Jakarta_Micro_All.php
80. Framework Comparison 2/2
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: „Battle Of The Microservice Frameworks: Micronaut versus Quarkus edition! by Michel Schudel“ https://www.youtube.com/watch?v=hnEXOqcNXPs
„Micronaut 2.0 vs Quarkus 1.3.1 vs Spring Boot 2.3 Performance on JDK 14“ https://www.youtube.com/watch?v=rJFgdFIs_k8
• Native Image with GraalVM
• Build time
• Framework compile-time activities
• GraalV Native Image compilation time
• Native Image Size
• (Cold) Startup Time
• Request/Invocation duration
• Memory Consumption
• Plain
• Application Size
• (Cold) Start Time
• Request/Invocation duration
• Memory Consumption
82. Lambda Container Image Support
Use Cases:
• What about the support of the current Java version?
• Only Long Term Support (LTS) by AWS
• Java 8 and Java 11, next Java 17 Fall 2021
• Use Container (Docker) Image with i.e. Java 15
• GraalVM without Native Image may be faster
than OpenJDK for your application
• Use Container (Docker) Image with GraalVM
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: https://aws.amazon.com/de/corretto/
https://renaissance.dev/
83. Conclusion 1/2
• GraalVM and Frameworks are really powerful with a lot of potential
• Micronaut and Quarkus will improve the cold starts of Java
applications significantly even without GraalVM and native image
• They use as little as possible reflection and other runtime optimizations
• Cold start duration will still be in the range of maximal single digit (e.g. 1.5) seconds
depending on your application and framework
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: „https://stackoverflow.com/questions/63328298/how-do-you-debug-a-no-instances-of-are-allowed-in-the-image-heap-when-buil
84. Conclusion 2/2
• But the combination of discussed frameworks with GraalVM and Native
Image currently not without challenges
• AWS Lambda Custom Runtime requires Linux executable only
• Managing Custom Runtime requires some additional effort
• You pay for the init-phase of the function packaged as AWS Lambda Custom Runtime
• Delivering execution runtime (i.e. Java) via Lambda Custom Runtime reduces performance
compared to the usage of the same managed Lambda Runtime
• Reduce cold starts to several hundred milliseconds and very low memory consumption
• Lots of experimentation with additional build arguments like “initialize-at-runtime” or “delay-
class-initialization-to-runtime“ required
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Source: https://stackoverflow.com/questions/63328298/how-do-you-debug-a-no-instances-of-are-allowed-in-the-image-heap-when-buil
85. Personal Recommendations (highly opinionated)
• By default use Java Long Term Support Version with Amazon Corretto and your
favourite Framework (Micronaut, Quarkus)
• If your function needs constantly low response times for the known period of time ?
• Use Provisioned Concurrency additionally
• If your function needs constantly low response time and low cost is a requirement?
• Use GraalVM Native Image with your favourite Framework (Micronaut, Quarkus, Spring Boot
GraalVM Native) and AWS Lambda Custom Runtime
• Define 2 pipelines: „standard“ for quick and local testing and „native image“ for production usage
• If you don‘t want to miss 3 years of innovation and use the newest Java Version?
• Use Lambda Container Image Support
Vadym Kazulkin @VKazulkin , ip.labs GmbH
86. Where AWS can help (highly opinionated again) 1/2
• Optimize Amazon Corretto for Lambda with Java on AWS
• Rethink garbage collection
• Find trade-off between increase of memory consumption because of the garbage and performance
loss of GC algorithms for the short running Lambda functions (max. 1 hour or so)
• Rethink HotSpot Just-In-Time Compiler optimizations
• Most JIT optimizations apply by default only after multiple thousands executions (i.e. 10.000) of the
function (so never), so find the write configuration for the short running Lambda functions
• Provide support for jlink
• jlink is tool that generates a custom Java runtime image that contains only the platform modules that
are required for a given application
• See what Oracle introduced for the Fn Project
Vadym Kazulkin @VKazulkin , ip.labs GmbH
Sources: „Guide to jlink” https://www.baeldung.com/jlink
„Open Source. Container-native. Serverless platform“ https://fnproject.io/