9. The serverless manifesto
• No machines, VMs, or containers visible in the programming model.
• Functions are the unit of deployment & scaling.
• Implicitly fault tolerant, as functions can run anywhere.
• Scales per request.
• Users can not over- or under-provision capacity.
• Permanent storage lives elsewhere.
• Never pay for idle, no cold servers or their costs.
• Bring your own code (BYOC).
• Metrics and logging are a universal right.
35. That’s not all
• No persistent state
• 5 minute execution limit
• No atomic deployments of a service
• Immature tooling
• Vendor control / lock-in
• Service Discovery
Testlio is a full-service app testing solution, that allows you to offload your QA with the most flexible requirements. All we need is your build and you will have a dedicated team of professional software testers.
A serverless application is one where the server-side logic runs in stateless compute containers that are event-triggered, ephemeral (may last only a single execution) and fully managed by a 3rd party. Essentially you can think about it as Functions as a Service (FaaS). There are multiple service providers today - Googles Cloud Functions, Azures Service Fabric and Auth0’s webtasks, but the first and perhaps the most popular implementation today is AWS’ Lambda.
First introduced in November of 2014, Lambda integrated with AWS’ DynamoDB (a managed key value store), S3 (cloud file storage) and Kinesis (data processing) as event sources. A year later AWS expanded support across their services (CloudFormation, SWF, CloudTrail, SES, Cognito, API Gateway, SNS, CloudWatch). Lets focus on some of these to see how they enable us to build serverless cloud services.
API Gateway - Allows us to map individual Lambda functions to HTTP endpoints to create Restful services.
DynamoDB - A fully managed database service to enable persistent data storage and lookup without the overhead of database maintenance.
CloudWatch - Centralised logging for all Lambda functions and periodic triggers.
CloudFormation - Enables to define and deploy a complete sandboxed stack.
Updating servers (OS’), Deploying code to a cluster, Horizontal scaling is completely automatic, pay for what you use (subsecond billing w/ 100ms accuracy)
You might be hyped. But before you jump into writing everything using the serverless paradigm, lets look at some of the bad problems and ugly workarounds.
In general your serverless service will be slower.
• Cold boots add about ~100ms depending on the size of your function and runtime (Node, Python)
• JVM functions may take significantly longer (up to ~10s)
• The ugly workaround is to ping your endpoint periodically or set a periodic event
Actually pretty easy with serverless framework, which abstracts some of the complexity away - just one configuration line.
A schedule does not completely solve the problem though.
Even if your function is under constant load the VM seems to be recycled every ~4 hours.
IOPipe data - AWS does not say.
Coldstarts are not the only problem.
Latency in the same region (us-east-1) is similar. The traditional approach comes in just under ~100ms, while the serverless is just over it. Both pretty consistent.
When we look at cross region requests things get a little funky though.
Lambda/APIGW approach is at times much slower and also a lot more inconsistent.
APIGW is based on CloudFront, you reach the Amazon network quickly, but the Amazon network seems at times more unreliable than the internet.
time_connect - from the start until the TCP connect to the remote host
APIGW/Lambda very fast - latency seems to happen in the Amazon network.
API Gateway - I have a love - hate relationship with that Amazon Service. At times it seems to follow the old Top Gear moto - “Ambitious, but rubbish”.
If you look at the service offering, it tries to do so much.
Some of these things are better remained in application code, as they are hard to debug, version control and sometimes even define.
Cache does not help with Amazon network problems, since it’s not implemented on the edge.
API Gateway might timeout on coldstart.
No certificate manager integration, works with ELB and CloudFront.
What you do in this case is crawl the AWS forums for help…
Where you will be assured it’s a high priority item in the backlog.
Even though it probably isn’t.
Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.
The twelve-factor app stores config in environment variables.
1. Bake the configuration into the package (either as a file or in code)
2. Use S3
3. Use DynamoDB
The first one is a bad idea for reasons which are out of the scope of this talk (see 12 Factor App), to mention a few:
• you will not be able to use your build artifacts across deployments to different stages, if the configuration between them varies at all
• redeploy to update configuration
S3 and Dynamo increase latency even more.
You can load configuration only once on cold start, but then you don’t know when your config change propagates.
No general FaaS abstraction
Local environment lots of manual work
Containers exist for AWS services
All event sources are possible to emulate, but impractical
Resort to development in the cloud
Rely on what your service provider gives you. Meaning there is no SSH access to containers or remote debugging. Debugging happens by investigating metrics and lots and lots of logging.
By default, AWS Lambda limits the total concurrent executions across all functions within a given region to 100. The default limit is a safety limit that protects you from costs due to potential runaway or recursive functions during initial development and testing.
This means otherwise self contained services can impact each other. If different environments (dev, test, prod) run under the same account, then those can also affect each other.
Load testing dev/test env
Processing big data
Results in unresponsive API
database connection pools / persistent HTTP connections to external services are also persistent state
recursive or spawn multiple lambdas to avoid execution time limit (can lead to fun stuff with the concurrent execution limit)
might have to disable event triggers to deploy a service due to no atomic deploys of a service
Still immature
Be aware of the limits
Sweet spot is currently along with servers
Very powerful and will become more prominent