Optimizing Your Serverless Applications (SRV401-R2) - AWS re:Invent 2018

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverless applications SERVICES (ANYTHING) Changes in data state Requests to endpoints Changes in resource state EVENT SOURCE FUNCTION Node.js Python Java C# Go

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Anatomy of a Lambda function Your function Language runtime Execution Environment Compute substrate Places where you can impact performance

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Anatomy of a Lambda function Handler() function Function to be executed upon invocation Event object Data sent during Lambda Function Invocation Context object Methods available to interact with runtime information (request ID, log group, etc.) public String handleRequest(Book book, Context context) { saveBook(book); return book.getName() + " saved!"; }

10. Anatomy of a Lambda function Function myhandler(event, context) { <Event handling logic> { result = SubfunctionA() }else { result = SubfunctionB() } return result; } Function subFunctionA(thing){ ## logic here } Function subFunctionB(thing){ ## logic here }

12. Import sdk Import http-lib Import ham-sandwich Pre-handler-secret-getter() Pre-handler-db-connect() Function myhandler(event, context) { <Event handling logic> { result = SubfunctionA() }else { result = SubfunctionB() } return result; } Function Pre-handler-secret-getter() { } Function Pre-handler-db-connect(){ } Function subFunctionA(thing){ ## logic here } Function subFunctionA(thing){ ## logic here }

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ephemeral function environment • Lambda processes a single event per execution environment • No need for non-blocking execution on the frontend • REMEMBER – execution environments are reused • Lazily load variables in the global scope • Don’t load it if you don’t need it – cold starts are affected Import sdk Import http-lib Import ham-sandwich Pre-handler-secret-getter() Pre-handler-db-connect() Function myhandler(event, context) { ....

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Environment variables • Key-value pairs that you can dynamically pass to your function • Available via standard environment variable APIs such as process.env for Node.js or os.environ for Python • Can optionally be encrypted via AWS Key Management Service (KMS) • Allows you to specify in IAM what roles have access to the keys to decrypt the information • Useful for creating environments per stage (i.e. dev, testing, production)

17. AWS Systems Manager – Parameter Store Centralized store to manage your configuration data • supports hierarchies • plain-text or encrypted with KMS • Can send notifications of changes to Amazon SNS/ AWS Lambda • Can be secured with IAM • Calls recorded in CloudTrail • Can be tagged • Available via API/SDK Useful for: centralized environment variables, secrets control, feature flags from __future__ import print_function import json import boto3 ssm = boto3.client('ssm', 'us-east-1') def get_parameters(): response = ssm.get_parameters( Names=['LambdaSecureString'],W ithDecryption=True ) for parameter in response['Parameters']: return parameter['Value'] def lambda_handler(event, context): value = get_parameters() print("value1 = " + value) return value # Echo back the first key value

18. AWS Systems Manager – Parameter Store Centralized store to manage your configuration data • supports hierarchies • plain-text or encrypted with KMS • Can send notifications of changes to Amazon SNS/ AWS Lambda • Can be secured with IAM • Calls recorded in CloudTrail • Can be tagged • Available via API/SDK Useful for: centralized environment variables, secrets control, feature flags from __future__ import print_function import json import boto3 ssm = boto3.client('ssm', 'us-east-1') def get_parameters(): response = ssm.get_parameters( Names=['LambdaSecureString'],W ithDecryption=True ) for parameter in response['Parameters']: return parameter['Value'] def lambda_handler(event, context): value = get_parameters() print("value1 = " + value) return value # Echo back the first key value

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Concise function logic • Separate Lambda handler (entry point) from core logic • Use functions to TRANSFORM, not TRANSPORT • Dynamic logic via configuration •Per function – Environment variables •Cross function – Amazon Parameter Store/Secrets Manager • Read only what you need. For example: •Properly indexed databases •Query filters in Aurora •Use S3 select

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions “Serverless” workflow management with zero administration: Makes it easy to coordinate the components of distributed applications and microservices using visual workflows Automatically triggers and tracks each step, and retries when there are errors, so your application executes in order and as expected Can handle custom failure messages from Lambda Task Choice Failure capture Parallel Tasks

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Project & repository scoping 1. If functions share an event source they can go in the same repo, if not they go in their own repo as separate “applications”  Simplifies permissions 2. If functions share an event source but require varying different imported packages, make them their own function files/jars/etc.  Keep dependency bloat minimized per function Monorepo == anti-pattern for FaaS Two guiding principles:

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The function lifecycle Bootstrap the runtime Start your code Full cold start Partial cold start Warm start Download your code Start new Execution environment AWS optimization Your optimization

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS X-Ray Integration with Serverless • Lambda instruments incoming requests for all supported languages and can capture calls made in code • API Gateway inserts a tracing header into HTTP calls as well as reports data back to X-Ray itself var AWSXRay = require(‘aws-xray-sdk-core‘); var AWS = AWSXRay.captureAWS(require(‘aws- sdk’)); S3Client = AWS.S3();

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tweak your function’s computer power Lambda exposes only a memory control, with the % of CPU core and network capacity allocated to a function proportionally Is your code CPU, Network or memory-bound? If so, it could be cheaper to choose more memory.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Smart resource allocation Match resource allocation (up to 3 GB!) to logic Stats for Lambda function that calculates 1000 times all prime numbers <= 1000000 128 MB 11.722965sec $0.024628 256 MB 6.678945sec $0.028035 512 MB 3.194954sec $0.026830 1024 MB 1.465984sec $0.024638 Green==Best Red==Worst

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Smart resource allocation Match resource allocation (up to 3 GB!) to logic Stats for Lambda function that calculates 1000 times all prime numbers <= 1000000 128 MB 11.722965sec $0.024628 256 MB 6.678945sec $0.028035 512 MB 3.194954sec $0.026830 1024 MB 1.465984sec $0.024638 Green==Best Red==Worst +$0.00001-10.256981sec

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multithreading? Maybe! • <1.8GB is still single core • CPU bound workloads won’t see gains – processes share same resources • >1.8GB is multi core • CPU bound workloads will gains, but need to multi thread • I/O bound workloads WILL likely see gains • e.g. parallel calculations to return

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda execution model Synchronous (push) Amazon API Gateway AWS Lambda function /order Asynchronous (event) Amazon SNS AWS Lambda function Amazon S3 reqs Poll-based Amazon DynamoDB Amazon Kinesis changes AWS Lambda service function

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda API 1. Lambda directly invoked via invoke API SDK clients Lambda function API provided by the Lambda service Used by all other services that invoke Lambda across all models Supports sync and async Can pass any event payload structure you want Client included in every SDK

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. If you don’t need a response, execute async Use the Lambda APIs to start an asynchronous execution Built-in queue (SQS behind the scenes) Automatic retries Dead letter queue for failed events client = boto3.client("lambda") client.invoke_async( FunctionName="test" InvokeArgs=json_payload )

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The microservices “iceberg” Common question: “Should every service of mine talk to another using an API?” Maybe not!: Most microservices are internal only for a given product supporting their customer facing features. They may only need to pass messages to each other that are simple events and not need a full fledged interactive API. Public interface Internal services

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Gateways and routers • Choose suitable entry point for client applications • Single, custom client? Use the AWS SDK • In region only public API: Use regional endpoints on API Gateway • Calls from private microservices in a VPC: Use private endpoints on API Gateway • No need for a custom interface: look at a non API Gateway sources • Discard uninteresting events ASAP • S3 – Event prefix • SNS – Message filtering

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Per Function Concurrency controls • Concurrency a shared pool by default • Separate using per function concurrency settings • Acts as reservation • Also acts as max concurrency per function • Especially critical for data sources like RDS • “Kill switch” – set per function concurrency to zero

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scaling/Concurrency Controls Service Scaling controls Lambda API Concurrency is point in time, not TPS, can go to 0 up through maximum for account per region and is shared for all functions in a region. By default no per function concurrency throttle is set. SNS Service automatically scales, use Lambda Per Function Concurrency setting to control downstream consumption. SQS Service automatically scales, use Lambda trigger Batch size setting and Per Function Concurrency setting to control downstream consumption. Kinesis Streams Shards in a stream: One shard provides ingest capacity of 1MB/sec or 1000 records/sec, up to 2MB/sec of data output.

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Persistence Service Persistence of requests “in flight” Lambda API No formal persistence model SNS No formal persistence model beyond delivery retry logic that extends up through potentially 13 hours SQS By default messages are stored for 4 days. This can be modified to as little as 60 seconds up to 14 days by configuring a queue’s MessageRetentionPeriod attribute Kinesis Streams By default data is stored for 24 hours. You can increase this up to 168 hours (7 days). Extended data retention costs $0.02 per Shard Hour above 24 hours

47. Retry/failure handling Service Retry/failure capabilities Lambda API Retry/failure logic is client dependent for synchronous invocations. For asynchronous invocations are retried twice by the Lambda service. SNS If Lambda is not available, SNS will retry 2 times at 1 seconds apart, then 10 times exponentially backing off from 1 seconds to 20 minutes and finally 38 times every 20 minutes for a total 50 attempts over more than 13 hours before the message is discarded from SNS. SQS Messages remain in the queue until deleted. They are prevented by being accessed by other consumers during a period of time known as the “visibility timeout”. Successful Lambda invocations will cause deletions of messages automatically. If an invocation fails or doesn’t delete a message during the visibility timeout window it is made available again for other consumers. Kinesis Streams When using the Kinesis Client Library (KCL) it maintains a checkpoint/cursor of processed records and will retry records from the same shard in order until the cursor shows completion.

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Dead Letter Queues “By default, a failed Lambda function invoked asynchronously is retried twice, and then the event is discarded.” – https://docs.aws.amazon.com/lambda/latest/dg/dlq.html • Turn this on! (for async use-cases) • Monitor it via an SQS Queue length metric/alarm • If you use SNS, send the messages to something durable and/or a trusted endpoint for processing •Can send to Lambda functions in other regions • If and when things go “boom” DLQ can save your invocation event information ☠️ ✉️ Q

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking region AWS Lambda VPC Lambda function execution environment Invocations can only come in via the AWS Lambda API

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking region AWS Lambda VPC Lambda function execution environment Today that API is available publicly in the region Lambda is running Invocations can only come in via the AWS Lambda API

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking with a customer configured VPC region AWS Lambda VPC Customer VPC elastic network interface Lambda function execution environment

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking with a customer configured VPC region AWS Lambda VPC Customer VPC elastic network interface Lambda function execution environment Completely managed by the AWS Lambda team Customer configured/managed VPC. Customer controls Security Groups, Network ACLs, Route Tables

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking with a customer configured VPC region AWS Lambda VPC Customer VPC elastic network interface Lambda function execution environment Invocations still can only come in via the AWS Lambda API

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda networking with a customer configured VPC region AWS Lambda VPC Customer VPC elastic network interface Lambda function execution environment Invocations still can only come in via the AWS Lambda API Even with a private API Gateway endpoint or a VPC Endpoint provided service

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Do I need to put my functions in an Amazon VPC? Putting your functions inside of a VPC provides little extra security benefit to your AWS Lambda functions

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Do I need a VPC? Should my Lambda function be in a VPC? Does my function need to access any specific resources in a VPC? Does it also need to access resources or services in the public internet? Don’t put the function in a VPC Put the function in a private subnet Put the function in a subnet with a NAT’d route to the internet Yes Yes No No

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Do I need a VPC? Should my Lambda function be in a VPC? Do I need to restrict outbound access from my function to the internet? Don’t put the function in a VPC Put the function in a private subnetYes No

60. Basic VPC Design Lambda Subnets ---------> Other Subnets ---------> VPC Availability Zone A Availability Zone B Subnet Subnet Subnet Subnet NAT per <----- AZ -----> VPC NAT gateway VPC NAT gateway

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Basic VPC Design • ALWAYS configure a minimum of 2 Availability Zones • Give your Lambda functions their own subnets • Give your Lambda subnets a large IP range to handle potential scale • If your functions need to talk to a resource on the internet, you need a NAT! • ENIs are a pain, we know, we’re working on it

63. Lambda permissions model Fine grained security controls for both execution and invocation: Execution policies: • Define what AWS resources/API calls can this function access via IAM • Used in streaming invocations • E.g. “Lambda function A can read from DynamoDB table users” Function policies: • Used for sync and async invocations • E.g. “Actions on bucket X can invoke Lambda function Z" • Resource policies allow for cross account access

65. https://github.com/awslabs/aws-serverless-samfarm/blob/master/api/saml.yaml <-THIS BECOMES THIS-> AWS SAM Templates

66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SAM Policy Templates MyQueueFunction: Type: AWS::Serverless::Function Properties: ... Policies: # Gives permissions to poll an SQS Queue - SQSPollerPolicy: queueName: !Ref MyQueue ... MyQueue: Type: AWS::SQS::Queue

70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Anatomy of a Lambda function Your function Language runtime Execution Environment Compute substrate Places where you can impact performance

73. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Learn more at sessions and breakouts Thursday, November 29 Leadership Session: Using DevOps, Microservices, and Serverless to Accelerate Innovation (SRV325) 12:15 – 1:15 PM | Venetian Theatre (Level 2) Thursday, November 29 A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1) 1:45 PM - 2:45 PM | Numerous overflow rooms Thursday, November 29 Inside AWS: Technology Choices for Modern Applications (SRV305-R1) 2:30 PM - 3:30 PM | Numerous overflow rooms