Serverless in Action on AWS
•
7 j'aime•756 vues
The slides from my talk on Serverless - more geared towards developers and architects, or innovators. Well actually to anyone interested really :)
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Serverless in Action on AWS
Similaire à Serverless in Action on AWS (20)
Plus de Adrian Hornsby
Plus de Adrian Hornsby (19)
Dernier
Dernier (20)
Serverless in Action on AWS
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Adrian Hornsby, Technical Evangelist Serverless Applications in Action on AWS @adhorn
- 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Technical Evangelist, Developer Advocate, … Software Engineer • Own bed in Finland • Previously: • Solutions Architect @AWS • Lead Cloud Architect @Dreambroker • Director of Engineering, Software Engineer, DevOps, Manager, ... @Hdm • Researcher @Nokia Research Center • and a bunch of other stuff. • Climber, like Ginger shots. @adhorn
- 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless key concepts
- 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No servers to provision or manage Scales with usage Never pay for idle Availability and fault tolerance built in Serverless means…
- 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Event driven A B CEvent on B by A triggers C Invocation Lambda functions Action
- 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How Lambda works S3 event notifications DynamoDB Streams Kinesis events Cognito events SNS events Custom events CloudTrail events LambdaDynamoDB Kinesis S3 Any custom Invoked in response to events - Changes in data - Changes in state Redshift SNS Access any service, including your own Such as… Lambda functions CloudWatch events
- 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Event-driven using Lambda AWS Lambda: Resize Images Users upload photos S3: Source Bucket S3: Destination Bucket Triggered on PUTs
- 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How does a Lambda function looks like? def handler(event, context): return { "message": ”Hello World!", "event": event } 'use strict'; exports.handler = (event, context, callback) => { console.log('LOG: Name is ' + event.name); callback(null, "Hello " + event.name); }
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Anatomy of a Lambda function Handler() function Function to be executed upon invocation Event object Data sent during Lambda Function Invocation Context object Methods available to interact with runtime information (request ID, log group, etc.) def handler(event, context): return { "message": ”Hello World!", "event": event }
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using AWS Lambda Bring your own code • Node.js, Java, Python, C# • Bring your own libraries (even native ones) Simple resource model • Select power rating from 128 MB to 1.5 GB • CPU and network allocated proportionately • Reports actual usage Flexible authorization • Securely grant access to resources, including VPCs • Fine-grained control over who can call your functions Flexible use • Call or send events • Integrated with other AWS services • Build whole serverless ecosystems
- 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using AWS Lambda Programming model • AWS SDK built in (Python and Node.js) • Lambda is the front end • Use processes, threads, /tmp, sockets normally Stateless • Persist data using Amazon DynamoDB, S3, or ElastiCache • No affinity to infrastructure (can’t “log in to the box”) Authoring functions • Author directly using the console WYSIWYG editor • Package code as a .zip and upload to Lambda or S3 • Plugins for Eclipse and Visual Studio • Command line tools Monitoring and logging • Built-in metrics for requests, errors, latency, and throttles • Built-in logs in Amazon CloudWatch Logs
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda execution model Synchronous (push) Asynchronous (event) Stream-based Amazon API Gateway AWS Lambda function Amazon DynamoDB Amazon SNS /api/hello AWS Lambda function Amazon S3 reqs Amazon Kinesis changes AWS Lambda service function
- 13. Chalice awslabs/aws-serverless-express awslabs/aws-serverless-java-container Serverless Frameworks
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example usage of frameworks
- 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless in action
- 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Event processing applications
- 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Event-driven actions AWS Lambda: Resize Images Users upload photos S3: Source Bucket S3: Destination Bucket Triggered on PUTs
- 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source data Source data Source data Source data Data Staging Layer /data/source-raw Data Staging Layer /data/source-validated Input Validation and Conversion layer Input Tracking layer State Management Store Event-driven controls
- 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. (scheduled) Automation applications
- 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Auto tagging resources as they start AWS Lambda: Update Tag Amazon CloudWatch Events: Rule Triggered Amazon EC2 Instance State Changes Amazon DynamoDB: EC2 Instance Properties Tag: N/A Amazon EC2 Instance State Changes Tag: Owner=userName PrincipalID=aws:userid • AMI • Instances • Snapshot • Volume
- 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CapitalOne Cloud Custodian AWS Lambda: Policy & Compliance Rules Amazon CloudWatch Events: Rules Triggered AWS CloudTrail: Events Amazon SNS: Alert Notifications Amazon CloudWatch Logs: Logs Read more here: http://www.capitalone.io/cloud-custodian/docs/index.html
- 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scheduled backup operation AWS Lambda: Backup Rules Amazon CloudWatch Events: Scheduled Trigger Amazon Redshift Cluster XYZ Snapshot
- 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Web applications
- 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3
- 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS SAM: Serverless Application Model
- 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. index.handler
- 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SAM Local: Local Serverless testing with Docker
- 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SAM package + deploy
- 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless template xxxxxxxx
- 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. gets.hello
- 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. posts.create
- 32. Deploy
- 33. Call
- 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Voice enabled applications
- 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Polly: Text In, Life-like Speech Out Amazon Polly “Today in Seattle, WA it’s 11°F” “Today in Seattle Washington it’s 11 degrees Fahrenheit”
- 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. <API> Amazon Polly </API> aws polly synthesize-speech --text "It was nice to live such a wonderful live show" --output-format mp3 --voice-id Joanna --text-type text johanna.mp3
- 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. <demo> PollyCast </demo> * Initial project by James Siri, Piotr Lewalski https://github.com/adhorn/pollycast
- 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pollycast: Serverless Template
- 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Computer vision powered applications
- 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Rekognition Object and scene detection Facial analysis Face comparison Celebrity recognition Image moderation Text in pictures Faces in crowd Rekognition: Images In, Rich Metadata Out NEW!!!
- 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. <API> Amazon Rekognition </API> aws rekognition detect-faces --image '{"S3Object":{"Bucket":"adhorn-reko","Name":"horse.jpg"}}' --attributes "ALL" aws rekognition detect-labels --image '{"S3Object":{"Bucket":"adhorn-reko","Name":"horse.jpg"}}'
- 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Step Functions: Orchestrate a Serverless processing workflow using AWS Lambda
- 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. <demo> Image Recognition and Processing Backend Step Functions </demo> https://github.com/awslabs/lambda-refarch-imagerecognition
- 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IoT powered applications
- 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Lambda Notifications: Amazon SNS DynamoDB AWS IoTSensors Control System Anomaly Detection Using AWS Lambda
- 48. AWS IoT IoT shadow Amazon Cognito MQTT over WebSockets AWS LambdaAlexa Amazon S3 Interactive Serverless applications
- 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Real-time streaming applications
- 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Real-time Analytics Demo http://quad.adhorn.me
- 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. More info: https://aws.amazon.com/serverless/
- 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thanks you!
Notes de l'éditeur
- Bring your own code. Simple resource model: only one thing that needs to be configured -> memory. CPU and Network is allocated proportionately which means tha t a 256MB function will have twice the CPU and Network as a 128MB one. Flexible use: trigger or invoke synchronously or asynchonously. Hook up with many othe other AWS services Use IAM roles under the hood. So you can very fine grain security so you can for example say my lambda function can access only one particular S3 bucket. VPC integration makes it even more control over what your lambda funciton can and cannot do.
- Build your function the same way you would do in your standard enviroment (threads.. ) Deploy using existing tools and plugins, cli tools and frameworks (demo) Lambda function are stateless so you need to use S3, elasticache or dynamodb to persist the state so you can excahneg data betweene functions. Use Amazon Cloudwatch for monitoring
- Also a classic in Data Lake designs. Amazon S3 is a simple key-based object store whose scalability and low cost make it ideal for storing large datasets. S3 to provide excellent performance for storing and retrieving objects based on a known key. Taking advantage of AWS Lambda event-driven triggers from S3. S3 storage tier – lifecycle policies Modern businesses typically collect data from internal and external sources at various frequencies throughout the day. These data sources could be franchise stores, subsidiaries, or new systems integrated as a result of merger and acquisitions. For example, a retail chain might collect point-of-sale (POS) data from all franchise stores three times a day to get insights into sales as well as to identify the right number of staff at a given time in any given store. As each franchise functions as an independent business, the format and structure of the data might not be consistent across the board. Depending on the geographical region, each franchise would provide data at a different frequency and the analysis of these datasets should wait until all the required data is provided (event-driven) from the individual franchises. In most cases, the individual data volumes received from each franchise are usually small but the velocity of the data being generated and the collective volume can be challenging to manage.
- The basics are pretty simple, but the service has deep functionality. You can send the service a simple string of text, and it will generate the life like voice in your choice of 47 different voices. But it’s not naive of the context of the text. For example, the text here - ‘WA’ and ‘degree F’, that would sound strange if it were spoken out loud. Instead, Polly will automatically expand the text strings ‘WA’ and ‘degree F’, to ‘Washington’ and ‘degrees fahrenheit’, to create more life like speech. The developer doesn’t have to do anything - just send the text, and get life like voice back.
- 30
- 24
- Amazon Rekognition currently supports the JPEG and PNG image formats. You can submit images either as an S3 object or as a byte array. Amazon Rekognition supports image file sizes up to 15MB when passed as an S3 object, and up to 5MB when submitted as an image byte array. Amazon Rekognition is currently available in US East (Northern Virginia), US West (Oregon) and EU (Ireland) regions. Mxnet convolutional deep neural networks (CNNs),
- 26
- 24
- Sometimes, though, you may want to analyze the anomalies or at least be notified of their presence. such algo could be : calculating an average and standard deviation of the time-series data Data is sent from our sensor to AWS IoT, where it is routed to AWS Lambda through the AWS IoT Rules Engine. Lambda executes the logic for anomaly detection and because the algorithm requires knowledge of previous measurements, uses Amazon DynamoDB as a key-value store. The Lambda function republishes the message along with parameters extracted from the PEWMA algorithm. The results can be viewed in your browser through a WebSocket connection to AWS IoT on your local machine
- To build that, no genius was involved .. but like Edison, we stitched innovations together to create a Wow experience for our customers. (security, scalable, reliable..) And this is exactly what the cloud is about! Just few years ago, it would have taken months or even years to build a scalable, reliable and secure application like that from scratch.