Best of re:Invent
•
4 j'aime•719 vues
The document summarizes announcements from AWS re:Invent 2016 related to compute, storage, artificial intelligence, serverless computing, databases, migration tools, and developer tools. Key announcements included new EC2 instance types, cost reductions, Elastic GPUs, AWS Batch for batch processing, Aurora PostgreSQL, Athena for analytics on S3 data, VMware on AWS, AWS X-Ray for tracing distributed applications, and expanded machine learning capabilities through services like Polly, Lex, and Rekognition as well as support for MXNet as an AI framework.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Best of re:Invent
Similaire à Best of re:Invent (20)
Plus de Amazon Web Services
Plus de Amazon Web Services (20)
Dernier
Dernier (18)
Best of re:Invent
- 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Rich Cowper, Manager of Solutions Architecture, Solution Architecture 2/1/2017 AWS Loft San Francisco Best of re:Invent 2016
- 2. Agenda My Goal: Have an interactive discussion about AWS product strategy by diving into re:Invent announcements. Agenda • Attendance Trends • Core themes • Features • Questions
- 4. Themes of re:Invent 2016 Compute / Networking / Storage Security / Management Table Stakes
- 5. Themes of re:Invent 2016 DevOps Data Compute / Networking / Storage Security / Management Enterprise Focus Table Stakes Migration and Operations Enablement
- 6. Themes of re:Invent 2016 Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Big Bets Enterprise Focus Table Stakes Migration and Operations Enablement
- 7. Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Migration and Operations Enablement Listening to customers and simplifying Priority 0
- 8. Compute Enhancements New Instances: T2 R4 I3 C5 P2 F1 Cost reductions: T2 / M4 / C4 The latest generation in a family is usually cheaper and has better specs Elastic GPUs (announced) • Attach GPU’s like EBS volumes • Will be available to all 9 instance families • Changes the economics of GPU compute in the cloud
- 9. Making EC2 easier VPS on AWS Predictable pricing model Instances run in separate VPC VPC peering to integrate with existing VPC based resources
- 10. Batch computing made easy AWS Batch (preview) • Fully-managed batch processing at any scale • Dynamic resource provisioning, scaling, spot fleet integration • Priority queuing and scheduling of jobs with dependencies Financial Services Life Sciences Digital Media Use Cases
- 11. Security and Account Management AWS Shield • Managed DDoS protection • Standard and ‘Advanced’ protection • Advanced protection includes WAF integration (CloudFront/ELB) AWS Organizations (Preview) • Replacing consolidating billing • Provides ability to • Create accounts • Invite existing accounts • Pay a single bill
- 12. AWS Big Data Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Migration and Operations Enablement
- 13. Clustered DB Engines Conventional Big Data Flow on AWS Data Lake S3 RedShiftEMR BI Tools Data Scientists ETL
- 14. Clustered DB Engines Making big data more attractive on AWS Amazon Athena • Serverless query service that runs on top of S3 • Presto engine (ANSI- SQL) • JSON, CSV, Parquet, ORC • Pay per query • Fast performance Data Lake S3 RedShiftEMR Athena BI Tools Data Scientists ETL
- 15. Disrupting ‘enterprise grade’ databases Aurora for PostgreSQL (Preview) • Full Postgres 9.6.1 compatibility • Aurora benefits: • Failover time < 30 sec • 6 copies across 3 AZs • Single-digit millisecond replica lag • At-rest and in-transit encryption SQL Txns Caching StoreStore StoreStore AZ 1 AZ 2 StoreStore AZ 3 What makes Aurora Special?
- 16. What about RDS?
- 17. Migrating data… Storage Gateway Enhancements • POSIX compliant on-prem NFS • Automatically stores data onto S3 • Automatically caches Snowball Edge • 100 TB Capacity • S3/NFS Compliant End/Mountpoints • Clustering mode • Lambda functions on-board
- 18. Migrating data… literally by the ton AWS Snowmobile • 45 Feet Long • 100 PB of Capacity • 256-bit KMS Managed encryption • Rich chain-of-custody options • $0.005 / GB per month
- 19. Shrinking migration barriers Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Migration and Operations Enablement
- 20. VMware on AWS (Preview) Fully managed VMware environment on the AWS Cloud Includes the VMware technologies customers are currently running on- premises Combines the enterprise features of VMware with the elasticity and security of AWS Components of VMWare Cloud on AWS
- 21. Configuration Management Amazon EC2 Systems Manager • Tools for configuration management of EC2 and on- premises systems • OS configuration • Ad-hoc configuration changes • Run scheduled administrative tasks AWS OpsWorks for Chef Automate • Fully managed Chef server and a suite of automation tools • Compatible Chef cookbooks from the Chef community • Auto scaling groups for instance registration and bootstrapping
- 22. Continuous Delivery AWS CodeBuild Fully managed build service that compiles source code, runs tests, and produces software packages that are ready to deploy
- 23. State Machines and ETL AWS Step Functions • Successor to Amazon Simple Workflow Service • Amazon States Language Specification • Lambda and ‘Activity’ Tasks • States: AWS Glue(pre-announced) • Successor to AWS Data Pipeline • Fully managed ETL • Source detection and transform suggestions • Built-in integration with • Any JDBC-compliant driver
- 24. Custom Scheduling Framework • Enhanced CloudWatch events for ECS state changes • Container instances • Tasks • Allows for custom scheduling • Cluster-state-service and daemon-scheduler ECS CLI • For development Containers https://github.com/blox • Tools and utilities to allow extensibility of the EC2 Container Service to better support emerging requirements
- 25. Pioneer tax cuts Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Migration and Operations Enablement
- 26. Serverless Improvements X-Ray (preview) • Debug and trace end-to-end requests in distributed apps • Graphical topology of services • Agent and API based integration (Java/Node/.Net) • Pricing will most likely be for trace storage and retrieval Serverless Application Models • Declarative CloudFormation syntax for Serverless • AWS::Serverless Primitives • Lambda Functions • DynamoDB Tables • API Gateway • (Swagger support) • Infrastructure as code
- 31. Other Notable Improvements Lambda@Edge (preview) Greengrass (limited preview) Dead Letter Queue C# Support
- 32. Widening the AI spectrum on AWS Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Migration and Operations Enablement
- 33. On one end of the spectrum: AI Enabled Services
- 34. AI Enabled Services Polly • Natural sounding speech • Accurate text processing • SSML support • 24 Languages, 47 voices Lex • Build Intelligent chatbots • Powered by same technology as Alexa • Deploy to multiple platforms Rekognition • Search, Verify and Organize images at scale • Face Analysis, Face Compare, Face Recognition, Object & Scene detection
- 35. At the other end of the spectrum: ‘DIY’ Deep Learning
- 37. AWS has picked its AI racehorse MXNet (Mixed Network Models) Programmability Python, JS, R, Matlab Portability 1000-layer network can fit inside 4GB Performance 88% efficiency scaling up to 256 GPUs
- 38. What about the middle of the spectrum?
- 39. Wrapping up… Artificial Intelligence Serverless DevOps Data Compute / Networking / Storage Security / Management Big Bets Enterprise Focus Table Stakes Migration and Operations Enablement
Notes de l'éditeur
- New R4 Instances – The R4 instances are designed for today’s memory-intensive Business Intelligence, in-memory caching, and database applications and offer up to 488 GiB of memory. The R4 instances improve on the popular R3 instances with a larger L3 cache and higher memory speeds. On the network side, the R4 instances support up to 20 Gbps of ENA-powered network bandwidth when used within a Placement Group, along with 12 Gbps of dedicated throughput to EBS. Instances are available in six sizes, with up to 64 vCPUs and 488 GiB of memory. Expanded T2 Instances – The T2 instances offer great performance for workloads that do not need to use the full CPU on a consistent basis. Our customers use them for general purpose workloads such as application servers, web servers, development environments, continuous integration servers, and small databases. We’ll be adding the t2.xlarge (16 GiB of memory) and the t2.2xlarge (32 GiB of memory). Like the existing T2 instances, the new sizes will be offer a generous amount of baseline performance (up to 4x that of the existing instances), along with the ability to burst to entire core when you need more compute power. Coming in early 2017, I3 instances are ideal for the most demanding input/output (I/O) intensive relational databases, NoSQL databases, transactional systems, and analytics workloads. I3 instances have 15.2 TB of fast, low latency locally attached storage backed by Non Volatile Memory Express (NVMe) based SSDs to deliver up to nine times the IOPs as the previous generation with 3.3 million random IOPS at 4 KB block size with a total I/O throughput of 16 GB per sec. Coming in early 2017, C5 instances include the next generation of Intel’s Xeon processors (code named Skylake) with AVX 512 and up to 72 vCPUs (twice that of previous generation compute-optimized instances) and 144 GiB of memory, making them the highest price to compute performance of any Amazon EC2 instance. C5 instances also feature new AWS hardware acceleration that delivers three times the Amazon EBS bandwidth of C4 instances. C5 instances are ideal for compute-intensive scientific modeling, financial operations, machine learning, and distributed analytics that require high performance for floating point calculations. Amazon EC2 F1 is a compute instance with field programmable gate arrays (FPGAs) that you can program to create custom hardware accelerations for your application. F1 instances are easy to program and come with everything you need to develop, simulate, debug, and compile your hardware acceleration code, including an FPGA Developer AMI and Hardware Developer Kit (HDK). Once your FPGA design is complete, you can register it as an Amazon FPGA Image (AFI), and deploy it to your F1 instance in just a few clicks. You can reuse your AFIs as many times, and across as many F1 instances as you like. Amazon EC2 F1 instances are currently in preview in two different instance sizes that include up to eight FPGAs per instance. F1 instances include the latest 16 nm Xilinx UltraScale Plus FPGA. Each FPGA includes local 64 GiB DDR4 ECC protected memory, with a dedicated PCIe x16 connection. Each FPGA contains approximately 2.5 million logic elements and approximately 6,800 Digital Signal Processing (DSP) engines.
- Quickly launch Virtual Private Servers on AWS Include compute, SSD-based storage, data transfer, DNS management, and a static IP – for a low, predictable price. Start simple, but use more AWS services as projects grows larger and more sophisticated over time.
- For applications in fields like life sciences, activities like computational chemistry, clinical modelling, and genomic sequencing are enabled by batch computing. There are many challenges involved in batch computing today even in the cloud: provisioning and maintaining a cluster, installing your batch processing software, managing dependencies between jobs, scaling, and job management. AWS Batch allows developers access to cloud resources for batch processing without having to provision, manage, monitor, or maintain clusters. Batch can automatically scale a cluster of managed servers to process a queue of jobs which can have their own priorities and complex dependencies. AWS Batch takes care of the undifferentiated heavy lifting and allows you to run your container images and applications on a dynamically scaled set of EC2 instances. This is in preview right now in us-east-1 and will roll out to other regions soon. More features are also on the way including the ability to leverage AWS Lambda to run batch jobs.
- Cost: Monthly Fee 3000 + .025 / .05 GB data transfer Comprehensive protections from large and sophisticated attacks Available on Elastic Load Balancing, Amazon CloudFront, Amazon Route 53 Advanced attack mitigation on Layer 3/ Layer 4 attacks Includes AWS WAF for proactive rules Near real-time notification via Amazon CloudWatch, with access to post-event analysis and investigation Specialized support - 24X7 access to DDoS Response Team AWS Bill protection against usage spikes due to a DDoS attack for ELB, CF and R53
- At-rest and in-transit encryption – KMS integration for at rest and AES-256 for SSL.
- Fun fact, if you had 100PB of data on site and fully utilized 4 10Gpbs DX ports, it would still take 215 days to complete the transfer
- Sort of the final pieces to our code* services. Chargted by the minutes
- Different from simple workflow in that you don’t have to implement providers, the states json definition does that. Team that manages SWF and Step are the same. Amazon States Language Specification – similar to a Business process markup language (BPML or BPAL). Similar to IBM, tibco, apian, etc Step - you pay 2.5 cents per 1000 state transitions
- cluster-state-service The cluster-state-service consumes events from a stream of all changes to containers and instances across your Amazon ECS clusters, persists the events in a local data store, and provides APIs (e.g., search, filter, list, etc.) that enable you to query the state of your cluster so you can respond to changes in real-time. The cluster-state-service tracks your Amazon ECS cluster state locally, and manages any drift in state by periodically reconciling state with Amazon ECS. daemon-scheduler The daemon-scheduler is a scheduler that allows you to run exactly one task per host across all nodes in a cluste. The scheduler can be used as a reference for how to use the cluster-state-service to build custom scheduling logic, and we plan to add additional scheduling capabilities for different use cases. ECS CLI does not support defining application load balancers or autoscaling groups
- - Every one at the re:Invent hackathon used AWS Lambda in some form or the other
- Amazon Polly supports Speech Synthesis Markup Language (SSML), a W3C standard, XML-based markup language for speech synthesis applications, and supports common SSML tags for phrasing, emphasis, and intonation. This flexibility helps you create lifelike speech that will attract and hold the attention of your audience. Lex: Automatic speech recognition (ASR) for converting speech to text, and natural language understanding (NLU) to recognize the intent of the text, to enable you to build applications with highly engaging user experiences and lifelike conversational interactions. With Amazon Lex, the same deep learning technologies that power Amazon Alexa are now available to any developer, enabling you to quickly and easily build sophisticated, natural language, conversational bots (“chatbots”). Rekognition: Amazon Rekognition is based on the same proven, highly scalable, deep learning technology developed by Amazon’s computer vision scientists to analyze billions of images daily for Prime Photos. Amazon Rekognition uses deep neural network models to detect and label thousands of objects and scenes in your images, and we are continually adding new labels and facial recognition features to the service.
- Mention the deep learning ami and cloudformation template that autscales