Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
•
3 j'aime•1,086 vues
Miles Ward's presentation at the AWS Australian Summit 2012, Sydney, Architect Track
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (19)
Similaire à Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
Similaire à Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia (20)
Plus de Amazon Web Services
Plus de Amazon Web Services (20)
Dernier
Dernier (20)
Serving Media From The Edge - Miles Ward - AWS Summit 2012 Australia
- 1. AWS Summit 2012 | Melbourne Welcome Serving Media and Entertainment from the Edge
- 2. AWS Summit 2012 | Melbourne Welcome Miles Ward – Solutions Architect @milesward
- 3. Today’s Agenda Examine common needs and challenges we’ve seen from Media and Entertainment customers. Provide examples of how customers have met these needs using AWS. Dig in on patterns for Storage, Processing, and Distribution. Learn how to get started.
- 4. Media Storage • Library storage for processing • Archiving Media Processing • Encode/Transcode • Encryption, watermarking • Applying individualization and monetization mechanisms (advertising, PPV, subscription, purchase, etc.) Media Distribution • Audio and video streaming of Live and VOD content • Managed distribution to set top boxes • Over the top (OTT) distribution • Internet distribution • Rich media web applications
- 5. Media Industry Trends and Needs • Media industry is at an inflection point • Infrastructure requirements growing exponentially • As the media industry has “gone digital”, content distribution requirements have also evolved • Scalable and secure media storage, processing and distribution • Anytime, anywhere, any device consumption • Personalized content and experiences • Expectation of low latency, global distribution • Support multiple business models (purchase, PPV, subscription) • Media has unusual performance, security and reliability characteristics compared to other industries
- 6. Today’s Ubiquitous Media is Complex and Challenging High Performance Computing Server Farms Content Distribution Networks DAM, DRM, Encoding, E diting, and Video Player Software Large Number of File Formats Massive Storage Networks
- 7. On-Premise Infrastructure is Costly & Complex Large Capital Expenditures Underutilized IT Assets Software Maintenance Out of Datacenter Space Budget Constraints Manual Deployments Contract negotiation Limited Capacity Expensive IT products “ Difficulties managing physical growth IT spends 80% of its time and resources keeping the lights on
- 8. The Cloud Provides a Better way Instant Elasticity Rapid Time to Value Focus on Innovation not On-Demand Infrastructure Pricing No CapEx
- 9. How the Cloud Can Reduce Digital Lifecycle Complexity Massive Scale Rapid File Transfer Lower IT Costs High Reliability Accredited Security
- 10. Massive Scale Makes it Easier to Hit Deadlines Problem: Timeline overruns to support PS3 launch Solution: AWS’s Scale, 1200 virtual machines on-demand Benefits:17,000 titles (80TB of data) transcoded in days, able to support launch
- 11. Massive Scale Makes it Easier to Focus on Your Core Business Problem: Massive global growth in online gaming Solution: AWS’s Global Infrastructure Benefits: Scaled to handle 4 million players in 8 weeks
- 12. Storage S3 – Storage for Source / Mezzanine files Exceptional Durability: 99.999999999% Secure Economical EBS – Storage for Media during Processing POSIX Block Device Dynamic Persistent for non-contiguous processing Direct Connect – Leverage your existing storage systems via a low latency Nx10Gbps link to AWS
- 13. S3 Use Cases and Users • Media Serving • Media Sharing • File Storage / Backup • Static Content • Big Data
- 14. Media Ingest and Storage Architecture 14
- 15. Moving Files to and From Amazon S3 is Fast Symantec NetBackup S3n Internet Cloudberry Pro Riverbed Whitewater Multi-part Upload Up to 700 Mb/s direct 1Gb and 10 Gb to Amazon S3 Connections per node
- 16. New! Storage Gateway Your Datacenter Amazon Elastic Compute Cloud (EC2) AWS Storage Gateway Clients VM SSL Internet On-premises Host or Direct AWS Storage Amazon Simple Connect Gateway Service Storage Service (S3) Application Servers Amazon Elastic Block Storage (EBS) Direct Attached or Storage Area Network Disks
- 17. Moving Files Faster Also Makes it Easier to Hit Deadlines 9 Minutes: 7 Seconds: Other than Amazon S3 Amazon S3
- 18. Processing EC2 – Servers in the Cloud by the Hour High-Performance, Scalable, Secure Origin servers, custom application servers, monetization and DRM systems EMR – A coordination framework for distributed computing Minimizes Administration Automates distribution Simplifies workflows CG1 – CUDA – GPGPU-enabled instances for super high FP performance, transcoding.
- 19. Batch Processing with EC2
- 20. Easily Scale Up and Down Internet Video App on Amazon EC2 From 50 to 5,000 servers in 3 days 5,000 Scaled to peak of Number of EC2 Instances 5,000 instances in 3 days Launch of Facebook application 0 Monday Tuesday Wednesday Thursday Friday Saturday Sunday The Animoto Blog
- 21. AWS Reduces IT Costs Saved 70% OUTAGE OPPORTUNITY COST Saved 50% Traditional Hardware Actual Demand Automated Virtualization Reduced costs by over 50%
- 22. Why Vimeo Chose AWS “ Our users don't care whether it's a peak time or not, and we prefer not “ to pay for transcoding machines the five days a week we aren't using them. This workload is perfect for EC2
- 23. CG1 – GPGPU is here = 10x faster, 2x higher res, lower cost?!
- 24. Distribution Cloudfront – Next-Gen Content Delivery Network Performance-oriented Multi-format Secure Economical Route 53 – DNS API controlled, allows programmatic shifts Weighted Records Alias records for ELB EC2– Origin servers, custom application servers, monetization and DRM systems
- 25. CloudFront in a nutshell Our Content Delivery Network Delivers: • Low latency. Improves media load times. • High bandwidth. Enables high bit rate HD video and other media applications. • Redundant. Eliminates single points of failure. • Scalable. Ensure great experience as number of end users grows. • Global. Worldwide network provides great experience regardless of geography. • Cost-effective. Pay as you go model provides flexibility for your business. 25
- 26. Delivery of Non-Cached Video Download Request S3 bucket or Object routed to most custom origin with requested CloudFront optimal edge your stored video content from origin Edge location 0 End-user requests 2 Locations 1 video Media stored 3 in cache 4 Data transfer of 5 End-user enjoys Data transfer of video to video CloudFront edge cached video location to cache download to end user
- 27. Delivery of Cached Video Download S3 bucket or Request routed custom origin with CloudFront to most optimal your stored video content Edge Locations edge location 0 End-user requests video 1 Skipping these steps means lower latency for your users! Media read from cache 2 of Data transfer 3 End-user enjoys video cached video download to end user
- 28. AWS Content and Media Serving Architecture 28
- 29. New Feature! CloudFront for Dynamic Websites • Deliver your entire site from CloudFront • Fast Performance using reliable AWS network connections • Simple, Self-Configuration using API or our Management Console • Cost-Effective with no minimum commits or required long-term contracts
- 30. • Different origins – same distribution • Persistent TCP Connections • Large (10) TCP Window • Full support for Query Strings • 0 second minimum TTL
- 31. AWS’ Reliable Infrastructure Makes it Easier to Focus on Other Tasks “As with all the AWS services we leverage, using Amazon CloudFront is so simple and reliable that the team doesn’t have to think about it. It all just works, freeing us to focus on building cool applications.” >1Pb/mo through Cloudfront
- 32. AWS and CloudFront’s global presence AWS Regions US East (Northern Virginia) US West (Northern California) US West (Northern Oregon) SA East (Sao Paulo) Europe (Dublin) Asia Pacific (Singapore) Asia Pacific (Toyko) AWS CloudFront Locations United States: New York, NY (2) Europe: Asia: South America: Ashburn, VA Palo Alto, CA Amsterdam Hong Kong Sao Paulo Dallas, TX San Jose, CA Dublin Tokyo Jacksonville, FL Seattle, WA Frankfurt Singapore Los Angeles, CA (2) South Bend, IN London Osaka! Miami, FL St. Louis, MO Paris Newark, NJ Stockholm Milan! 32
- 33. What do M&E customers expect from a CDN: • Great performance to a global audience. • Reliable delivery to wide range of clients. • Scalability for unpredictable spikes in demand • Control for secure content delivery • Support for HD high-bandwidth streaming • Reporting and analytics to track & analyze viewing patterns • Easy to configure and manage solutions • Cost-effective solutions 33
- 34. Great performance to a global audience. • Amazon is a metrics driven company. Gomez Large Object Test • We focus on metrics Japan that capture the end 12.000 Download Time (Seconds) 9.673 10.000 user experience: the 8.000 “last mile,” not internet 6.000 3.688 3.441 “backbone” data 4.000 3.126 2.000 centers. 0.000 Last Mile • Your customers do CloudFront Comparison A not live in data Comparison B Comparison C centers. Last mile data based on 7,907 observations taken between 17-SEP-2011 and 1-Oct-2011 *The Gomez tests were designed and conducted by Amazon using the Compuware Corporation performance network. The test results have not been reviewed, approved or endorsed by Compuware Corporation
- 35. Reliable delivery to wide range of clients. • Multiple delivery protocols for Adobe FMS 4.5 different platforms and devices • Adobe RTMP • HTTP Streaming for iOS ? • Microsoft Silverlight • Options for live and on-demand video • Full control over origin for live streaming • Reliability backed by CloudFront Service Level Agreement 35
- 36. Example: Video banner ad customer. 80 70 Delivered (Gbps) Volume of Data 60 Peak usage over 60 Gbps 50 40 30 20 10 0 8:00 AM 9:00 AM 10:00 11:00 12:00 PM 1:00 PM 2:00 PM 3:00 PM 4:00 PM 5:00 PM 6:00 PM 7:00 PM 8:00 PM 9:00 PM AM AM 36
- 37. secure content delivery controls • Private Content Feature authenticate users with signed URLS • Uses policy-driven access controls for control and flexibility • Restrict on resource or path, time, source IP • Signatures generated using asymmetric encryption • SSL delivery and RTMPE streaming encrypt bytes on the wire • Origin Access Identities secure your content in Amazon S3 • Identify and Access Management (IAM) to control who can configure your CloudFront distributions
- 38. • Self service signup and configuration – anytime, from anywhere • AWS Management Console to create and manage AWS infrastructure like S3, EC2, CloudFront • REST APIs for integration into your own systems and workflow 38
- 39. Getting Started • Simply sign up for AWS at http://aws.amazon.com/ • Store some content in S3, put a CloudFront distribution on it, and compare! • Try your media processing applications on EC2 • Take advantage of the Free Tier to experiment with more advanced services
Notes de l'éditeur
- In today’s highly connected world where consumers typically have multiple devices that they consume video content on, consumer’s access to content in an on-demand “where you want it, when you want it, how you want it” world is really quite frictionless, at least from their point of view. But supporting frictionless consumption from a technology and scale perspective is more difficult than ever.We’ve listed some of the technical challenges here around the need for HPC, server farms, DAM, encoding, encryption, editing, and others, but the biggest issue is that you need to be able to do this at scale due to the large number of files that get processed.
- The answer is the cloud!Cloud computing removes many of the physical limitations related to media processing. The scale of cloud resources can change the problem from a linear one to non-linear and enable companies to optimize how they process and distribute media in ways that have never before been available. Cloud computing:Gives you the agility to respond to opportunities more quickly than competitors.Enables efficient use of resources and allows you to focus on your core competencies and adding real value vs. managing infrastructureIn these times of tight budgets, it turns the conversation from capex to opex. Hardware depreciates quickly. For unpredictable demand, a rental model maximizes efficient use of resources.Moving to a service oriented model with utility billing enables you to deal with unexpected workloads via elastic scaling, with a pay-as-you-go model. And it optimizes the efficiency by handling the right tasks in the right place at the right timeExamples:Encoding can be done on cluster GPU instances to maximize speed or Spot instances to maximize budget. Enables you to make businesses decision vs. being beholden to technical or capacity limitations.In the future we expect additional focus on efficient use of resources for media processing and delivery. For example, from an efficiency perspective, once content is in the cloud, all media processing can happen while minimizing file movement.Get ProRes/mezzanine files into cloud storage, then have various processes act on those files.Also, there is a network effect: As content providers put more content into the cloud, more partners will move to a model where they can participate in the value chain.
- Over the next few slides I’m going to talk about some of the ways that Cloud Computing can help, including…
- Proof Point of how scales solves the growth problem (Netflix)Netflix wanted to make it’s library of content available to Sony Playstation 3 users when the SP3 launchedDue to timeline overruns, Netflix’s window of time to transcode their library of 17,000 videos was rapidly closing before the PS3 launch date.Netflix decided to make use of AWS’s massive scale and started 1200 EC2 instances to do the encoding.In just a few days Netflix was able to transcode over 80TB of video in time for the PS3 launch. This is a great example of how cloud computing such as Amazon Web Services can enable non-linear solutions to problems that traditionally could only be solve in a linear fashion
- Because their games are often based on social networking, the demand can grow significantly in a short period of time, sometimes quite unexpectedlyThe beta launch of Restaurant City saw the audience grow to more than 4 million players in the first 8 weeks. They initially projected between 100k to 250k players…Able to scale their infrastructure seamlessly while keeping everyone’s concentration on their core business.
- Before, taking advantage of AWS’s massive scale was limited by transfer speeds in to Amazon S3 and file size limitations.At Amazon we really focus on listening to our customers and innovating on their behalf, so we launch large object support , up-to 5TB, and Multi-part uploadAspera has created technology that takes advantage of multi-part uploads to transfer data to S3 at up-to 700 Mb/s (you will likely get a chance to see this live later today).We also launched Amazon Direct Connect, so companies can connect directly to AWS with 1Gb and 10Gb ports, bypassing the internet.
- The ability to move files faster is possible due to our low-latency internal transfer. This is an efficiency gained by having content and processing both in the AWS cloud infrastructure. As an example, one of our encoding partners, Zencoder, has an encoding service that runs on top of AWS.In order to process the files, they must first be transferred to Zencoder. Those files can live on your own hard drive, a file server, or cloud based storage.Example is a 1.5 GB file to Zencoder for processingAs you can see, transferring files that already live in Amazon Web Services’ S3 storage is much faster than transferring files from storage locations outside of AWS…nearly 80X faster!
- This is a graph of IT capacity over timeIn a traditional IT situation where you run the HW on premise, you can see how you would typically increase capacity in very discrete steps, likely aligning with when you get your new annual budget allocationsThe problem is that actual demand doesn’t usually follow that pattern. As you can see, there may be points in time where you’ve got excess capacity going unused – which is an opportunity cost, and at other times you may not have enough capacity – which leads to a poor customer experienceWith automated scalability of Amazon Web Services, your capacity can match your demand as it grows and shrinksWith On-Demand, Reserved, and Spot EC2 instances, you can choose what works best for you.Low cost: reduced pricing 15 times in last 5 yearsWe’ve removed the S3 data ingress cost as of July 2011Tiered pricing, rates go down as volume increasesFlexible: Pay-as-you-go pricing with or without commitments Transparent pricing: you get what you pay for
- AWS’ Reliable Infrastructure Makes it Easier to Focus on Other Tasks. An example customer that has seen the benefits here is PBS.PBS is using Amazon’s CloudFront Content Delivery Network for its streamed media.They were using another CDN initially, but were having reliability issues so they wrote a monitoring tool to test other CDNs, including CloudFront.What they found was that CloudFront had a significantly lower error rate than the incumbent CDN, so within a matter of weeks they made the switch.PBS is now delivering over 1PB/mo through CloudFront.