Amazon Web Services
•Télécharger en tant que PPTX, PDF•
1 j'aime•5,608 vues
By Leveraging AWS Cloud and its services it not only help in reducing the cost but also brings agility and innovation. One of such service BigData provides a paradigm shift by putting smart in everything we do today including smart home, smart city, smart health, smart campus and many more. We will talk about how AWS services can help in reducing the cost and bring agility by leveraging Big Data to bring in innovation to campus.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Amazon Web Services
Similaire à Amazon Web Services (20)
Plus de Jisc
Plus de Jisc (20)
Dernier
Dernier (20)
Amazon Web Services
- 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Matt Johnson, Solutions Architect, UK Public Sector March 2017 Big Data Services on AWS JISC Digifest
- 3. Cloud is the New Normal
- 4. Equipment Resources and Administration Contracts Cost No Up Front Expense Pay for what you Use Improve Time to Market & Agility Scale Up and Down Self-Service Infrastructure Traditional Infrastructure AWS Cloud
- 5. What is Big Data?
- 6. Then…
- 7. When your data sets become so large and complex you have to start innovating around how to collect, store, process, analyze, and share it.
- 8. …Now
- 9. Scale as your data and business grows The volume, variety, and velocity at which data are being generated are leaving organisations with new questions:
- 10. Big Data is all about getting high value, actionable insights from your data assets
- 11. Unlocking Data Most organizations are embarking on ambitious innovation initiatives to unlock their data. The data already exists but goes unused or is locked away from complimentary data sets in isolated data silos.
- 12. Enter Data Lake Architectures Data Lake is a new and increasingly popular architecture to store and analyze massive volumes and heterogeneous types of data.
- 13. Benefits of a Data Lake – All Data in One Place Store and analyze all of your data, from all of your sources, in one centralized location. “Why is the data distributed in many locations? Where is the single source of truth ?”
- 14. Benefits of a Data Lake – Quick Ingest Quickly ingest data without needing to force it into a pre-defined schema. “How can I collect data quickly from various sources and store it efficiently?”
- 15. S3 is the Data Lake
- 16. Implementing a Data Lake using Amazon S3
- 17. AWS Direct Connect AWS Snowball ISV Connectors Amazon Kinesis Firehose S3 Transfer Acceleration AWS Storage Gateway Data Ingestion into Amazon S3
- 18. AWS Snowball: PB-scale Data Transport E-ink shipping label Ruggedized case “8.5G Impact” All data encrypted end-to-end 80TB & 100TB 10G network Rain & dust resistant Tamper-resistant case & electronics
- 19. Amazon Kinesis Streams Amazon Kinesis Firehose Amazon Kinesis Analytics Work with streaming data using Amazon Kinesis
- 20. Implementing IoT on AWS
- 21. Summary of AWS Analytics, Database & AI Tools Amazon Redshift Enterprise Data Warehouse Amazon EMR Hadoop/Spark Amazon Athena Clusterless SQL Amazon Glue Clusterless ETL Amazon Aurora Managed Relational Database Amazon Machine Learning Predictive Analytics Amazon Quicksight Business Intelligence/Visualization Amazon ElasticSearch Service ElasticSearch Amazon ElastiCache Redis In-memory Datastore Amazon DynamoDB Managed NoSQL Database Amazon Rekognition Deep Learning-based Image Recognition Amazon Lex Voice or Text Chatbots
- 22. Building a Big Data Application web clients mobile clients DBMS Amazon Redshift Amazon QuickSight AWS Cloud Transform your and explore your data at scale with Amazon EMR corporate data center Amazon EMR Structured Data In Amazon S3 Raw data In Amazon S3
- 23. Upload/user your existing data Discover & build models Make predictions over new data Integrated with AWS services for easy data access Model evaluation and interpretation tools Data visualization and exploration Machine Learning Algorithms APIs for batch and real-time predictions Modeling APIs Amazon Machine Learning
- 24. Building a Big Data Application web clients mobile clients DBMS Amazon Redshift Amazon QuickSight AWS Cloud React intelligently in real-time with Amazon Machine Learning corporate data center Amazon Kinesis Firehose Amazon Kinesis Analytics AWS Lambda Amazon Kinesis Streams Reference data in Amazon S3 Amazon Machine Learning Amazon SNS Amazon Athena
- 25. Amazon SQS apps Streaming KCL apps Amazon Redshift COLLECT STORE CONSUMEPROCESS / ANALYZE Amazon Machine Learning Presto Amazon EMR Amazon Elasticsearch Service Apache Kafka Amazon SQS Amazon Kinesis Streams Amazon Kinesis Firehose Amazon DynamoDB Amazon S3 Amazon ElastiCache Amazon RDS Amazon DynamoDB Streams HotHotWarm FastSlowFast SearchSQLNoSQLCacheFileMessageStream Amazon EC2 Amazon EC2 Mobile apps Web apps Devices Messaging Message Sensors & IoT platforms AWS IoT Data centers AWS Direct Connect AWS Import/Export Snowball Logging Amazon CloudWatch AWS CloudTrail RECORDS DOCUMENTS FILES MESSAGES STREAMS Amazon QuickSight Apps & Services Analysis&visualizationNotebooksIDEAPI LoggingIoTApplicationsTransportMessaging ETL BatchMessageInteractiveStreamML Amazon EMR AWS Lambda Amazon Kinesis Analytics Amazon Athena
- 26. Why AWS for Academic Computing?
- 27. University of Chicago Uses AWS to Reduce Costs and Provide Greater Than 99% Availability • The University of Chicago’s Computation Institute builds solutions for big data projects like genomics analysis and other scientific projects • Needed a cost-effective way to provide big-data analysis to labs around the world while providing always-on service • By using AWS, the Institute now experiences more than 99% availability and can focus on delivering capabilities to scientists rather than building and maintaining infrastructure
- 28. What sets AWS apart? Building and managing cloud since 2006 90+ services to support any cloud workload Over 2,000 customer-driven service/feature releases 16 regions, 42 availability zones, 70 edge locations 59 proactive price reductions to date Thousands of partners; 2,700+ Marketplace products Experience Service Breadth & Depth Pace of Innovation Global Footprint Pricing Philosophy Ecosystem *as of June 2016
- 29. JustGiving Supports 24 Million Users on Charity Site Using AWS • Needed a new platform to support general operations and new analytics service • Moved to AWS, using a wide range of services • Can scale system faster in response to unanticipated spikes in traffic • Receives query results in seconds compared to 30 minutes under old system • Obtains deeper insights into billions of data points, using information to deliver better services Using the new AWS tools, we can extract much finer-grained data points based on millions of donations and billions of visits, and then use that information to provide a better platform for our visitors. Richard Atkinson Chief Information Officer, JustGiving ” “ JustGiving is a major online platform that supports charitable giving. The organization is based in London.
- 30. Summary Build decoupled systems • Data → Store → Process → Store → Analyze → Answers Use the right tool for the job • Data structure, latency, throughput, access patterns Leverage AWS managed services • Scalable/elastic, available, reliable, secure, no/low admin Use log-centric design patterns • Immutable log, batch, interactive & real-time views Be cost-conscious • Big data ≠ big cost
- 31. Labs and training on cloud topics and AWS products Open course content by leading professors and AWS Grants for free usage of AWS services Communities that share best practices virtually and in person We invite you to join AWS Educate Accelerate cloud learning with AWS credits, cloud training, course content, and collaboration tools Not a student or educator? Help extend AWS grants to more students by inviting your network to participate (#awseducate). Learn more at: www.awseducate.com
- 32. Matt Johnson Solutions Architect, UK Public Sector E: mhj@amazon.com T: @mhj_work https://aws.amazon.com/government-education/government-uk/
Notes de l'éditeur
- So, you probably know what traditional infra look like…servers, routers, all of this equipment that you must provision, manage and secure…. In the Traditional Model – pay for everything up front, nothing is on demand, there are significant long term contracts – you undergo a long planning process to determine what you need, buy that, set up, and hope you’ve planned correctly for the future. There is a lot of internal administration to get this right. With cloud computing, you turn things on as you need or use them and no longer than that. Instead of planning and seeking funding to pay for all of this equipment, you can focus on building apps and services to accomplish your mission, to meet your customer’s needs, on a timely basis…and be able to change when they change….be able to start small, and based on demand, you can scale with your customers….give your developers the tools they need at their fingertips to manage your IT needs and advance your mission. You can move more quickly, be more agile, scale as your business requires, and provide the infrastructure to the people in your org who are building things. In cloud computing, you are giving that capability to the actual engineers to build on your behalf – this is powerful!
- Companies Will Use Data More Expansively 8th March 1939. Some of the 4 Million tickets collected from Underground Passengers examined to discover the most least used routes to help future infrastructure development.
- Companies Will Use Data More Expansively 8th March 1939. Some of the 4 Million tickets collected from Underground Passengers examined to discover the most least used routes to help future infrastructure development.
- STORY BACKGROUND The Computation Institute is a joint initiative between the University of Chicago and Argonne National Laboratory The University of Chicago serves more than 15,000 students, is located in Chicago, IL, and was founded in in 1891 Argonne National Laboratory is a multidisciplinary science and engineering research center located outside Chicago, IL. Computation Institute customers are researchers who perform big-data analysis in labs around the world SOLUTION AND BENEFITS Experiencing more than 99% availability Significant reduction in costs for large-scale genomics analysis Hosts its Globus Transfer service on AWS, helping more than 12,000 users to move data Also uses AWS for Global Genomics, which helps researchers do large-scale genomics analysis on AWS HIPAA compliance Business Associate Agreement (BAA) Using Amazon EC2, Amazon EBS, Amazon S3, Amazon RDS, Amazon Glacier
- Account Manager: See the following link for PR approved AWS Stats and Update This Slide accordingly: https://w.amazon.com/index.php/AWS_Communications_Portal#Public_AWS_Stats At AWS, we have been building and refining our services since 2006. At AWS we have been continually expanding our services to support virtually any cloud workload - we now have more than 90 services (ranging from compute, storage, networking, database, analytics, application services, deployment, management and mobile). Customers have selected AWS because we have proven ourselves committed to customer success. Our services are more mature than any other platform…and our portfolio continues to scale to meet the needs of the diverse and growing needs of our customers. As a subsidiary of Amazon, AWS thinks very long term – this benefits our customers so that we bring this value to them Because we have such a breadth and depth of services, we provide greater agility and flexibility than any on premise provider or cloud competitor can offer. This is enabled by our global footprint. We offer a very transparent pricing model, and have offered 50+ price reductions since our inception in 2006. That is because we are committed to doing what is right for our customers. I also want to highlight our partner network - our ecosystem attracts the best partners in the market that have been using us for a long time. Our customers can leverage this partner network that includes system integrators, tech partners, consulting partners, and our marketplace. Our partner network has grown over 50% over the past 12 months.
- STORY BACKGROUND JustGiving is one of the world’s largest online platforms supporting charitable causes Needed a new IT platform to support its operations and a new analytics service SOLUTION & BENEFITS Uses wide range of AWS services for general operations and for new analytics service Can scale faster in response to spikes in traffic For analytics service, receives query results in seconds instead of 30 minutes or longer under previous system, enabling faster decision making Can obtain deeper insights into billions of data points to provide better customer service CONTENT TAGS Main use case: Big Data, Analytics and Business Intelligence (BI) Additional use case(s): Website/Web App Keywords (seperated by commas): Charities, nonprofit All AWS Services used by the customer: Amazon EC2, Amazon Redshift, Amazon EMR, Amazon Kinesis, AWS Lambda, Amazon DynamoDB, Amazon S3, Amazon SNS, Amazon SQS Benefits Realized: Agility, Better Performance, Lower Time to Market, Reliability, Scalability/Elasticity, Speed