Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (10)
Similaire à Emc ecs 2 technical deep dive workshop
Similaire à Emc ecs 2 technical deep dive workshop (20)
Plus de solarisyougood
Plus de solarisyougood (16)
Dernier
Dernier (20)
Emc ecs 2 technical deep dive workshop
- 1. 1© Copyright 2015 EMC Corporation. All rights reserved. ELASTIC CLOUD STORAGE 2.0 ASD PRODUCT MANAGEMENT
- 2. 2© Copyright 2015 EMC Corporation. All rights reserved. • Student Guide & Workshop & Internal Training & Confidential Update Daily • https://goo.gl/VVmVZ0
- 3. 3© Copyright 2015 EMC Corporation. All rights reserved. UNSTRUCTURED DATA GROWTH Total Capacity Shipped, Worldwide Unstructured Data 80% 106 EB 133 EB71 EB 75% 78% Source: March 2014, IDC Structured vs. Unstructured Data, The balance of power continues to shift
- 4. 4© Copyright 2015 EMC Corporation. All rights reserved. SCALE-OUT OBJECT MARKET $- $0.2 $0.4 $0.6 $0.8 $1.0 $1.2 2014 2015 2016 2017 2018 $0.4 $0.5 $0.7 $0.9 $1.1 Source: IDC Worldwide File and Object Based Storage 2014–2018 Forecast, Oct. 2014 Revenue (in $B) 4© Copyright 2014 EMC Corporation. All rights reserved.© Copyright 2014 EMC Corporation. All rights reserved.
- 5. 5© Copyright 2015 EMC Corporation. All rights reserved. MODERN APPS ARE DRIVING CHANGE REQUIRE A DIFFERENT ARCHITECTURE/NEW CAPABILITIES Multi-tenant Universal Access/Geo-Distributed Extremely Scalable “Cloud-like” Attributes
- 6. 6© Copyright 2015 EMC Corporation. All rights reserved. CHANGE CREATES CHALLENGE MANAGING DATA GROWTH Private Cloud Public Cloud Hybrid Cloud CLOUD STRATEGY GAIN INSIGHTS FROM DATA
- 7. 7© Copyright 2015 EMC Corporation. All rights reserved. Hyper Scale - Sales Out to Billions of Objects “Public Cloud-Like” – Secure Access, Anytime, Anywhere Comprehensive Multi-Tenant Management Active/Active Geo-Distributed Architecture Multiple Protocol Support – REST and HDFS Ready Compelling Economics –Appliance or SW Only (DIY) ECS HYPERSCALE CLOUD STORAGE
- 8. 8© Copyright 2015 EMC Corporation. All rights reserved. ECS USE CASES Cloud Modern/ Apps Global Content Repository ArchiveAnalytics Global Namespace Internet of Things Sensor data SITE 1 SITE 2 SITE 3
- 9. 9© Copyright 2015 EMC Corporation. All rights reserved. ECS USE CASES UPDATE
- 10. 10EMC CONFIDENTIAL—INTERNAL USE ONLYEMC CONFIDENTIAL—INTERNAL USE ONLY GLOBAL CONTENT REPOSITORY ON-PREMISE UNSTRUCTURED STORAGE PLATFORM PROBLEM • Can’t cost-effectively manage or scale storage to support explosive growth in unstructured content • Traditional storage not suited for new Web, mobile and cloud applications • Difficult and costly to manage data lifecycle and retention policies across archive silos and sites SOLUTION EMC ECS Appliance (Object and HDFS) VALUE • Reduce complexity and cost–one globally accessible, geo-efficient archive that serves multiple applications and content types at lower cost than public cloud • Anywhere data access – All data globally accessible by Web, mobile and cloud apps • Enterprise-grade data protection – Efficient geo-protection and policy-based retention for basic compliance and governance https://accesspoint.yourcompany.com U.K.L.A. Memphis Applications Tiering, Archiving, Backup
- 11. 11EMC CONFIDENTIAL—INTERNAL USE ONLYEMC CONFIDENTIAL—INTERNAL USE ONLY MODERN APPLICATION PLATFORM EFFICIENT GEO-CAPABLE STORAGE & ANYWHERE ACCESS https://accesspoint.yourcompany.com U.K.L.A. Memphis PROBLEM • Traditional storage architecture not optimized for multi-site, mobile access to content • Writing to multiple file systems and proprietary APIs complicates development • Can’t access or process large data sets SOLUTION EMC ECS Appliance (Object and HDFS) VALUE • Anywhere access - Provides anywhere access to geo-replicated content • Simpler, faster development - Supports multiple industry standard APIs/protocols and anywhere access with strong consistency • Unmatched access and efficiency - Geo- protection, active-active architecture optimizes both access and storage efficiency for Big Data – large and small files
- 12. 12EMC CONFIDENTIAL—INTERNAL USE ONLYEMC CONFIDENTIAL—INTERNAL USE ONLY GEO-SCALE BIG DATA ANALYTICS EFFICIENT GEO-SCALE STORAGE & GLOBAL BIG DATA ANALYTICS https://accesspoint.yourcompany.com U.K.L.A. New York ANALYTICS PROBLEM • Large (and growing) data volumes lead to exponential storage costs • Traditional Hadoop replication leads to unmanageable DC footprint with data growth • Always have to move data to the analytics cluster SOLUTION EMC ECS Appliance (Object and HDFS) VALUE • Cost Efficient Storage – ECS is 65% cheaper than even public cloud • HDFS Archive – ECS brings state of the art patented technology to provide highly dense storage for Hadoop • Global Analytics –Bring analytics to geo- distributed data and archives
- 13. 13EMC CONFIDENTIAL—INTERNAL USE ONLYEMC CONFIDENTIAL—INTERNAL USE ONLY PROBLEM • Unstructured data growth - Reclaim costly Tier 1 storage • Current solutions aren’t scalable or cost efficient • Instant access to cold-stored data is required • “No Public Cloud” policy - Data needs to be on-premises SOLUTION EMC ECS Appliance (Object and HDFS) VALUE Costs less than public cloud - Provides on-premises security U.K.L.A. Memphis LAN/ WAN Video Unstructured Data Sensory Data Images COLD ARCHIVE COST EFFECTIVE LONG TERM RETENTION
- 14. 14EMC CONFIDENTIAL—INTERNAL USE ONLYEMC CONFIDENTIAL—INTERNAL USE ONLY PROBLEM • Need cost effective solution to store hue amounts of unstructured data generated by IOT and sensors • “No Public Cloud” policy - Data needs to be on-premises • Data collection via modern cloud applications requires compatibility with APIS’s like S3 and OpenStack • Analytics workflow is slow, expensive and complicated using Hadoop direct attach or public cloud storage SOLUTION EMC ECS Appliance (Object and HDFS) VALUE • ECS cost per GB is less than public clouds • ECS provides high availability with on-premises security • ECS is compatible with S3, OpenStack, and other popular API’s • ECS is HDFS compatible and enables a streamlined Hadoop workflow for “data in place” analytics ‘IOT’ CLOUD STORAGE PLATFORM ‘INTERNET OF THINGS’ – SENSORY & TELEMETRY DATA COLLECTION
- 15. 15© Copyright 2015 EMC Corporation. All rights reserved. UPCOMING ECS INTEGRATIONS
- 16. 16© Copyright 2015 EMC Corporation. All rights reserved. ISILON CLOUDPOOLS SMART TIERING TO OBJECT STORES Key Features Benefits Stub to Cloud of choice Extending SmartPools workflow to CloudPools Ability to send encrypted data to the cloud Compression for efficient transport Simple policy based management Combine file & object store benefits Use stubs to optimize local storage space, with offsite archive protection Seamless placement and availability of data per policy One Accessible namespace SmartPools -> CloudPools Clients SMB | NFS | REST| HDFS | SWIFT OneFS Service Provider Public Cloud ECS
- 17. 17© Copyright 2015 EMC Corporation. All rights reserved. EMC CLOUDBOOST LONG TERM RETENTION TO THE CLOUD LAN LAN/WAN CloudBoost appliance Desktops Laptops Files NAS/NDMP VMware & Hyper-V Databases Email Applications DB ROBO Protected by NetWorker, Avamar, NetBackup Key Features Benefits Long-term retention to ECS for NetWorker, Avamar, NetBackup Inline variable de-duplication and compression Data encrypted in-flight and at rest Cloud choice: private/public clouds Appliance cache for ROBO Capacity of up to 6PB logical per appliance Central management via a cloud portal Lower storage cost per TB Efficiency: Reduced network and storage consumption Lower risk, operational overhead than tape. Airtight security
- 18. 18© Copyright 2015 EMC Corporation. All rights reserved.© Copyright 2015 EMC Corporation. All rights reserved. ECS FREE downloadNew Monitoring Diagnostics Better Multi-Site Performance/Availability
- 19. 19© Copyright 2015 EMC Corporation. All rights reserved. Quota Management Lock/unlock user/bucket capabilities Retention Management Replication policy across tenants and buckets Operational management efficiency Support for short and long term data retention Enterprise Features FEATURES KEY BENEFITS
- 20. 20© Copyright 2015 EMC Corporation. All rights reserved. Improved failover scheme, Handling of Temporary Outages Data is now shipped as it is written Quick recovery from network outages Access to data even in the event of a temporary outage Improved RPO granularity Higher Availability FEATURES KEY BENEFITS
- 21. 21© Copyright 2015 EMC Corporation. All rights reserved. Geo Caching Better Multi-site Performance Reduced latency FEATURES KEY BENEFITS Better Multi-Site Performance
- 22. 22© Copyright 2015 EMC Corporation. All rights reserved. Improved tenant/bucket views Comprehensive capacity reporting/graphs Real-time Metering for tenants and buckets Increased visibility, determine capacity across various metrics Effectively monitor usage across your enterprise Monitoring FEATURES KEY BENEFITS
- 23. 23© Copyright 2015 EMC Corporation. All rights reserved. New storage and replication metrics Drill down trend charts, usage reports See performance/system health Flexible access GUI or API Simplified management Comprehensive metering and reporting Diagnostics FEATURES KEY BENEFITS
- 24. 24© Copyright 2015 EMC Corporation. All rights reserved. Self Contained /controller-less design Easy, single command install Improved User Experience Rapid Deployment Simplified Deployment FEATURES KEY BENEFITS
- 25. 25© Copyright 2015 EMC Corporation. All rights reserved. FREE ECS SOFTWARE 2.0 • Free Unlimited capacity • Free Unlimited time • Go from application prototype to production faster • Container-ized download available on github.com/emcecs/ • Community driven support
- 26. 26© Copyright 2015 EMC Corporation. All rights reserved. ECS ECOSYSTEM Enterprise Information Archiving Enterprise Content Management Analytics Cloud Gateways Migration Sync & Share 26© Copyright 2015 EMC Corporation. All rights reserved. Analytics Protocols CAS
- 27. 27© Copyright 2015 EMC Corporation. All rights reserved. ECS IN ACTION Analytics: Data in Place Analysis Required ECS created a multi-protocol data lake that is geo-distributed and can accept huge volumes of data from a variety of sources Analytics - Ability to add dense storage without more compute ECS scales compute and storage independently to ensure that the data in the HDFS archive is accessible to analytics applications Modern Applications: Multi protocol object support ECS delivers multi protocol object support to offer customized tax software services using S3 and Swift Global Content Repository – Massive Scale Required ECS is being used to Digitize 40 million pages, and deliver the content to anyone, anywhere on any device Tensor Company LTD *NDA Only – Not public references
Notes de l'éditeur
- Welcome to ECS (Elastic Cloud Storage 2.0) This presentation is a 4 part story: The explosive growth of unstructured data How modern “born-in-the-cloud” applications are driving changes in both application development and storage How this change is creating challenges for IT managers Introduction to ECS 2.0, use cases, benefits, and new features
- The growth in unstructured data in this IDC forecast shows that by 2017, of the 133 exabytes in enterprise storage shipped, 80% of that will be storage for unstructured data. That 80% will be made up of both scale-out file and object storage. Today’s presentation will focus on how EMC is meeting customer’s object storage needs with a solution called EMC Elastic Cloud Storage.
- Feedback- Total market numbers rather than object.
- What’s driving this “Tipping point” of explosive growth in unstructured data are modern applications. Modern or “cloud” applications have different needs than traditional platform 2 applications (ie Oracle or Microsoft enterprise apps). Multi-Tenant Modern applications require multi-tenancy. In many cases the business is operating as a service provider so customers need the ability to manage and report on various tenants while keeping the data segregated and secure. Besides requiring a multi-tenant architecture, customers also need the ability to easily provision tenants and report on usage and service levels. Universal Access/Geo Distributed Modern applications are different from traditional apps in that they need to be accessed from anywhere and on any device (preferable mobile). Its no secret that cloud applications like Uber airbnnb, and Dropbox are designed for cloud and mobile use, but this universal access is driving an entirely new app development model where common API’s are being used to deliver this accessibility. To keep service levels high, these modern cloud applications need to be geo-distributed to get the data as geo graphically close to the consumer as practical. “Cloud-like” Attributes Self-service portals have become a very convenient method to enable application developers to easily prototype, test, and proof of concept modern applications. IT requires a cloud storage platform that delivers this self-service convenience which developers have become accustomed to with public cloud services. The days of waiting weeks for storage to be provisioned for a new application are gone. The popular “Dev/Ops model empowers developers to self-serve from provisioning to production and launch.
- This sea-change has created a significant challenge for IT Managers: 1) How am I going to manage this explosive data growth being driven by the cloud? 2) What is our companies official cloud strategy? Many IT managers have elected a “no public cloud allowed” strategies to keep company data safe and secure behind the corporate firewall. Unfortunately, developers are bypassing this strategy by opening public cloud accounts because they don’t have a similar or better “company sanctioned” option. This is known as “Shadow IT” 3) As customers consider their options for a cloud storage platform, is their new solution “Hadoop ready”, allowing customers to leverage their data for it most significant value?
- EMC ECS (Elastic Cloud Storage) is a hyper scale, object-based, cloud storage infrastructure which leverages commodity components and software-defined intelligence to deliver a turnkey solution. ECS delivers the benefits of the “public cloud” in a solution that can be purchased as a turnkey system, or as software which can be deployed over an EMC approved 3rd party storage.
- ECS supports 4 primary use cases: Global content repository Cloud/modern applications Hadoop Analytics and Big Data IOT applications Cost Effective Archive
- ECS 2.0 features a host of improvements and new features. To start ECS 2.0 is now controller-less and does not require the installation of a ViPR Controller instance. In addition ECS 2.0 can now be installed via bare metal installation or with VMware virtual machine. New packaging scripts have been included for a simpler deployment experience.
- Situation Unstructured content repositories containing images, videos etc. are currently stored in high cost storage systems making it impossible for businesses to cost-effectively manage massive data growth. Desire for on-premise clouds to manage and store cold/archive data with ease. Newer applications e.g. Uber, Instagram are being written to take advantage of massive data availability, anytime, anywhere through open APIs. Enterprise developers are creating shadow IT by deploying applications in public clouds. Other 3rd party solution not enterprise production ready. Solution ECS Object Appliance
- Our next use case is really just a sub-category of the complete cloud storage platform. In addition to analytics, enterprises and service providers can use their complete cloud storage platform to support Web, mobile and Cloud applications. Problem: Traditional storage was never architected for new Web, mobile and cloud applications. They were built for access over a LAN for specific applications. Provisioning and access is driven by IT, it’s difficult if not impossible to provide self-service access to traditional storage in an IT-as-a-Service model. Writing to multiple file systems and proprietary APIs increases development time and cost Data locked into on-prem file systems is not accessible by Web-based and mobile applications Developers find ti easier to go to public cloud alternatives Solution: ECS HDFS Appliance, with support for industry standard APIs Value: ECS supports multiple access methods and a very simple geo-capability. Developers only have to worry about the apps, not the ops. ECS is made to support next-gen Web, mobile and Cloud applications. Multi-site read/writes with strong consistency make a developers job much easier. As the ECS capacity changes and grows, developers never need to recode their apps. Again, the target audience are C-level and IT leadership that are looking to deploy new Web, mobile and cloud applications, They may have some apps already deployed in a public cloud. ECS Software and/or ECS appliance lets them deploy on their own infrastructure. The VP of Apps/App architecture will also be interested – especially if they are not able to use public cloud – they can be influencers in an account since ECS appliance will make their development efforts less risky and speed time to production.
- Our next use case performance trending and host reporting is specifically focused on host performance troubleshooting. Problem: Inability to unlock business insights from complex datasets. Large (and growing) data volumes prevent timely analysis and insights. Struggle with storing and accessing PBs of data, billions of small files and/or large media files being generated. Data volume & velocity make it costly to store persistently on traditional storage platforms. Need for on-premise, enterprise ready data analytics to meet business requirements. Unmanageable Data center footprint increase due to 3X replication of standard HDFS Solution: ECS HDFS Appliance, ECS Software HDFS Service Value: Time to Market - Improve time to market for new products & applications leveraging Objects and HDFS delivered as a service. “In-place” analytics capabilities reduce risk, resources and time-to-results. Storage Efficiencies - Efficiently store PBs of data, billions of small files and/or large media files in a low cost, state-of-art, commodity-based storage system Future proof Architecture - Addresses challenges with traditional HDFS enabling enterprise features like erasure coding and geo replication with reduced storage overhead. Industry accepted standard API support for all interfaces. Reduce Risk/Deliver Value on existing Infrastructure – Enables analytics on your existing storage infrastructure without moving your data. For analytics, again, this can be a c-level discussion or even in the business units that are trying to better understand their data and extract business insight. Do they have projects for information-based applications? Data scientists are also targets - they are responsible for business intelligence and analytics and are trying to tap new data sources.
- Situation Organizations are seeing a massive growth specifically in unstructured data and need to move inactive content off of Tier 1 storage to drive down cost and fully optimize existing storage resources Public cloud archive storage services have unpredictable cost structures and often take an extremely long time to retrieve data. SMB’s and Enterprises cannot afford to wait days/weeks to gain access to archived data. Tape solutions have a low hardware cost but the dollar spend on servicing and storage tape and managing the library can become more expensive. Cloud solutions provide a much easier experience with a much lower price point at scale. Solution EMC Elastic Cloud Storage
- The Internet of Things offers a new revenue opportunity for businesses who can extract value from customer data. ECS offers an efficient platform for data collection at massive scale. ECS also streamlines analytics because the data can be analyzed directly on the ECS platform without requiring time consuming ETL (Extract, transform, load) processes. Just point the Hadoop cluster at ECS to start running queries. No expensive DAS is required on your Hadoop cluster.
- ECS 2.0 features a host of improvements and new features. To start ECS 2.0 is now controller-less and does not require the installation of a ViPR Controller instance. In addition ECS 2.0 can now be installed via bare metal installation or with VMware virtual machine. New packaging scripts have been included for a simpler deployment experience.
- CloudPools is targeted for GA in the Riptide release. The CloudPools beta is available today for selected customers. Contact product management if a customer is interested. The current CloudPools beta is a special build based on the JAWS code but newest targets such as AWS S3 and ECS will be available for testing as part of Riptide beta. So what the next six year boom if the last one was enterprise scale out NAS? Focusing on cloud as one of those areas where we want to lead and differentiate, let’s walk through the impact of ‘cloud’ on our platform… 1st – new applications require new semantics, first RAN then something else – rest transport, platform value prop, looking at swift and more S3 like semantics AND SOON OpenStack Swift access for all apps developed to this api/protocol 2nd – think of a smart pool, but out of the four walls of the box, also different in that it caches not just moves…. (weave in ever increasing performance here and tier 1.5 ambitions…) First, tier to an on prem isilon cluster – why? Little fast cluster, tiering to cold giant core. Then – what about delivering as a service – think rack space – X on prem, Y off prem Then what about deploying this same thing to CSPs – and leveraging CSPs – cooperating and competing; but offering services on top of Amazon (hat tip to Panzura here) Think more into the future, this isn’t just tiering…. The cloud version of the assets isn’t just data objects – when viewed through the Isilon software we are creating it’s a distributed filesystem.
- ECS 2.0 features a host of improvements and new features. To start ECS 2.0 is now controller-less and does not require the installation of a ViPR Controller instance. In addition ECS 2.0 can now be installed via bare metal installation or with VMware virtual machine. New packaging scripts have been included for a simpler deployment experience.
- Feedback- This should have features as well as benefits
- Feedback- This should have features as well as benefits
- Feedback- This should have features as well as benefits Please refer data lake slide deck
- Feedback- This should have features as well as benefits
- Feedback- This should have features as well as benefits
- Feedback- This should have features as well as benefits
- ECS 2.0 will be available as a FREE container-zed download on emc.com on June 22nd. It is a full functioning version with no time or capacity limits. Software developers can download ECS and install it on their notebook to provide S3 compatible cloud storage for non-production application testing and development.
- EMC Elastic Cloud has a fast growing ecosystem of partners who provide various solutions to augment its uses.
- NDA Only Remove for public presentation – These customers are not public references Partner Healthcare built their data lake with ECS to accept large amounts of data from medical imaging, customer record keeping DigitalVaticana – Used ECS to digitize over 40 million pages and make the library available to anyone globally GE uses ECS to support their advanced Hadoop data science department. Only ECS can scale compute and storage independently enabling the flexibility to purchase exactly what the application requires Tensor is a large Russian IT company that uses ECS to support modern tax applications which were created with S3 and Swift