Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

S106195 cos-use cases-istanbul-v1902a

50 vues

Publié le

IBM Cloud Object Storage, Use Cases

Publié dans : Technologie
  • Soyez le premier à commenter

S106195 cos-use cases-istanbul-v1902a

  1. 1. 2019 IBM Systems Technical University February 6-8 Istanbul, Turkey IBM Cloud Object Storage Features and Use Cases Tony Pearson Master Inventor, Senior IT Architect IBM Systems Lab Services
  2. 2. Abstract 2 © Copyright IBM Corporation 2019 Can't afford to save your world's collection of data on your IT budget? This session explains why object storage is becoming so popular and how it differs from block or file-level storage. The speaker will provide an overview of IBM Cloud Object Storage System and its clever, highly scalable, highly reliable use of Erasure Coding to reduce storage costs by up to 70 percent from traditional disk arrays
  3. 3. Agenda 3 Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases © Copyright IBM Corporation 2019
  4. 4. How is Object Storage Different? 4 Block and File Storage – Decide where to put it • For block, which array/volume/LUN • For file, which filer/subdirectory – Remember where it is to get it back – Don’t let anyone else move it – Read/Write records, append data – Limits on LUN size, number of files Object Storage – Provide data over to the Object storage • Get “claim stub” reference locator – Use or share “claim stub” to access data HTTP, Openstack Swift, S3 – Get/Put/Delete object in its entirety – Effectively “unlimited” scalability © Copyright IBM Corporation 2019
  5. 5. Object Storage for Static and Stable data 5 Hot Data High-IOPS and Low-Latency All-Flash and Hybrid Flash/Disk Information Lifecycle Management (ILM) Structured data / Random-Access Active logs and traces Virtual Machines and VDI Single-Tenancy Static and Stable data Backups, Files, Archives Seismic, Research, Telemetry, HPC Video, Animation, Body Cams Photos, Images, CAD/CAM, GIS Music, Audio Genomic, Medical Images Multi-tenancy Object Store provides a • Secure • Reliable • Scalable • Cost Effective Platform For Unstructured data Object Store is not designed for • High IOPS workflows • Transaction Processing • Inherent ILM © Copyright IBM Corporation 2019
  6. 6. Object Storage is Simpler for Application Development 6 POSIX – over 60 commands NFS – over 30 commands HEAD • Read metadata GET • Read content and metadata PUT • Write content and metadata DELETE • Remove object or empty container POST • Update metadata Object – 5 commands SMB – over 80 commands © Copyright IBM Corporation 2019
  7. 7. Why Object Storage? 7 It enables my business and reduces costs! • Handles 80% of my data • Scales easily, growth does not impact my business plan • Less expensive than traditional flash and disk arrays It is easy to deploy and manage! • Easily provision to users and teams • Location-independent, Flat namespace • Software-Defined or Pre-built systems • Ethernet-based and Cloud friendly • Online backup/archive alternative to tape and virtual tape libraries It is great for developing software! • GET/PUT interface reduces coding effort • Robust metadata for search of documents, images, and videos • Can be used for Analytics, IoT, Mobile, Web and Cloud Applications IT Director Software Developer Storage Administrator © Copyright IBM Corporation 2019
  8. 8. Agenda 8 Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases © Copyright IBM Corporation 2019
  9. 9. Moderate Performance, Moderate Cost 9 Unified file and object storage. Optimized for high performance, across flash and disk Flash 15K Unified file and object storage on tape Information Lifecycle Management (ILM) across tiers HighestPerformance Lowest cost Tape IBM was looking to offer easy to manage, scalable disk-based object storage for unstructured data • Moderate performance • Moderate cost 10K 7200 rpm © Copyright IBM Corporation 2019
  10. 10. RAID versus Erasure Coding 10 Tolerate 1 drive failure Tolerate 2 drive failures Tolerate “M” failures RAID-1 / RAID-10 K pieces : 2 x K slices RAID-5 K pieces : K + 1 slices 2.0X 1.2X 3.0X 1.5X 1.7X Triplication K pieces : 3 x K slices RAID-6 K pieces : K + 2 slices Erasure Coding K pieces : K+M = N slices © Copyright IBM Corporation 2019
  11. 11. Data Growth at Petabyte (PB) Scale 11 PB of data 3 to 5x Data Protection RAID, Mirrors, Replication, Tape Data Protection High Availability & Disaster Recovery Geo-Distribution & Erasure Coding Infrastructure Proprietary, specialized hardware, multiple systems Operations More than 1 FTE per PB Maintenance outages Infrastructure Software Defined, Commodity Hardware, Single System Operations Less than 1 FTE per 6 PB Single system, Secure Self-healing 1.7 x 60% Less Hardware & Rack space Traditional Approach IBM Cloud Object Storage Approach 70% Lower TCO © Copyright IBM Corporation 2019
  12. 12. IBM Cloud Object Storage System economics beat legacy NAS storage and Amazon Web Services (AWS) 12 $8,400 $4,210 $1,613 $1,053 Legacy NAS DR protected Legacy NAS single copy IBM Cloud Object + NAS gateway dsNet object protected IBM Cloud Object IBM Cloud Object vs NAS Cost: 80% lower $0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 480 TB 960 TB 1920 TB 3840 TB Cost: 10 to 60% lower IBM Cloud Object vs AWS S3 IBM Cloud Object Amazon S3 © Copyright IBM Corporation 2019
  13. 13. IBM Cloud Object Storage System – Terminology 13 HTTP Application Server IBM COS Manager Accesser Slicestor Device Set Accesser Pool Vault Load Balancer Site A Site B Site C End user Global Namespace IBM Cloud Object Storage System organizes objects into “Vaults” and “Containers” Storage Pools can hold multiple vaults Vaults can have millions of containers OpenStack Swift S3 Protocol © Copyright IBM Corporation 2019
  14. 14. The Math Behind Reed-Solomon Erasure Coding 14 “K” variables of input data: a, b, c, d, e Example: K=5 M=3 K+M= 8 Here we create “K+M” equations, adding and subtracting by different co-factors Results in “K+M” slices that can be geographically dispersed We can tolerate losing up to “M” slices of data, and still solve for the original “K” pieces of data. © Copyright IBM Corporation 2019
  15. 15. Information Dispersal Algorithm (IDA) 15 01 02 03 04 05 06 07 08 09 10 11 12 A B C D E F G IDA is represented as Width/Read/Write This example is 12/7/9 Original data is received by IBM COS Accesser and chopped up into 7 pieces (Read threshold) Erasure Coding: 7 variables in, 12 equation results out (Width) 12 slices written in parallel to IBM COS Slicestor devices A B C D E F G IBM SmartWrite™ At least 9 slices (Write Threshold) must be written for the I/O to be considered complete, rest will be written later. IBM COS periodically scans and rebuilds slices as needed IBM SmartRead™ To read the data back, only 7 slices are used to solve 7 equations (Read threshold) Original data is re-assembled © Copyright IBM Corporation 2019
  16. 16. Dedicated (Private) IBM Cloud Object Storage System – Single-site, Two-site and Multi-Site 16 Single Site Better performance, when site-tolerance not a factor, better than traditional RAID-5 / RAID-6 Two Site Vault Mirroring Allows customers to leverage existing two-site infrastructure. Provides concurrent reads and writes despite communication disruption between locations. Local data better than traditional RAID-5 / RAID-6 Geographically Dispersed Three to Nine Sites Consider adding IBM Cloud or leverage existing datacenter locations to provide a broader distribution of data for higher availability, site- tolerance and scalability © Copyright IBM Corporation 2019
  17. 17. Two Dispersal Modes 17 — Standard Dispersal — One or more Accessers — Each slice on its own Slicestor — Concentrated Dispersal (CD Mode) — Entry Level System (1-3 sites) — Accessers can be embedded into Slicestor (Optional) — Each slice stored on separate HDD within Slicestor for availability 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12 © Copyright IBM Corporation 2019
  18. 18. Scalability 18 Site 1 Site 2 Site 3 S3 Compatible API Openstack Swift Compatible API Simple Object API Slicestor Software Accesser Software Scalability – Scale performance and/or capacity at any time with no downtime to operations Need more Performance? Add more Accesser nodes Need more Capacity? Add more disks to existing Slicestor nodes, add more device sets to existing storage pools, or add more storage pools –All nodes in single device set must have same amount of storage • New device sets can have different size drives than existing drives sets –All device sets in a storage pool must have the same width • Standard and Concentrated IDA can be intermixed in same storage pool –Different storage pools can have different widths Device Set 1 Device Set 2 Device Set 3 © Copyright IBM Corporation 2019
  19. 19. Encryption Options for IBM Cloud Object Storage 19 A A B C D E F G Data A A# AONT Data SecureSlice™ System-generated key Used to encrypt data Data A # Hash taken of encrypted data Key wrapped with hash makes All-or-Nothing Transform (AONT) Information Dispersal Algorithm (IDA) User sends data Data B Data B Server-Side Encryption with Customer key System-generated key used to encrypt data Key wrapped with customer-supplied key User sends data and key SSE-C Data CData C AMK Application-managed key User sends encrypted data Encryption disabled in IBM COS © Copyright IBM Corporation 2019
  20. 20. Compliance-Enabled Vault 20 Standard Vault Compliance Enabled Vault Government Regulations specify that “electronic storage must preserve the records exclusively in a non-erasable, non-rewriteable, format” (SEC 17a-4) Various Terms have been used: • Write Once Read Many (WORM) for tape and optical media • Fixed-Content or Content-Addressable for disk- based storage • Immutable for file systems Non-erasable, Non-rewriteable (NENR) adopted to cover all of the above Compliance-Enabled Vaults prevent objects from being deleted or modified for a specified retention period Vaults can co-exist in same storage pool © Copyright IBM Corporation 2019
  21. 21. IBM Cloud Object Storage System – Software, Pre-Built Systems or Cloud Services 21 Software • ClevOS – IBM Cloud Object Storage System software packaged with Debian Linux OS • Software-defined, hardware- aware model for flexibility of x86 platform choice • Can be deployed on qualified vendor hardware Pre-built Systems • Fully integrated appliance models for easy deployment and support • On-premises object storage solution Cloud Services • IBM Cloud Object Storage System as a service for customers that want security and controlled performance © Copyright IBM Corporation 2019
  22. 22. IBM Spectrum Storage Suite 22 Bundle license for flash, disk, tape and storage- rich servers Per TiB IBM Spectrum Storage Suite Control Protect Archive Virtualize Accelerate Scale IBM Cloud Object Storage I’ve deployed Spectrum Storage Suite. Now what? 1. Well, since you already own entitlement to all capabilities in the Spectrum Storage suite, you are free to download any of the software you like, including IBM Cloud Object Storage! 2. To help facilitate quick adoption of additional use cases, Spectrum Storage Suite licensing offers the ability to perform extended tests in an evaluation sandbox proving ground without additional charge. Tebibytes (TiB) in this sandbox are not counted. So go ahead, experiment with your next use case. Prove it, become familiar with it, pay for it only when it’s deployed for productive use. © Copyright IBM Corporation 2019
  23. 23. IBM Cloud Object Storage System – Deployment Options 23 Infrastructure Dedicated (Private) Public Local (Private) On-premises Off-premises Cloud Infrastructure Off-premises (IBM Cloud) Public • Standard regional • Nearline regional • Nearline geo- dispersed Consumed pricing Dedicated • IBM Managed • Client managed • Hybrid / Mixed Allocated pricing Client infrastructure On-premises Locally managed • Software • Pre-built systems Entry Level System as small as 72 TB © Copyright IBM Corporation 2019
  24. 24. Pre-Built Systems for IBM Cloud Object Storage System 24 IBM COS Manager 2105 / 3105 Accesser 2100 / 3105 / 4105 Slicestor 2212 Slicestor 2448 Health and performance monitoring GUI and API access Configuration and security Zero downtime upgrades 12 drives @ 4, 6, 8 or 10 TB 48 to 120 TB Nearline HDD in 2U rack space 16, 32 or 48 drives @ 4, 6, 8 or 10 TB 64 to 480 TB Nearline HDD in 4U rack space Slices, Disperses and Retrieves data © Copyright IBM Corporation 2019
  25. 25. VersaStack Solution for Object Storage 25 Cisco UCS S3260 Storage Server Dual nodes in 4U space 56 hot-swappable 3.5” LFF HDD 4, 6, 8, or 10 TB 7200-rpm NL-SAS (28 drives per COS Slicestor) Cisco UCS 6300 Fabric Interconnect Low-latency, lossless 10 and 40 GbE Cisco UCS 220 M4 servers 1U with 36 cores, 24 DDR4 memory DIMMs -- For COS Manager and Accessers Cisco Validated Design (CVD) 784 to 1960 TB Usable Capacity © Copyright IBM Corporation 2019
  26. 26. IBM Cloud Object Storage public cloud services 26 IBM Cloud Object Storage Flex DataAccessPattern IBM Cloud Object Storage Standard IBM Cloud Object Storage Vault Variable data access pattern / Mixed workloads Predictable data access pattern / workload Frequent data access Less frequent data access Minimal data access IBM Cloud Object Storage Cold Vault • High Availability – Cross Regional USA • Low Latency – Regional (Dallas) © Copyright IBM Corporation 2019
  27. 27. Agenda 27 Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases © Copyright IBM Corporation 2019
  28. 28. A broad spectrum of strategies to build, buy, or assemble an end to end solution Integrations, Connectors, Gateways Ready to RunCustom Applications Off-PremisesHybridOn-Premises and/or and/or How to Consume IBM Cloud Object Storage 28 © Copyright IBM Corporation 2019
  29. 29. Large multimedia files Video, Audio, Images Store telemetric, instrumentatio n and sensor data Web generated and social media data Backup, Active archive, DR, inactive and compliance data Security audits, Log files and Analytics Collaboration and content distribution Global file services, mobile access, and StaaS Genomics, HPC, Geo- spatiial 8 7 6 5 4 1 2 3 Object Storage Typical Use Cases Storage Services 29 © Copyright IBM Corporation 2019
  30. 30. Market Industry Verticals 30 Secure customer trust and business compliance . Financial Services & Insurance Incorporate into IoT processes, analytics Industrial, Distribution Manage the data essential to serving the good of the public. Public Sector, Government Scale your market offering without worry. Telco & Service Providers Create and Deliver Content on demand. Media & Entertainment Heath Care & Life Sciences Put medical progress before everything else. Every industry does backup, all have some amount of static-and-stable unstructured data, most need file or content archive… © Copyright IBM Corporation 2019
  31. 31. Client Experience – IBM Cloud Object Storage 31 Users upload photo and video content via web based application Photo and video objects are sent to IBM COS via REST based protocols Metadata is captured and stored • Scale – 150 petabytes and growing: more than 50 Billion images stored • Manageability – 3 Administrators manage entire environment • Security – 50,000+ uploads per minute with zero touch security • Always-on availability – SLA of 100% download on demand – even during California to Nevada datacenter move • Economics – Operating costs reduced by more than 70% • Key decision makers – Technical team backed by financial cost cutting mandates © Copyright IBM Corporation 2019
  32. 32. IBM Spectrum Protect – Cloud and Object Store Capabilities 32 Client nodes Off-premises: • IBM Cloud • Amazon Web Services • Microsoft Azure On-premises IBM Spectrum Protect Server • IBM Cloud Object Storage System (using S3-compatible API) © Copyright IBM Corporation 2019
  33. 33. IBM Spectrum Storage and IBM Cloud Object 33 Unified file and object storage. Optimized for high performance, across flash, disk and object store Flash Object Store 15K Object storage on disk ( File, backup and archive interfaces available through variety of options ) IBM Cloud Amazon Web Services Microsoft Azure Swift S3 emulation OpenStack Swift Unified file and object storage on tape Transparent Cloud Tiering Information Lifecycle Management (ILM) across tiers HighestPerformance Lowest cost Tape10K 7200 © Copyright IBM Corporation 2019
  34. 34. Summary 34 Object Storage scales easily to handle your unstructured data, easy to manage, ideal for software development IBM is the #1 leading vendor of Object Storage, with over 400 patents, and several clients >100PB IBM Cloud Object Storage can be used on-premises, hybrid, or in the Cloud Integrates with the rest of the IBM Spectrum Storage suite © Copyright IBM Corporation 2019
  35. 35. Thank you! 35 © Copyright IBM Corporation 2019 Please complete the Session Evaluation! Speaker: Tony Pearson Session: s106195 IBM Cloud Object Storage Features and Use Cases
  36. 36. About the Speaker 36 Tony Pearson is a Master Inventor and Senior IT Architect for the IBM Storage product line. Tony joined IBM Corporation in 1986 in Tucson, Arizona, USA, and has lived there ever since. In his current role, Tony presents briefings on storage topics covering the entire IBM Storage product line, IBM Spectrum Storage software products, and topics related to Cloud Computing, Analytics and Cognitive Solutions. He interacts with clients, speaks at conferences and events, and leads client workshops to help clients with strategic planning for IBM’s integrated set of storage management software, hardware, and virtualization solutions. Tony writes the “Inside System Storage” blog, which is read by thousands of clients, IBM sales reps and IBM Business Partners every week. This blog was rated one of the top 10 blogs for the IT storage industry by “Networking World” magazine, and #1 most read IBM blog on IBM’s developerWorks. The blog has been published in series of books, Inside System Storage: Volume I through V. Over the past years, Tony has worked in development, marketing and consulting for various storage hardware and software products. Tony has a Bachelor of Science degree in Software Engineering, and a Master of Science degree in Electrical Engineering, both from the University of Arizona. Tony is an inventor or co-inventor of 19 patents in the field of electronic data storage. 9000 S. Rita Road Bldg 9032 Floor 1 Tucson, AZ 85744 +1 520-799-4309 (Office) tpearson@us.ibm.com Tony Pearson Master Inventor Senior IT Architect IBM Storage © Copyright IBM Corporation 2019
  37. 37. Special Thanks for the following contributors to this presentation — John Shubeck IBM Client Technical Specialist for IBM COS 37 © Copyright IBM Corporation 2019
  38. 38. Additional Resources from Tony Pearson 38 © Copyright IBM Corporation 2019 Email: tpearson@us.ibm.com Twitter: twitter.com/az990tony Blog: ibm.co/Pearson Books: www.lulu.com/spotlight/990_tony IBM Expert Network on Slideshare: www.slideshare.net/az990tony Facebook: www.facebook.com/tony.pearson.16121 LinkedIn: https://www.linkedin.com/in/az990tony
  39. 39. This presentation uses the IBM Plex™ font 39 © Copyright IBM Corporation 2019 IBM Plex™ is our new typeface. It’s global, it’s versatile and it’s distinctly IBM. IBM Plex Sans The IBM company is freeing itself from the cold, modernist cliché and replacing Helvetica with a new corporate typeface. Also replaces Arial, Calibri, Lucida Grande, Trebuchet, etc. IBM Plex Mono A little something for developers. Replaces Courier New, Letter Gothic, Lucida Console, etc. IBM Plex Serif A hybrid of the third kind (combining the best of Plex, Bodoni, and Janson into a contemporary serif). Replaces Cambria, Garamond, Lucida Bright, Times New Roman, etc. IBM Plex is freely available as TrueType and OpenType at: https://github.com/IBM/plex/releases
  40. 40. Notices and disclaimers © 2019 International Business Machines Corporation. No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event, shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted per the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer follows any law. 40 © Copyright IBM Corporation 2019
  41. 41. Notices and disclaimers, continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products about this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a purpose. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com and [names of other referenced IBM products and services used in the presentation] are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml. . 41 © Copyright IBM Corporation 2019