Open stack in action openstack storage - new and upcoming from ibm research

IBM and OpenStackOpenStack Summit(San Diego 2012) by Jeff BorekLink http://www.openstack.org/summit/san-diego-2012/openstack-summit- sessions/presentation/enabling-choice-for-the-openstack-community IBM Research - Haifa2 © 2012 IBM Corporation

IBM and OpenStack IBM has a strong history of working with open standards and open source E.g., Linux, Apache, Eclipse Community approach to developing software can help meet clients’ needs faster Drive agreement on needed cloud standards and help remove threat of proprietary lock in IBM’s value Maintaining and supported by IBM IBM embraces OpenStack, contributes code and resources and consumes code Providing support for IBM platforms Leveraging IBM software and management for OpenStack IBM Research - Haifa3 © 2012 IBM Corporation

Contributions to OpenStack Success Deliver Value Globalization & localization enablement Localization for Simplified Chines Crowd-sourced translation capability Implementations of 2 important open cloud standards API, quotas, Nova integration PowerVM driver Drivers for IBM SVC & XIV Dynamic hypervisor support Membership Services from HSLT Legal support for drafting Legal support for drafting bylaws bylaws IBM Power Systems IBM Storwize V7000 IBM XIV Improvements to stability and Improvements to stability and quality quality Community sprint days Community sprint days Permission building in the Permission building in the China market China market Three IBMers named “core Three IBMers named “core contributors” contributors” 124 IBMers collaborating 124 IBMers collaborating across divisional lines across divisional lines IBM Research - Haifa4 © 2012 IBM Corporation

Different cloud workloads need different classes of storage High-performance, co-located storage for XaaS • Blocks/file to support compute General purpose data center NAS extension • Files Fixed content depot • Objects IBM Research - Haifa5 © 2012 IBM Corporation

IBM Block Storage Enablement for OpenStackOpenStack volume management drivers New in • SAN Volume Controller and Storwize family FOLSOM New in • IBM XIV FOLSOM Flex System SAN Volume Storwize V7000 V7000 Controller Unified IBM XIV Storwize V3700 Storwize V7000 IBM Research is exploring additional drivers enhancements… IBM Research - Haifa6 © 2012 IBM Corporation

Adding Functionality Beyond “just drivers” Not all storage was made equal Allocate data by business needs and requirements Enable QoS and prioritization for storage Enable IaaS provider to “hand out” different storage types Mechanism Cinder Volume types – new in Folsom Cinder Scheduler – support for “filters” and back-end capabilities Supporting volume differentiation in drivers IBM Research - Haifa7 © 2012 IBM Corporation

Adding Functionality Beyond “just drivers” Fibre Channel support (FC and FCoE) Enable use of OpenStack in enterprises IBM Research is investigating potential features such as Federation of storage systems Storage system support for fast VM provisioning High availability and QoS options for volumes Backup and DR Data reduction for Openstack storage Etc… IBM Research - Haifa8 © 2012 IBM Corporation

VISION CloudVirtualized Storage Services Foundation for the Future Internet Architect and build the next generation, standard-based, scalable, low-cost and secure cloud storage system Key Innovations: • Raise Abstraction Level of Storage • Computational Storage • Content-Centric Storage • Advanced Capabilities • Data Mobility and Federation Four use cases to demonstrate data-intensive services • Telco, Media, Healthcare and Enterprise A 3-year project, European project led by IBM • Started Oct 2010 Now considering features to port to OpenStack Swift… IBM Research - Haifa 9 www.visioncloud.eu © 2012 IBM Corporation

Cloud Data Management Interface (CDMI):An emerging standard interface for storage cloud RESTful HTTP(s) Interface: Create, Retrieve, Update and Delete objects and containers (along with other abstractions) ►Capabilities: Allow implementation to define which subset it is supports ►CDMI-aware and non-CDMI-aware clients Client issues: HTTP(s) GET, PUT, POST, DELETE CDMI Cloud Requests/Responses can include: Mime-type, data, metadata Implementation responds: HTTP(s) Status Defined by SNIA (Storage Networking Industry Association) and v1.01 in process of ISO standardization April 2009 April 2010 March 2011 April 2011 September Current: Cloud TWG CDMI V1.0 CDMI Submitted 2011 CDMI Work on launched published reference for ISO v1.0.1 CDMI 1.1 implemen- standard errata tation published IBM Research - Haifa10 © 2012 IBM Corporation

Rich Meta Data Support for ObjectsDescription • Metadata integral part of objects • Can describe content and how handled • Provide queries over metadataBenefits • Increases the value of object stores as an infrastructure for building value-add applications over the stored data, e.g. for healthcare, telco and media. IBM Research - Haifa11 © 2012 IBM Corporation

Rich Meta Data Support for ObjectsIndex and queries for user metadataA catalog maintains for each object in a container a list of MyContainer the attributes and attribute-value pairs Obj 1 • A content-centric query requires a look-up in the catalog Obj 3Example (schematic) – list all red objects GET /MyContainer/ HTTP/1.1 Obj 2 . . . x-Match-md: x-Attribute=‘color’ x- Value=‘red’Response (schematic) Attribute Value Object HTTP/1.1 200 OK Content-Type: application/json color red Obj 3 { shape square Obj 2 "children" : [ “Obj 2", shape triangle Obj 1 “Obj 3" ] color blue Obj 1 } color red Obj 2 shape square Obj 3 IBM Research - Haifa12 © 2012 IBM Corporation

Computational Support via Storlet EngineDescription • “Stored procedures” for a storage cloud • Provide ability to run computations (storlets) safely and securely, close to the dataBenefits • Reduce bandwidth, prevent exposure of sensitive data • Enables extending Swift without changing its code • Create customized solutions IBM Research - Haifa13 © 2012 IBM Corporation

Storlets are the “stored procedure” of object clouds PUT Pudong Feb 2012Storlets provide a safe and secure way to execute mimetype = jpegcomputations in a storage cloud category = vacation picture • Typically run in a sandbox location = ShanghaiStorlets are uploaded as objects Thumbnail Creator • Distinguished from other objects by metadata Object-type = storletStorlets are triggered by events on objects (e.g., Put object trigger:put/get) and associated metadata attributes mimetype = jpeg category = vacation picture • Synchronous or asynchronous Code:Benefits .... • Locality – avoid network overhead • Security – avoid transferring data outside of cloud • Timeliness Pudong Feb 2012 thumbnail • Automated execution mimetype = jpeg • Stronger provenance category = vacation pictureUse cases Location = Shanghai • Transformations on data, e.g., transcoding, computing thumbnails • Extraction/derivation of metadata • Simple computations IBM Research - Haifa14 © 2012 IBM Corporation

Meta data and Storlets in actionManaging and Serving Content Transcode into additional formats. Metadata indicates formats Telco use case Use metadata to Upload select optimum format MPEG-4 for device/browser Media use case Low-res copy IBM Research - Haifa15 © 2012 IBM Corporation

Supporting Secure Multi-TenancyDescription • Provide secure logical isolation between tenants to enable hosting of many tenants over the same shared infrastructure • User of one tenant cannot access storage of another tenant • Security breach in one tenant cannot be leveraged to breach another tenantBenefits • Feature required in order to provide secure public object cloud IBM Research - Haifa16 © 2012 IBM Corporation

We want to allow secure lightweight isolation between tenants whileallowing complete sharing of physical resources Approach Run time model and security Model • Principle of least privilege: Every sub component should operate using the least set of privileges required for the job completion. Tenant Multi-tenancy and isolation • Build a system with separate tenant privileges. If the system is compromised the damage should be confined to a single User tenant. • There should be a complete isolation of all tenant related information to prevent any cross-tenant leakage. • All data-at-rest should be encrypted with a per tenant key Container Scalability and performance • Security that can scale Object • Limit the performance affecting overheads. IBM Research - Haifa17 © 2012 IBM Corporation

Secure WAN De-duplicationDescription • Phase 1: Support full object deduplication in the storage and over the network. • Phase 2: Add Proof of Ownership (PoW) mechanisms to enable secure WAN deduplicationBenefits • Capacity and bandwidth efficiency for applications like Mail and content depots • Security for client use is unique IBM Research - Haifa18 © 2012 IBM Corporation

Client-side deduplication in a cloud has a potential for significant savings,capacity and bandwidth but entails security challengesAllows savings both bandwidth and capacity Content is new Content already existsBasic protocol: Client computes a deterministic short hash of the data Client Swift Client Swift Client sends hash value to the cloud server Data Content Data Content Cloud asks for the actual data only if the hash SHA SHA (and data) are unfamiliar 1 1 2fd4e1c6 2fd4e1c6One problem to address is spoofing uploads Create: object1 Create: object2 Attacker obtains hash of victim’s file Content-id: 2fd4e1c6 Content-id: 2fd4e1c6 Attempts to upload a file, but swaps the hash value with that of the victim’s file. Object w/ content ID 2fd4e1c6 doesn’t exist Object w/ content ID 2fd4e1c6 exists File is now registered to attacker Download file… Special error response Success response Create: object1 Content-id: 2fd4e1c6 Only need to get hold of a very small (not necessarily secret) piece of information Data Content Example: 160 bits to get hold of a 1.5GB object from a popular backup server Create ‘object1’ Success responseWe have developed a solution called Proofs of Ownership (PoW) A challenge response phase during uploads IBM Research - Haifa19 © 2012 IBM Corporation

Extending to a Global Storage PoolDescription • Tie together multiple Swift clusters into a single namespace • Enable a geographically distributed Swift installation • Support active/active replication • Support geographic placement constraintsBenefits • Reduce TCO • Increase availability • Enable large cloud-based implementations IBM Research - Haifa20 © 2012 IBM Corporation

Extending Swift to a Global Storage Pool enables large cloud-basedimplementationsGlobal Distribution • Replicate objects in different data centers • Simplify failure recovery/DR • Use for availability, recovery and performance IBM Research - Haifa21 © 2012 IBM Corporation

FI-WARE – Foundation for Future Internet Mission: provide core platform for FI applications in multiple industries* (‘Usage Area’ projects*) * FI-WARE * * * * * * *FI-WARE Budget: €40M (FI-PPP in total: €300M over 5 years) IBM Research - Haifa 22 © 2012 IBM Corporation

Open stack in action openstack storage - new and upcoming from ibm research

by eNovance on Dec 03, 2012

Accessibilité

Détails de l'import

Droits d'utilisation

1 intégré 2

Statistiques

Open stack in action openstack storage - new and upcoming from ibm research Presentation Transcript