1. OpenStack Storage
New and Upcoming from IBM Research
Ronen Kat, IBM Research – Haifa
ronenkat@il.ibm.com
© 2012 IBM Corporation

2. IBM and OpenStack
OpenStack Summit
(San Diego 2012)
by Jeff Borek
Link
http://www.openstack.org/summit/san-diego-2012/openstack-summit-
sessions/presentation/enabling-choice-for-the-openstack-community
IBM Research - Haifa
2 © 2012 IBM Corporation

3. IBM and OpenStack
IBM has a strong history of working with open standards and open
source
E.g., Linux, Apache, Eclipse
Community approach to developing software can help meet clients’
needs faster
Drive agreement on needed cloud standards and help remove threat of
proprietary lock in
IBM’s value
Maintaining and supported by IBM
IBM embraces OpenStack, contributes code and resources and consumes code
Providing support for IBM platforms
Leveraging IBM software and management for OpenStack
IBM Research - Haifa
3 © 2012 IBM Corporation

4. Contributions to OpenStack Success Deliver Value
Globalization & localization enablement
Localization for Simplified Chines
Crowd-sourced translation capability Implementations of 2 important open
cloud standards
API, quotas, Nova integration
PowerVM driver Drivers for IBM SVC & XIV
Dynamic hypervisor
support
Membership Services from HSLT
Legal support for drafting
Legal support for drafting
bylaws
bylaws
IBM Power Systems IBM Storwize V7000 IBM XIV Improvements to stability and
Improvements to stability and
quality
quality
Community sprint days
Community sprint days
Permission building in the
Permission building in the
China market
China market
Three IBMers named “core
Three IBMers named “core
contributors”
contributors”
124 IBMers collaborating
124 IBMers collaborating
across divisional lines
across divisional lines
IBM Research - Haifa
4 © 2012 IBM Corporation

5. Different cloud workloads need different classes of storage
High-performance, co-located storage for XaaS
• Blocks/file to support compute
General purpose data center NAS extension
• Files
Fixed content depot
• Objects
IBM Research - Haifa
5 © 2012 IBM Corporation

6. IBM Block Storage Enablement for OpenStack
OpenStack volume management drivers
New in
• SAN Volume Controller and Storwize family FOLSOM
New in
• IBM XIV FOLSOM
Flex System SAN Volume
Storwize V7000
V7000 Controller
Unified IBM XIV
Storwize V3700 Storwize V7000
IBM Research is exploring additional drivers enhancements…
IBM Research - Haifa
6 © 2012 IBM Corporation

7. Adding Functionality Beyond “just drivers”
Not all storage was made equal
Allocate data by business needs and requirements
Enable QoS and prioritization for storage
Enable IaaS provider to “hand out” different storage types
Mechanism
Cinder Volume types – new in Folsom
Cinder Scheduler – support for “filters” and back-end capabilities
Supporting volume differentiation in drivers
IBM Research - Haifa
7 © 2012 IBM Corporation

8. Adding Functionality Beyond “just drivers”
Fibre Channel support (FC and FCoE)
Enable use of OpenStack in enterprises
IBM Research is investigating potential features
such as
Federation of storage systems
Storage system support for fast VM provisioning
High availability and QoS options for volumes
Backup and DR
Data reduction for Openstack storage
Etc…
IBM Research - Haifa
8 © 2012 IBM Corporation

9. VISION Cloud
Virtualized Storage Services Foundation for the Future Internet
Architect and build the next generation, standard-based,
scalable, low-cost and secure cloud storage system
Key Innovations:
• Raise Abstraction Level of Storage
• Computational Storage
• Content-Centric Storage
• Advanced Capabilities
• Data Mobility and Federation
Four use cases to demonstrate data-intensive services
• Telco, Media, Healthcare and Enterprise
A 3-year project, European project led by IBM
• Started Oct 2010
Now considering features to port to OpenStack Swift…
IBM Research - Haifa
9 www.visioncloud.eu © 2012 IBM Corporation

10. Cloud Data Management Interface (CDMI):
An emerging standard interface for storage cloud
RESTful HTTP(s) Interface: Create, Retrieve, Update and Delete objects and containers
(along with other abstractions)
►Capabilities:
Allow implementation to define which subset it is supports
►CDMI-aware and non-CDMI-aware clients
Client issues:
HTTP(s) GET, PUT, POST, DELETE
CDMI Cloud
Requests/Responses can include:
Mime-type, data, metadata
Implementation responds:
HTTP(s) Status
Defined by SNIA (Storage Networking Industry Association) and v1.01 in process of
ISO standardization
April 2009 April 2010 March 2011 April 2011 September Current:
Cloud TWG CDMI V1.0 CDMI Submitted 2011 CDMI Work on
launched published reference for ISO v1.0.1 CDMI 1.1
implemen- standard errata
tation published
IBM Research - Haifa
10 © 2012 IBM Corporation

11. Rich Meta Data Support for Objects
Description
• Metadata integral part of objects
• Can describe content and how handled
• Provide queries over metadata
Benefits
• Increases the value of object stores as an infrastructure for building value-add
applications over the stored data, e.g. for healthcare, telco and media.
IBM Research - Haifa
11 © 2012 IBM Corporation

12. Rich Meta Data Support for Objects
Index and queries for user metadata
A catalog maintains for each object in a container a list of MyContainer
the attributes and attribute-value pairs
Obj 1
• A content-centric query requires a look-up in the
catalog
Obj 3
Example (schematic) – list all red objects
GET /MyContainer/ HTTP/1.1
Obj 2
. . .
x-Match-md: x-Attribute=‘color’ x-
Value=‘red’
Response (schematic) Attribute Value Object
HTTP/1.1 200 OK
Content-Type: application/json color red Obj 3
{ shape square Obj 2
"children" : [
“Obj 2", shape triangle Obj 1
“Obj 3" ] color blue Obj 1
}
color red Obj 2
shape square Obj 3
IBM Research - Haifa
12 © 2012 IBM Corporation

13. Computational Support via Storlet Engine
Description
• “Stored procedures” for a storage cloud
• Provide ability to run computations (storlets) safely and securely, close to the
data
Benefits
• Reduce bandwidth, prevent exposure of sensitive data
• Enables extending Swift without changing its code
• Create customized solutions
IBM Research - Haifa
13 © 2012 IBM Corporation

14. Storlets are the “stored procedure” of object clouds
PUT Pudong Feb 2012
Storlets provide a safe and secure way to execute mimetype = jpeg
computations in a storage cloud category = vacation picture
• Typically run in a sandbox location = Shanghai
Storlets are uploaded as objects Thumbnail Creator
• Distinguished from other objects by metadata Object-type = storlet
Storlets are triggered by events on objects (e.g., Put object trigger:
put/get) and associated metadata attributes mimetype = jpeg
category = vacation picture
• Synchronous or asynchronous Code:
Benefits ....
• Locality – avoid network overhead
• Security – avoid transferring data outside of cloud
• Timeliness
Pudong Feb 2012 thumbnail
• Automated execution mimetype = jpeg
• Stronger provenance category = vacation picture
Use cases Location = Shanghai
• Transformations on data, e.g., transcoding,
computing thumbnails
• Extraction/derivation of metadata
• Simple computations IBM Research - Haifa
14 © 2012 IBM Corporation

15. Meta data and Storlets in action
Managing and Serving Content
Transcode into
additional formats.
Metadata indicates formats
Telco use case
Use metadata to
Upload select optimum format
MPEG-4 for device/browser
Media use case
Low-res copy
IBM Research - Haifa
15 © 2012 IBM Corporation

16. Supporting Secure Multi-Tenancy
Description
• Provide secure logical isolation between tenants to enable hosting of many
tenants over the same shared infrastructure
• User of one tenant cannot access storage of another tenant
• Security breach in one tenant cannot be leveraged to breach another tenant
Benefits
• Feature required in order to provide secure public object cloud
IBM Research - Haifa
16 © 2012 IBM Corporation

17. We want to allow secure lightweight isolation between tenants while
allowing complete sharing of physical resources
Approach
Run time model and security
Model • Principle of least privilege: Every sub component should
operate using the least set of privileges required for the job
completion.
Tenant Multi-tenancy and isolation
• Build a system with separate tenant privileges. If the system
is compromised the damage should be confined to a single
User
tenant.
• There should be a complete isolation of all tenant related
information to prevent any cross-tenant leakage.
• All data-at-rest should be encrypted with a per tenant key
Container
Scalability and performance
• Security that can scale
Object
• Limit the performance affecting overheads.
IBM Research - Haifa
17 © 2012 IBM Corporation

18. Secure WAN De-duplication
Description
• Phase 1: Support full object deduplication in the storage and over the network.
• Phase 2: Add Proof of Ownership (PoW) mechanisms to enable secure WAN
deduplication
Benefits
• Capacity and bandwidth efficiency for applications like Mail and content depots
• Security for client use is unique
IBM Research - Haifa
18 © 2012 IBM Corporation

19. Client-side deduplication in a cloud has a potential for significant savings,
capacity and bandwidth but entails security challenges
Allows savings both bandwidth and capacity
Content is new Content already exists
Basic protocol:
Client computes a deterministic short hash of the
data Client Swift Client Swift
Client sends hash value to the cloud server Data Content Data Content
Cloud asks for the actual data only if the hash SHA SHA
(and data) are unfamiliar 1 1
2fd4e1c6 2fd4e1c6
One problem to address is spoofing uploads
Create: object1 Create: object2
Attacker obtains hash of victim’s file Content-id: 2fd4e1c6 Content-id: 2fd4e1c6
Attempts to upload a file, but swaps the hash value
with that of the victim’s file. Object w/ content ID 2fd4e1c6
doesn’t exist
Object w/ content ID
2fd4e1c6 exists
File is now registered to attacker
Download file… Special error response Success response
Create: object1
Content-id: 2fd4e1c6
Only need to get hold of a very small
(not necessarily secret) piece of information Data Content
Example: 160 bits to get hold of a 1.5GB object
from a popular backup server Create ‘object1’
Success response
We have developed a solution called Proofs of Ownership (PoW)
A challenge response phase during uploads
IBM Research - Haifa
19 © 2012 IBM Corporation

20. Extending to a Global Storage Pool
Description
• Tie together multiple Swift clusters into a single namespace
• Enable a geographically distributed Swift installation
• Support active/active replication
• Support geographic placement constraints
Benefits
• Reduce TCO
• Increase availability
• Enable large cloud-based implementations
IBM Research - Haifa
20 © 2012 IBM Corporation

21. Extending Swift to a Global Storage Pool enables large cloud-based
implementations
Global Distribution
• Replicate objects in different
data centers
• Simplify failure recovery/DR
• Use for availability,
recovery and performance
IBM Research - Haifa
21 © 2012 IBM Corporation

22. FI-WARE – Foundation for Future Internet
Mission: provide core platform for FI applications in multiple industries* (‘Usage Area’ projects*)
*
FI-WARE
*
*
*
*
* * *
FI-WARE Budget: €40M (FI-PPP in total: €300M over 5 years)
IBM Research - Haifa
22 © 2012 IBM Corporation

23. More OpenStack focus at IBM Research - Haifa
Object Storage
Network
Virtualization
Block Storage
Compute
HA, powerVM, VM placement
IBM Research - Haifa
23 © 2012 IBM Corporation

24. Questions…
Thank you…
IBM Research - Haifa
24 © 2012 IBM Corporation