Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Red Hat Storage Day Atlanta - Why Software Defined Storage Matters
1. RED HAT STORAGE DAY
ATLANTA
Irshad Raihan
Red Hat Storage Marketing
August 2016
2. RED HAT STORAGE DAY - ATLANTA
AGENDA
2:30 - 2:35: Welcome
2:35 - 2:50: Why Software Defined Storage Matters
2:50 - 3:30: Red Hat Ceph Storage on Intel
3:30 - 4:00: Partner Spotlight: Supermicro
4:00 - 4:30: Storage for Containerized Applications
4:30 - 4:50: Red Hat Gluster Storage vs. Traditional Storage Appliances
4:50 - 5:00: Q&A - Wrap Up
5:00 - 6:00: Cocktail Reception
3. 3
HOW DID WE GET HERE?
DEVELOPMENT
MODEL
APPLICATION
ARCHITECTURE
DEPLOYMENT
AND PACKAGING
APPLICATION
INFRASTRUCTURE
STORAGE
Waterfall
Agile
DEVOPS
Monolithic
N-Tier
MICROSERVICES
Virtual Services
CONTAINERS
Bare Metal
Hosted
HYBRID CLOUD
Data Center
Scale Out
SOFTWARE-DEFINED
STORAGE
Scale Up
4. ORGANIZATIONS ARE RETHINKING STORAGE
38% of IT decision
makers report
inadequate storage
capabilities as one
of their top three
weekly pain points
70% of IT decision
makers admit that
their organization’s
current storage can’t
cope with emerging
workloads
98% of IT decision
makers believe a
more agile storage
solution could
benefit their
organization
6. 6
THE EVOLUTION OF STORAGE
OPEN, SOFTWARE-DEFINED STORAGE
Standardized, unified, open platforms
Standard Computers
and Disks
Ceph Gluster +++
Control Plane (API, GUI)
Custom GUI
Proprietary Hardware
Proprietary
Software
Custom GUI
Proprietary Hardware
Proprietary
Software
USER USER USER
ADMIN ADMIN ADMIN
Custom GUI
Proprietary Hardware
Proprietary
Software
USER ADMIN
Standard
Hardware
OpenSource
Software
TRADITIONAL STORAGE
Complex proprietary silos
7. A RISING TIDE
“By 2020, between 70%-80% of unstructured data
will be held on lower-cost storage managed by
SDS.”
Innovation Insight: Separating Hype From Hope for Software-Defined Storage
“By 2019, 70% of existing storage array products
will also be available as software only versions.”
Innovation Insight: Separating Hype From Hope for Software-Defined Storage
Software-Defined Storage is leading a
shift in the global storage industry, with
far-reaching effects.
SDS-P MARKET SIZE BY SEGMENT
Block Storage
File Storage
Object Storage
Hyperconverged
2013 2014 2015 2016 2017 2018 2019
Source: IDC
$475M
$1,395M
$1,195M
$1,029M
$859M
$705M
$592M
8. 8
WHAT IS OPEN SOFTWARE-DEFINED
STORAGE?
HARDWARE
INDEPENDENCE
SOFTWARE-BASED
ORCHESTRATION
OPEN
INNOVATION
9. 9
SOFTWARE-BASED ORCHESTRATION
ON DEMAND AND PROGRAMMATICALLY
Web Console API Command Line
A browser interface designed for
managing distributed storage
A full API for automation and
integration with outside
systems
A robust, scriptable command-line
interface for expert operators
PROVISION INSTALL CONFIGURE TUNE MONITOR
11. 11
INDUSTRY STANDARD HARDWARE
Standardization makes storage more convenient
Customers can build clusters using standard hardware
from existing vendors that’s perfect for their workload.
• Clusters can be performance-optimized, capacity-
optimized, or throughput-optimized
• Need capacity? Add more disks. Too slow? Add more
servers.
• Clusters can become larger or smaller with no downtime
12. 12
OPEN INNOVATION
• Contributions from Intel, SanDisk, CERN, and Yahoo
• Presenting Ceph Days in cities around the world and
quarterly virtual Ceph Developer Summit events
• Over 11M downloads in the last 12 months
• Increased development velocity, authorship, and
discussion has resulted in rapid feature expansion
97 AUTHORS/MO
2,453 COMMITS/MO
260 POSTERS/MO
33 AUTHORS/MO
97 COMMITS/MO
138 POSTERS/MO
13. 13
THE RED HAT STORAGE PORTFOLIO
Gluster
management
Ceph data service Gluster data service
Ceph management
OPENSOURCE
SOFTWARE
STANDARD
HARDWARE
Share-nothing, scale-out architecture provides
durability and adapts to changing demands
Self-managing and self-healing features
reduce operational overhead
Standards-based interfaces and full APIs
ease integration with applications and
systems
Supported by the experts at Red Hat
14. 14
PETABYTE SCALABILITY
Open, software-
defined storage
is the best way
to manage data
cost-effectively
at web scale
Storage clusters of inexpensive,
standard x86 hardware
• Adds more disks to increase
capacity, more servers for
performance
• Intelligently uses various kinds of
media (SSD, HDD, Flash)
Distributed, parallelized data access
• Provides performance that scales
as capacity does, to petabytes
and beyond
Policy-based data distribution
• Respects defined failure domain
• Reacts when hardware changes
Hash-based data distribution
• Simplifies architecture
• Ensures linear performance
scaling
15. 15
UNPARALLELED AGILITY
Built with
open source and
standard
hardware, Red
Hat Storage
solutions provide
unparalleled
flexibility
Deployments come in all shapes
and sizes
• Runs on hardware optimized for
capacity, performance, or IOPS (or a
blend)
• Deploys on-premise, in the cloud,
or spanning the two
Broad, commercial ecosystems
prevent lock-in
• Supports a wide variety of systems
from most major providers
Compatible and extensible
• Supports clients written for S3/Swift
• Provides powerful native object API
Multiprotocol support
• Supports NFS, CIFS, object,
native
• Offers multi-protocol data access
21. WHY RED HAT CEPH STORAGE FOR
OBJECTS?
21
Proven at Web-Scale for Object Storage
Flexible Storage for Your Applications
Data Protection, Reliability & Availability
Enterprises Demand
Open, Community Based Software Defined
Storage for Object Storage
Cost Effective Object Storage to Help You
Maximize Your Storage Budge
22. IDC ON RED HAT CEPH STORAGE
Highlights
“The recent release of Red Hat Ceph Storage
2.0 brings many improvements that make
Ceph truly enterprise worthy.”
“With the release of Red Hat Ceph Storage 2.0,
Red Hat is squarely targeting enterprise use
cases that showcase the versatility of Ceph
as an object store in the enterprise.”
Research sponsored by Red Hat - Available publicly on redhat.com/storage
Networked storage services are generally classified in the way they are consumed and interfaced with on the client side:
File:
The most traditional service type, as the name implies offers to multiple clients the ability to access a single shared folder.
Unrelated data blocks managed by a native file system that places data on disk in a hierarchical manner
File system protocols to access data are NFS and SMB/CIFS.
Rich media, backups, archives
Block:
Block storage provides network access to the equivalent of raw block devices. Data organized as an array of unrelated blocks and the host file system places data on disk
Block devices are usually exported over Fibre Channel, iSCSI or FCoE (Fibre Channel over Ethernet)
A client machine connects to a specific volume on the storage service and formats it as if it were a local block device.
Object:
Object storage is a relatively new storage type, designed for unstructured data such as images, videos, documents, logs, backups, application binaries and VM images.
The typical quality of this class of data is “write-once-read-many.” Hundreds of millions or even billions of objects vs. millions of files
Immutable data vs. amendable data
No hierarchy
API access to data vs. file or block storage. REST APIs – Swift, S3
Metadata driven, policy-based
Valet parking analogy - you hand over your ticket to valet, have no idea where your car is or how many times its been moved but you get it back every time just with the number on your ticket, after a sumptuous meal
Global object storage clusters
Support for authentication using Active Directory, LDAP & OpenStack Keystone v3
Greater compatibility with the Amazon S3 and OpenStack Swift object storage APIs
AWS v4 signatures, object versioning, bulk deletes