The document discusses various topics related to data storage technologies and trends. It covers storage growth trends, new storage protocols like FCoE, high density storage technologies, SSD and RAM-based storage, storage virtualization, deduplication technologies, new storage vendors, differences between SAN and NAS storage, and considerations around differentiating between high-end and mid-range storage solutions. The document provides analysis and perspectives from a storage technology expert on these various topics.
STKI Summit 2014 Infra Trends - How CIO Deliver - complete infra trends
Storage Growth and Protocols
1. ;
The Gap!
Pini Cohen
EVP
pini@stki.info
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
2. Agenda
• Major Trends and Issues
• Development and SOA
• ESM BSM CMDB
• DBMS and DATA
• Platforms – Servers
• Clients
• Storage
Source: http://astonguild.org.uk/files/NEW_MENU_FRONT_RGB%5B1%5D.jpg
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
3. Technologies Maturity Model
2011 Storage
Business Value
Investment Sophisticated
snaps for creating
to make money CDP environments
Cut costs, Increase Green Dedup for
VTL Dedup Storage Primary
productivity for Backup
storage
Storage
Thin
Virtualization
Provisioning
Commodity IT NAS and FCOE
Services SAN
Flash
Drives Cloud
Email Email Storage
Investment Archiving
Archiving
for regulations as a service
Using Implementing Looking
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
4. Mini agenda- Storage
• Storage growth
• Storage protocols – FCOE
• High density storage
• RAM and SSD storage
• Storage virtualization
• VTL-Deduplication status
• New type of storage vendors
• SAN vs. NAS Highend vs. Midrange
• Staffing ratios
• Vendor ratings
Pini Cohen’s work Copyright 2011 @STKI 4
Do not remove source or attribution from any graphic or portion of graphic
5. Storage growth – you have not
seen nothing yet…
Pini Cohen’s work Copyright 2011 @STKI 5
Do not remove source or attribution from any graphic or portion of graphic
6. Ubiquitous computing
pervasive computing
Audio-Video Cluster Computer as Hub of cluster
Local Network
User Interaction
Internal Comms.
Source: http://www.elec.qmul.ac.uk/people/stefan/ubicom/slides/ubicom-ch02-slides.ppt
Processing:
edit, annotate, compose
Management:
Resources & Content
Storage
Email
Web
Ubiquitous computing: smart devices, environments and inte
Wide network Chat
VoD
Audio-Video Stream
VoIP
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
7. Video
Video is a
SCATTERED IT
TECHNOLOGY
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
8. Age of Internet Video
Pini Cohen’s work Copyright 2011 @STKI
Source: Cisco VNI Usage, 2010
Do not remove source or attribution from any graphic or portion of graphic
9. Streaming Video is Gaining Market
Share
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
10. Storage Size and Growth
in Selected Industries
Industry 2011 1Q 2010 1Q Planned
Size RAW Size RAW Growth per
year
Defense 500T-6P 500TB-4P 50%- 100%
Finance 600T-1.3P 400TB-1P 40% - 75%
Health 140T-550T 140TB-350TB 30%-50%
Manufacturing – 100T-250T 40TB-200TB 20%-50%
Retail
Telco 2P-3P 900TB-2.5 30%-50%
PETA
Governmental 100T-300T 10TB-30TB 25%-100%
Public
High Tech 150T-550T 40TB-150TB 20%-30%
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
11. Protocol Comparisons
FC over Ethernet
(no TCP/IP)
App
App Applications
Applications
SCSI
SCSI SCSI SCSI
iSCSI FC FC FC
FC FC SRP
Encapsulation
Encapsulation iFCP FCIP
TCP
Layer
Layer TCP TCP
IP IP IP FCoE
FCoE
Base
Base FC Infiniband
Ethernet
Ethernet
Transport
Transport
Block storage FC replication
New transport and
with TCP/IP over IP FC management drivers
Low latency, high
Source: EMC http://www.google.co.il/url?sa=t&source=web&cd=1&ved=0CBkQFjAA&url=http%3A%2F%2Fwww.scalar.ca%2Fnewsevents%2Fdownload%2FFCOE-
Aug7.ppt&rct=j&q=%22Data%20center%20Ethernet%22%20%20%22Converged%20Enhanced%20Ethernet%22%20%20FCOE%20filetype%3Appt&ei=cJt4TafEF8Kg8QOx883ABA&usg=AFQjCNEjDc
bandwidth
AKpuQjNVJ1-HfyhzhCGqmMcw&cad=rja
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
12. Lossless Ethernet
AKpuQjNVJ1-HfyhzhCGqmMcw&cad=rja
Aug7.ppt&rct=j&q=%22Data%20center%20Ethernet%22%20%20%22Converged%20Enhanced%20Ethernet%22%20%20FCOE%20filetype%3Appt&ei=cJt4TafEF8Kg8QOx883ABA&usg=AFQjCNEjDc
Source: EMC http://www.google.co.il/url?sa=t&source=web&cd=1&ved=0CBkQFjAA&url=http%3A%2F%2Fwww.scalar.ca%2Fnewsevents%2Fdownload%2FFCOE-
• Limit the environment only to the Data Center
– FCoE is Layer 2 only
• IEEE 802.1 Data Center Bridging (DCB) is the standards task
group
• Converged Enhanced Ethernet (CEE) is an industry consensus
term which covers three link level features
– Priority Flow Control (PFC, IEEE 802.1Qbb)
– Enhanced Transmission Selection (ETS, IEEE 802.1Qaz)
– Data Center Bridging Exchange Notification (DCBX, currently part of IEEE 802.1Qaz, leverages 802.1AB
(LLDP))
• Data Center Ethernet is a Cisco term for CEE plus additional
functionality including Congestion Notification (IEEE 802.1Qau)
Enhanced Ethernet provides the Lossless Infrastructure
which will enable FCoE and augment iSCSI storage traffic .
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
13. RT storage
• Users are certain that FCOE will change dramatically
the storage landscape and the relationship between
Storage and Networking
• However users believe that FCOE will mature only in
about 2-3 years or more
Pini Cohen’s work Copyright 2011 @STKI 13
Do not remove source or attribution from any graphic or portion of graphic
14. High density- low electricity
• MAID - Many Array of Idle Disks (SGI)
Pini Cohen’s work Copyright 2011 @STKI 14
Do not remove source or attribution from any graphic or portion of graphic
15. FCoE Extends FC on a Single
Network
Server sees storage traffic as FC
Network FC Ethernet
Driver Driver Network
FCoE SW Stack
Standard Converged
10G NIC Network Adapter FC storage
2 Lossless Ethernet
options Links FC network
Ethernet Converged
FC
Network Switch
Source: EMC
http://www.google.co.il/url?sa=t&source=web&cd=1&ved=0CBkQFjAA&url=http%3A%2F%2Fwww.scalar.ca%2Fnewsevents%2Fdownload%2FF
COE-
Aug7.ppt&rct=j&q=%22Data%20center%20Ethernet%22%20%20%22Converged%20Enhanced%20Ethernet%22%20%20FCOE%20filetype%3A
ppt&ei=cJt4TafEF8Kg8QOx883ABA&usg=AFQjCNEjDcAKpuQjNVJ1-HfyhzhCGqmMcw&cad=rja
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
SAN sees host as FC
16. GS: Price gap is closing. We expect 80GB-100GB SSDs to be
within $100 of a HDD in the next 24 months.
Source: GS http://bg.panlv.net/file/2010/01/10/087c22dd5c96e25f.pdf
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
17. Kaminario K2- DRAM Storage Appliance
Most demanding
business application
Up to 25x
performance
Less than 1 day
deployment
Kaminario K2
• Performance – eliminate the I/O bottleneck!
Enables millions of IOPS, 10s GB/s, low latency
• High Availability – data protection, no SPoF
• Scalability – start small and grow
• Easy – any application, no tuning
18. Kaminario K2 Capabilities - Performance
• Eliminates I/O bottlenecks
• Millions of IOPs
– 300K to 1.5M IOPs
• high throughput
– 3.2GB/S to 16GB/S
• Application performance
– 2x - 25x improvement
• Consistent performance for any workload, and
with no degradation over time
Page 18
19. Storage virtualization- the next big
thing?!
• We have expected many years? Will it ever
come?!
• Looks like it is progressing (abroad)
Pini Cohen’s work Copyright 2011 @STKI 19
Do not remove source or attribution from any graphic or portion of graphic
20. RT storage – VTLDedup
• VTL Dedup has matured a lot this year
• There are occasionally Dedup product and Backup software
compatibility issues. In many cases license update for the backup SW is
needed.
• Also the integrator should have good knowledge both in the Backup SW
and the DedupVTL solution
• Accessing the VTL directly (without the backup SW) might be very
useful in call center environments for recording hearing
• Currently not all source type are appropriate for Dedup. Example of
problematic situations – DBMS log files, Exchange EDB, etc.
• Users believe that one algorithmmethod should be used for all Dedup
purposes – both source and target Dedup. Symantec is leading the pack
in this area.
Pini Cohen’s work Copyright 2011 @STKI 20
Do not remove source or attribution from any graphic or portion of graphic
21. RT VTL DEDUP
• Actual Dedup ratio in Backup environment is
about 1 to 10. Vendors claim much more…
• The solution throughput and scalability is also
an issue
• Some clients has discontinued physical tapes
completely! But most clients to backup to
physical tapes once in a while.
Pini Cohen’s work Copyright 2011 @STKI 21
Do not remove source or attribution from any graphic or portion of graphic
22. RT VTL cont.
• Backup of virtual server environment is still an issue.
Backing up with traditional agents on each guest
might cause issues (CPU, network overloaded)
• Users are implanting VMware Data Recovery (VDR)
orwith Dedup products (Avanar , Puredisk). Backup
with storage snaps is also an option used
• Who is responsible to the virtualization storage
needs? The System or Storage groups ?
Pini Cohen’s work Copyright 2011 @STKI 22
Do not remove source or attribution from any graphic or portion of graphic
23. Production Dedup and
Compression
• Backup of Prod data is promising technology.
Currently users are using mainly Netapp and to
some extend EMC’s technologies
• Users have witnesses even 35% reduction of
space in NAS environments
• The combination of production Dedup and
Snaps is not always appropriate. Dedup and
compression might work better.
• IBM purchase of Storwize is also a sign for the
importance of these technologies
Pini Cohen’s work Copyright 2011 @STKI 23
Do not remove source or attribution from any graphic or portion of graphic
24. New type of vendors
• Local, Agile, Knowledgeable, Hands on
and Brave!
• Example: OpenStorage representing
many small vendorssolutions such as
Nexenta, Arkeia, Gluster, SANS DIGITAL
and also established solutions such as HP
and EMC
• Assembling locally storage solution
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
25. Users perspective: Should we
differentiate SAN and NAS?
Source: STKI
Pini Cohen’s work Copyright 2011 @STKI 25
Do not remove source or attribution from any graphic or portion of graphic
27. Users perspective: Should we differentiate
Highend and Midrange Storage?
Pini Cohen’s work Copyright 2011 @STKI
Source: STKI
Do not remove source or attribution from any graphic or portion of graphic
28. Which vendors are considered as
Highend storage vendors?
Source: STKI
Pini Cohen’s work Copyright 2011 @STKI 28
Do not remove source or attribution from any graphic or portion of graphic
29. High End Storage vs. Midrange Storage
• The majority of IT organizations differentiate High End and Midrange
Storage solutions. What differentiate according to users:
– Performance (example: “500 to 900 MB per second is HIGHEND”)
– Availability – Reliability
– Higher risk
– Upgrade firmware without downtime
– Functionality (number of copies, failback, etc.)
– Price
– More difficult to manageoperate Source:
http://blog.oregonlive.com/clackamascounty_impact/2009/07/large
_Worthington-Testifies.JPG
• Big storage shops tend to differentiate
• Higher level managers tend not to differentiate
• New argument this year “Do not need midrange storage – we can use
SATA in highend”!
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
30. STKI’s take
• VTLDedup is maturing fast and can bring real benefit now.
• Soon we will have “general storage” category. Specialized “HighEnd” or
NAS will be a niche.
• Managing the storage will be key priority
• Automation Automation Automation (which also means standardization)
• Users should have established storage metrics such as:
– Storage managed per administrator
– Storage in inventory - unused storage divided by total storage
– Data multiplier (number of copies made of primary data )
– Data availability (number of hours storage is
– Mean time to recovery (MTTR)
• IAAS (Storage services) will become mainstream for long term data
retention needs in about 5 years
Pini Cohen’s work Copyright 2011 @STKI 30
Do not remove source or attribution from any graphic or portion of graphic
31. Competition in Storage is the
toughest!
• User to STKI “I am in shock. From 60% price
reduction in two months!”
Source: http://t2.gstatic.com/images?q=tbn:ANd9GcRry6HPDj21W6jLMe6hIeJFDJtuA0JLGJFIah_GGbelxmC5DNCL
Pini Cohen’s work Copyright 2011 @STKI 31
Do not remove source or attribution from any graphic or portion of graphic
32. Historical Cost of Memory and
Storage
http://organdi.net
Pini Cohen’s work Copyright 2011 @STKI 32
Do not remove source or attribution from any graphic or portion of graphic
33. Storage Ratios
• Number of Raw TB and Usable TB per Storage Staff Member FTE
(including backup and DRP of storage):
Per FTE RAW Storage Usable Storage
25 percentile 97TB 42TB
Median 225TB 150TB
75 percentile 350TB 238TB
• Moderate 25% increase from last years data
Source: STKI
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
34. Usable/Raw storage ratio
• Net Storage in this research
– usable for applications:
– After Raids
– After replication to DRP
– Without VTL’s NETRAW Ratio
– The term “Usable storage” is 25 percentile 50%
tricky since with snapshots Median 60%
application can see more 75 percentile 71%
storage then “Raw storage”
Source: STKI
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
35. Market Status and Trends
• Survey has shown that these integrators are used:
(providers – from service point of view) in Storage
area:
• EMC
• Netapp
• HP IBM Hilan (WE+Ankor)
• Malam-Team Emet
• HDS Matrix Bynet
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
36. Market Status and Trends
• Survey has shown that these integrators are used:
(providers – from service point of view) in backup
area:
• GlassHouse
• IBM
• HP
• Emet
• Team RealNetworks
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
37. Israel Market Positioning High-end Storage (total
including MF)
EMC
Local Support
IBM
HDS
HP
This analysis should be
used with its supporting
documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
38. Israel Market Positioning Midrange Storage SCSI
EMC
Vendors to Watch:
NETAPP
IBM XIV N DS
DataDirect
Local Support
Networks
HP
Midrange storage share is
rising on the expenses of
highend storage
HDS
This analysis should be
used with its supporting
documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
39. Israel Market Positioning General Storage
Solution – starting point for general use
Leaders
EMC
NETAPP
HP
Local Support
IBM XIV N DS
HDS
This analysis should be
used with its supporting
documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
40. Israel Market Positioning Storage NAS
NETAPP
Local Support
EMC
This analysis should be
used with its supporting
documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
41. Israel Market Positioning
(Enterprise Backup SW)
Backup to
Symantec
Cloud is an
option:
example
Asigra
Local Support
EMC
IBM
HP
This analysis should be
used with its supporting
documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
42. Selected Installations 2010-1Q11 Partial List of
selected wins –Storage
EMC- IDF, Hot, Teva, Partner, Cellcom, Bezeq, Pelephone, 012,
Yes, ITEAM, VISA CAL, Discount, Clal, Comverse, Amdocs, IEC,
Zim, Motorolla
NETAPP (some via IBM) – Clalit, IAI, Elbit, Postil, Rafael,
Pelephone (upgrade), Cellcom (upgrade) , Clal, Ministry of
Transportation, Netafim
HDS:HDS: Walla, Kul Al Arab, H&M, Aeroscout, Office Core,
DIDWW, Shararey Zedek hospital, Mataf(upgrade),
Poalim(upgrade), Igud (upgrade), Bezeqint (Upgrade), Bituach
Leumi (updrade), Shaam(upgrade), Ministry of Social Affairs
(Revaha) upgrade, Migdal (upgrade), Teva (upgrade), TASE
(upgrade), Clalit (upgrade)
Pini Cohen’s work Copyright 2011 @STKI 42
Do not remove source or attribution from any graphic or portion of graphic
43. Selected Installations 2010-1Q11 Partial List of
selected wins –Storage
IBM – IAA (DS), Ayalon (DS), Menora (DS). Isracard (DS)
Poslim (DS), IEC (XIV) , Mn. Of Education (XIC), Mn. Of
Transportation (XIV), Hotzaa Lapoal (XIV), Central
Statiscial Bereau (XIV), LivePerson (XIV) Pointis (XIV),
Hachsharat Hayeshuv (XIV), 012 (XIV) Bank Leumi (XIV
upgrade) Leumit (XIV upgrade), Meucheded
Kaminario – Digitaltrowel, other installations
Pini Cohen’s work Copyright 2011 @STKI 43
Do not remove source or attribution from any graphic or portion of graphic
44. Selected Installations 2010-1Q11
Partial List of selected wins –Storage
• HP (wins and upgrades and customers) Kenshoo ; ECI ; Amdocs; Assaf Harofeh
; Bank- Hapoalim ; Bank- Yahav ; Bank of Palestine ; Bezeq International ;
Bezeq, Brom ; Dead Sea Works ;Egged;Elisra ; Flextronics ; Hadassah Hospital ;
Haaretz ; Hillel Yaffe ; Intel Israel Police ; JaJah ; Keter-Plastic ; Logic ; Microsoft
; Maccan Ericson ; Mifal HaPais ; Ministry of Justice ; Ministry of Finance ; Nilit ;
Nice ; Osem ; Strauss-Elite ; Peker Plada ; Zoglovek ; TOTO ; JCE; Big ; Fox ;
Rami-Levi ; Tuttnauer ; Ramat- Gan Municipality ; Ministry of Interior ;
Altshuler Shaham;Migdal ; Shirionit- Hosem ; Haifa Municipality ; Ruppin
Collage ; Ministry of Health ; Flying Cargo ; Tamnun ; Bakara- Geva ; Shagrir ;
Tel Aviv Municipality ; Jerusalem Municipality Comverse ; Star-Home ; Sapir
Collage ; Amiad; RH Electronics ; Orbotech ; Random Logic ;Numonix ;
Sintecmedia ; Carmel Olefins ; Maman Cargo ; Hillel Yafe Hospital ; Ziv Hospital
; Hadasa
Pini Cohen’s work Copyright 2011 @STKI 44
Do not remove source or attribution from any graphic or portion of graphic
45. Selected Installations 2010-1Q11 Partial List of
selected wins –VTLsource-target DEDUPBackup
EMC – Datadomain – Orbotech, Pelephone, Discount,
Harel, Asuta, Clal, Migda, Yad Vashem, ITEAM, MOD
EMC – Avamar –Visa Cal, Clal, Bezeq, Intel, NDS, Coca-
Cola, Co-Op, Comverse
IBM – Protectier - IEC, Mataf (updrage), Amdocs
EMC – Mirs, Partner (major upgrade)
Symantec (NB) – Nice, Isracard, Hani (Israel ports),
Israel Foreign Affairs (Puredisk), 888, Liveperson,
Diplomat, Exlibris , Bezeq ing.
Pini Cohen’s work Copyright 2011 @STKI 45
Do not remove source or attribution from any graphic or portion of graphic
46. Storage Selected Vendors
TEAM /
M alam/
Bezeq omnitech/ Glass
Int. netcom WE House CCC M atrix Bynet EM ET ONE1 Taldor SGI Kavey Zahav
EM C x x x x x
IBM x x x x x x
HP x x x
NETAPP x x x x x x x
HDS x x x
DDN x
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
47. Selected Vendors and SI’s in Israel – Enterprise B/R
Glass T eam
Vendor EMET House Malam CCC Matrix Yael ANKOR One1
(we) T errasky Simplimentia
HP DP x x
EMC Legato x X x x
Symantec (NB) x x x x x x
BM TSM X x
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic
48. Thank you
Pini Cohen
Pini Cohen’s work Copyright 2011 @STKI 48
Do not remove source or attribution from any graphic or portion of graphic