DataCore™ Virtual SAN introduces the next evolution in Software-defined Storage (SDS) by creating high-performance and highly-available shared storage pools using the disks and flash storage in your servers. It addresses
the requirements for fast and reliable access to storage across a cluster of servers at remote sites as well as in high-performance applications.
DataCore Virtual SAN virtualizes the local storage on two or more physical x86-64 servers. It can leverage any combination of magnetic disks (SAS, SATA) and optionally flash, to provide persistent storage services as close to the application as possible without having to go out over the wire (network or fabric). Virtual disks provisioned from DataCore Virtual SAN can also be shared across the cluster to support the dynamic migration and failover of applications between hosts.
DataCore Virtual SAN addresses the challenges that exist today within many IT organisations such as single points of failure, poor application performance (particularly within virtualized environments), low storage efficiency and utilisation, and high infrastructure costs.
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Virtual SAN - A Deep Dive into Converged Storage (technical whitepaper)
1. DataCore™Virtual SAN introduces the next evolution in Software-defined
Storage (SDS) by creating high-performance and highly-available shared
storage pools using the disks and flash storage in your servers. It addresses
the requirements for fast and reliable access to storage across a cluster of
servers at remote sites as well as in high-performance applications.
DataCoreVirtual SAN virtualizes the local storage on two or more physical x86-
64 servers. It can leverage any combination of magnetic disks (SAS, SATA) and
optionally flash, to provide persistent storage services as close to the application
as possible without having to go out over the wire (network or fabric).Virtual disks
provisioned from DataCoreVirtual SAN can also be shared across the cluster to
support the dynamic migration and failover of applications between hosts.
DataCoreVirtual SAN addresses the challenges that exist today within
many IT organizations such as single points of failure, poor application
performance (particularly within virtualized environments), low storage
efficiency and utilization, and high infrastructure costs.
Let’s explore how DataCoreVirtual SAN addresses each of these challenges
with a proven, 10th generation software-defined storage solution, purpose-
built for maximizing storage resources across the enterprise.
DataCoreVirtual SAN
Technical Whitepaper
A Deep Dive into Converged Storage
Virtual SAN Benefits
Only 2 servers needed
for High-Availability
RAM for I/OAcceleration
Flash is Optional
50M IOPS, 32PB with
32 Host Cluster
Works with all major
Hypervisors
Integrate Converged,
SAN, and Cloud Storage
WHITEPAPER
2. 2
Synchronous Mirroring
The availability of business-critical data is extremely important
for your organization. DataCoreVirtual SAN provides protection
against catastrophic events; outages that fault-tolerance
measures alone cannot adequately protect against. Synchronous
mirroring provides data-level redundancy to maximize data
availability and keep applications running without interruption.
DataCoreVirtual SAN accomplishes high-availability within the
cluster by synchronously mirroring every data block between
two fully active cluster nodes, where others may require three or
more hosts to accomplish that same capability.The two nodes
maintain redundant copies of the data ensuring that even if
one cluster node becomes inoperable, the applications will not
experience any interruption, and the data is always available.
Below is a diagram that outlines the order of operations
that occurs during synchronous mirroring:
Synchronous mirroring also plays a role in providing redundant
write cache.When running in a mirrored configuration, a high-
speed write cache is enabled.Writes are only confirmed when
both nodes acknowledge they have received the data. If a partner
node becomes unavailable due to maintenance for example,
then write data is still safe and destaged to persistent storage.
Unlike other solutions, where performance degrades significantly
when a loss of redundancy occurs, DataCoreVirtual SAN read
cache and write-coalescing functions are still enabled, providing
read acceleration and optimized destaging operations.
WHITEPAPER
1. Inbound write I/O received
from Host at Site A
2. Inbound write I/O committed to
high-speed RAM cache on SSV-1
3. Inbound write I/O transferred to SSV-2
4. Inbound write I/O committed to
high-speed RAM cache on SSV-2
5. Acknowledgement of write I/O
receipt from SSV-2 to SSV-1
6. Acknowledgement of write I/O receipt
from SSV-1 to Host at Site A
7. Inbound write I/O is destaged from
cache to internal storage
Challenge #1: Elimate Single Points of Failure and Downtime
Data availability involves achieving data redundancy and maintaining accessibility even
in the event of a catastrophic failure within the storage architecture due to hardware
malfunctions, site failures, regional disasters, and user errors. DataCoreVirtual SAN provides
full data redundancy and eliminates interruptions using synchronous mirroring.
Synchronous Mirroring Order of Operations
3. 3
High-speed Caching
The first feature responsible for storage acceleration within
the cluster is high-speed cache. Many other solutions today
require the use of flash as a caching device. DataCore
Virtual SAN leverages RAM as high-speed cache, along with
the hosts’ local processors and I/O resources, to execute
sophisticated multi-threaded read and write caching
algorithms across all storage devices under its management.
High-speed caching drastically improves overall application
performance by pre-fetching and storing the most frequently
accessed data in local memory, optimizing data transfers to
storage through write coalescing (sequential ordering), and
acting as a speed-matching write buffer to alleviate inbound
channel congestion. Up to 1Terabyte (TB) of RAM can be
reserved per node forming cluster-wide mega-caches.
Using RAM as a high-speed cache offers several unique
advantages since RAM is many orders of magnitude faster
than flash technologies. RAM is mirrored for inbound write
data, is not impacted by wearing or write-amplification
effects like flash, and resides closest to the CPU to minimize
latency. It is the fastest storage component in the storage
architecture, helping to deliver up to 10X performance
boost to applications and freeing up hosts to perform
other tasks. It also extends the life of traditional magnetic
storage components by minimizing the stress experienced
from disk thrashing due to I/O intensive workloads.
While flash is inadequate to perform high-speed caching
functions, it can be used as persistent storage within a disk
pool and to accelerate frequently accessed data provided via
the auto-tiering feature. In this manner, its performance and
capacity attributes contribute to the system, as opposed to
other solutions which only use flash for a system-wide cache.
WHITEPAPER
Challenge #2: Improve Application Performance
Today’s demanding applications, especially within virtualized environments, require high
performance from storage to keep up with the rate of data acquisition and unpredictable
demands of enterprise workloads. DataCoreVirtual SAN provides substantial storage
acceleration through two primary features: high-speed caching & automated storage tiering.
4. 4
Automated StorageTiering
The second feature responsible for storage acceleration within
the cluster is automated storage tiering. Automated storage
tiering is designed to move data to higher performing storage
tiers based on access frequency.When a high-capacity disk
is combined with high-speed storage devices such as flash,
it yields markedly better application performance at scale.
Unlike other auto-tiering solutions, which tend to be manual
or at best scheduled, DataCoreVirtual SAN auto-tiering is a
fully-automated real-time mechanism. In the background, it
analyzes data block requests and chooses the best placement
for your data based on access frequency. Both read and write
access patterns can influence the auto-tiering mechanism.
Also, auto-tiering policies apply on a per- virtual disk basis
granting you granular control over optimizations.
Automated storage tiering provides seamless integration of
high-speed storage devices, such as optional flash, with different
types of magnetic high-capacity devices (such as SAS and
SATA drives). A very compelling reason to use auto-tiering is
that only a small amount of flash disk is needed to significantly
improve application performance and user experience. Adding
just a small amount of flash, typically 8-10% of the total storage
pool capacity, yields significant performance enhancements.
DataCoreVirtual SAN supports many different tiers of
storage technology including cloud storage for infrequently
accessed, long-term archival data. As more advanced
technologies become available, tiers can be modified or
added as necessary to maintain performance service levels.
CPU-Storage Proximity
An important characteristic of DataCoreVirtual SAN is
that high-speed storage services reside as close to the
CPU and applications as possible.This type of architecture
reduces the number of components and the distance
involved in accessing the data.The result is greatly
improved application responsiveness and performance.
In general, RAM is the fastest and closest component to the
CPU. It is followed by PCIe flash, storage connected via an
internal controller, and finally external storage arrays. By
minimizing the number of components that must be traversed
to access the data, context switching is minimized, and
therefore latency. DataCoreVirtual SAN uses RAM as high-
speed cache, providing the closest proximity to storage services.
The locality of persistent storage is also an important design
consideration. As shown above, latency increases the further the
data resides from the CPU. Leveraging PCIe flash with internal
disks eliminates the inherent latency attributed with traversing
the external network or fabric.This combined with auto-tiering
provides a highly-optimized, high-performance storage solution.
WHITEPAPER
5. 5
DataCoreVirtual SAN uses thin provisioning which allocates
storage only when data is written resulting in an increase
of storage utilization to as high as 95%.Thin provisioning
is accomplished by presenting a standard storage volume
which appears to have been fully provisioned and allocated,
when in fact nothing has been allocated. Only when the
application writes data are the storage blocks granularly
allocated for that particular volume. Conversely, automatic
zero-block reclamation constantly analyzes the thin
provisioned storage blocks looking for those which have
been zeroed out due to data removal by the host.
But what about scalability? Expanding storage resources
separately from compute resources is another important
advantage of DataCoreVirtual SAN over other solutions.
To scale storage, you can choose to:
• Expand capacity by adding more internal
storage to existing hosts (scale up)
• Add hosts which contribute more storage (scale out)
• Tapping into external storage on a central
SAN and/or cloud storage
• Supplement on-premise storage with cloud storage
Note: Other alternatives confine you to options A
and B.They do not permit access to capacity in your
SAN (option C) or cloud storage (option D)
To scale out processing power, you may:
• Expand compute resources by adding
more hosts without data storage
• Expand compute and storage resources
concurrently by adding hosts
• Substitute faster hosts
In all cases, you have full control over how best to scale of
and storage resources independently within the cluster.
WHITEPAPER
Challenge #3: Increase Efficiency and Scalability
DataCoreVirtual SAN RAM automatically improves storage efficiency and utilization through
thin provisioning.With storage systems that do not utilize thin provisioning, storage is
allocated on a per-volume per-application basis.This results in much of the available storage
being wasted since the unused space on each volume is inaccessible by other applications.
The industry average storage utilization without thin provisioning is around 35%.
6. 6
Ultimate Storage Convergence:
As noted earlier, the DataCoreVirtual SAN can also take
advantage of existing central SANs and cloud storage
for additional capacity without having to add extra
components to the cluster. Unify all your storage under
DataCore control and manage it all from the same console.
This gives you the best of both worlds.The fastest storage
is right next to your applications directly on the host, yet
you can access more capacity or a different class of storage
from the central SAN or cloud storage as needs arise.The
auto-tiering software can do this for you without manual
intervention. No other product offer this capability!
You may also choose to mirror certain virtual disks
between your hosts and the central SAN / cloud storage,
rather than between hosts. Or, replicate between a
DataCoreVirtual SAN at remote locations to a central
DataCore SAN at the main data center or to cloud
storage.The point is that applications clusters don’t
need to become isolated as others would have you do.
They can be fully integrated into the broader storage
infrastructure to harness all the assets available to you.
WHITEPAPER
7. 7
Deploying a Converged, Flexible Architecture
Consolidate your architecture to one-third of its traditional
size and introduce your choice of storage components
necessary to deliver the performance and capacity your
applications require. And as conditions change over time, deploy
additional storage components to meet the new resource
demands, all without interruption or risk to your data.
Achieving High Performance at Lower Cost
The use of low cost RAM as high-speed cache gives you a
significant performance-cost advantage. Additionally, over
time it also reduces the number of device failures since
the backend storage systems are not being overwhelmed
with constant high I/O activity.The use of RAM along with
less stress on storage components results in a significantly
lower total operating cost. And unlike other solutions,
the use of flash is optional with DataCoreVirtual SAN.
Integrate DataCoreVirtual SAN with
Central SAN / Cloud Storage
Whether you have an existing central SAN / Cloud
Storage architecture or you need to expand to one in
the future, DataCoreVirtual SAN seamlessly unifies
both architectures and provides a single pane of glass
management interface for the entire environment.
OnlyTwo Nodes NeededTo Start
DataCoreVirtual SAN only requires two nodes to get
started and can be incremented one node at a time. Other
solutions need three or more nodes to be fully functional.
Cross Hypervisor Support
DataCoreVirtual SAN supports all industry hypervisors
such as vSphere, Hyper-V, XenServer, and KVM. Different
hypervisors offer different advantages. It may be optimal
to deploy one type of hypervisor for critical enterprise-class
workloads such as line-of-business applications or databases
and another type for non-critical workloads such as virtual
desktops.Whether deployed at headquarters, at a branch
office, or both, DataCoreVirtual SAN will provide the same
enterprise class storage and management services.
WHITEPAPER
Challenge #4: Reduce Infrastructure Costs
DataCoreVirtual SAN significantly lowers yourTCO by:
8. 8
Applicable Use Cases
DataCoreVirtual SAN opens up many new possibilities
within the infrastructure. Below are some of the most
common use cases:
Databases
Database applications such as SQL, Oracle, and SAP run
best in a low latency high-throughput storage environment.
DataCoreVirtual SAN delivers low latency storage access by
providing storage services as close to the application as possible.
Additionally, high-speed caching functions leverage RAM, which
is the fastest storage component in the system.The result is
significantly improved database application performance.
Virtual Desktop (VDI) Deployments
Run more virtual desktops (bothVMware Horizon
and Citrix XenDesktop) on each hypervisor host
and scale them out across more servers without the
complexity or expense of an external SAN.
Compact Server Clusters at Remote
Sites and Branch Offices
Put the internal storage capacity of your application
servers to work as a shared resource while protecting
your data against outages. Industries such as retail,
hospitality, manufacturing, and healthcare benefit greatly
from converged highly-available architectures.
Summary of Benefits
DataCoreVirtual SAN provides many benefits to your
infrastructure such as:
Highest Performance
Highest storage performance through the use of
high-speed RAM as cache.
Achieve end-to-end hardware independence
Just as server virtualization decouples ‘computing’
from the server hardware, storage virtualization
decouples the ‘data’ from the storage hardware,
resulting in complete infrastructure-wide flexibility.
Low upfront capital expenditure
Add resources as performance and capacity demands increase.
Conclusion
DataCoreVirtual SAN is the next evolution in software-defined storage which provides
high-performance and highly-available shared storage pools using the disks and flash
storage in your application servers; and if necessary, integrate external storage.
DataCoreVirtual SAN will significantly improve application performance and responsiveness, eliminate
single points of failure, increase storage efficiency and utilization, and lower infrastructure costs. It
provides complete storage flexibility, leveraging any make and model of storage device, all while
providing blazing performance and the most advanced forms of data protection. And as conditions
change over time, expand and adapt to fulfill new demands without interrupting access to your data.
WHITEPAPER