SlideShare une entreprise Scribd logo
1  sur  14
Télécharger pour lire hors ligne
© RED HAT, INC.1
Red Hat Gluster Storage
Historical Tick Data Repository
Narendra N. Narang
Sr. Cloud Storage Solutions Architect
narnar@redhat.com
October 2016
PROPRIETARY
HARDWARE
SCALE-UP
ARCHITECTURE
HARDWARE-BASED
INTELLIGENCE
CLOSED DEVELOPMENT
PROCESS
Common,
off-the-shelf hardware
Scale-out
architecture
Software-based
intelligence
Open development
process
Lower cost, standardized supply chain
Increased operational flexibility
More programmability, agility,
and control
More flexible, well-integrated
technology
FOUR IMPORTANT DIFFERENCES
3
RED HAT GLUSTER STORAGE
• Software-defined, distributed, scale-out and resilient file store
• Cost-efficient and performant at scale
• Easy to consume, deploy, scale and manage in public,
private, and hybrid cloud architectures
• Offers mature NFS, SMB and HDFS interfaces for enterprise
applications
Nimble file storage for petabyte-scale workloads
Analytics
• Machine analytics with
Splunk
• Big data analytics
Enterprise File Sharing
• Media streaming
• Active Archives
Enterprise Virtualization
Rich Media and Archival
TARGETUSECASES
GLUSTER ARCHITECTURE
CLOUD
VOLUME
MANAGER
CLOUD
VOLUME
MANAGER
ADMINISTRATOR
USER
RED HAT
GLUSTER STORAGE CLI
NFS • CIFS • FUSE • OPENSTACK
SWIFT
RED HAT GLUSTER STORAGE
POOL
SERVER
DISKS
SERVER
DISKS
File system mount point
Tick Data
What is a Tick?
A “tick” is the minimum upward or downward movement (any change) in the price of a security
as measured over a period of time.
An "uptick" refers to a trade where the current transaction occurred at a price higher than the
Previous transaction and a "downtick" refers to a transaction that has occurred at a lower price
than the previous transaction. Consequently, a “zerotick” refers to a trade where the current
transaction occurred at a price higher than the previous transaction.
What is Tick Data?
Tick data is time series data containing price, volume and many other dimensions (bid/ask prices,
bid/ask sizes, quote time, trade time, exchange information) for each point of granularity.
Tick Data and Storage
The higher the resolution of tick data collected, the larger will be the dataset size and hence,
the amount of storage capacity required.
Customer Challenge
Standardize on an economical, scale-out, distributed, shared filestore as an alternative to their existing expensive
Isilon storage system for Historical Tick Data.
Financial compliance and regulations require banks, brokerages, and trading institutions to store each
transaction tick. The historical tick data is also used to create richer and more accurate risk models, to
perform backtesting and to run trading simulations for quantitative analytics.
A scalable, cost-efficient solution is imperative since storage capacity is a direct function of the number of data
dimensions and the resolution of tick data collected.
High-level Tick Data Workflow
Data Feed 1 Data Feed 2 Data Feed 3
Market Data Servers
(Aggregation of Feed Handlers)
Data Feed N
KDB+
In-memory
TickDB
(Real-time)
Tick
LogFile
Historical Tick Database
EndofDay(EOD)
Intraday
EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file
for that day. This file is typically written as a large sequential stream of blocks.
News
Social
Media
Use Case: Historical Tick Database on
Red Hat Gluster Storage
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
GLUSTERFSRHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
Data Feed 1 Data Feed 2 Data Feed 3
Market Data Servers
(Aggregation of Feed Handlers)
Data Feed N
In-memory
TickDB
(Real-time)
Tick
LogFile
EndofDay(EOD)
Intraday
EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file
for that day. This file is typically written as a large sequential stream of blocks.
News
Social
Media
Scale Out Architecture
Mathematical, Algorithmic
Low frequency Calculations:
Risk Modeling, P&L Reporting,
Mark-to-market, Stress Tests,
MC Simulations
Historical Tick Database on
Red Hat Gluster Storage –
Disaster Recovery Implementation
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
GLUSTERFSRHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
Data Feed 1 Data Feed 2 Data Feed 3
Market Data Servers
(Aggregation of Feed Handlers)
Data Feed N
In-memory
TickDB
(Real-time)
Tick
LogFile
EOD
Intraday
EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file
for that day. This file is typically written as a large sequential stream of blocks.
News
Social
Media
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
GLUSTERFSRHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
SITE A SITE B
Async
Geo-rep
Hadoop Analytics on an
Historical Tick Database
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
GLUSTERFSRHGS
NODE
RHGS
NODE
RHGS
NODE
RHGS
NODE
Data Feed 1 Data Feed 2 Data Feed 3
Market Data Servers
(Aggregation of Feed Handlers)
Data Feed N
In-memory
TickDB
(Real-time)
Tick
LogFile
Historical Tick Database
EndofDay(EOD)
Intraday
News
Social
Media
Hadoop YARN
Batch
(MR)
Script
(Pig)
SQL
(Hive)
Online
(HBase)
Real-time
(Storm)
In-memory
(Spark)
* Risk Analysis
* Trading Risk Analysis
* Ticker Plant Data
* Archival/retention
© RED HAT, INC.10
Analytics on Red Hat Gluster Storage
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Spark
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Red Hat Gluster Storage Cloud
Kubernetes Orchestration for Docker Containers
Message Bus for Microservices
Figure1. Shared Storage
Spark
Spark
Spark
Spark
Spark
Spark
Spark
Spark
Spark
Spark
Spark
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Spark
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
Linux Kernel
RHEL Atomic
Container Engine
Physical, Virtual, Cloud
Linux Kernel
RHEL Atomic
Container Engine
SSD, HDD 15K, HDD, 7.2K (Self-healing,Tiering, Replicas, Geo-replication, Erasure Coding)
Kubernetes Orchestration for Docker Containers
Message Bus for Microservices
Figure2. Containerized Storage
Spark
Spark
Storage
Spark
Spark
Spark
Spark
Spark
Storage
Storage
Storage
11
Comparing Throughput
and Costs at Scale
STORAGE PERFORMANCE
SCALABILITY
STORAGE COSTS
SCALABILITY
Number of Storage Nodes Number of Storage Nodes
TotalStorageCosts($)
Reads/WritesThroughput(mBps)
Software Defined
Scale out Storage
Traditional
Enterprise NAS
Storage
Traditional
Enterprise NAS
Storage
Software Defined
Scale out Storage
12
3 Year TCO
(incl. support)
TCO COMPARISON
Throughput Configuration
• HDD-only media
• Higher CPU-to-media ratio
than archive configurations
• 2x replication with RHGS
• 8:3 Erasure Coding with
EMC Isilon
X-210 12LFF 12LFF
(small)
X-410 36LFF 36LFF
(medium)
For 1PB Usable Capacity Throughput-optimized Solutions
Pricing Sources: EMC Isilon: Gartner Competitive Profiles, as of 2/16/16) & Supermicro: Thinkmate, as of 1/13/16)
13
HEAD TO HEAD FEATURE COMPARISON
STANDARD
VS. ADD ON
FEATURES
TOTAL COST
OF OWNERSHIP
HARDWARE
CONFIGURATION
CHOICES
FAULT
TOLERANCE
COMPLEX
PRICING
14
RED HAT GLUSTER STORAGE
 Data Services
 Erasure Coding
 Tiering
 Bit-Rot Detection
 Snapshots
 Quotas
 Geo Replication
 Protocols
 Active/Active NFSv4
 SMB3 (protocol negotiation, in-flight
encryption, server-side copy)
 Management
 Device Management
 Dashboard
 Monitoring
 Command and Control
 Security & Data Integrity
 SSL based in-flight encryption
 At-rest encryption using dm-crypt
 SELinux Enforcing Mode
 Self Healing
Half the price for comparable features & greater flexibility

Contenu connexe

Tendances

Red Hat Storage Day Boston - Supermicro Super Storage
Red Hat Storage Day Boston - Supermicro Super StorageRed Hat Storage Day Boston - Supermicro Super Storage
Red Hat Storage Day Boston - Supermicro Super StorageRed_Hat_Storage
 
Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Acc...
Implementation of Dense Storage Utilizing  HDDs with SSDs and PCIe Flash  Acc...Implementation of Dense Storage Utilizing  HDDs with SSDs and PCIe Flash  Acc...
Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Acc...Red_Hat_Storage
 
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application Red Hat Storage Day Dallas - Gluster Storage in Containerized Application
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application Red_Hat_Storage
 
Red Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red Hat Storage Day Seattle: Why Software-Defined Storage MattersRed Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red Hat Storage Day Seattle: Why Software-Defined Storage MattersRed_Hat_Storage
 
Red Hat Storage Day Dallas - Defiance of the Appliance
Red Hat Storage Day Dallas - Defiance of the Appliance Red Hat Storage Day Dallas - Defiance of the Appliance
Red Hat Storage Day Dallas - Defiance of the Appliance Red_Hat_Storage
 
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed_Hat_Storage
 
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...Red_Hat_Storage
 
Seagate Implementation of Dense Storage Utilizing HDDs and SSDs
Seagate Implementation of Dense Storage Utilizing HDDs and SSDsSeagate Implementation of Dense Storage Utilizing HDDs and SSDs
Seagate Implementation of Dense Storage Utilizing HDDs and SSDsRed_Hat_Storage
 
Why Software-Defined Storage Matters
Why Software-Defined Storage MattersWhy Software-Defined Storage Matters
Why Software-Defined Storage MattersColleen Corrice
 
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...Red_Hat_Storage
 
Red Hat Storage: Emerging Use Cases
Red Hat Storage: Emerging Use CasesRed Hat Storage: Emerging Use Cases
Red Hat Storage: Emerging Use CasesRed_Hat_Storage
 
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...Red_Hat_Storage
 
Red Hat Storage Day New York - Persistent Storage for Containers
Red Hat Storage Day New York - Persistent Storage for ContainersRed Hat Storage Day New York - Persistent Storage for Containers
Red Hat Storage Day New York - Persistent Storage for ContainersRed_Hat_Storage
 
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...Red_Hat_Storage
 
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...Red_Hat_Storage
 
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...Red_Hat_Storage
 
Why Software-Defined Storage Matters
Why Software-Defined Storage MattersWhy Software-Defined Storage Matters
Why Software-Defined Storage MattersRed_Hat_Storage
 
Red Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed_Hat_Storage
 
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...Red_Hat_Storage
 

Tendances (20)

Red Hat Storage Day Boston - Supermicro Super Storage
Red Hat Storage Day Boston - Supermicro Super StorageRed Hat Storage Day Boston - Supermicro Super Storage
Red Hat Storage Day Boston - Supermicro Super Storage
 
Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Acc...
Implementation of Dense Storage Utilizing  HDDs with SSDs and PCIe Flash  Acc...Implementation of Dense Storage Utilizing  HDDs with SSDs and PCIe Flash  Acc...
Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Acc...
 
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application Red Hat Storage Day Dallas - Gluster Storage in Containerized Application
Red Hat Storage Day Dallas - Gluster Storage in Containerized Application
 
Red Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red Hat Storage Day Seattle: Why Software-Defined Storage MattersRed Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red Hat Storage Day Seattle: Why Software-Defined Storage Matters
 
Red Hat Storage Day Dallas - Defiance of the Appliance
Red Hat Storage Day Dallas - Defiance of the Appliance Red Hat Storage Day Dallas - Defiance of the Appliance
Red Hat Storage Day Dallas - Defiance of the Appliance
 
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
 
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...
Red Hat Storage Day New York - QCT: Avoid the mess, deploy with a validated s...
 
Seagate Implementation of Dense Storage Utilizing HDDs and SSDs
Seagate Implementation of Dense Storage Utilizing HDDs and SSDsSeagate Implementation of Dense Storage Utilizing HDDs and SSDs
Seagate Implementation of Dense Storage Utilizing HDDs and SSDs
 
Why Software-Defined Storage Matters
Why Software-Defined Storage MattersWhy Software-Defined Storage Matters
Why Software-Defined Storage Matters
 
Containerized Storage
Containerized StorageContainerized Storage
Containerized Storage
 
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
 
Red Hat Storage: Emerging Use Cases
Red Hat Storage: Emerging Use CasesRed Hat Storage: Emerging Use Cases
Red Hat Storage: Emerging Use Cases
 
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...
Red Hat Storage Day LA - Why Software-Defined Storage Matters and Web-Scale O...
 
Red Hat Storage Day New York - Persistent Storage for Containers
Red Hat Storage Day New York - Persistent Storage for ContainersRed Hat Storage Day New York - Persistent Storage for Containers
Red Hat Storage Day New York - Persistent Storage for Containers
 
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...
Red Hat Storage Day Seattle: Supermicro Solutions for Red Hat Ceph and Red Ha...
 
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...
Red Hat Storage Day Seattle: Stabilizing Petabyte Ceph Cluster in OpenStack C...
 
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...
Red Hat Storage Day Seattle: Stretching A Gluster Cluster for Resilient Messa...
 
Why Software-Defined Storage Matters
Why Software-Defined Storage MattersWhy Software-Defined Storage Matters
Why Software-Defined Storage Matters
 
Red Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed Hat Storage for Mere Mortals
Red Hat Storage for Mere Mortals
 
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...
Red Hat Storage Day New York - Penguin Computing Spotlight: Delivering Open S...
 

En vedette

Red Hat Storage Day - When the Ceph Hits the Fan
Red Hat Storage Day -  When the Ceph Hits the FanRed Hat Storage Day -  When the Ceph Hits the Fan
Red Hat Storage Day - When the Ceph Hits the FanRed_Hat_Storage
 
Red Hat Ceph Storage Acceleration Utilizing Flash Technology
Red Hat Ceph Storage Acceleration Utilizing Flash Technology Red Hat Ceph Storage Acceleration Utilizing Flash Technology
Red Hat Ceph Storage Acceleration Utilizing Flash Technology Red_Hat_Storage
 
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...Red_Hat_Storage
 
Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red_Hat_Storage
 
Red Hat Storage Day Boston - Why Software-defined Storage Matters
Red Hat Storage Day Boston - Why Software-defined Storage MattersRed Hat Storage Day Boston - Why Software-defined Storage Matters
Red Hat Storage Day Boston - Why Software-defined Storage MattersRed_Hat_Storage
 
Red Hat Storage Day New York - New Reference Architectures
Red Hat Storage Day New York - New Reference ArchitecturesRed Hat Storage Day New York - New Reference Architectures
Red Hat Storage Day New York - New Reference ArchitecturesRed_Hat_Storage
 
Red Hat Storage Day New York - Welcome Remarks
Red Hat Storage Day New York - Welcome Remarks Red Hat Storage Day New York - Welcome Remarks
Red Hat Storage Day New York - Welcome Remarks Red_Hat_Storage
 
Red Hat Storage Day Boston - OpenStack + Ceph Storage
Red Hat Storage Day Boston - OpenStack + Ceph StorageRed Hat Storage Day Boston - OpenStack + Ceph Storage
Red Hat Storage Day Boston - OpenStack + Ceph StorageRed_Hat_Storage
 
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...Red_Hat_Storage
 
Efficient data maintaince in GlusterFS using Databases
Efficient data maintaince in GlusterFS using DatabasesEfficient data maintaince in GlusterFS using Databases
Efficient data maintaince in GlusterFS using DatabasesJoseph Elwin Fernandes
 
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph Enterprise
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph EnterpriseRed Hat Enterprise Linux OpenStack Platform on Inktank Ceph Enterprise
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph EnterpriseRed_Hat_Storage
 
What's new in Jewel and Beyond
What's new in Jewel and BeyondWhat's new in Jewel and Beyond
What's new in Jewel and BeyondSage Weil
 
Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016Red_Hat_Storage
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsRed_Hat_Storage
 
Distributed Storage and Compute With Ceph's librados (Vault 2015)
Distributed Storage and Compute With Ceph's librados (Vault 2015)Distributed Storage and Compute With Ceph's librados (Vault 2015)
Distributed Storage and Compute With Ceph's librados (Vault 2015)Sage Weil
 
The State of Ceph, Manila, and Containers in OpenStack
The State of Ceph, Manila, and Containers in OpenStackThe State of Ceph, Manila, and Containers in OpenStack
The State of Ceph, Manila, and Containers in OpenStackSage Weil
 
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageCeph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageSage Weil
 

En vedette (17)

Red Hat Storage Day - When the Ceph Hits the Fan
Red Hat Storage Day -  When the Ceph Hits the FanRed Hat Storage Day -  When the Ceph Hits the Fan
Red Hat Storage Day - When the Ceph Hits the Fan
 
Red Hat Ceph Storage Acceleration Utilizing Flash Technology
Red Hat Ceph Storage Acceleration Utilizing Flash Technology Red Hat Ceph Storage Acceleration Utilizing Flash Technology
Red Hat Ceph Storage Acceleration Utilizing Flash Technology
 
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...
Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...
 
Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers
 
Red Hat Storage Day Boston - Why Software-defined Storage Matters
Red Hat Storage Day Boston - Why Software-defined Storage MattersRed Hat Storage Day Boston - Why Software-defined Storage Matters
Red Hat Storage Day Boston - Why Software-defined Storage Matters
 
Red Hat Storage Day New York - New Reference Architectures
Red Hat Storage Day New York - New Reference ArchitecturesRed Hat Storage Day New York - New Reference Architectures
Red Hat Storage Day New York - New Reference Architectures
 
Red Hat Storage Day New York - Welcome Remarks
Red Hat Storage Day New York - Welcome Remarks Red Hat Storage Day New York - Welcome Remarks
Red Hat Storage Day New York - Welcome Remarks
 
Red Hat Storage Day Boston - OpenStack + Ceph Storage
Red Hat Storage Day Boston - OpenStack + Ceph StorageRed Hat Storage Day Boston - OpenStack + Ceph Storage
Red Hat Storage Day Boston - OpenStack + Ceph Storage
 
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...
Red Hat Storage Day New York -Performance Intensive Workloads with Samsung NV...
 
Efficient data maintaince in GlusterFS using Databases
Efficient data maintaince in GlusterFS using DatabasesEfficient data maintaince in GlusterFS using Databases
Efficient data maintaince in GlusterFS using Databases
 
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph Enterprise
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph EnterpriseRed Hat Enterprise Linux OpenStack Platform on Inktank Ceph Enterprise
Red Hat Enterprise Linux OpenStack Platform on Inktank Ceph Enterprise
 
What's new in Jewel and Beyond
What's new in Jewel and BeyondWhat's new in Jewel and Beyond
What's new in Jewel and Beyond
 
Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016Red Hat Ceph Storage Roadmap: January 2016
Red Hat Ceph Storage Roadmap: January 2016
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
 
Distributed Storage and Compute With Ceph's librados (Vault 2015)
Distributed Storage and Compute With Ceph's librados (Vault 2015)Distributed Storage and Compute With Ceph's librados (Vault 2015)
Distributed Storage and Compute With Ceph's librados (Vault 2015)
 
The State of Ceph, Manila, and Containers in OpenStack
The State of Ceph, Manila, and Containers in OpenStackThe State of Ceph, Manila, and Containers in OpenStack
The State of Ceph, Manila, and Containers in OpenStack
 
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud StorageCeph, Now and Later: Our Plan for Open Unified Cloud Storage
Ceph, Now and Later: Our Plan for Open Unified Cloud Storage
 

Similaire à Red Hat Storage Day New York - Red Hat Gluster Storage: Historical Tick Data Repository

Scalable POSIX File Systems in the Cloud
Scalable POSIX File Systems in the CloudScalable POSIX File Systems in the Cloud
Scalable POSIX File Systems in the CloudRed_Hat_Storage
 
Red hat storage el almacenamiento disruptivo
Red hat storage el almacenamiento disruptivoRed hat storage el almacenamiento disruptivo
Red hat storage el almacenamiento disruptivoNextel S.A.
 
Red Hat Storage 2014 - Product(s) Overview
Red Hat Storage 2014 - Product(s) OverviewRed Hat Storage 2014 - Product(s) Overview
Red Hat Storage 2014 - Product(s) OverviewMarcel Hergaarden
 
Real-time Analytics with Redis
Real-time Analytics with RedisReal-time Analytics with Redis
Real-time Analytics with RedisCihan Biyikoglu
 
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...HostedbyConfluent
 
Getting Started with Splunk Breakout Session
Getting Started with Splunk Breakout SessionGetting Started with Splunk Breakout Session
Getting Started with Splunk Breakout SessionSplunk
 
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015Andrey Vykhodtsev
 
HPE Hadoop Solutions - From use cases to proposal
HPE Hadoop Solutions - From use cases to proposalHPE Hadoop Solutions - From use cases to proposal
HPE Hadoop Solutions - From use cases to proposalDataWorks Summit
 
Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Andrew Underwood
 
Common MongoDB Use Cases
Common MongoDB Use CasesCommon MongoDB Use Cases
Common MongoDB Use CasesDATAVERSITY
 
Hopsworks in the cloud Berlin Buzzwords 2019
Hopsworks in the cloud Berlin Buzzwords 2019 Hopsworks in the cloud Berlin Buzzwords 2019
Hopsworks in the cloud Berlin Buzzwords 2019 Jim Dowling
 
Architecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big DataArchitecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big DataRichard McDougall
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big datasolarisyourep
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big dataxKinAnx
 
HPE Solutions for Challenges in AI and Big Data
HPE Solutions for Challenges in AI and Big DataHPE Solutions for Challenges in AI and Big Data
HPE Solutions for Challenges in AI and Big DataLviv Startup Club
 
Saviak lviv ai-2019-e-mail (1)
Saviak lviv ai-2019-e-mail (1)Saviak lviv ai-2019-e-mail (1)
Saviak lviv ai-2019-e-mail (1)Lviv Startup Club
 
Cost Savings at High Performance with Redis Labs and AWS
Cost Savings at High Performance with Redis Labs and AWSCost Savings at High Performance with Redis Labs and AWS
Cost Savings at High Performance with Redis Labs and AWSAmazon Web Services
 
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageBringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageMapR Technologies
 

Similaire à Red Hat Storage Day New York - Red Hat Gluster Storage: Historical Tick Data Repository (20)

Scalable POSIX File Systems in the Cloud
Scalable POSIX File Systems in the CloudScalable POSIX File Systems in the Cloud
Scalable POSIX File Systems in the Cloud
 
Red hat storage el almacenamiento disruptivo
Red hat storage el almacenamiento disruptivoRed hat storage el almacenamiento disruptivo
Red hat storage el almacenamiento disruptivo
 
Red Hat Storage 2014 - Product(s) Overview
Red Hat Storage 2014 - Product(s) OverviewRed Hat Storage 2014 - Product(s) Overview
Red Hat Storage 2014 - Product(s) Overview
 
Hdf5
Hdf5Hdf5
Hdf5
 
Real-time Analytics with Redis
Real-time Analytics with RedisReal-time Analytics with Redis
Real-time Analytics with Redis
 
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...
Distributed Data Storage & Streaming for Real-time Decisioning Using Kafka, S...
 
Ceph as software define storage
Ceph as software define storageCeph as software define storage
Ceph as software define storage
 
Getting Started with Splunk Breakout Session
Getting Started with Splunk Breakout SessionGetting Started with Splunk Breakout Session
Getting Started with Splunk Breakout Session
 
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
 
HPE Hadoop Solutions - From use cases to proposal
HPE Hadoop Solutions - From use cases to proposalHPE Hadoop Solutions - From use cases to proposal
HPE Hadoop Solutions - From use cases to proposal
 
Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016
 
Common MongoDB Use Cases
Common MongoDB Use CasesCommon MongoDB Use Cases
Common MongoDB Use Cases
 
Hopsworks in the cloud Berlin Buzzwords 2019
Hopsworks in the cloud Berlin Buzzwords 2019 Hopsworks in the cloud Berlin Buzzwords 2019
Hopsworks in the cloud Berlin Buzzwords 2019
 
Architecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big DataArchitecting Virtualized Infrastructure for Big Data
Architecting Virtualized Infrastructure for Big Data
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big data
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big data
 
HPE Solutions for Challenges in AI and Big Data
HPE Solutions for Challenges in AI and Big DataHPE Solutions for Challenges in AI and Big Data
HPE Solutions for Challenges in AI and Big Data
 
Saviak lviv ai-2019-e-mail (1)
Saviak lviv ai-2019-e-mail (1)Saviak lviv ai-2019-e-mail (1)
Saviak lviv ai-2019-e-mail (1)
 
Cost Savings at High Performance with Redis Labs and AWS
Cost Savings at High Performance with Redis Labs and AWSCost Savings at High Performance with Redis Labs and AWS
Cost Savings at High Performance with Redis Labs and AWS
 
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale StorageBringing Structure, Scalability, and Services to Cloud-Scale Storage
Bringing Structure, Scalability, and Services to Cloud-Scale Storage
 

Dernier

The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementNuwan Dias
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdfPaige Cruz
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 

Dernier (20)

The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API Management
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 

Red Hat Storage Day New York - Red Hat Gluster Storage: Historical Tick Data Repository

  • 1. © RED HAT, INC.1 Red Hat Gluster Storage Historical Tick Data Repository Narendra N. Narang Sr. Cloud Storage Solutions Architect narnar@redhat.com October 2016
  • 2. PROPRIETARY HARDWARE SCALE-UP ARCHITECTURE HARDWARE-BASED INTELLIGENCE CLOSED DEVELOPMENT PROCESS Common, off-the-shelf hardware Scale-out architecture Software-based intelligence Open development process Lower cost, standardized supply chain Increased operational flexibility More programmability, agility, and control More flexible, well-integrated technology FOUR IMPORTANT DIFFERENCES
  • 3. 3 RED HAT GLUSTER STORAGE • Software-defined, distributed, scale-out and resilient file store • Cost-efficient and performant at scale • Easy to consume, deploy, scale and manage in public, private, and hybrid cloud architectures • Offers mature NFS, SMB and HDFS interfaces for enterprise applications Nimble file storage for petabyte-scale workloads Analytics • Machine analytics with Splunk • Big data analytics Enterprise File Sharing • Media streaming • Active Archives Enterprise Virtualization Rich Media and Archival TARGETUSECASES
  • 4. GLUSTER ARCHITECTURE CLOUD VOLUME MANAGER CLOUD VOLUME MANAGER ADMINISTRATOR USER RED HAT GLUSTER STORAGE CLI NFS • CIFS • FUSE • OPENSTACK SWIFT RED HAT GLUSTER STORAGE POOL SERVER DISKS SERVER DISKS File system mount point
  • 5. Tick Data What is a Tick? A “tick” is the minimum upward or downward movement (any change) in the price of a security as measured over a period of time. An "uptick" refers to a trade where the current transaction occurred at a price higher than the Previous transaction and a "downtick" refers to a transaction that has occurred at a lower price than the previous transaction. Consequently, a “zerotick” refers to a trade where the current transaction occurred at a price higher than the previous transaction. What is Tick Data? Tick data is time series data containing price, volume and many other dimensions (bid/ask prices, bid/ask sizes, quote time, trade time, exchange information) for each point of granularity. Tick Data and Storage The higher the resolution of tick data collected, the larger will be the dataset size and hence, the amount of storage capacity required. Customer Challenge Standardize on an economical, scale-out, distributed, shared filestore as an alternative to their existing expensive Isilon storage system for Historical Tick Data. Financial compliance and regulations require banks, brokerages, and trading institutions to store each transaction tick. The historical tick data is also used to create richer and more accurate risk models, to perform backtesting and to run trading simulations for quantitative analytics. A scalable, cost-efficient solution is imperative since storage capacity is a direct function of the number of data dimensions and the resolution of tick data collected.
  • 6. High-level Tick Data Workflow Data Feed 1 Data Feed 2 Data Feed 3 Market Data Servers (Aggregation of Feed Handlers) Data Feed N KDB+ In-memory TickDB (Real-time) Tick LogFile Historical Tick Database EndofDay(EOD) Intraday EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file for that day. This file is typically written as a large sequential stream of blocks. News Social Media
  • 7. Use Case: Historical Tick Database on Red Hat Gluster Storage RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE GLUSTERFSRHGS NODE RHGS NODE RHGS NODE RHGS NODE Data Feed 1 Data Feed 2 Data Feed 3 Market Data Servers (Aggregation of Feed Handlers) Data Feed N In-memory TickDB (Real-time) Tick LogFile EndofDay(EOD) Intraday EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file for that day. This file is typically written as a large sequential stream of blocks. News Social Media Scale Out Architecture Mathematical, Algorithmic Low frequency Calculations: Risk Modeling, P&L Reporting, Mark-to-market, Stress Tests, MC Simulations
  • 8. Historical Tick Database on Red Hat Gluster Storage – Disaster Recovery Implementation RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE GLUSTERFSRHGS NODE RHGS NODE RHGS NODE RHGS NODE Data Feed 1 Data Feed 2 Data Feed 3 Market Data Servers (Aggregation of Feed Handlers) Data Feed N In-memory TickDB (Real-time) Tick LogFile EOD Intraday EOD data stored as a distinct Historical Database Partitioned Format “hdpf” file for that day. This file is typically written as a large sequential stream of blocks. News Social Media RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE GLUSTERFSRHGS NODE RHGS NODE RHGS NODE RHGS NODE SITE A SITE B Async Geo-rep
  • 9. Hadoop Analytics on an Historical Tick Database RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE RHGS NODE GLUSTERFSRHGS NODE RHGS NODE RHGS NODE RHGS NODE Data Feed 1 Data Feed 2 Data Feed 3 Market Data Servers (Aggregation of Feed Handlers) Data Feed N In-memory TickDB (Real-time) Tick LogFile Historical Tick Database EndofDay(EOD) Intraday News Social Media Hadoop YARN Batch (MR) Script (Pig) SQL (Hive) Online (HBase) Real-time (Storm) In-memory (Spark) * Risk Analysis * Trading Risk Analysis * Ticker Plant Data * Archival/retention
  • 10. © RED HAT, INC.10 Analytics on Red Hat Gluster Storage Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Spark Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Red Hat Gluster Storage Cloud Kubernetes Orchestration for Docker Containers Message Bus for Microservices Figure1. Shared Storage Spark Spark Spark Spark Spark Spark Spark Spark Spark Spark Spark Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Spark Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine Linux Kernel RHEL Atomic Container Engine Physical, Virtual, Cloud Linux Kernel RHEL Atomic Container Engine SSD, HDD 15K, HDD, 7.2K (Self-healing,Tiering, Replicas, Geo-replication, Erasure Coding) Kubernetes Orchestration for Docker Containers Message Bus for Microservices Figure2. Containerized Storage Spark Spark Storage Spark Spark Spark Spark Spark Storage Storage Storage
  • 11. 11 Comparing Throughput and Costs at Scale STORAGE PERFORMANCE SCALABILITY STORAGE COSTS SCALABILITY Number of Storage Nodes Number of Storage Nodes TotalStorageCosts($) Reads/WritesThroughput(mBps) Software Defined Scale out Storage Traditional Enterprise NAS Storage Traditional Enterprise NAS Storage Software Defined Scale out Storage
  • 12. 12 3 Year TCO (incl. support) TCO COMPARISON Throughput Configuration • HDD-only media • Higher CPU-to-media ratio than archive configurations • 2x replication with RHGS • 8:3 Erasure Coding with EMC Isilon X-210 12LFF 12LFF (small) X-410 36LFF 36LFF (medium) For 1PB Usable Capacity Throughput-optimized Solutions Pricing Sources: EMC Isilon: Gartner Competitive Profiles, as of 2/16/16) & Supermicro: Thinkmate, as of 1/13/16)
  • 13. 13 HEAD TO HEAD FEATURE COMPARISON STANDARD VS. ADD ON FEATURES TOTAL COST OF OWNERSHIP HARDWARE CONFIGURATION CHOICES FAULT TOLERANCE COMPLEX PRICING
  • 14. 14 RED HAT GLUSTER STORAGE  Data Services  Erasure Coding  Tiering  Bit-Rot Detection  Snapshots  Quotas  Geo Replication  Protocols  Active/Active NFSv4  SMB3 (protocol negotiation, in-flight encryption, server-side copy)  Management  Device Management  Dashboard  Monitoring  Command and Control  Security & Data Integrity  SSL based in-flight encryption  At-rest encryption using dm-crypt  SELinux Enforcing Mode  Self Healing Half the price for comparable features & greater flexibility