SlideShare une entreprise Scribd logo
1  sur  3
Télécharger pour lire hors ligne
Introduce
leo_redundant_manager
Yosuke Hara @yosukehara
May, 2014
storage_0
storage_1
storage_2
[ Actual Nodes ]
RING
2 ^ 128
# of replicas = 3
0
2^128 -1
clock-wise
Filename = “bucket/leofs.key”
Hash = crypto:md5(Filename)
Primary node
Secondary(1) node
Secondary(2) node
“Virtual nodes”
[Actual Node : Virtual node = 1 : 128]
- For restrain unbalanced disc-usage and transaction (requests from gateways)
- For restrain effect of data-rebalance
Redundant-Manager 1/2
2
Primary node
Secondary(1) node
Secondary(2) node
Object of Replica-nodes
1 “Consisten-hashing”
storage_0
storage_1
storage_2
[ Actual Nodes ]
RING
2 ^ 128
# of replicas = 3
0
2^128 -1
“Virtual nodes”
[Actual Node : Virtual node = 1 : 128]
- For restrain unbalanced disc-usage and transaction (requests from gateways)
- For restrain effect of data-rebalance
Redundant-Manager 2/2
storage_3
Filename = “bucket/leofs.key”
Hash = crypto:md5(Filename)
clock-wise
Primary node
Secondary(1) node
Secondary(2) node
2
Primary node
Secondary(1) node
Secondary(2) node
Object of Replica-nodes
an attached node
Primary node
Secondary(1) node
Secondary(2) node
Object of Replica-nodes
<before>
<after>
1 “Consisten-hashing”

Contenu connexe

Tendances

MongoDB Replication (Dwight Merriman)
MongoDB Replication (Dwight Merriman)MongoDB Replication (Dwight Merriman)
MongoDB Replication (Dwight Merriman)MongoSF
 
Replication MongoDB Days 2013
Replication MongoDB Days 2013Replication MongoDB Days 2013
Replication MongoDB Days 2013Randall Hunt
 
MongoDB - External Authentication
MongoDB - External AuthenticationMongoDB - External Authentication
MongoDB - External AuthenticationJason Terpko
 
1 m+ qps on mysql galera cluster
1 m+ qps on mysql galera cluster1 m+ qps on mysql galera cluster
1 m+ qps on mysql galera clusterOlinData
 
MongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster TutorialMongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster TutorialJason Terpko
 
MongoDB: Comparing WiredTiger In-Memory Engine to Redis
MongoDB: Comparing WiredTiger In-Memory Engine to RedisMongoDB: Comparing WiredTiger In-Memory Engine to Redis
MongoDB: Comparing WiredTiger In-Memory Engine to RedisJason Terpko
 
Chaining 7 vulnerabilities in Citrix ShareFile On-Premise
Chaining 7 vulnerabilities in Citrix ShareFile On-PremiseChaining 7 vulnerabilities in Citrix ShareFile On-Premise
Chaining 7 vulnerabilities in Citrix ShareFile On-PremiseJohanna Curiel
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisKnoldus Inc.
 
Paris Redis Meetup Introduction
Paris Redis Meetup IntroductionParis Redis Meetup Introduction
Paris Redis Meetup IntroductionGregory Boissinot
 
Designate Installation Workshop
Designate Installation WorkshopDesignate Installation Workshop
Designate Installation WorkshopGraham Hayes
 
Search and analyze data in real time
Search and analyze data in real timeSearch and analyze data in real time
Search and analyze data in real timeRohit Kalsarpe
 
Scalable Socket Server by Aryo
Scalable Socket Server by AryoScalable Socket Server by Aryo
Scalable Socket Server by AryoAgate Studio
 
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...Ontico
 
Sessionization with Spark streaming
Sessionization with Spark streamingSessionization with Spark streaming
Sessionization with Spark streamingRamūnas Urbonas
 
Introduction to apache zoo keeper
Introduction to apache zoo keeper Introduction to apache zoo keeper
Introduction to apache zoo keeper Omid Vahdaty
 
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)Intro KaKao MRTE (MySQL Realtime Traffic Emulator)
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)I Goo Lee
 

Tendances (20)

MongoDB Replication (Dwight Merriman)
MongoDB Replication (Dwight Merriman)MongoDB Replication (Dwight Merriman)
MongoDB Replication (Dwight Merriman)
 
Replication MongoDB Days 2013
Replication MongoDB Days 2013Replication MongoDB Days 2013
Replication MongoDB Days 2013
 
MongoDB-SESSION03
MongoDB-SESSION03MongoDB-SESSION03
MongoDB-SESSION03
 
Threads and Node.js
Threads and Node.jsThreads and Node.js
Threads and Node.js
 
MongoDB - External Authentication
MongoDB - External AuthenticationMongoDB - External Authentication
MongoDB - External Authentication
 
1 m+ qps on mysql galera cluster
1 m+ qps on mysql galera cluster1 m+ qps on mysql galera cluster
1 m+ qps on mysql galera cluster
 
MongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster TutorialMongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster Tutorial
 
MongoDB: Comparing WiredTiger In-Memory Engine to Redis
MongoDB: Comparing WiredTiger In-Memory Engine to RedisMongoDB: Comparing WiredTiger In-Memory Engine to Redis
MongoDB: Comparing WiredTiger In-Memory Engine to Redis
 
Chaining 7 vulnerabilities in Citrix ShareFile On-Premise
Chaining 7 vulnerabilities in Citrix ShareFile On-PremiseChaining 7 vulnerabilities in Citrix ShareFile On-Premise
Chaining 7 vulnerabilities in Citrix ShareFile On-Premise
 
Disruptor
DisruptorDisruptor
Disruptor
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Paris Redis Meetup Introduction
Paris Redis Meetup IntroductionParis Redis Meetup Introduction
Paris Redis Meetup Introduction
 
Unqlite
UnqliteUnqlite
Unqlite
 
Designate Installation Workshop
Designate Installation WorkshopDesignate Installation Workshop
Designate Installation Workshop
 
Search and analyze data in real time
Search and analyze data in real timeSearch and analyze data in real time
Search and analyze data in real time
 
Scalable Socket Server by Aryo
Scalable Socket Server by AryoScalable Socket Server by Aryo
Scalable Socket Server by Aryo
 
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...
Новые возможности полнотекстового поиска в PostgreSQL / Олег Бартунов (Postgr...
 
Sessionization with Spark streaming
Sessionization with Spark streamingSessionization with Spark streaming
Sessionization with Spark streaming
 
Introduction to apache zoo keeper
Introduction to apache zoo keeper Introduction to apache zoo keeper
Introduction to apache zoo keeper
 
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)Intro KaKao MRTE (MySQL Realtime Traffic Emulator)
Intro KaKao MRTE (MySQL Realtime Traffic Emulator)
 

Similaire à Introduce leo-redundant-manager

Spil Games @ FOSDEM: Galera Replicator IRL
Spil Games @ FOSDEM: Galera Replicator IRLSpil Games @ FOSDEM: Galera Replicator IRL
Spil Games @ FOSDEM: Galera Replicator IRLspil-engineering
 
Automated Application Management with SaltStack
Automated Application Management with SaltStackAutomated Application Management with SaltStack
Automated Application Management with SaltStackinovex GmbH
 
Process Address Space: The way to create virtual address (page table) of user...
Process Address Space: The way to create virtual address (page table) of user...Process Address Space: The way to create virtual address (page table) of user...
Process Address Space: The way to create virtual address (page table) of user...Adrian Huang
 
Monitoring with Prometheus
Monitoring with PrometheusMonitoring with Prometheus
Monitoring with PrometheusShiao-An Yuan
 
Zing Database – Distributed Key-Value Database
Zing Database – Distributed Key-Value DatabaseZing Database – Distributed Key-Value Database
Zing Database – Distributed Key-Value Databasezingopen
 
Zing Database
Zing Database Zing Database
Zing Database Long Dao
 
Multitenancy: Kafka clusters for everyone at LINE
Multitenancy: Kafka clusters for everyone at LINEMultitenancy: Kafka clusters for everyone at LINE
Multitenancy: Kafka clusters for everyone at LINEkawamuray
 
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent Sandbox
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent SandboxHITBSecConf 2017-Shadow-Box-the Practical and Omnipotent Sandbox
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent SandboxSeunghun han
 
Input and Output Devices and Systems
Input and Output Devices and SystemsInput and Output Devices and Systems
Input and Output Devices and SystemsNajma Alam
 
Zookeeper Introduce
Zookeeper IntroduceZookeeper Introduce
Zookeeper Introducejhao niu
 
Painless ruby deployment on shelly cloud
Painless ruby deployment on shelly cloudPainless ruby deployment on shelly cloud
Painless ruby deployment on shelly cloudGiedrius Rimkus
 
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...Equnix Business Solutions
 
MySQL for Large Scale Social Games
MySQL for Large Scale Social GamesMySQL for Large Scale Social Games
MySQL for Large Scale Social GamesYoshinori Matsunobu
 
Linux Du Jour
Linux Du JourLinux Du Jour
Linux Du Jourmwedgwood
 

Similaire à Introduce leo-redundant-manager (20)

Spil Games @ FOSDEM: Galera Replicator IRL
Spil Games @ FOSDEM: Galera Replicator IRLSpil Games @ FOSDEM: Galera Replicator IRL
Spil Games @ FOSDEM: Galera Replicator IRL
 
Apache Cassandra at Macys
Apache Cassandra at MacysApache Cassandra at Macys
Apache Cassandra at Macys
 
Automated Application Management with SaltStack
Automated Application Management with SaltStackAutomated Application Management with SaltStack
Automated Application Management with SaltStack
 
Process Address Space: The way to create virtual address (page table) of user...
Process Address Space: The way to create virtual address (page table) of user...Process Address Space: The way to create virtual address (page table) of user...
Process Address Space: The way to create virtual address (page table) of user...
 
Monitoring with Prometheus
Monitoring with PrometheusMonitoring with Prometheus
Monitoring with Prometheus
 
Hadoop admin
Hadoop adminHadoop admin
Hadoop admin
 
Zing Database – Distributed Key-Value Database
Zing Database – Distributed Key-Value DatabaseZing Database – Distributed Key-Value Database
Zing Database – Distributed Key-Value Database
 
Zing Database
Zing Database Zing Database
Zing Database
 
Multitenancy: Kafka clusters for everyone at LINE
Multitenancy: Kafka clusters for everyone at LINEMultitenancy: Kafka clusters for everyone at LINE
Multitenancy: Kafka clusters for everyone at LINE
 
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent Sandbox
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent SandboxHITBSecConf 2017-Shadow-Box-the Practical and Omnipotent Sandbox
HITBSecConf 2017-Shadow-Box-the Practical and Omnipotent Sandbox
 
Apache Spark
Apache SparkApache Spark
Apache Spark
 
Scaling search with SolrCloud
Scaling search with SolrCloudScaling search with SolrCloud
Scaling search with SolrCloud
 
Input and Output Devices and Systems
Input and Output Devices and SystemsInput and Output Devices and Systems
Input and Output Devices and Systems
 
Zookeeper Introduce
Zookeeper IntroduceZookeeper Introduce
Zookeeper Introduce
 
Painless ruby deployment on shelly cloud
Painless ruby deployment on shelly cloudPainless ruby deployment on shelly cloud
Painless ruby deployment on shelly cloud
 
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...
PGConf.ASIA 2019 Bali - Mission Critical Production High Availability Postgre...
 
MySQL for Large Scale Social Games
MySQL for Large Scale Social GamesMySQL for Large Scale Social Games
MySQL for Large Scale Social Games
 
Linux Du Jour
Linux Du JourLinux Du Jour
Linux Du Jour
 
Hdfs high availability
Hdfs high availabilityHdfs high availability
Hdfs high availability
 
Hdfs high availability
Hdfs high availabilityHdfs high availability
Hdfs high availability
 

Introduce leo-redundant-manager

  • 2. storage_0 storage_1 storage_2 [ Actual Nodes ] RING 2 ^ 128 # of replicas = 3 0 2^128 -1 clock-wise Filename = “bucket/leofs.key” Hash = crypto:md5(Filename) Primary node Secondary(1) node Secondary(2) node “Virtual nodes” [Actual Node : Virtual node = 1 : 128] - For restrain unbalanced disc-usage and transaction (requests from gateways) - For restrain effect of data-rebalance Redundant-Manager 1/2 2 Primary node Secondary(1) node Secondary(2) node Object of Replica-nodes 1 “Consisten-hashing”
  • 3. storage_0 storage_1 storage_2 [ Actual Nodes ] RING 2 ^ 128 # of replicas = 3 0 2^128 -1 “Virtual nodes” [Actual Node : Virtual node = 1 : 128] - For restrain unbalanced disc-usage and transaction (requests from gateways) - For restrain effect of data-rebalance Redundant-Manager 2/2 storage_3 Filename = “bucket/leofs.key” Hash = crypto:md5(Filename) clock-wise Primary node Secondary(1) node Secondary(2) node 2 Primary node Secondary(1) node Secondary(2) node Object of Replica-nodes an attached node Primary node Secondary(1) node Secondary(2) node Object of Replica-nodes <before> <after> 1 “Consisten-hashing”