SlideShare une entreprise Scribd logo
1  sur  3
Télécharger pour lire hors ligne
High Availability via
Asynchronous Virtual
Machine Replication
Review by Mário Almeida (EMDC)

Summary
High availability requires the usage of redundancy techniques that are capable of maintaining
and switching to backups in case of failure. Commercial high availability systems generally use
specialized hardware and/or customized software to achieve this purpose.

This paper describes a system called Remus. It provides OS and application agnostic high
availability on commodity hardware. It performs virtualization to migrate running VMs between
physical hosts, and extends the technique to replicate snapshots of an entire running OS
instance at very high frequencies between a pair of physical machines. It discretizes the system
into a serie of replicated snapshots.

Any transmitted network packets is not released until the system state that produced it has been
replicated. It allows a single host to execute speculatively and then checkpoint and replicate
its state asynchronously. System state is not made externally visible until the checkpoint is
committed.

Remus ensures that regardless of the moment at which the primary fails, no externally visible
state is ever lost. It aims to make mission-critical availability accessible to mid- and low-end
systems.

Remus goals:
  ● Generality - High availability should be provided as a low-level service, with common
     mechanisms that apply regardless of the application being protected or the hardware on
     which it runs.

   ●   Transparency - High availability should not require that OS or application code be
       modified to support facilities such as failure detection or state recovery.
●   Seamless failure recovery - No externally visible state should ever be lost in the case
       of single-host failure. Failure recovery should be fast. Established TCP connections
       should not be lost or reset.

Remus runs paired servers in an active-passive configuration. Speculative execution decouples
external output from synchronization points. Synchronization with the replicated server is
performed asynchronously. The basic stages of operation in Remus are the following:




Some characteristics:

   ●   VM-based whole-system replication.

   ●   Speculative execution - Replication may be achieved either by copying the state of a
       system. The state of the replica is synchronized with the primary only when the output of
       the primary has become externally visible. It buffers output until a more convenient time,
       performing computation speculatively ahead of synchronization points.

   ●   Asynchronous replication - due to buffering output at the primary server. The primary
       host can resume execution when its machine state has been captured, without waiting
       for an ack.

Remus failure model provides the following properties:

   ●   The fail-stop failure of any single host is tolerable.
●   Should both the primary and backup hosts fail concurrently, the protected system’s data
       will be left in a crash-consistent state.

   ●   No output will be made externally visible until the associated system state has been
       committed to the replica.

It uses a simple failure detector integrated in the checkpointing stream. A timeout of the backup
responding to commit requests will result in the primary assuming that the backup has crashed
and disabling protection. Similarly, a timeout of new checkpoints being transmitted from the
primary will result in the backup assuming that the primary has crashed and resuming execution
from the most recent checkpoint.

Remus also has pipelined checkpoints since it uses an epoch-based system in which execution
of the active VM is bounded by brief pauses in execution in which changed state is atomically
captured, and external output is released when that state has been propagated to the backup.


Lesson
High availability is possible through virtual machine replication using existing software and
running on commodity hardware. Remus performs frequent global checkpoints to replicate the
state of a single speculatively executing virtual machine.


Critique
It comes with the price of introducing a small performance overhead due to the network
buffering required to ensure consistent replication.

Contenu connexe

En vedette

1phase induction motors
1phase induction motors1phase induction motors
1phase induction motorsVijay Raskar
 
Electric drives
Electric drivesElectric drives
Electric drivesraj_e2004
 
Op amp(operational amplifier)
Op amp(operational amplifier)Op amp(operational amplifier)
Op amp(operational amplifier)Kausik das
 
Illumination Lighting
Illumination LightingIllumination Lighting
Illumination LightingVijay Raskar
 
Circuit switching packet switching
Circuit switching  packet  switchingCircuit switching  packet  switching
Circuit switching packet switchingSneha Dalvi
 
Electric drives
Electric drivesElectric drives
Electric drivesSamsu Deen
 
Power Supply Project
Power Supply ProjectPower Supply Project
Power Supply Projectusmanhadi91
 

En vedette (11)

1phase induction motors
1phase induction motors1phase induction motors
1phase induction motors
 
Electric drives
Electric drivesElectric drives
Electric drives
 
Op amp(operational amplifier)
Op amp(operational amplifier)Op amp(operational amplifier)
Op amp(operational amplifier)
 
Illumination Lighting
Illumination LightingIllumination Lighting
Illumination Lighting
 
Circuit switching packet switching
Circuit switching  packet  switchingCircuit switching  packet  switching
Circuit switching packet switching
 
8051 MICROCONTROLLER
8051 MICROCONTROLLER 8051 MICROCONTROLLER
8051 MICROCONTROLLER
 
Electric drives
Electric drivesElectric drives
Electric drives
 
Circuit breaker
Circuit breakerCircuit breaker
Circuit breaker
 
Power Supply Project
Power Supply ProjectPower Supply Project
Power Supply Project
 
Induction motor
Induction motorInduction motor
Induction motor
 
Electric traction
Electric tractionElectric traction
Electric traction
 

Plus de Mário Almeida

Empirical Study of Android Alarm Usage for Application Scheduling
Empirical Study of Android Alarm Usage for Application SchedulingEmpirical Study of Android Alarm Usage for Application Scheduling
Empirical Study of Android Alarm Usage for Application SchedulingMário Almeida
 
Android reverse engineering - Analyzing skype
Android reverse engineering - Analyzing skypeAndroid reverse engineering - Analyzing skype
Android reverse engineering - Analyzing skypeMário Almeida
 
High-Availability of YARN (MRv2)
High-Availability of YARN (MRv2)High-Availability of YARN (MRv2)
High-Availability of YARN (MRv2)Mário Almeida
 
Flume impact of reliability on scalability
Flume impact of reliability on scalabilityFlume impact of reliability on scalability
Flume impact of reliability on scalabilityMário Almeida
 
Dimemas and Multi-Level Cache Simulations
Dimemas and Multi-Level Cache SimulationsDimemas and Multi-Level Cache Simulations
Dimemas and Multi-Level Cache SimulationsMário Almeida
 
Self-Adapting, Energy-Conserving Distributed File Systems
Self-Adapting, Energy-Conserving Distributed File SystemsSelf-Adapting, Energy-Conserving Distributed File Systems
Self-Adapting, Energy-Conserving Distributed File SystemsMário Almeida
 
Smith waterman algorithm parallelization
Smith waterman algorithm parallelizationSmith waterman algorithm parallelization
Smith waterman algorithm parallelizationMário Almeida
 
Man-In-The-Browser attacks
Man-In-The-Browser attacksMan-In-The-Browser attacks
Man-In-The-Browser attacksMário Almeida
 
Flume-based Independent News Aggregator
Flume-based Independent News AggregatorFlume-based Independent News Aggregator
Flume-based Independent News AggregatorMário Almeida
 
Exploiting Availability Prediction in Distributed Systems
Exploiting Availability Prediction in Distributed SystemsExploiting Availability Prediction in Distributed Systems
Exploiting Availability Prediction in Distributed SystemsMário Almeida
 
High Availability of Services in Wide-Area Shared Computing Networks
High Availability of Services in Wide-Area Shared Computing NetworksHigh Availability of Services in Wide-Area Shared Computing Networks
High Availability of Services in Wide-Area Shared Computing NetworksMário Almeida
 
Instrumenting parsecs raytrace
Instrumenting parsecs raytraceInstrumenting parsecs raytrace
Instrumenting parsecs raytraceMário Almeida
 
Architecting a cloud scale identity fabric
Architecting a cloud scale identity fabricArchitecting a cloud scale identity fabric
Architecting a cloud scale identity fabricMário Almeida
 

Plus de Mário Almeida (15)

Empirical Study of Android Alarm Usage for Application Scheduling
Empirical Study of Android Alarm Usage for Application SchedulingEmpirical Study of Android Alarm Usage for Application Scheduling
Empirical Study of Android Alarm Usage for Application Scheduling
 
Android reverse engineering - Analyzing skype
Android reverse engineering - Analyzing skypeAndroid reverse engineering - Analyzing skype
Android reverse engineering - Analyzing skype
 
Spark
SparkSpark
Spark
 
High-Availability of YARN (MRv2)
High-Availability of YARN (MRv2)High-Availability of YARN (MRv2)
High-Availability of YARN (MRv2)
 
Flume impact of reliability on scalability
Flume impact of reliability on scalabilityFlume impact of reliability on scalability
Flume impact of reliability on scalability
 
Dimemas and Multi-Level Cache Simulations
Dimemas and Multi-Level Cache SimulationsDimemas and Multi-Level Cache Simulations
Dimemas and Multi-Level Cache Simulations
 
Self-Adapting, Energy-Conserving Distributed File Systems
Self-Adapting, Energy-Conserving Distributed File SystemsSelf-Adapting, Energy-Conserving Distributed File Systems
Self-Adapting, Energy-Conserving Distributed File Systems
 
Smith waterman algorithm parallelization
Smith waterman algorithm parallelizationSmith waterman algorithm parallelization
Smith waterman algorithm parallelization
 
Man-In-The-Browser attacks
Man-In-The-Browser attacksMan-In-The-Browser attacks
Man-In-The-Browser attacks
 
Flume-based Independent News Aggregator
Flume-based Independent News AggregatorFlume-based Independent News Aggregator
Flume-based Independent News Aggregator
 
Exploiting Availability Prediction in Distributed Systems
Exploiting Availability Prediction in Distributed SystemsExploiting Availability Prediction in Distributed Systems
Exploiting Availability Prediction in Distributed Systems
 
High Availability of Services in Wide-Area Shared Computing Networks
High Availability of Services in Wide-Area Shared Computing NetworksHigh Availability of Services in Wide-Area Shared Computing Networks
High Availability of Services in Wide-Area Shared Computing Networks
 
Instrumenting parsecs raytrace
Instrumenting parsecs raytraceInstrumenting parsecs raytrace
Instrumenting parsecs raytrace
 
Architecting a cloud scale identity fabric
Architecting a cloud scale identity fabricArchitecting a cloud scale identity fabric
Architecting a cloud scale identity fabric
 
SOAP vs REST
SOAP vs RESTSOAP vs REST
SOAP vs REST
 

Dernier

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimizationarrow10202532yuvraj
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...Daniel Zivkovic
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 

Dernier (20)

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 

High availability via asynchronous virtual machine replication

  • 1. High Availability via Asynchronous Virtual Machine Replication Review by Mário Almeida (EMDC) Summary High availability requires the usage of redundancy techniques that are capable of maintaining and switching to backups in case of failure. Commercial high availability systems generally use specialized hardware and/or customized software to achieve this purpose. This paper describes a system called Remus. It provides OS and application agnostic high availability on commodity hardware. It performs virtualization to migrate running VMs between physical hosts, and extends the technique to replicate snapshots of an entire running OS instance at very high frequencies between a pair of physical machines. It discretizes the system into a serie of replicated snapshots. Any transmitted network packets is not released until the system state that produced it has been replicated. It allows a single host to execute speculatively and then checkpoint and replicate its state asynchronously. System state is not made externally visible until the checkpoint is committed. Remus ensures that regardless of the moment at which the primary fails, no externally visible state is ever lost. It aims to make mission-critical availability accessible to mid- and low-end systems. Remus goals: ● Generality - High availability should be provided as a low-level service, with common mechanisms that apply regardless of the application being protected or the hardware on which it runs. ● Transparency - High availability should not require that OS or application code be modified to support facilities such as failure detection or state recovery.
  • 2. Seamless failure recovery - No externally visible state should ever be lost in the case of single-host failure. Failure recovery should be fast. Established TCP connections should not be lost or reset. Remus runs paired servers in an active-passive configuration. Speculative execution decouples external output from synchronization points. Synchronization with the replicated server is performed asynchronously. The basic stages of operation in Remus are the following: Some characteristics: ● VM-based whole-system replication. ● Speculative execution - Replication may be achieved either by copying the state of a system. The state of the replica is synchronized with the primary only when the output of the primary has become externally visible. It buffers output until a more convenient time, performing computation speculatively ahead of synchronization points. ● Asynchronous replication - due to buffering output at the primary server. The primary host can resume execution when its machine state has been captured, without waiting for an ack. Remus failure model provides the following properties: ● The fail-stop failure of any single host is tolerable.
  • 3. Should both the primary and backup hosts fail concurrently, the protected system’s data will be left in a crash-consistent state. ● No output will be made externally visible until the associated system state has been committed to the replica. It uses a simple failure detector integrated in the checkpointing stream. A timeout of the backup responding to commit requests will result in the primary assuming that the backup has crashed and disabling protection. Similarly, a timeout of new checkpoints being transmitted from the primary will result in the backup assuming that the primary has crashed and resuming execution from the most recent checkpoint. Remus also has pipelined checkpoints since it uses an epoch-based system in which execution of the active VM is bounded by brief pauses in execution in which changed state is atomically captured, and external output is released when that state has been propagated to the backup. Lesson High availability is possible through virtual machine replication using existing software and running on commodity hardware. Remus performs frequent global checkpoints to replicate the state of a single speculatively executing virtual machine. Critique It comes with the price of introducing a small performance overhead due to the network buffering required to ensure consistent replication.