SlideShare une entreprise Scribd logo
1  sur  49
Télécharger pour lire hors ligne
Resilient Algorithms and
Data Structures
Giuseppe F. Italiano
Università di Roma “Tor Vergata”
Some advertising first
School on Graph Theory, Algorithms and Applications
Erice (Sicily), September 25 - October 3, 2011
Please send your best students /postdocs /junior ….
Memory Errors
 Common in Practice 
3
Outline of the Talk
1.  Motivation and Model
2.  Resilient Algorithms:
•  Sorting and Searching
3.  Resilient Data Structures
•  Priority Queues
•  Dictionaries
4.  Conclusions and Open Problems
4
Memory Errors
Memory error: one or multiple bits read differently from
how they were last written.
Many possible causes:
•  electrical or magnetic interference (cosmic rays)
•  hardware problems (bit permanently damaged)
•  corruption in data path between memories and
processing units
Errors in DRAM devices concern for a long time 
[May & Woods 79, Ziegler et al 79, Chen & Hsiao 84, Normand 96,
O’Gorman et al 96, Mukherjee et al 05, … ]
5
Memory Errors
Soft Errors: 
Randomly corrupt bits, but do not leave any physical
damage --- cosmic rays
Hard Errors: 
Corrupt bits in a repeatable manner because of a
physical defect (e.g., stuck bits) --- hardware problems
6
Error Correcting Codes (ECC)
Error correcting codes (ECC) allow detection and
correction of one or multiple bit errors 
Typical ECC is SECDED (i.e., single error correct,
double error detect)
Chip-Kill can correct up to 4 adjacent bits at once
ECC has several overheads in terms of performance
(33%), size (20%) and money (10%).
ECC memory chips are mostly used in memory systems
for server machines rather than for client computers
7
Impact of Memory Errors
Consequence of a memory error is system dependent
1. Correctable errors : fixed by ECC
2. Uncorrectable errors :
2.1. Detected : Explicit failure (e.g., a machine reboot)
2.2. Undetected : 

 2.2.1. Induced failure (e.g., a kernel panic)

 2.2.2. Unnoticed (but application corrupted,

 
 e.g., segmentation fault, file not found,

 
 file not readable, … )
8
Impact of Memory Errors
9
How Common are Memory Errors?
10
How Common are Memory Errors?
11
How Common are Memory Errors?
[Schroeder et al 2009] experiments 2.5 years (Jan 06 – Jun 08) 
on Google fleet (104 machines, ECC memory)
Memory errors are NOT rare events!
12
Memory Errors
Not all machines (clients) have ECC memory chips. 
Increased demand for larger capacities at low cost
just makes the problem more serious – large clusters
of inexpensive memories 
Need of reliable computation in the presence of
memory faults
13
Memory Errors
•  Memory errors can cause security vulnerabilities: 
Fault-based cryptanalysis [Boneh et al 97, Xu et al 01, Bloemer & Seifert 03]
Attacking Java Virtual Machines [Govindavajhala & Appel 03]
Breaking smart cards [Skorobogatov & Anderson 02, Bar-El et al 06] 
• Avionics and space electronic systems:
Amount of cosmic rays increase with altitude (soft errors)
Other scenarios in which memory errors have impact (and
seem to be modeled in an adversarial setting):
14
Memory Errors in Space
15
Memory Errors in Space
16
Memory Errors in Space
17
Recap on Memory Errors
1. Memory errors are NOT rare: even a small cluster
of computers with few GB per node can experience
one bit error every few minutes.
18
I know my PIN number:
it’s my name I can’t remember…
Memory Errors
Mem. size
 Mean Time Between Failures
512 MB
 2.92 hours
1 GB
 1.46 hours
16 GB
 5.48 minutes
64 GB
 1.37 minutes
1 TB
 5.13 seconds
In the field study, Google researchers observed mean error
rates of 2,000 – 6,000 per GB per year (25,000 – 75,000
FIT/Mbit)
19
Recap on Memory Errors
2. Memory errors can be harmful: uncorrectable
memory errors cause some catastrophic event (reboot,
kernel panic, data corruption, …)
20
I’m thinking of getting back into crime, Luigi.
Legitimate business is too corrupt…
A small example
Classical algorithms may not be correct in the
presence of (even very few) memory errors
1
 2
 3
 4
 5
 6
 7
 8
 9
 10
11
 12
 13
 14
 15
 16
 17
 18
 19
 20
A
B
Out
An example: merging two ordered lists
Θ(n)
 Θ(n)
Θ(n2)
inversions
...
11
 12
 20
13
80
...
2
 3
 4
 9
 10
80
21
Recap on Memory Errors
3. ECC may not be
available (or may not be
enough): No ECC in
inexpensive memories.
ECC does not guarantee
complete fault coverage;
expensive; system halt
upon detection of
uncorrectable errors;
service disruption; etc…
etc…
22
Resilient Algorithms and Data Structures
Resilient Algorithms and Data Structures:
Capable of tolerating memory errors on data (even
throughout their execution) without sacrificing
correctness, performance and storage space
Make sure that the algorithms and data structures we
design are capable of dealing with memory errors
23
Faulty- Memory Model [Finocchi, I. 04]
•  Memory fault = the correct data stored in a memory
location gets altered (destructive faults)
•  Faults can appear
at any time
in any memory location
simultaneously
•  Assumptions: 
–  Only O(1) words of reliable memory (safe memory)
–  Corrupted values indistinguishable from correct ones
Wish to produce correct output on
uncorrupted data (in an adversarial model)
•  Even recursion may be problematic in this model.
24
Terminology
δ = 
 upper bound known on the number of memory

 errors (may be function of n)
α = actual number of memory errors

 (happen during specific execution)
Note: typically α ≤ δ
All the algorithms / data structure described here need to
know δ in advance
25
Other Faulty Models
Design of fault-tolerant alg’s received attention for 50+ years
Liar Model [Ulam 77, Renyi 76,…]

 Comparison questions answered by a possibly lying
adversary. Can exploit query replication strategies.
Fault-tolerant sorting networks [Assaf Upfal 91, Yao Yao 85,…]

 Comparators can be faulty. Exploit substantial data
replication using fault-free data replicators.
Parallel Computations [Huang et al 84, Chlebus et al 94, …]

 Faults on parallel/distributed architectures: PRAM or
DMM simulations (rely on fault-detection mechanisms) 
26
Other Faulty Models
  Robustness in Computational Geometry [Schirra 00, …] 
  Faults from unreliable computation (geometric precision)
rather than from memory errors
  Noisy / Unreliable Computation [Bravermann Mossel 08] 
  Faults (with given probability) from unreliable primitives
(e.g., comparisons) rather than from memory errors
  Memory Checkers [Blum et al 93, Blum et al 95, …] 
  Programs not reliable objects: self-testing and self-correction.
Essential error detection and error correction mechanisms. 
  ………………………………………
27
Outline of the Talk
1.  Motivation and Model
2.  Resilient Algorithms:
•  Sorting and Searching
3.  Resilient Data Structures
•  Priority Queues
•  Dictionaries
4.  Conclusions and Open Problems
28
Resilient Sorting
We are given a set of n keys that need to be sorted
Q1. Can sort efficiently correct values in presence of 

 memory errors? 
Q2. How many memory errors can tolerate in the worst

 case if we wish to maintain optimal time and space?
Value of some keys may get arbitrarily corrupted
We cannot tell which is faithful and which is corrupted
29
Terminology
•  Faithfully ordered sequence = ordered except for
corrupted keys
•  Resilient sorting algorithm = produces a faithfully
ordered sequence (i.e., wish to sort correctly all the
uncorrupted keys)
•  Faithful key = never corrupted
1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 ordered
Faithfully
80
•  Faulty key = corrupted
30
Trivially Resilient
Resilient variable: consists of (2δ+1) copies
x1, x2, …, x2δ+1 of a standard variable x
Value of resilient variable given by majority of its copies:
•  cannot be corrupted by faults
•  can be computed in linear time and constant space
[Boyer Moore 91]
Trivially-resilient algorithms and data structures
have Θ(δ) multiplicative overheads in terms of
time and space
Note: Trivially-resilient does more than ECC
(SECDED, Chip-Kill, ….)
31
Trivially Resilient Sorting
Can trivially sort in O(δ n log n) time during δ
memory errors
Trivially Resilient Sorting
O(n log n) sorting algorithm able to tolerate only
O (1) memory errors
32
Resilient Sorting
Comparison-based sorting algorithm that takes
O(n log n + δ2) time to run during δ memory errors
O(n log n) sorting algorithm able to tolerate up
to O ((n log n)1/2) memory errors

 Any comparison-based resilient O(n log n) sorting

 algorithm can tolerate the corruption of at most

 O ((n log n)1/2) keys
Upper Bound [Finocchi, Grandoni, I. 05]: 
Lower Bound [Finocchi, I. 04]: 
33
Resilient Sorting (cont.)
Randomized integer sorting algorithm that takes
O(n + δ2) time to run during δ memory errors
O(n) randomized integer sorting algorithm able
to tolerate up to O(n1/2) memory errors
Integer Sorting [Finocchi, Grandoni, I. 05]: 
34
search(5) = false
Resilient Binary Search
2
 3
 4
 5
 8
 9
 13
 20
 26
1
 7
80
 10
Wish to get correct answers at least on correct keys:
search(s) either finds a key equal to s,
or determines that no correct key is equal to s
If only faulty keys are equal to s, answer uninteresting
(cannot hope to get trustworthy answer) 
35
Trivially Resilient Binary Search
Can search in O(δ log n) time during δ memory errors
Trivially Resilient Binary Search
36
Resilient Searching

 Randomized algorithm with O(log n + δ) expected time
[Finocchi, Grandoni, I. 05] 

 

 Deterministic algorithm with O(log n + δ) time
[Brodal et al. 07] 
Upper Bounds : 
Lower Bounds : 

 Ω(log n + δ) lower bound (deterministic)
[Finocchi, I. 04]

 Ω(log n + δ) lower bound on expected time 
[Finocchi, Grandoni, I. 05] 
37
Resilient Dynamic Programming

 Running time O(nd + δd+1) and space usage O(nd + nδ)
Can tolerate up to δ = O(nd/(d+1)) memory errors
[Caminiti et al. 10] 
d-dim. Dynamic Programming
38
Outline of the Talk
1.  Motivation and Model
2.  Resilient Algorithms:
•  Sorting and Searching
3.  Resilient Data Structures
•  Priority Queues
•  Dictionaries
4.  Conclusions and Open Problems
39
Resilient Data Structures
Algorithms affected by errors during execution
Data structures affected by errors in lifetime

 Data structures more vulnerable to memory errors
than algorithms:
40
Resilient Priority Queues
Maintain a set of elements under insert and deletemin
insert adds an element
deletemin deletes and returns either the minimum
uncorrupted value or a corrupted value
Consistent with resilient sorting
41
Resilient Priority Queues
Upper Bound : 

 Both insert and deletemin can be implemented
in O(log n + δ) time
[Jorgensen et al. 07]

 (based on cache-oblivious priority queues) 

 Lower Bound : 

 A resilient priority queue with n > δ elements must
use Ω(log n + δ) comparisons to answer an insert
followed by a deletemin
[Jorgensen et al. 07] 
42
Resilient Dictionaries
Maintain a set of elements under insert, delete
and search
insert and delete as usual, search as in
resilient searching:
Again, consistent with resilient sorting
search(s) either finds a key equal to s,
or determines that no correct key is equal to s
43
Resilient Dictionaries
Randomized resilient dictionary implements each
operation in O(log n + δ) time
[Brodal et al. 07] 
More complicated deterministic resilient dictionary
implements each operation in O(log n + δ) time
[Brodal et al. 07] 
44
Resilient Dictionaries

 Pointer-based data structures

 Faults on pointers likely to be more problematic
than faults on keys
Randomized resilient dictionaries of Brodal et al.
built on top of traditional (non-resilient) dictionaries

 Our implementation built on top of AVL trees
45
Outline of the Talk
1.  Motivation and Model
2.  Resilient Algorithms:
•  Sorting and Searching
3.  Resilient Data Structures
•  Priority Queues
•  Dictionaries
4.  Conclusions and Open Problems
46
Concluding Remarks
•  Need of reliable computation in the presence of
memory errors
•  Investigated basic algorithms and data structures in
the faulty memory model: do not wish to detect /
correct errors, only produce correct output on
correct data
•  Tight upper and lower bounds in this model
•  After first tests, resilient implementations of
algorithms and data structures look promising
47
Future Work and Open Problems
•  More (faster) implementations, engineering and
experimental analysis?
•  Resilient graph algorithms? 
•  Lower bounds for resilient integer sorting? 
•  Better faulty memory model?
•  Resilient algorithms oblivious to δ?
•  Full repertoire for resilient priority queues
(delete, decreasekey, increasekey)? 
48
Thank You!
49
My memory’s terrible these days…

Contenu connexe

Similaire à Resilient algorithms data structures intro by Giuseppe F.Italiano

Debugging Complex Systems - Erlang Factory SF 2015
Debugging Complex Systems - Erlang Factory SF 2015Debugging Complex Systems - Erlang Factory SF 2015
Debugging Complex Systems - Erlang Factory SF 2015lpgauth
 
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel....NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...Karel Zikmund
 
DieHard: Probabilistic Memory Safety for Unsafe Languages
DieHard: Probabilistic Memory Safety for Unsafe LanguagesDieHard: Probabilistic Memory Safety for Unsafe Languages
DieHard: Probabilistic Memory Safety for Unsafe LanguagesEmery Berger
 
Major Elements Of Memory Management
Major Elements Of Memory ManagementMajor Elements Of Memory Management
Major Elements Of Memory ManagementApril Bell
 
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATA
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATAMATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATA
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATAVLSICS Design
 
Cost-effective software reliability through autonomic tuning of system resources
Cost-effective software reliability through autonomic tuning of system resourcesCost-effective software reliability through autonomic tuning of system resources
Cost-effective software reliability through autonomic tuning of system resourcesVincenzo De Florio
 
Debugging multiplayer games
Debugging multiplayer gamesDebugging multiplayer games
Debugging multiplayer gamesMaciej Siniło
 
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...Automatic Differentiation and SciML in Reality: What can go wrong, and what t...
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...Chris Rackauckas
 
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02Yazeed Khalid
 
Debugging With Id
Debugging With IdDebugging With Id
Debugging With Idguest215c4e
 
Good ideas that we forgot
Good ideas that we forgot   Good ideas that we forgot
Good ideas that we forgot J On The Beach
 
Machine Learning: Past, Present and Future - by Tom Dietterich
Machine Learning: Past, Present and Future - by Tom DietterichMachine Learning: Past, Present and Future - by Tom Dietterich
Machine Learning: Past, Present and Future - by Tom DietterichBigML, Inc
 
M.tech admission in india
M.tech admission in indiaM.tech admission in india
M.tech admission in indiaEdhole.com
 
M.tech admission in india
M.tech admission in indiaM.tech admission in india
M.tech admission in indiaEdhole.com
 
International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)IJERD Editor
 

Similaire à Resilient algorithms data structures intro by Giuseppe F.Italiano (20)

Debugging Complex Systems - Erlang Factory SF 2015
Debugging Complex Systems - Erlang Factory SF 2015Debugging Complex Systems - Erlang Factory SF 2015
Debugging Complex Systems - Erlang Factory SF 2015
 
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel....NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...
.NET Core Summer event 2019 in Linz, AT - War stories from .NET team -- Karel...
 
DieHard: Probabilistic Memory Safety for Unsafe Languages
DieHard: Probabilistic Memory Safety for Unsafe LanguagesDieHard: Probabilistic Memory Safety for Unsafe Languages
DieHard: Probabilistic Memory Safety for Unsafe Languages
 
Major Elements Of Memory Management
Major Elements Of Memory ManagementMajor Elements Of Memory Management
Major Elements Of Memory Management
 
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATA
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATAMATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATA
MATRIX CODE BASED MULTIPLE ERROR CORRECTION TECHNIQUE FOR N-BIT MEMORY DATA
 
Computer awareness
Computer awarenessComputer awareness
Computer awareness
 
Cost-effective software reliability through autonomic tuning of system resources
Cost-effective software reliability through autonomic tuning of system resourcesCost-effective software reliability through autonomic tuning of system resources
Cost-effective software reliability through autonomic tuning of system resources
 
Three things that rowhammer taught me by Halvar Flake
Three things that rowhammer taught me by Halvar FlakeThree things that rowhammer taught me by Halvar Flake
Three things that rowhammer taught me by Halvar Flake
 
Debugging multiplayer games
Debugging multiplayer gamesDebugging multiplayer games
Debugging multiplayer games
 
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...Automatic Differentiation and SciML in Reality: What can go wrong, and what t...
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...
 
Acm icpc-briefing-prof-nbv
Acm icpc-briefing-prof-nbvAcm icpc-briefing-prof-nbv
Acm icpc-briefing-prof-nbv
 
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02
Ibpscomputerknowledgepaperswithanswers 130123104735-phpapp02
 
Computer question
Computer questionComputer question
Computer question
 
Debugging With Id
Debugging With IdDebugging With Id
Debugging With Id
 
Good ideas that we forgot
Good ideas that we forgot   Good ideas that we forgot
Good ideas that we forgot
 
Machine Learning: Past, Present and Future - by Tom Dietterich
Machine Learning: Past, Present and Future - by Tom DietterichMachine Learning: Past, Present and Future - by Tom Dietterich
Machine Learning: Past, Present and Future - by Tom Dietterich
 
M.tech admission in india
M.tech admission in indiaM.tech admission in india
M.tech admission in india
 
M.tech admission in india
M.tech admission in indiaM.tech admission in india
M.tech admission in india
 
ECC memory : Notes
ECC memory : NotesECC memory : Notes
ECC memory : Notes
 
International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)
 

Plus de Stephen Larroque

Brain-Computer Interface and States of Vigilance
Brain-Computer Interface and States of VigilanceBrain-Computer Interface and States of Vigilance
Brain-Computer Interface and States of VigilanceStephen Larroque
 
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysis
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysisA (quick) introduction to Magnetic Resonance Imagery preprocessing and analysis
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysisStephen Larroque
 
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousness
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousnessDiffusion Tensor Imaging (DTI) for the study of disorders of consciousness
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousnessStephen Larroque
 
Open Anti-Cheat System (OACS)
Open Anti-Cheat System (OACS)Open Anti-Cheat System (OACS)
Open Anti-Cheat System (OACS)Stephen Larroque
 
Recursive Neural Network summary
Recursive Neural Network summaryRecursive Neural Network summary
Recursive Neural Network summaryStephen Larroque
 
Pathway Evolution Algorithm in Netlogo
Pathway Evolution Algorithm in NetlogoPathway Evolution Algorithm in Netlogo
Pathway Evolution Algorithm in NetlogoStephen Larroque
 
Sma presentation-bani-hotel-larroque
Sma presentation-bani-hotel-larroqueSma presentation-bani-hotel-larroque
Sma presentation-bani-hotel-larroqueStephen Larroque
 
Birdsong variability is modulated by LMAN in anterior forebrain pathway
Birdsong variability is modulated by LMAN in anterior forebrain pathwayBirdsong variability is modulated by LMAN in anterior forebrain pathway
Birdsong variability is modulated by LMAN in anterior forebrain pathwayStephen Larroque
 
Glial cells and astrocytes: neural networks modulators?
Glial cells and astrocytes: neural networks modulators?Glial cells and astrocytes: neural networks modulators?
Glial cells and astrocytes: neural networks modulators?Stephen Larroque
 
Tagged network (colored clique network) COGNITIVE 2015 by Stephen Larroque
Tagged network (colored clique network) COGNITIVE 2015 by Stephen LarroqueTagged network (colored clique network) COGNITIVE 2015 by Stephen Larroque
Tagged network (colored clique network) COGNITIVE 2015 by Stephen LarroqueStephen Larroque
 
Simulation des raisonnements éthiques par logiques non-monotones
Simulation des raisonnements éthiques par logiques non-monotonesSimulation des raisonnements éthiques par logiques non-monotones
Simulation des raisonnements éthiques par logiques non-monotonesStephen Larroque
 
Financement de la recherche en france, europe et dans le monde en 2014
Financement de la recherche en france, europe et dans le monde en 2014Financement de la recherche en france, europe et dans le monde en 2014
Financement de la recherche en france, europe et dans le monde en 2014Stephen Larroque
 

Plus de Stephen Larroque (12)

Brain-Computer Interface and States of Vigilance
Brain-Computer Interface and States of VigilanceBrain-Computer Interface and States of Vigilance
Brain-Computer Interface and States of Vigilance
 
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysis
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysisA (quick) introduction to Magnetic Resonance Imagery preprocessing and analysis
A (quick) introduction to Magnetic Resonance Imagery preprocessing and analysis
 
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousness
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousnessDiffusion Tensor Imaging (DTI) for the study of disorders of consciousness
Diffusion Tensor Imaging (DTI) for the study of disorders of consciousness
 
Open Anti-Cheat System (OACS)
Open Anti-Cheat System (OACS)Open Anti-Cheat System (OACS)
Open Anti-Cheat System (OACS)
 
Recursive Neural Network summary
Recursive Neural Network summaryRecursive Neural Network summary
Recursive Neural Network summary
 
Pathway Evolution Algorithm in Netlogo
Pathway Evolution Algorithm in NetlogoPathway Evolution Algorithm in Netlogo
Pathway Evolution Algorithm in Netlogo
 
Sma presentation-bani-hotel-larroque
Sma presentation-bani-hotel-larroqueSma presentation-bani-hotel-larroque
Sma presentation-bani-hotel-larroque
 
Birdsong variability is modulated by LMAN in anterior forebrain pathway
Birdsong variability is modulated by LMAN in anterior forebrain pathwayBirdsong variability is modulated by LMAN in anterior forebrain pathway
Birdsong variability is modulated by LMAN in anterior forebrain pathway
 
Glial cells and astrocytes: neural networks modulators?
Glial cells and astrocytes: neural networks modulators?Glial cells and astrocytes: neural networks modulators?
Glial cells and astrocytes: neural networks modulators?
 
Tagged network (colored clique network) COGNITIVE 2015 by Stephen Larroque
Tagged network (colored clique network) COGNITIVE 2015 by Stephen LarroqueTagged network (colored clique network) COGNITIVE 2015 by Stephen Larroque
Tagged network (colored clique network) COGNITIVE 2015 by Stephen Larroque
 
Simulation des raisonnements éthiques par logiques non-monotones
Simulation des raisonnements éthiques par logiques non-monotonesSimulation des raisonnements éthiques par logiques non-monotones
Simulation des raisonnements éthiques par logiques non-monotones
 
Financement de la recherche en france, europe et dans le monde en 2014
Financement de la recherche en france, europe et dans le monde en 2014Financement de la recherche en france, europe et dans le monde en 2014
Financement de la recherche en france, europe et dans le monde en 2014
 

Dernier

Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...soginsider
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfodunowoeminence2019
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
Test of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxTest of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxHome
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxwendy cai
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchrohitcse52
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfRedhwan Qasem Shaddad
 
Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Bahzad5
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Modelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsModelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsYusuf Yıldız
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Akarthi keyan
 
ASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderjuancarlos286641
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Amil baba
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 

Dernier (20)

Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
Test of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxTest of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptx
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptx
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdf
 
Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Modelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovationsModelling Guide for Timber Structures - FPInnovations
Modelling Guide for Timber Structures - FPInnovations
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part A
 
ASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entender
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
計劃趕得上變化
計劃趕得上變化計劃趕得上變化
計劃趕得上變化
 

Resilient algorithms data structures intro by Giuseppe F.Italiano

  • 1. Resilient Algorithms and Data Structures Giuseppe F. Italiano Università di Roma “Tor Vergata”
  • 2. Some advertising first School on Graph Theory, Algorithms and Applications Erice (Sicily), September 25 - October 3, 2011 Please send your best students /postdocs /junior ….
  • 3. Memory Errors Common in Practice 3
  • 4. Outline of the Talk 1.  Motivation and Model 2.  Resilient Algorithms: •  Sorting and Searching 3.  Resilient Data Structures •  Priority Queues •  Dictionaries 4.  Conclusions and Open Problems 4
  • 5. Memory Errors Memory error: one or multiple bits read differently from how they were last written. Many possible causes: •  electrical or magnetic interference (cosmic rays) •  hardware problems (bit permanently damaged) •  corruption in data path between memories and processing units Errors in DRAM devices concern for a long time [May & Woods 79, Ziegler et al 79, Chen & Hsiao 84, Normand 96, O’Gorman et al 96, Mukherjee et al 05, … ] 5
  • 6. Memory Errors Soft Errors: Randomly corrupt bits, but do not leave any physical damage --- cosmic rays Hard Errors: Corrupt bits in a repeatable manner because of a physical defect (e.g., stuck bits) --- hardware problems 6
  • 7. Error Correcting Codes (ECC) Error correcting codes (ECC) allow detection and correction of one or multiple bit errors Typical ECC is SECDED (i.e., single error correct, double error detect) Chip-Kill can correct up to 4 adjacent bits at once ECC has several overheads in terms of performance (33%), size (20%) and money (10%). ECC memory chips are mostly used in memory systems for server machines rather than for client computers 7
  • 8. Impact of Memory Errors Consequence of a memory error is system dependent 1. Correctable errors : fixed by ECC 2. Uncorrectable errors : 2.1. Detected : Explicit failure (e.g., a machine reboot) 2.2. Undetected : 2.2.1. Induced failure (e.g., a kernel panic) 2.2.2. Unnoticed (but application corrupted, e.g., segmentation fault, file not found, file not readable, … ) 8
  • 9. Impact of Memory Errors 9
  • 10. How Common are Memory Errors? 10
  • 11. How Common are Memory Errors? 11
  • 12. How Common are Memory Errors? [Schroeder et al 2009] experiments 2.5 years (Jan 06 – Jun 08) on Google fleet (104 machines, ECC memory) Memory errors are NOT rare events! 12
  • 13. Memory Errors Not all machines (clients) have ECC memory chips. Increased demand for larger capacities at low cost just makes the problem more serious – large clusters of inexpensive memories Need of reliable computation in the presence of memory faults 13
  • 14. Memory Errors •  Memory errors can cause security vulnerabilities: Fault-based cryptanalysis [Boneh et al 97, Xu et al 01, Bloemer & Seifert 03] Attacking Java Virtual Machines [Govindavajhala & Appel 03] Breaking smart cards [Skorobogatov & Anderson 02, Bar-El et al 06] • Avionics and space electronic systems: Amount of cosmic rays increase with altitude (soft errors) Other scenarios in which memory errors have impact (and seem to be modeled in an adversarial setting): 14
  • 15. Memory Errors in Space 15
  • 16. Memory Errors in Space 16
  • 17. Memory Errors in Space 17
  • 18. Recap on Memory Errors 1. Memory errors are NOT rare: even a small cluster of computers with few GB per node can experience one bit error every few minutes. 18 I know my PIN number: it’s my name I can’t remember…
  • 19. Memory Errors Mem. size Mean Time Between Failures 512 MB 2.92 hours 1 GB 1.46 hours 16 GB 5.48 minutes 64 GB 1.37 minutes 1 TB 5.13 seconds In the field study, Google researchers observed mean error rates of 2,000 – 6,000 per GB per year (25,000 – 75,000 FIT/Mbit) 19
  • 20. Recap on Memory Errors 2. Memory errors can be harmful: uncorrectable memory errors cause some catastrophic event (reboot, kernel panic, data corruption, …) 20 I’m thinking of getting back into crime, Luigi. Legitimate business is too corrupt…
  • 21. A small example Classical algorithms may not be correct in the presence of (even very few) memory errors 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 A B Out An example: merging two ordered lists Θ(n) Θ(n) Θ(n2) inversions ... 11 12 20 13 80 ... 2 3 4 9 10 80 21
  • 22. Recap on Memory Errors 3. ECC may not be available (or may not be enough): No ECC in inexpensive memories. ECC does not guarantee complete fault coverage; expensive; system halt upon detection of uncorrectable errors; service disruption; etc… etc… 22
  • 23. Resilient Algorithms and Data Structures Resilient Algorithms and Data Structures: Capable of tolerating memory errors on data (even throughout their execution) without sacrificing correctness, performance and storage space Make sure that the algorithms and data structures we design are capable of dealing with memory errors 23
  • 24. Faulty- Memory Model [Finocchi, I. 04] •  Memory fault = the correct data stored in a memory location gets altered (destructive faults) •  Faults can appear at any time in any memory location simultaneously •  Assumptions: –  Only O(1) words of reliable memory (safe memory) –  Corrupted values indistinguishable from correct ones Wish to produce correct output on uncorrupted data (in an adversarial model) •  Even recursion may be problematic in this model. 24
  • 25. Terminology δ = upper bound known on the number of memory errors (may be function of n) α = actual number of memory errors (happen during specific execution) Note: typically α ≤ δ All the algorithms / data structure described here need to know δ in advance 25
  • 26. Other Faulty Models Design of fault-tolerant alg’s received attention for 50+ years Liar Model [Ulam 77, Renyi 76,…] Comparison questions answered by a possibly lying adversary. Can exploit query replication strategies. Fault-tolerant sorting networks [Assaf Upfal 91, Yao Yao 85,…] Comparators can be faulty. Exploit substantial data replication using fault-free data replicators. Parallel Computations [Huang et al 84, Chlebus et al 94, …] Faults on parallel/distributed architectures: PRAM or DMM simulations (rely on fault-detection mechanisms) 26
  • 27. Other Faulty Models   Robustness in Computational Geometry [Schirra 00, …]   Faults from unreliable computation (geometric precision) rather than from memory errors   Noisy / Unreliable Computation [Bravermann Mossel 08]   Faults (with given probability) from unreliable primitives (e.g., comparisons) rather than from memory errors   Memory Checkers [Blum et al 93, Blum et al 95, …]   Programs not reliable objects: self-testing and self-correction. Essential error detection and error correction mechanisms.   ……………………………………… 27
  • 28. Outline of the Talk 1.  Motivation and Model 2.  Resilient Algorithms: •  Sorting and Searching 3.  Resilient Data Structures •  Priority Queues •  Dictionaries 4.  Conclusions and Open Problems 28
  • 29. Resilient Sorting We are given a set of n keys that need to be sorted Q1. Can sort efficiently correct values in presence of memory errors? Q2. How many memory errors can tolerate in the worst case if we wish to maintain optimal time and space? Value of some keys may get arbitrarily corrupted We cannot tell which is faithful and which is corrupted 29
  • 30. Terminology •  Faithfully ordered sequence = ordered except for corrupted keys •  Resilient sorting algorithm = produces a faithfully ordered sequence (i.e., wish to sort correctly all the uncorrupted keys) •  Faithful key = never corrupted 1 2 3 4 5 6 7 8 9 10 ordered Faithfully 80 •  Faulty key = corrupted 30
  • 31. Trivially Resilient Resilient variable: consists of (2δ+1) copies x1, x2, …, x2δ+1 of a standard variable x Value of resilient variable given by majority of its copies: •  cannot be corrupted by faults •  can be computed in linear time and constant space [Boyer Moore 91] Trivially-resilient algorithms and data structures have Θ(δ) multiplicative overheads in terms of time and space Note: Trivially-resilient does more than ECC (SECDED, Chip-Kill, ….) 31
  • 32. Trivially Resilient Sorting Can trivially sort in O(δ n log n) time during δ memory errors Trivially Resilient Sorting O(n log n) sorting algorithm able to tolerate only O (1) memory errors 32
  • 33. Resilient Sorting Comparison-based sorting algorithm that takes O(n log n + δ2) time to run during δ memory errors O(n log n) sorting algorithm able to tolerate up to O ((n log n)1/2) memory errors Any comparison-based resilient O(n log n) sorting algorithm can tolerate the corruption of at most O ((n log n)1/2) keys Upper Bound [Finocchi, Grandoni, I. 05]: Lower Bound [Finocchi, I. 04]: 33
  • 34. Resilient Sorting (cont.) Randomized integer sorting algorithm that takes O(n + δ2) time to run during δ memory errors O(n) randomized integer sorting algorithm able to tolerate up to O(n1/2) memory errors Integer Sorting [Finocchi, Grandoni, I. 05]: 34
  • 35. search(5) = false Resilient Binary Search 2 3 4 5 8 9 13 20 26 1 7 80 10 Wish to get correct answers at least on correct keys: search(s) either finds a key equal to s, or determines that no correct key is equal to s If only faulty keys are equal to s, answer uninteresting (cannot hope to get trustworthy answer) 35
  • 36. Trivially Resilient Binary Search Can search in O(δ log n) time during δ memory errors Trivially Resilient Binary Search 36
  • 37. Resilient Searching Randomized algorithm with O(log n + δ) expected time [Finocchi, Grandoni, I. 05] Deterministic algorithm with O(log n + δ) time [Brodal et al. 07] Upper Bounds : Lower Bounds : Ω(log n + δ) lower bound (deterministic) [Finocchi, I. 04] Ω(log n + δ) lower bound on expected time [Finocchi, Grandoni, I. 05] 37
  • 38. Resilient Dynamic Programming Running time O(nd + δd+1) and space usage O(nd + nδ) Can tolerate up to δ = O(nd/(d+1)) memory errors [Caminiti et al. 10] d-dim. Dynamic Programming 38
  • 39. Outline of the Talk 1.  Motivation and Model 2.  Resilient Algorithms: •  Sorting and Searching 3.  Resilient Data Structures •  Priority Queues •  Dictionaries 4.  Conclusions and Open Problems 39
  • 40. Resilient Data Structures Algorithms affected by errors during execution Data structures affected by errors in lifetime Data structures more vulnerable to memory errors than algorithms: 40
  • 41. Resilient Priority Queues Maintain a set of elements under insert and deletemin insert adds an element deletemin deletes and returns either the minimum uncorrupted value or a corrupted value Consistent with resilient sorting 41
  • 42. Resilient Priority Queues Upper Bound : Both insert and deletemin can be implemented in O(log n + δ) time [Jorgensen et al. 07] (based on cache-oblivious priority queues) Lower Bound : A resilient priority queue with n > δ elements must use Ω(log n + δ) comparisons to answer an insert followed by a deletemin [Jorgensen et al. 07] 42
  • 43. Resilient Dictionaries Maintain a set of elements under insert, delete and search insert and delete as usual, search as in resilient searching: Again, consistent with resilient sorting search(s) either finds a key equal to s, or determines that no correct key is equal to s 43
  • 44. Resilient Dictionaries Randomized resilient dictionary implements each operation in O(log n + δ) time [Brodal et al. 07] More complicated deterministic resilient dictionary implements each operation in O(log n + δ) time [Brodal et al. 07] 44
  • 45. Resilient Dictionaries Pointer-based data structures Faults on pointers likely to be more problematic than faults on keys Randomized resilient dictionaries of Brodal et al. built on top of traditional (non-resilient) dictionaries Our implementation built on top of AVL trees 45
  • 46. Outline of the Talk 1.  Motivation and Model 2.  Resilient Algorithms: •  Sorting and Searching 3.  Resilient Data Structures •  Priority Queues •  Dictionaries 4.  Conclusions and Open Problems 46
  • 47. Concluding Remarks •  Need of reliable computation in the presence of memory errors •  Investigated basic algorithms and data structures in the faulty memory model: do not wish to detect / correct errors, only produce correct output on correct data •  Tight upper and lower bounds in this model •  After first tests, resilient implementations of algorithms and data structures look promising 47
  • 48. Future Work and Open Problems •  More (faster) implementations, engineering and experimental analysis? •  Resilient graph algorithms? •  Lower bounds for resilient integer sorting? •  Better faulty memory model? •  Resilient algorithms oblivious to δ? •  Full repertoire for resilient priority queues (delete, decreasekey, increasekey)? 48
  • 49. Thank You! 49 My memory’s terrible these days…