SlideShare a Scribd company logo
1 of 29
Distributed Computing Seminar Lecture 3: Distributed Filesystems Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Google, Inc.  Summer 2007 Except as otherwise noted, the content of this presentation is © Copyright University of Washington and licensed under the Creative Commons Attribution 2.5 License.
Outline ,[object Object],[object Object],[object Object]
Filesystems Overview ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Distributed Filesystems ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
NFS  ,[object Object],[object Object],[object Object]
NFS Protocol ,[object Object],[object Object],[object Object],[object Object]
Server-side Implementation ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS Locking ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS Client Caching ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS: Tradeoffs ,[object Object],[object Object],[object Object],[object Object]
The Google File System Sanjay Ghemawat, Howard Gobioff,  and Shun-Tak Leung SOSP 2003 (These slides by Alex Moshchuk, University of Washington – used with permission)
Motivation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Assumptions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
GFS Design Decisions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
GFS Architecture ,[object Object],[object Object],… Can anyone see a potential weakness in this design?
Single master ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Metadata (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Metadata (2/2) ,[object Object],[object Object],[object Object],[object Object]
Mutations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Atomic record append ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Relaxed consistency model (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Relaxed consistency model (2/2) ,[object Object],[object Object],[object Object],[object Object]
Master’s responsibilities (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Master’s responsibilities (2/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Fault Tolerance ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Performance
Deployment in Google ,[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Next Time... ,[object Object],[object Object]

More Related Content

What's hot

dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood
Susan Wu
 
Porting dmtcp mac_slides
Porting dmtcp mac_slidesPorting dmtcp mac_slides
Porting dmtcp mac_slides
yuliang_neu
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
Ravindra Raju Kolahalam
 

What's hot (20)

dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood
 
20150207 howes-gpgpu8-dark secrets
20150207 howes-gpgpu8-dark secrets20150207 howes-gpgpu8-dark secrets
20150207 howes-gpgpu8-dark secrets
 
.ppt
.ppt.ppt
.ppt
 
Interprocess communication (IPC) IN O.S
Interprocess communication (IPC) IN O.SInterprocess communication (IPC) IN O.S
Interprocess communication (IPC) IN O.S
 
IPC
IPCIPC
IPC
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory Systems
 
PThreads Vs Win32 Threads
PThreads  Vs  Win32 ThreadsPThreads  Vs  Win32 Threads
PThreads Vs Win32 Threads
 
ITFT_Inter process communication
ITFT_Inter process communicationITFT_Inter process communication
ITFT_Inter process communication
 
WiMAX implementation in ns3
WiMAX implementation in ns3WiMAX implementation in ns3
WiMAX implementation in ns3
 
IPC
IPCIPC
IPC
 
NS-3
NS-3 NS-3
NS-3
 
Porting dmtcp mac_slides
Porting dmtcp mac_slidesPorting dmtcp mac_slides
Porting dmtcp mac_slides
 
Capturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspaceCapturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspace
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
 
INET for Starters
INET for StartersINET for Starters
INET for Starters
 
Multiple processor systems
Multiple processor systemsMultiple processor systems
Multiple processor systems
 
Threads (operating System)
Threads (operating System)Threads (operating System)
Threads (operating System)
 
Chap 4
Chap 4Chap 4
Chap 4
 
Lecture 5 inter process communication
Lecture 5 inter process communicationLecture 5 inter process communication
Lecture 5 inter process communication
 

Similar to Distributed computing seminar lecture 3 - distributed file systems

Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
Sri Prasanna
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
Sri Prasanna
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
Ntu
 
Dfs (Distributed computing)
Dfs (Distributed computing)Dfs (Distributed computing)
Dfs (Distributed computing)
Sri Prasanna
 
Distributed Filesystems Review
Distributed Filesystems ReviewDistributed Filesystems Review
Distributed Filesystems Review
Schubert Zhang
 
Ch16 OS
Ch16 OSCh16 OS
Ch16 OS
C.U
 

Similar to Distributed computing seminar lecture 3 - distributed file systems (20)

Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
 
tittle
tittletittle
tittle
 
Distributed File Systems
Distributed File SystemsDistributed File Systems
Distributed File Systems
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
 
Google
GoogleGoogle
Google
 
Dfs (Distributed computing)
Dfs (Distributed computing)Dfs (Distributed computing)
Dfs (Distributed computing)
 
CS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMSCS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMS
 
Gfs介绍
Gfs介绍Gfs介绍
Gfs介绍
 
Kosmos Filesystem
Kosmos FilesystemKosmos Filesystem
Kosmos Filesystem
 
GOOGLE FILE SYSTEM
GOOGLE FILE SYSTEMGOOGLE FILE SYSTEM
GOOGLE FILE SYSTEM
 
Distributed File System.ppt
Distributed File System.pptDistributed File System.ppt
Distributed File System.ppt
 
Distributed Filesystems Review
Distributed Filesystems ReviewDistributed Filesystems Review
Distributed Filesystems Review
 
Advance google file system
Advance google file systemAdvance google file system
Advance google file system
 
Hadoop Distributed File System for Big Data Analytics
Hadoop Distributed File System for Big Data AnalyticsHadoop Distributed File System for Big Data Analytics
Hadoop Distributed File System for Big Data Analytics
 
Hadoop
HadoopHadoop
Hadoop
 
Chapter 17 - Distributed File Systems
Chapter 17 - Distributed File SystemsChapter 17 - Distributed File Systems
Chapter 17 - Distributed File Systems
 
OSCh16
OSCh16OSCh16
OSCh16
 
Ch16 OS
Ch16 OSCh16 OS
Ch16 OS
 
OS_Ch16
OS_Ch16OS_Ch16
OS_Ch16
 

Recently uploaded

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 

Distributed computing seminar lecture 3 - distributed file systems

  • 1. Distributed Computing Seminar Lecture 3: Distributed Filesystems Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Google, Inc. Summer 2007 Except as otherwise noted, the content of this presentation is © Copyright University of Washington and licensed under the Creative Commons Attribution 2.5 License.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11. The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung SOSP 2003 (These slides by Alex Moshchuk, University of Washington – used with permission)
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 27.
  • 28.
  • 29.