google file systems

1. Google File System

2. Overview
 NFS
 Introduction-Design Overview
 Architecture
 System Interactions
 Fault tolerance
 Conclusion

3. NFS
 Conventional Client/Server model
 Low performance
 Security Issues

4. Introduction
Need For GFS:
Large Data Files
Scalability
Reliability
Automation
Replication of data
Fault Tolerance

5. Design Overview:
 Component’s Monitoring
 Reading and writing of data
 Storing of huge data
Assumptions:
Interface:
 Not POSIX compliant
 Additional operations
o Snapshot
o Record append

6. Architecture:
 Single Master
 Multiple Chunk Servers
 Multiple clients
 Stores 64 bit file chunks

8. Single Master , Chunk size & Meta data
Single Master:
Minimal Master Load.
Fixed chunk Size.
The master also predicatively provide chunk
locations immediately following those requested by
unique id.

9. Single Master , Chunk size & Meta data
Chunk Size :
 64 MB size.
Read and write operations on same chunk.
Reduces network overhead and size of metadata in
the master.

10. Metadata :
 Types of Metadata:
o File and chunk namespaces
o Mapping from files to chunks
o Location of each chunks replicas

11.  Chunk Locations:
o Master polls chunk server for the information.
o Client request data from chunk server.
 Operation Log:
o Keeps track of activities.
o It is central to GFS.
o It stores on multiple remote locations.

12. System Interactions:
 Write Operations
o Master picks one replica as primary.
o Primary defines serial order for
mutations.
o Replicas follow same serial order.
o Minimize management overhead at
the master.

13.  Atomic Record Appends:
o GFS offers Record Append .
o Clients on di erent machines append to the sameff
file concurrently.
o The data is written at least once as an atomic unit.

14. Snapshot:
o It creates quick copy of files or a directory .
o Master revokes lease for that file
o Duplicate metadata
o On first write to a chunk after the snapshot operation
o All chunk servers create new chunk
o Data can be copied locally

15. Fault Tolerance
High availability:
o Fast recovery.
o Chunk replication.
o Shadow Masters.

16. Conclusion
GFS meets Google storage requirements:
Incremental growth
Regular check of component failure
Data optimization from special operations
Simple architecture
Fault Tolerance

17. THANK YOU!!!