The Presentation given by Brad Childs on April 25, 2013 at the Austin Hadoop Users Group (AHUG): Fun with Hadoop File Systems

1. FUN WITH
HADOOP
FILE SYSTEMS
© Bradley Childs / bdc@redhat.com

2. HISTORY
• Distributed file systems have been around for a long time
• DFS battle optimizing the CAP theorem
• Hadoops DFS implementation is called HDFS
• Wide adoption of hadoop, users forced to use HDFS as the
only alternative
• HDFS has technical trade offs and limitations

3. HDFS ARCHITECTURE
client
Name
Node
client
Data
Node
client
Data
Node
client
Data
Node
Store & Compute

4. HDFS ISSUES
Handy
• Locking around metadata operations permitted by single name
node
• File locking permitted by single name node
Frustrating
• Difficult to get data in and out (ingest)
• Name Node is single point of failure
• Name Node is system bottleneck

5. GLUSTER FILE
SYSTEM
Gluster is an open source multi purpose DFS
Features:
• Data Striping
• Global elastic hashing for file placement
• Basic and GEO Replication
• Full POSIX Compliant Interface
• Flexible architecture
• Supports Storage Resident Apps – Compute and Data on
same machine
More Info: www.gluster.org

6. GLUSTER
ARCHITECTURE
client
Trusted Peers
client
Data
Brick
client
Data
Brick
client
Data
Brick
VolumeVolume
Store & Compute

7. HCFS
HCFS: Hadoop Compatible File System
• Implementing the o.a.h.fs.FileSystem interface not enough for
existing hadoop jobs to run on a different file system
• HDFS architecture created semantics and assumptions
• HCFS defines these semantics so any file system can replace
HDFS without fear of compatibility
• Open ongoing effort to define file system semantics decoupled
from architecture
JIRA:
issues.apache.org/jira/browse/HADOOP-9371

8. COMMON FILESYSTEM
ATTRIBUTES
• Hierarchical structure of directories containing directories and
files
• File contain between 0 and MAX_SIZE data
• Directories contain 0 or more files or directories
• Directories have no data, only child elements

9. NETWORK
ASSUMPTIONS
• The final state of a file system after a network failure is
undefined
• The immediate consistency state of a file system after a
network failure is undefined
• If a network failure can be reported to the client, the failure
MUST be an instance of IOException

10. NETWORK FAILURE
• Any operation with a file system MAY signal an error by
throwing an instance of IOException
• File system operations MUST NOT throw RuntimeException
exceptions on the failure of a remote operations, authentication
or other operational problems
• Stream read operations MAY fail if the read channel has been
idle for a file system specific period of time
• Stream write operations MAY fail if the write channel has been
idle for a file system specific period of time
• Network failures MAY be raised in the Stream close() operation

11. ATOMICITY
• Rename of a file MUST be atomic
• Rename of a directory SHOULD be atomic
• Delete of a file MUST be atomic
• Delete of an empty directory MUST be atomic
• Recursive directory deletion MAY be atomic. Although HDFS
offers atomic recursive directory deletion, none of the other file
systems that Hadoop supports offers such a guarantee -
including the local file systems
• mkdir() SHOULD be atomic
• mkdirs() MAY be atomic. [It is currently atomic on HDFS, but
this is not the case for most other filesystems -and cannot be
guaranteed for future versions of HDFS]

12. CONCURRENCY
• The data added to a file during a write or append MAY be visible
while the write operation is in progress
• If a client opens a file for a read() operation while another read()
operation is in progress, the second operation MUST succeed.
Both clients MUST have a consistent view of the same data
• If a file is deleted while a read() operation is in progress, the read()
operation MAY complete successfully. Implementations MAY
cause read() operations to fail with an IOException instead
• Multiple writers MAY open a file for writing. If this occurs, the
outcome is undefined
• Undefined: action of delete() while a write or append operation is
in progress

13. CONSISTENCY
The consistency model of a Hadoop file system is one-copy-update-semantics; partially
generally that of a traditional Posix file system.
• Create: once the close() operation on an output stream writing a newly created file has
completed, in-cluster operations querying the file metadata and contents MUST
immediately see the file and its data
• Update: Once the close() operation on an output stream writing a newly created file has
completed, in-cluster operations querying the file metadata and contents MUST
immediately see the new data
• Delete: once a delete() operation is on a file has completed, listStatus() , open() ,
rename() and append() operations MUST fail
• When file is deleted then overwritten, listStatus() , open() , rename() and append()
operations MUST succeed: the file is visible
• Rename: after a rename has completed, operations against the new path MUST succeed;
operations against the old path MUST fail
• The consistency semantics out of cluster client MUST be the same as in-cluster clients: All
clients calling read() on a closed file MUST see the same metadata and data until it is
changed from a create() , append() , rename() and append() operation

14. REFERENCES
Apache HCFS Wiki:
wiki.apache.org/hadoop/HCFS
Apache file Systems semantics JIRA:
issues.apache.org/jira/browse/HADOOP-9371
Some of this text is taken from the working draft linked in above Jira, credit Steve Loughran et al.
The opinions expressed do not necessarily represent those of RedHat Inc. or any of its affiliates.
© Bradley Childs / bdc@redhat.com