We adopt the basically structure of hdfs raid from Facebook, and we have migrated these feature to our cluster both in hadoop 1.x(0.20.203) and hadoop 2.x(hadoop 2.4) in Sina. We made some change during our development. Here is the theory of Rc code and the structure of hfs raid, besides we compared the efficiency of different Rc code and make a final conclusion.
8. !
!
!
! ! ! ! ! ! raid.xml配置!
!
! raid.xml⽰示例:!
Page of8 13
key value Description
policy name policy的名称,唯⼀一标识
srcPath prefix 该policy对应的⺫⽬目录
codecId 与HDFS Codec对应
property 可以配置额外的property,如
raid后的副本数,meta副本
数等
<configuration>!
<policy name = "rs">!
<srcPath prefix="hdfs://yz522.hadoop.data.sina.com.cn:8020/test"></srcPath>!
<shouldRaid>true</shouldRaid>!
<codecId>rs</codecId>!
<property>!
<name>targetReplication</name>!
<value>1</value>!
<description> after RAIDing, decrease the replication factor of a file to!
this value.!
</description>!
</property>!
<property>!
<name>metaReplication</name>!
<value>1</value>!
<description> the replication factor of the RAID meta file!
</description>!
</property>!
<property>!
<name>modTimePeriod</name>!
<value>1000</value>!
<description> time (milliseconds) after a file is modified to make it a!
candidate for RAIDing!
</description>!
</property>!
</policy>!
</configuration>