![this.toString() ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Nextag talk
Similaire à Nextag talk (20)
Nextag talk
- 1. HBase and Hive a random walk down H street Joydeep Sen Sarma
- 4. Hadoop Ecosystem HDFS ZooKeeper HBase Map-Reduce Scheduler Hive Metastore Hive Pig
- 8. HIVE: Components HDFS Hive CLI DDL Queries Browsing Map Reduce MetaStore Thrift API SerDe Thrift Jute JSON.. Execution Hive QL Parser Planner Mgmt. Web UI
- 9. Data Model Hive Entity Sample Metastore Entity Sample HDFS Location Table T /wh/T Partition date=d1 /wh/T/date=d1 Bucketing column userid /wh/T/date=d1/part-0000 … /wh/T/date=d1/part-1000 (hashed on userid) External Table extT /wh2/existing/dir (arbitrary location)
- 18. HBase png s via Lars George
- 22. System Picture (thx. Lars)
- 25. Appendix
- 26. Looks like this .. Disks Node Disks Node Disks Node Disks Node Disks Node Disks Node 1 Gigabit 4-8 Gigabit Node = DataNode + Map-Reduce
- 28. In pictures .. NameNode Disks 32GB RAM Secondary NameNode Disks 32GB RAM DataNode DataNode DataNode DFS Client DataNode DataNode DataNode getLocations locations
Notes de l'éditeur
- Offline and Near-Real time data processing Not online
- Simple map-reduce is easy – but it can get complicated very quickly.
- Assume users know about Hadoop Streaming
- Nomenclature: Core switch and Top of Rack
- Compare to a standard unix file system
- Rack local and node local access rocks Scalability is bound by switches
- Point out that now we know how HDFS works – we can run maps close to data
- Point out how data local computing is useful in this example Exposes some of the features we need in hadoop – output of reducer can be directly sent to another reducer As an exercise to the reader – the results from the shell do not equal those from hadoop – interesting to find why.