An Introduction to Apache Hama, what is it ? Why is it useful for scientific computations on big data ?

1. Apache Hama
● What is it ?
● What is it used for ?
● How does it work ?
● Architecture
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

2. Hama – What is it ?
● A Distributed computing framework
● For massive scientific calculations
● For distributed computing
● Uses Bulk Synchronous Parallel Computing ( BSP )
● Written in Java
● Licensed via Apache
● Available for Hadoop greater than 1.0.x
● Requires Java 1.6.x or higher
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

3. Hama – What is it used for ?
● large scale computations that
– exceed the abilities of Map reduce
● Machine learning
● Matrix
● Graph
● Network Algorithms
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

4. Hama – How does it work ?
● A series of super steps
● Each of which is
– Concurent computation on local data
– Point to point communication
– Synchronise all processors
● Uses Hadoop RPC for communication
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

5. Hama – Architecture
Hama consists of BSP Master, GroomServers, ZooKeeper
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

6. Hama – Architecture
● Role of BSPMaster
– Maintain groom server statuses
– Controlling cluster super steps
– Maintaining job progress information
– Scheduling Jobs
– Assign tasks to groom servers
– Fault control
– Provide cluster control to users
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

7. Hama – Architecture
● Role of GroomServer
– Performs BSP tasks
– Reports status to master
– Runs on a single HDFS data node
– Starts with a BSP Peer instance
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

8. Hama – Architecture
● Role of ZooKeeper
– Manages barrier synchronisation of BSP Peers
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

9. Contact Us
● Feel free to contact us at
– www.semtech-solutions.co.nz
– info@semtech-solutions.co.nz
● We offer IT project consultancy
● We are happy to hear about your problems
● You can just pay for those hours that you need
● To solve your problems

10. Contact Us
● Feel free to contact us at
– www.semtech-solutions.co.nz
– info@semtech-solutions.co.nz
● We offer IT project consultancy
● We are happy to hear about your problems
● You can just pay for those hours that you need
● To solve your problems