In this presentation, I use the tools of a 4th grade classroom to illustrate the basics of Hadoop MapReduce. You can do these exercises with any heterogeneous collection of objects, such as condiment packs from a cafeteria, teabags from a breakroom, pebbles and marbles, etc. You can do these exercises with any number of people and any volume of materials, illustrating the scale-out nature of Hadoop.

1. The Cottonball
Demo
"I just don't get this MapReduce thing"

2. The File
This isn't a bag
of cottonballs.
It's a file.

3. Input Splits
One split (~HDFS block) is kept
on one node
The other split (~HDFS block) is
kept on another node.
Will two students please each
take one input split?

4. map() -> partition by size
On your input split, please
group by size.
Note, you can group your split
by size at the same time
someone else can do their split.

5. reduce()
Items of the same size are grouped
together into a partition. Each partition is
then passed to a reducer.
The black line indicates the splits to the
mappers. The blue lines delineate the
partitions to the reducers.
The reducer can do anything: it can count,
aggregate, explode, whatever.
Also with this example, note the skew. The
reducers with the small balls have more
work.

6. map() -> Partition by color
Because we have flexibility in
our storage format and our
query structure, we can run a
completely different mapper on
the same data at a later time.
Will two students please now
group the input splits by color?

7. Another reducer
Like in the last example,
groups from each mapper are
passed to the same reducer.
In this example, there are more
partitions, but less skew. That's
okay.

8. We like MapReduce because...
● Flexibility: We can process the data as it's
stored, rather than have to anticipate how it'll
be queried
● Scalability: The same simple instructions will
work on dump-trucks full of cottonballs and
more people to sort through them.
● Economy: We can predict the performance
of larger operations processes with smaller
amounts of data and workers.
● Versatility: The same demo works on legos,
candies, and baby animals.

9. MapReduce has issues because...
● Schema is applied during the query, so this
makes for a more complex query.
● Issues like skew require developers to be
mindful.

10. Thank you!
jwfbean@cloudera.com