MapReduce facilitates concurrent processing by splitting petabytes of data into smaller chunks, and processing them in parallel on Hadoop commodity servers.

1. MapReduce

2. Introduction to MapReduce
• MapReduce is a programming model and an
associated implementation for processing
and generating large data sets with a
parallel, distributed algorithm on a cluster.
• Map() procedure that performs filtering and
sorting.
• Reduce() procedure that performs a
summary operation (such as statistical
operations)

3. What is MapReduce in Hadoop?
• Hadoop is a free, Java-based programming framework that supports
the processing of large data sets in a distributed computing
environment. It is part of the Apache project sponsored by the
Apache Software Foundation.
• MapReduce is defined as the framework of Hadoop, which is used
to process a huge amount of data parallelly on large clusters of
commodity hardware in a reliable manner. It allows the application
to store the data in the distributed form and process large dataset
across groups of computers using simple programming models

4. Why use MapReduce?

5. MapReduce Architecture

6. How MapReduce in Hadoop works?

7. Steps of MapReduce in Hadoop
• Map stage − The map or mapper’s job is to process the input
data. Generally, the input data is in the form of file or directory
and is stored in the Hadoop file system (HDFS). The input file is
passed to the mapper function line by line. The mapper
processes the data and creates several small chunks of data.
• Reduce stage − This stage is the combination of
the Shuffle stage and the Reduce stage. The Reducer’s job is to
process the data that comes from the mapper. After processing,
it produces a new set of output, which will be stored in the
HDFS.

8. Algorithm Behind MapReduce

9. MapReduce Jobs
Application Master
• Responsible for the
execution of a single
application or MapReduce
job
• Divides job requests into
tasks and assigns them to
Node-Managers running on
the slave node
Node-Manager
• Has many dynamic
resource containers
which executes each
active map or reduce
task
• Communicates regularly
with the Application
Master

10. MapReduce jobs workflow

11. Advantage of MapReduce
• Fault tolerance: It can handle failures without downtime.
Speed: It splits, shuffles, and reduces the unstructured data in a short
time.
• Cost-effective: Hadoop MapReduce has a scale-out feature that enables
users to process or store the data in a cost-effective manner.
• Scalability: It provides a highly scalable framework. MapReduce allows
users to run applications from many nodes.
• Parallel processing: As Hadoop storage data in the distributed file system
and the MapReduce program’s working, it divides tasks task map and
reduce and that could execute in parallel. And again, because of the
parallel execution, it reduces the entire run time.

12. Limitations Of MapReduce
• MapReduce cannot cache the intermediate data in memory
for a further requirement which diminishes the performance
of Hadoop.
• It is only suitable for Batch Processing of a Huge amounts
of Data, and it is not flexible, means the MapReduce framework
is rigid.
• A lot of manual coding is required, even for common operations
such as join, filter, projection, aggregates, sorting, distinct...
• Semantics are hidden inside the map and reduce functions, so it
is difficult to maintain, extend and optimize them.