Hadoop is an important tool for data science when the volume of data exceeds the system memory or when the business case requires data to be distributed across multiple servers.

1. Session 1
1

2. Copyright © Global Tech Council www.globaltechcouncil.org 2
Data science is a blend of various algorithms, tools, and machine learning
principles that operate with the goal of discovering hidden patterns from raw data.
It is used to make decisions and predictions by using prescriptive analysis,
predictive causal analysis, and machine learning. Data science experts work in
the realm of the unknown. Some of the data science techniques are regression
analysis, classification analysis, clustering analysis, association analysis, and
anomaly detection.
In this article, we will analyze the importance of Hadoop for the field of data
science.

3. Copyright © Global Tech Council www.globaltechcouncil.org
A Brief Introduction To Hadoop
3
Apache Hadoop is an open-source framework that facilitates a network of
computers to solve problems requiring massive computational power and
datasets. It processes the datasets across clusters of computers with the help of
simple programming models.
Three Main Components of Hadoop
Let us now understand the three major components of Hadoop.

4. Copyright © Global Tech Council www.globaltechcouncil.org 4
● Map-Reduce- This component is responsible for high-level data processing.
It helps process a large amount of data over the cluster of nodes.
● Hadoop Distributed File system- This is the storage component of Hadoop,
which is a collection of master-slave networks. A Hadoop Distributed
FileSystem has two daemons such as namenode and datanode running on
the master and slave nodes, respectively.
● YARN- This component is used for resource management and job
scheduling. It is difficult to allocate, manage, and release resources in a multi-
node cluster. Hadoop YARN helps manage and control these resources
efficiently.

5. Copyright © Global Tech Council www.globaltechcouncil.org
Role Of Hadoop In Data Science
We will now list down the various areas in which Hadoop plays a significant role in
data science.
1. Hadoop for Data Exploration
A data scientist spends 80% of his time in data preparation and data exploration.
Hadoop is good at data exploration as it helps data scientists discover the
complexities present in data, even if they cannot make sense of it. It allows data
scientists to store data as it is, and that is the whole idea of data exploration. It
does not need a data scientist to understand the data when they deal with large
volumes of data.
5

6. Copyright © Global Tech Council www.globaltechcouncil.org
2. Hadoop for Data Sampling
6
It is not possible for a data scientist to build a model by taking just the first 1000
records from the dataset because of the way in which the data is usually written-
grouping similar kinds of records together. A data scientist cannot get a good view
of what is there in the data as a whole, without sampling the data. Using Hadoop
for data sampling gives the data scientist a fair idea of the approaches that might
work and those that might not work for data modeling. The cool keyword utility
'Sample' of Hadoop lets users down the number of records.

7. Copyright © Global Tech Council www.globaltechcouncil.org
3. Hadoop for Summarization
7
Summarizing the data as a whole with Hadoop MapReduce will help data
scientists get a bird's eye view of better data building models. Hadoop
MapReduce is used for the summarization of data. Mappers get the data and
reducers summarize the data. Hadoop is also used in a significant part of the data
science process, which is data preparation. It is both important and useful for a
data scientist to familiarize himself with concepts like Hadoop MapReduce, Hive,
and Pig.

8. Copyright © Global Tech Council www.globaltechcouncil.org
Conclusion
8
Learning Hadoop would certainly prove to be useful for a data scientist as it helps
speed up the learning process. Hadoop will let data scientists look for novel ways
to leverage the big data of organizations.
To become an expert in data science and learn more about data science
certifications, check out Global Tech Council.

9. Copyright © Global Tech Council www.globaltechcouncil.org
Globaltech Council Certifications -
You can check out our certifications, and kick start your career.
● Certifies Artificial Intelligence Expert
● Certified Augmented Reality Developer
● Certified Chatbot Expert
● Certified Data Scientist Expert
● Certified Big Data Expert
● Certified Machine Learning Expert
● Certified Virtual Reality Expert
Learn more about GlobalTech Council click here
9

10. THANK YOU!
Any questions?
You can mail us at
hello@globaltechcouncil.org
Copyright © Global Tech Council www.globaltechcouncil.org 10