Want to learn Hadoop online? This PPT give you Introduction to Big Data Hadoop Training Online by expert trainers at ITJobZone.biz - Start your Hadoop Online training with this Presentation.

1. Bigdata and Hadoop Tutorial
By www.ITJobZone.biz

2. Audience
Students
Working Professionals
Developers
Database experts
Adminstrators
Those looking for career change.

3. Topics Covered
What is Bigdata?
Characteristics of Bigdata.
Use Cases and Need of Hadoop.
Apache Hadoop Ecosystem and its components
Brief History of Hadoop.
Hadoop Architecture.
Who Uses Hadoop

4. What is Bigdata?
Lots of Data (Terabytes or PetaBytes)
Bigdata is term for collection of datasets so large and
complex that it becomes difficult to process using on
hand database tools or Traditional data processing
applications.
The challenges include capture, curation, storage, search,
sharing, transfer, analysis and visualization.

5. What is Bigdata?
System /Enterprises generates huge amount of data from Terabytes to Petabytes of information
Stock Market generates around one terabyte of new data per day to
perform stock trading analysis and determine trends for optimal trades.

6. Bigdata Characteristics

7. Bigdata - Types of Data
● Structured Data
- Data from enterprise (ERP, CRM)
● Semi structured Data
- xml, json, csv, log files
● Unstructured Data
- audio, video, image, archive documents

8. Unstructured Data is Exploding

9. Bigdata Scenarios
Web and e-tailing
Recommendation Engines
Ad Targeting
Search Quality
Abuse and Click Fraud Detection
Telecommunication
Customer Churn Analysis and Prevention
Network Performance Optimization
Call Data Record (CDR) Analysis
Analysing Network to Predict Failure

10. Bigdata Scenarios (Cont…)
Government
Fraud Detection and Cyber Security
Welfare Schemes
Justice
Telecommunication
Health Information Exchange
Gene Sequencing
Serialization
Healthcare service Quality Improvements
Drug Safety

11. Brief history of Hadoop
Designed to answer the question
“ How to process big data with reasonable cost and time”?
Google ( published white papers)
GFS
2003
Mapreduce 2005
Yahoo( implementation by Doug Cutting)
HDFS 2006-07
Mapreduce 2007-08

12. What is Hadoop
Apache top level project, Opensource implementation of
frameworks for reliable, scalable, distributed computing
and storage.
It is a flexible and highly-available architecture for large
scale computation and data processing on a network of
commodity hardware.

13. Goals / Requirements
Abstract & facilitate the storage and processing of large and rapidly growing data.
• Structured and unstructured datasets
• Simple programming model
High scalability and availability
Use commodity hardware with little redundancy
Fault tolerance
Move computation rather data

14. Goals / Requirements
Hardware will fail.
Processing will be run in batches. Thus there is an emphasis on
high throughput as opposed to low latency.
Applications that run on HDFS have large data sets. A typical
file in HDFS is gigabytes to terabytes in size.
It should provide high aggregate data bandwidth and scale to
hundreds of nodes in a single cluster. It should support tens
of millions of files in a single instance.
Applications need a write-once-read-many access model.

15. Architecture
Distributed, with some centralization
Main nodes of cluster are where most of the computational power and storage
of the system lies
Main nodes run TaskTracker to accept and reply to MapReduce tasks, and
also DataNode to store needed blocks closely as possible
Central control node runs NameNode to keep track of HDFS directories &
files, and JobTracker to dispatch compute tasks to TaskTracker
Written in Java, also supports Python and Ruby

16. Architecture

17. Architecture

18. Architecture
Hadoop Distributed Filesystem
Tailored to needs of MapReduce
Targeted towards many reads of filestreams
Writes are more costly
High degree of data replication (3x by default)
No need for RAID on normal nodes
Large blocksize (64MB)
Location awareness of DataNodes in network

19. Architecture - NameNode:
Stores metadata for the files, like the directory structure of a typical FS.
The server holding the NameNode instance is quite crucial, as there is
only one.
Transaction log for file deletes/adds, etc. Does not use transactions for
whole blocks or file-streams, only metadata.
Handles creation of more replica blocks when necessary after a
DataNode failure

20. Architecture - DataNode:
Stores the actual data in HDFS
Can run on any underlying filesystem (ext3/4, NTFS, etc)
Notifies NameNode of what blocks it has
NameNode replicates blocks 2x in local rack, 1x elsewhere

21. Architecture

24. Some examples
•Yahoo! : More than 100,000 CPUs in ~20,000 computers running Hadoop;
biggest cluster: 2000 nodes (2*4cpu boxes with 4TB disk each); used to support
research for Ad Systems and Web Search
•AOL : Used for a variety of things ranging from statistics generation to running
advanced algorithms for doing behavioral analysis and targeting; cluster size is
50 machines, Intel Xeon, dual processors, dual core, each with 16GB Ram and
800 GB hard-disk giving us a total of 37 TB HDFS capacity.
•Facebook: To store copies of internal log and dimension data sources and use
it as a source for reporting/analytics and machine learning; 320 machine cluster
with 2,560 cores and about 1.3 PB raw storage;

25. Join Our Hadoop Online Training
http://www.itjobzone.biz/Big-Data-Hadoop-training.html
Get More Hadoop Tutorials on YouTube
Big Data and Hadoop Introduction Training Demo
Click - https://youtu.be/SHxf-v8ePk4
Apache Hadoop Standalone Installation Using Virtual box Training Demo
Click - https://youtu.be/u-YhaIkqubU