Hadoop ecosystem framework n hadoop in live environment

[object Object],[object Object]

Outline ,[object Object],[object Object],[object Object],[object Object]

Hadoop Ecosystem ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Job Scheduler ,[object Object],[object Object],[object Object],[object Object]

Overview of Oozie ,[object Object],[object Object],[object Object],[object Object],[object Object]

Overview of Oozie ,[object Object],[object Object],[object Object]

Overview of Oozie ,[object Object],[object Object]

Oozie vs Azkaban ,[object Object],[object Object],[object Object],[object Object],[object Object]

Chain MR ,[object Object],[object Object],[object Object],[object Object],[object Object]

Oozie Flow Start Map reduce Fork MR Streaming Pig Join Decision MR Pipes Java FileSystem End

Performance Tuning Parameters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Performance Tuning Parameters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Performance Tuning Parameters ,[object Object],[object Object],[object Object],[object Object],[object Object]

Tips ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Tips ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Tips ,[object Object],[object Object],[object Object],[object Object],[object Object]

Mahout (Machine learning library) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Different minds Different interpretation ,[object Object]

Hadoop in live environment ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

@Google ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

@Yahoo! ,[object Object],[object Object],[object Object],[object Object]

Hadoop is critical to Yahoo’s business ,[object Object],Ads Optimization Content Optimization Search Index Content Feed Processing Machine Learning (e.g. Spam filters)

Tremendous Impact on Productivity ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Search & Advertising Sciences Hadoop Applications: Search Assist™ ,[object Object],[object Object],[object Object],Before Hadoop After Hadoop Time 26 days 20 minutes Language C++ Python Development Time 2-3 weeks 2-3 days

Largest Hadoop Clusters in the Universe ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Large Hadoop-Based Applications 2008 2009 Webmap ~70 hours runtime ~300 TB shuffling ~200 TB output 1480 nodes ~73 hours runtime ~490 TB shuffling ~280 TB output 2500 nodes Sort benchmarks (Jim Gray contest) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Largest cluster ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

@Facebook ,[object Object],[object Object],[object Object],[object Object],[object Object]

@Nokia ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

@Clickable ,[object Object],[object Object],[object Object]

@Stumbleupon ,[object Object],[object Object],[object Object]

Hadoop ecosystem framework n hadoop in live environment

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Hadoop ecosystem framework n hadoop in live environment

Similaire à Hadoop ecosystem framework n hadoop in live environment (20)

Dernier

Dernier (20)

Hadoop ecosystem framework n hadoop in live environment