SlideShare a Scribd company logo

Hadoop & Zing

•Download as PPT, PDF•
•
Long Dao
Long Dao
Technology
1 of 29
PRESENTER: HUNGVV W: http://me.zing.vn/hung.vo E: [email_address] 2011-08 HADOOP & ZING
AGENDA Using Hadoop in Zing Rank Introduction to Hadoop, Hive A case study: Log Collecting, Analyzing & Reporting System ter Estimate Conclusion 1 3 2
Hadoop & Zing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Data Flow into Hadoop Web Servers Scribe MidTier Network Storage and Servers Hadoop Hive Warehouse MySQL
Hive – Data Warehouse ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Hive Architecture HDFS Map Reduce Web UI + Hive CLI + JDBC/ODBC Browse, Query, DDL Hive QL Parser Planner Optimizer Execution SerDe CSV Thrift Regex UDF/UDAF substr sum average FileFormats TextFile SequenceFile RCFile User-defined Map-reduce Scripts
Hive DDL ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Hive DML ,[object Object],[object Object],[object Object],[object Object]
Hive Query Language ,[object Object],[object Object],[object Object],[object Object],[object Object]
Multi-table Group-By/Insert ,[object Object],[object Object],[object Object],[object Object],[object Object]
File Formats ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SerDe ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
UDF/UDAF ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
What we use Hadoop for? ,[object Object],[object Object],[object Object],[object Object],[object Object]
Data Usage ,[object Object],[object Object],[object Object],[object Object]
Where is the data stored? ,[object Object],[object Object],[object Object],[object Object],[object Object]
Log Collecting, Analyzing & Reporting ,[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting
THANK YOU!

Recommended

Zing Database
Zing Database Zing Database
Zing Database
Long Dao
 
Intro to Apache Hadoop
Intro to Apache HadoopIntro to Apache Hadoop
Intro to Apache Hadoop
Sufi Nawaz
 
The Google Bigtable
The Google BigtableThe Google Bigtable
The Google Bigtable
Romain Jacotin
 
An introduction to Big-Data processing applying hadoop
An introduction to Big-Data processing applying hadoopAn introduction to Big-Data processing applying hadoop
An introduction to Big-Data processing applying hadoop
Amir Sedighi
 
Efficient in situ processing of various storage types on apache tajo
Efficient in situ processing of various storage types on apache tajoEfficient in situ processing of various storage types on apache tajo
Efficient in situ processing of various storage types on apache tajo
Hyunsik Choi
 
Introduction to apache nutch
Introduction to apache nutchIntroduction to apache nutch
Introduction to apache nutch
Sigmoid
 
Running MongoDB 3.0 on AWS
Running MongoDB 3.0 on AWSRunning MongoDB 3.0 on AWS
Running MongoDB 3.0 on AWS
MongoDB
 
No SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability MeetupNo SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability Meetup
Hyderabad Scalability Meetup
 

Recommended

Zing Database
Zing Database Zing Database
Zing Database
Long Dao
 
Intro to Apache Hadoop
Intro to Apache HadoopIntro to Apache Hadoop
Intro to Apache Hadoop
Sufi Nawaz
 
The Google Bigtable
The Google BigtableThe Google Bigtable
The Google Bigtable
Romain Jacotin
 
An introduction to Big-Data processing applying hadoop
An introduction to Big-Data processing applying hadoopAn introduction to Big-Data processing applying hadoop
An introduction to Big-Data processing applying hadoop
Amir Sedighi
 
Efficient in situ processing of various storage types on apache tajo
Efficient in situ processing of various storage types on apache tajoEfficient in situ processing of various storage types on apache tajo
Efficient in situ processing of various storage types on apache tajo
Hyunsik Choi
 
Introduction to apache nutch
Introduction to apache nutchIntroduction to apache nutch
Introduction to apache nutch
Sigmoid
 
Running MongoDB 3.0 on AWS
Running MongoDB 3.0 on AWSRunning MongoDB 3.0 on AWS
Running MongoDB 3.0 on AWS
MongoDB
 
No SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability MeetupNo SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability Meetup
Hyderabad Scalability Meetup
 
Connecting NodeJS & MongoDB
Connecting NodeJS & MongoDBConnecting NodeJS & MongoDB
Connecting NodeJS & MongoDB
Enoch Joshua
 
Introduction to HDFS
Introduction to HDFSIntroduction to HDFS
Introduction to HDFS
Siddharth Mathur
 
Mongo db dhruba
Mongo db dhrubaMongo db dhruba
Mongo db dhruba
Dhrubaji Mandal ♛
 
HDFS introduction
HDFS introductionHDFS introduction
HDFS introduction
injae yeo
 
Gluster.community.day.2013
Gluster.community.day.2013Gluster.community.day.2013
Gluster.community.day.2013
Udo Seidel
 
Web Crawling and Data Gathering with Apache Nutch
Web Crawling and Data Gathering with Apache NutchWeb Crawling and Data Gathering with Apache Nutch
Web Crawling and Data Gathering with Apache Nutch
Steve Watt
 
8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker
Fabio Fumarola
 
Hadoop
HadoopHadoop
Hadoop
Scott Leberknight
 
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Mydbops
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
Ravi Teja
 
Hbase an introduction
Hbase an introductionHbase an introduction
Hbase an introduction
Fabio Fumarola
 
MySQL database replication
MySQL database replicationMySQL database replication
MySQL database replication
PoguttuezhiniVP
 
Mongo DB
Mongo DBMongo DB
Mongo DB
Karan Kukreja
 
Web scraping with nutch solr part 2
Web scraping with nutch solr part 2Web scraping with nutch solr part 2
Web scraping with nutch solr part 2
Mike Frampton
 
Scaling with MongoDB
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDB
Rick Copeland
 
Postgresql Database Administration Basic - Day1
Postgresql Database Administration Basic - Day1Postgresql Database Administration Basic - Day1
Postgresql Database Administration Basic - Day1
PoguttuezhiniVP
 
MongoDB: Advance concepts - Replication and Sharding
MongoDB: Advance concepts - Replication and ShardingMongoDB: Advance concepts - Replication and Sharding
MongoDB: Advance concepts - Replication and Sharding
Knoldus Inc.
 
Introduction to hadoop ecosystem
Introduction to hadoop ecosystem Introduction to hadoop ecosystem
Introduction to hadoop ecosystem
Rupak Roy
 
BIG DATA: Apache Hadoop
BIG DATA: Apache HadoopBIG DATA: Apache Hadoop
BIG DATA: Apache Hadoop
Oleksiy Krotov
 
Zing Me & PHP
Zing Me & PHPZing Me & PHP
Zing Me & PHP
Long Dao
 
Feed Ranking in Zing Me
Feed Ranking in Zing MeFeed Ranking in Zing Me
Feed Ranking in Zing Me
Long Dao
 
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
AiTi Education
 

More Related Content

What's hot

HDFS introduction
HDFS introductionHDFS introduction
HDFS introduction
injae yeo
 

What's hot (19)

Connecting NodeJS & MongoDB
Connecting NodeJS & MongoDBConnecting NodeJS & MongoDB
Connecting NodeJS & MongoDB
 
Introduction to HDFS
Introduction to HDFSIntroduction to HDFS
Introduction to HDFS
 
Mongo db dhruba
Mongo db dhrubaMongo db dhruba
Mongo db dhruba
 
HDFS introduction
HDFS introductionHDFS introduction
HDFS introduction
 
Gluster.community.day.2013
Gluster.community.day.2013Gluster.community.day.2013
Gluster.community.day.2013
 
Web Crawling and Data Gathering with Apache Nutch
Web Crawling and Data Gathering with Apache NutchWeb Crawling and Data Gathering with Apache Nutch
Web Crawling and Data Gathering with Apache Nutch
 
8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker
 
Hadoop
HadoopHadoop
Hadoop
 
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Hbase an introduction
Hbase an introductionHbase an introduction
Hbase an introduction
 
MySQL database replication
MySQL database replicationMySQL database replication
MySQL database replication
 
Mongo DB
Mongo DBMongo DB
Mongo DB
 
Web scraping with nutch solr part 2
Web scraping with nutch solr part 2Web scraping with nutch solr part 2
Web scraping with nutch solr part 2
 
Scaling with MongoDB
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDB
 
Postgresql Database Administration Basic - Day1
Postgresql Database Administration Basic - Day1Postgresql Database Administration Basic - Day1
Postgresql Database Administration Basic - Day1
 
MongoDB: Advance concepts - Replication and Sharding
MongoDB: Advance concepts - Replication and ShardingMongoDB: Advance concepts - Replication and Sharding
MongoDB: Advance concepts - Replication and Sharding
 
Introduction to hadoop ecosystem
Introduction to hadoop ecosystem Introduction to hadoop ecosystem
Introduction to hadoop ecosystem
 
BIG DATA: Apache Hadoop
BIG DATA: Apache HadoopBIG DATA: Apache Hadoop
BIG DATA: Apache Hadoop
 

Viewers also liked

Zing Me & PHP
Zing Me & PHPZing Me & PHP
Zing Me & PHP
Long Dao
 
Feed Ranking in Zing Me
Feed Ranking in Zing MeFeed Ranking in Zing Me
Feed Ranking in Zing Me
Long Dao
 
Opportunities & Obligations
Opportunities & ObligationsOpportunities & Obligations
Opportunities & Obligations
Martin Rehm
 
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
Martin Rehm
 
Learning With Technology the Educator's Role revised
Learning With Technology the Educator's Role revisedLearning With Technology the Educator's Role revised
Learning With Technology the Educator's Role revised
tuchodi
 

Viewers also liked (20)

Zing Me & PHP
Zing Me & PHPZing Me & PHP
Zing Me & PHP
 
Feed Ranking in Zing Me
Feed Ranking in Zing MeFeed Ranking in Zing Me
Feed Ranking in Zing Me
 
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
[Vietnam Mobile Day 2013] - How did we launched an online mobile game in 3 mo...
 
User-tailored Inter-Widget Communication Extending the Shared Data Interface ...
User-tailored Inter-Widget Communication Extending the Shared Data Interface ...User-tailored Inter-Widget Communication Extending the Shared Data Interface ...
User-tailored Inter-Widget Communication Extending the Shared Data Interface ...
 
微言大益录2010
微言大益录2010微言大益录2010
微言大益录2010
 
Rik Panganiban's Keynote at SLCC 2011
Rik Panganiban's Keynote at SLCC 2011Rik Panganiban's Keynote at SLCC 2011
Rik Panganiban's Keynote at SLCC 2011
 
Double-Vision
Double-VisionDouble-Vision
Double-Vision
 
Opportunities & Obligations
Opportunities & ObligationsOpportunities & Obligations
Opportunities & Obligations
 
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
Unified in Learning –Separated by Space (S-ICT 2008 Conference Proceedings)
 
Issues and Considerations regarding Sharable Data Sets for Recommender System...
Issues and Considerations regarding Sharable Data Sets for Recommender System...Issues and Considerations regarding Sharable Data Sets for Recommender System...
Issues and Considerations regarding Sharable Data Sets for Recommender System...
 
تفريغ دورة تسويق الافكار
تفريغ دورة تسويق الافكارتفريغ دورة تسويق الافكار
تفريغ دورة تسويق الافكار
 
Unit 2.1 Part 3
Unit 2.1 Part 3Unit 2.1 Part 3
Unit 2.1 Part 3
 
Learning With Technology the Educator's Role revised
Learning With Technology the Educator's Role revisedLearning With Technology the Educator's Role revised
Learning With Technology the Educator's Role revised
 
Improve your Presentation Power
Improve your Presentation PowerImprove your Presentation Power
Improve your Presentation Power
 
Ethics privacy washington
Ethics privacy washingtonEthics privacy washington
Ethics privacy washington
 
Unit 2.7 Images
Unit 2.7 ImagesUnit 2.7 Images
Unit 2.7 Images
 
Tamgün Sunusu
Tamgün SunusuTamgün Sunusu
Tamgün Sunusu
 
Тренинг продаж: проход секретаря при холодном звонке
Тренинг продаж: проход секретаря при холодном звонкеТренинг продаж: проход секретаря при холодном звонке
Тренинг продаж: проход секретаря при холодном звонке
 
PresentacióN
PresentacióNPresentacióN
PresentacióN
 
Unit 2.12
Unit 2.12Unit 2.12
Unit 2.12
 

Similar to Hadoop & Zing

hadoop&zing
hadoop&zinghadoop&zing
hadoop&zing
zingopen
 
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Cloudera, Inc.
 
Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010
nzhang
 
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit JainApache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
Yahoo Developer Network
 
Hive ICDE 2010
Hive ICDE 2010Hive ICDE 2010
Hive ICDE 2010
ragho
 
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
Chester Chen
 
Hw09 Hadoop Development At Facebook Hive And Hdfs
Hw09 Hadoop Development At Facebook Hive And HdfsHw09 Hadoop Development At Facebook Hive And Hdfs
Hw09 Hadoop Development At Facebook Hive And Hdfs
Cloudera, Inc.
 

Similar to Hadoop & Zing (20)

hadoop&zing
hadoop&zinghadoop&zing
hadoop&zing
 
Hive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use CasesHive Training -- Motivations and Real World Use Cases
Hive Training -- Motivations and Real World Use Cases
 
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
 
Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010
 
Honu - A Large Scale Streaming Data Collection and Processing Pipeline__Hadoo...
Honu - A Large Scale Streaming Data Collection and Processing Pipeline__Hadoo...Honu - A Large Scale Streaming Data Collection and Processing Pipeline__Hadoo...
Honu - A Large Scale Streaming Data Collection and Processing Pipeline__Hadoo...
 
Hive Percona 2009
Hive Percona 2009Hive Percona 2009
Hive Percona 2009
 
Xadoop - new approaches to data analytics
Xadoop - new approaches to data analyticsXadoop - new approaches to data analytics
Xadoop - new approaches to data analytics
 
מיכאל
מיכאלמיכאל
מיכאל
 
Hands on Hadoop and pig
Hands on Hadoop and pigHands on Hadoop and pig
Hands on Hadoop and pig
 
Intro to hadoop ecosystem
Intro to hadoop ecosystemIntro to hadoop ecosystem
Intro to hadoop ecosystem
 
Introduction to Hive for Hadoop
Introduction to Hive for HadoopIntroduction to Hive for Hadoop
Introduction to Hive for Hadoop
 
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit JainApache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
Apache Hadoop India Summit 2011 talk "Hive Evolution" by Namit Jain
 
Basics of big data analytics hadoop
Basics of big data analytics hadoopBasics of big data analytics hadoop
Basics of big data analytics hadoop
 
Hive ICDE 2010
Hive ICDE 2010Hive ICDE 2010
Hive ICDE 2010
 
Hadoop data access layer v4.0
Hadoop data access layer v4.0Hadoop data access layer v4.0
Hadoop data access layer v4.0
 
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
 
Nextag talk
Nextag talkNextag talk
Nextag talk
 
Hw09 Hadoop Development At Facebook Hive And Hdfs
Hw09 Hadoop Development At Facebook Hive And HdfsHw09 Hadoop Development At Facebook Hive And Hdfs
Hw09 Hadoop Development At Facebook Hive And Hdfs
 
Datalake Architecture
Datalake ArchitectureDatalake Architecture
Datalake Architecture
 
Hadoop overview
Hadoop overviewHadoop overview
Hadoop overview
 

Recently uploaded

TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
Stephen Perrenod
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
Hiroshi SHIBATA
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Skynet Technologies
 

Recently uploaded (20)

WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Your enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4jYour enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4j
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
The Metaverse: Are We There Yet?
The Metaverse: Are We There Yet?The Metaverse: Are We There Yet?
The Metaverse: Are We There Yet?
 

Hadoop & Zing

  • 1. PRESENTER: HUNGVV W: http://me.zing.vn/hung.vo E: [email_address] 2011-08 HADOOP & ZING
  • 2. AGENDA Using Hadoop in Zing Rank Introduction to Hadoop, Hive A case study: Log Collecting, Analyzing & Reporting System ter Estimate Conclusion 1 3 2
  • 3.
  • 4. Data Flow into Hadoop Web Servers Scribe MidTier Network Storage and Servers Hadoop Hive Warehouse MySQL
  • 5.
  • 6. Hive Architecture HDFS Map Reduce Web UI + Hive CLI + JDBC/ODBC Browse, Query, DDL Hive QL Parser Planner Optimizer Execution SerDe CSV Thrift Regex UDF/UDAF substr sum average FileFormats TextFile SequenceFile RCFile User-defined Map-reduce Scripts
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.

Editor's Notes

  1. SequenceFile  is a flat file consisting of binary key/value pairs. It is extensively used in  MapReduce  as input/output formats. It is also worth noting that, internally, the temporary outputs of maps are stored using SequenceFile.
  2. Persistent connection One-way communication