SlideShare une entreprise Scribd logo

Hadoopソースコードリーディング MapReduce障害時のフロー

•Télécharger en tant que KEY, PDF•
•
Ryu Kobayashi
Ryu Kobayashi

This document discusses Hadoop and MapReduce. It describes the interactions between the JobTracker, TaskTrackers, and how tasks are assigned and monitored. It also mentions configuration options for Ubuntu/Debian for IPv6 and task failure handling in MapReduce jobs.

Technologie
1  sur  18
2 Hadoop MapReduce
• • Twitter @ryu_kobayashi • ...
• MapReuceTaskTracker JobTracker
• MapReuceTaskTracker JobTracker • Hadoop P176-178
• MapReuceTaskTracker JobTracker • Hadoop P176-178 •
•
• • Ubuntu Debian
• • Ubuntu Debian • Ubuntu IPv6 OS -Djava.net.preferIPv4Stack=true
• • Ubuntu Debian • Ubuntu IPv6 OS -Djava.net.preferIPv4Stack=true • OS •
MapReduce
• JobClient#runJob • JobClint#monitorAndPrintJob mapred-site.xml jobclient.output.filter FAILDE monitorAndPrintJob Job
TaskTracker • TaskTracker#run • TaskTracker#offerService • TaskTracker#markUnresponsiveTasks Task • JobTracker#statusUpdate • TaskInProgress#reportProgress • JobTracker TaskTracker#offerService HeartBeat
JobTracker • JobTracker#heartbeat heatbeat TaskTracker • JobTracker#offerService RetireJobs • Job Job • JobTracker#offerService ExpireTrackers • TaskTracker TaskTracker • JobTracker#offerService ExpireLaunchingTasks • 10 ( )heatbeat • RecoveryManager
Hadoopソースコードリーディング MapReduce障害時のフロー
Hadoopソースコードリーディング MapReduce障害時のフロー
Hadoopソースコードリーディング MapReduce障害時のフロー

Recommandé

Graduating To Go - A Jumpstart into the Go Programming Language
Graduating To Go - A Jumpstart into the Go Programming LanguageGraduating To Go - A Jumpstart into the Go Programming Language
Graduating To Go - A Jumpstart into the Go Programming Language
Kaylyn Gibilterra
 
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Puppet
 
Airflow Clustering and High Availability
Airflow Clustering and High AvailabilityAirflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
 
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Jarek Potiuk
 
Getting to Know Airflow
Getting to Know AirflowGetting to Know Airflow
Getting to Know Airflow
Rosanne Hoyem
 
Puppet at Spotify (stockholm)
Puppet at Spotify (stockholm)Puppet at Spotify (stockholm)
Puppet at Spotify (stockholm)
Puppet
 
New York GeoConn 2015 Presentation
New York GeoConn 2015 PresentationNew York GeoConn 2015 Presentation
New York GeoConn 2015 Presentation
Matthew Wilson
 
Python Dependency Management - PyconDE 2018
Python Dependency Management - PyconDE 2018Python Dependency Management - PyconDE 2018
Python Dependency Management - PyconDE 2018
Patrick Muehlbauer
 

Recommandé

Graduating To Go - A Jumpstart into the Go Programming Language
Graduating To Go - A Jumpstart into the Go Programming LanguageGraduating To Go - A Jumpstart into the Go Programming Language
Graduating To Go - A Jumpstart into the Go Programming Language
Kaylyn Gibilterra
 
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Puppet
 
Airflow Clustering and High Availability
Airflow Clustering and High AvailabilityAirflow Clustering and High Availability
Airflow Clustering and High Availability
Robert Sanders
 
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Manageable data pipelines with airflow (and kubernetes) november 27, 11 45 ...
Jarek Potiuk
 
Getting to Know Airflow
Getting to Know AirflowGetting to Know Airflow
Getting to Know Airflow
Rosanne Hoyem
 
Puppet at Spotify (stockholm)
Puppet at Spotify (stockholm)Puppet at Spotify (stockholm)
Puppet at Spotify (stockholm)
Puppet
 
New York GeoConn 2015 Presentation
New York GeoConn 2015 PresentationNew York GeoConn 2015 Presentation
New York GeoConn 2015 Presentation
Matthew Wilson
 
Python Dependency Management - PyconDE 2018
Python Dependency Management - PyconDE 2018Python Dependency Management - PyconDE 2018
Python Dependency Management - PyconDE 2018
Patrick Muehlbauer
 
How to become a Git power user
How to become a Git power userHow to become a Git power user
How to become a Git power user
Deveo
 
Simple Tips and Tricks with Ansible
Simple Tips and Tricks with AnsibleSimple Tips and Tricks with Ansible
Simple Tips and Tricks with Ansible
Keith Resar
 
Whirr devdown
Whirr devdownWhirr devdown
Whirr devdown
Puppet
 
Ember.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.jsEmber.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.js
gavinjoyce
 
Illustrator_Sample
Illustrator_SampleIllustrator_Sample
Illustrator_Sample
Saeid Saadatmand
 
How Openstack is Built
How Openstack is BuiltHow Openstack is Built
How Openstack is Built
Anton Weiss
 
Git in pills : git stash
Git in pills : git stashGit in pills : git stash
Git in pills : git stash
Federico Panini
 
Introducing Apache Airflow and how we are using it
Introducing Apache Airflow and how we are using itIntroducing Apache Airflow and how we are using it
Introducing Apache Airflow and how we are using it
Bruno Faria
 
Q&a
Q&aQ&a
Q&a
Subhabrata Mishra
 
Automating Application over OpenStack using Workflows
Automating Application over OpenStack using WorkflowsAutomating Application over OpenStack using Workflows
Automating Application over OpenStack using Workflows
Yaron Parasol
 
Integration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container serviceIntegration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container service
SaltStack
 
Kubernetes London - Musings on Kubernetes Operators
Kubernetes London - Musings on Kubernetes OperatorsKubernetes London - Musings on Kubernetes Operators
Kubernetes London - Musings on Kubernetes Operators
Stuart Leeks
 
AGES Presentation on Web, Python, Django and GeoServer
AGES Presentation on Web, Python, Django and GeoServerAGES Presentation on Web, Python, Django and GeoServer
AGES Presentation on Web, Python, Django and GeoServer
Ng'eno Victor
 
Git
GitGit
Git
IT Booze
 
From airflow to google cloud composer
From airflow to google cloud composerFrom airflow to google cloud composer
From airflow to google cloud composer
Bruce Kuo
 
Weightlifting at SimplySocial
Weightlifting at SimplySocialWeightlifting at SimplySocial
Weightlifting at SimplySocial
Bogdan Gaza
 
Hadoop MapReduce Introduction and Deep Insight
Hadoop MapReduce Introduction and Deep InsightHadoop MapReduce Introduction and Deep Insight
Hadoop MapReduce Introduction and Deep Insight
Hanborq Inc.
 
サンプルから見るMap reduceコード
サンプルから見るMap reduceコードサンプルから見るMap reduceコード
サンプルから見るMap reduceコード
Shinpei Ohtani
 
サンプルから見るMapReduceコード
サンプルから見るMapReduceコードサンプルから見るMapReduceコード
サンプルから見るMapReduceコード
Shinpei Ohtani
 
Using Riak for Events storage and analysis at Booking.com
Using Riak for Events storage and analysis at Booking.comUsing Riak for Events storage and analysis at Booking.com
Using Riak for Events storage and analysis at Booking.com
Damien Krotkine
 
BIG DATA ANALYSIS
BIG DATA ANALYSISBIG DATA ANALYSIS
BIG DATA ANALYSIS
Nitesh Singh
 
Quartz Scheduler
Quartz SchedulerQuartz Scheduler
Quartz Scheduler
Software Infrastructure
 

Contenu connexe

Tendances

Whirr devdown
Whirr devdownWhirr devdown
Whirr devdown
Puppet
 
Ember.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.jsEmber.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.js
gavinjoyce
 
Automating Application over OpenStack using Workflows
Automating Application over OpenStack using WorkflowsAutomating Application over OpenStack using Workflows
Automating Application over OpenStack using Workflows
Yaron Parasol
 

Tendances (16)

How to become a Git power user
How to become a Git power userHow to become a Git power user
How to become a Git power user
 
Simple Tips and Tricks with Ansible
Simple Tips and Tricks with AnsibleSimple Tips and Tricks with Ansible
Simple Tips and Tricks with Ansible
 
Whirr devdown
Whirr devdownWhirr devdown
Whirr devdown
 
Ember.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.jsEmber.js internals backburner.js and rsvp.js
Ember.js internals backburner.js and rsvp.js
 
Illustrator_Sample
Illustrator_SampleIllustrator_Sample
Illustrator_Sample
 
How Openstack is Built
How Openstack is BuiltHow Openstack is Built
How Openstack is Built
 
Git in pills : git stash
Git in pills : git stashGit in pills : git stash
Git in pills : git stash
 
Introducing Apache Airflow and how we are using it
Introducing Apache Airflow and how we are using itIntroducing Apache Airflow and how we are using it
Introducing Apache Airflow and how we are using it
 
Q&a
Q&aQ&a
Q&a
 
Automating Application over OpenStack using Workflows
Automating Application over OpenStack using WorkflowsAutomating Application over OpenStack using Workflows
Automating Application over OpenStack using Workflows
 
Integration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container serviceIntegration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container service
 
Kubernetes London - Musings on Kubernetes Operators
Kubernetes London - Musings on Kubernetes OperatorsKubernetes London - Musings on Kubernetes Operators
Kubernetes London - Musings on Kubernetes Operators
 
AGES Presentation on Web, Python, Django and GeoServer
AGES Presentation on Web, Python, Django and GeoServerAGES Presentation on Web, Python, Django and GeoServer
AGES Presentation on Web, Python, Django and GeoServer
 
Git
GitGit
Git
 
From airflow to google cloud composer
From airflow to google cloud composerFrom airflow to google cloud composer
From airflow to google cloud composer
 
Weightlifting at SimplySocial
Weightlifting at SimplySocialWeightlifting at SimplySocial
Weightlifting at SimplySocial
 

Similaire à Hadoopソースコードリーディング MapReduce障害時のフロー

サンプルから見るMap reduceコード
サンプルから見るMap reduceコードサンプルから見るMap reduceコード
サンプルから見るMap reduceコード
Shinpei Ohtani
 
Storm distributed processing
Storm distributed processingStorm distributed processing
Storm distributed processing
ducquoc_vn
 
Treasure Data on The YARN - Hadoop Conference Japan 2014
Treasure Data on The YARN - Hadoop Conference Japan 2014Treasure Data on The YARN - Hadoop Conference Japan 2014
Treasure Data on The YARN - Hadoop Conference Japan 2014
Ryu Kobayashi
 
London devops logging
London devops loggingLondon devops logging
London devops logging
Tomas Doran
 

Similaire à Hadoopソースコードリーディング MapReduce障害時のフロー (20)

Hadoop MapReduce Introduction and Deep Insight
Hadoop MapReduce Introduction and Deep InsightHadoop MapReduce Introduction and Deep Insight
Hadoop MapReduce Introduction and Deep Insight
 
サンプルから見るMap reduceコード
サンプルから見るMap reduceコードサンプルから見るMap reduceコード
サンプルから見るMap reduceコード
 
サンプルから見るMapReduceコード
サンプルから見るMapReduceコードサンプルから見るMapReduceコード
サンプルから見るMapReduceコード
 
Using Riak for Events storage and analysis at Booking.com
Using Riak for Events storage and analysis at Booking.comUsing Riak for Events storage and analysis at Booking.com
Using Riak for Events storage and analysis at Booking.com
 
BIG DATA ANALYSIS
BIG DATA ANALYSISBIG DATA ANALYSIS
BIG DATA ANALYSIS
 
Quartz Scheduler
Quartz SchedulerQuartz Scheduler
Quartz Scheduler
 
Map reduce
Map reduceMap reduce
Map reduce
 
Storm distributed processing
Storm distributed processingStorm distributed processing
Storm distributed processing
 
Airflow at WePay
Airflow at WePayAirflow at WePay
Airflow at WePay
 
Treasure Data on The YARN - Hadoop Conference Japan 2014
Treasure Data on The YARN - Hadoop Conference Japan 2014Treasure Data on The YARN - Hadoop Conference Japan 2014
Treasure Data on The YARN - Hadoop Conference Japan 2014
 
Is hadoop for you
Is hadoop for youIs hadoop for you
Is hadoop for you
 
Big data week presentation
Big data week presentationBig data week presentation
Big data week presentation
 
Introduction to hadoop
Introduction to hadoopIntroduction to hadoop
Introduction to hadoop
 
London devops logging
London devops loggingLondon devops logging
London devops logging
 
ELK stack at weibo.com
ELK stack at weibo.comELK stack at weibo.com
ELK stack at weibo.com
 
Devoxx france 2015 influxdb
Devoxx france 2015 influxdbDevoxx france 2015 influxdb
Devoxx france 2015 influxdb
 
Devoxx france 2015 influx db
Devoxx france 2015 influx dbDevoxx france 2015 influx db
Devoxx france 2015 influx db
 
DBA だってもっと効率化したい！〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい！〜最近の自動化事情とOracle Database〜DBA だってもっと効率化したい！〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい！〜最近の自動化事情とOracle Database〜
 
OC Big Data Monthly Meetup #5 - Session 1 - Altiscale
OC Big Data Monthly Meetup #5 - Session 1 - AltiscaleOC Big Data Monthly Meetup #5 - Session 1 - Altiscale
OC Big Data Monthly Meetup #5 - Session 1 - Altiscale
 
Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)
 

Plus de Ryu Kobayashi

Hadoop Conference Japan 2011 Fall
Hadoop Conference Japan 2011 FallHadoop Conference Japan 2011 Fall
Hadoop Conference Japan 2011 Fall
Ryu Kobayashi
 
Developers summit cassandraで見るNoSQL
Developers summit cassandraで見るNoSQLDevelopers summit cassandraで見るNoSQL
Developers summit cassandraで見るNoSQL
Ryu Kobayashi
 
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング第3回 Hadopo MR + CassandraHadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Ryu Kobayashi
 

Plus de Ryu Kobayashi (6)

PLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core engine
PLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core enginePLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core engine
PLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core engine
 
Huahin Framework for Hadoop, Hadoop Conference Japan 2013 Winter
Huahin Framework for Hadoop, Hadoop Conference Japan 2013 WinterHuahin Framework for Hadoop, Hadoop Conference Japan 2013 Winter
Huahin Framework for Hadoop, Hadoop Conference Japan 2013 Winter
 
Hadoop Conference Japan 2011 Fall
Hadoop Conference Japan 2011 FallHadoop Conference Japan 2011 Fall
Hadoop Conference Japan 2011 Fall
 
Developers summit cassandraで見るNoSQL
Developers summit cassandraで見るNoSQLDevelopers summit cassandraで見るNoSQL
Developers summit cassandraで見るNoSQL
 
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング第3回 Hadopo MR + CassandraHadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
 
AWSを使ったトラッキングログ収集
AWSを使ったトラッキングログ収集AWSを使ったトラッキングログ収集
AWSを使ったトラッキングログ収集
 

Dernier

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
Overkill Security
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
sammart93
 

Dernier (20)

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 

Hadoopソースコードリーディング MapReduce障害時のフロー

  • 1. 2 Hadoop MapReduce
  • 2. • • Twitter @ryu_kobayashi • ...
  • 3.
  • 4. • MapReuceTaskTracker JobTracker
  • 5. • MapReuceTaskTracker JobTracker • Hadoop P176-178
  • 6. • MapReuceTaskTracker JobTracker • Hadoop P176-178 •
  • 7.
  • 8. •
  • 9. • • Ubuntu Debian
  • 10. • • Ubuntu Debian • Ubuntu IPv6 OS -Djava.net.preferIPv4Stack=true
  • 11. • • Ubuntu Debian • Ubuntu IPv6 OS -Djava.net.preferIPv4Stack=true • OS •
  • 13. • JobClient#runJob • JobClint#monitorAndPrintJob mapred-site.xml jobclient.output.filter FAILDE monitorAndPrintJob Job
  • 14. TaskTracker • TaskTracker#run • TaskTracker#offerService • TaskTracker#markUnresponsiveTasks Task • JobTracker#statusUpdate • TaskInProgress#reportProgress • JobTracker TaskTracker#offerService HeartBeat
  • 15. JobTracker • JobTracker#heartbeat heatbeat TaskTracker • JobTracker#offerService RetireJobs • Job Job • JobTracker#offerService ExpireTrackers • TaskTracker TaskTracker • JobTracker#offerService ExpireLaunchingTasks • 10 ( )heatbeat • RecoveryManager

Notes de l'éditeur