Soumettre la recherche
Mettre en ligne
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
•
Télécharger en tant que ODP, PDF
•
2 j'aime
•
707 vues
Sugree Phatanapherom
Suivre
Presentation of academic research with 140-char limit
Lire moins
Lire la suite
Technologie
Business
Signaler
Partager
Signaler
Partager
1 sur 83
Télécharger maintenant
Recommandé
Event Scheduling
Event Scheduling
Ayesha Kanwal
Example Solutions for Scheduling and Work Planning
Example Solutions for Scheduling and Work Planning
SIS Group International
Why Average Response Time is not a right measure of your web application's pe...
Why Average Response Time is not a right measure of your web application's pe...
vodQA
Performance testing basics
Performance testing basics
Charu Anand
Mining model for hotel recommendations (Kaggle Challenge)
Mining model for hotel recommendations (Kaggle Challenge)
Arjun Varma
Time advance mehcanism
Time advance mehcanism
Nikhil Sharma
Predictive control 1 introduction
Predictive control 1 introduction
jamestpp
UC4 - One Automation
UC4 - One Automation
k1k2sdad
Recommandé
Event Scheduling
Event Scheduling
Ayesha Kanwal
Example Solutions for Scheduling and Work Planning
Example Solutions for Scheduling and Work Planning
SIS Group International
Why Average Response Time is not a right measure of your web application's pe...
Why Average Response Time is not a right measure of your web application's pe...
vodQA
Performance testing basics
Performance testing basics
Charu Anand
Mining model for hotel recommendations (Kaggle Challenge)
Mining model for hotel recommendations (Kaggle Challenge)
Arjun Varma
Time advance mehcanism
Time advance mehcanism
Nikhil Sharma
Predictive control 1 introduction
Predictive control 1 introduction
jamestpp
UC4 - One Automation
UC4 - One Automation
k1k2sdad
Data Mining and Analytics
Data Mining and Analytics
Nathaniel Palmer
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
CrimsonPublishersRDMS
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Editor IJCATR
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
genetic paper
genetic paper
Swathi Rampur
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
CSCJournals
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CUO VEERANAN VEERANAN
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CUO VEERANAN VEERANAN
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
ijceronline
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
SIMUL8 Corporation
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
Carasik BPM ECM
Carasik BPM ECM
Bob Carasik
Analytics for Process Excellence
Analytics for Process Excellence
Denis Gagné
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET Journal
performance
performance
manogallery
G017314249
G017314249
IOSR Journals
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
iosrjce
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
ijcsit
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
cscpconf
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Sugree Phatanapherom
@sugree and Twitter
@sugree and Twitter
Sugree Phatanapherom
Contenu connexe
Similaire à Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
Data Mining and Analytics
Data Mining and Analytics
Nathaniel Palmer
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
CrimsonPublishersRDMS
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Editor IJCATR
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
genetic paper
genetic paper
Swathi Rampur
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
CSCJournals
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CUO VEERANAN VEERANAN
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CUO VEERANAN VEERANAN
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
ijceronline
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
SIMUL8 Corporation
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
Carasik BPM ECM
Carasik BPM ECM
Bob Carasik
Analytics for Process Excellence
Analytics for Process Excellence
Denis Gagné
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET Journal
performance
performance
manogallery
G017314249
G017314249
IOSR Journals
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
iosrjce
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
ijcsit
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
cscpconf
Similaire à Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
(20)
Data Mining and Analytics
Data Mining and Analytics
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Workforce Management & BPM Integration
genetic paper
genetic paper
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Carasik BPM ECM
Carasik BPM ECM
Analytics for Process Excellence
Analytics for Process Excellence
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
performance
performance
G017314249
G017314249
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
Plus de Sugree Phatanapherom
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Sugree Phatanapherom
@sugree and Twitter
@sugree and Twitter
Sugree Phatanapherom
Behind the madness
Behind the madness
Sugree Phatanapherom
drupal.in.th
drupal.in.th
Sugree Phatanapherom
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Sugree Phatanapherom
Readme Read Sugree
Readme Read Sugree
Sugree Phatanapherom
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Sugree Phatanapherom
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Sugree Phatanapherom
Drupal: blog and beyond
Drupal: blog and beyond
Sugree Phatanapherom
The Spirit of Open Source
The Spirit of Open Source
Sugree Phatanapherom
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
Sugree Phatanapherom
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Sugree Phatanapherom
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Sugree Phatanapherom
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Sugree Phatanapherom
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Sugree Phatanapherom
Twitter Rules
Twitter Rules
Sugree Phatanapherom
Plus de Sugree Phatanapherom
(16)
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
@sugree and Twitter
@sugree and Twitter
Behind the madness
Behind the madness
drupal.in.th
drupal.in.th
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Readme Read Sugree
Readme Read Sugree
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Drupal: blog and beyond
Drupal: blog and beyond
The Spirit of Open Source
The Spirit of Open Source
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Twitter Rules
Twitter Rules
Dernier
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Igalia
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Neo4j
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
The Digital Insurer
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
V3cube
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
Pooja Nehwal
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
The Digital Insurer
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
gurkirankumar98700
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
Dernier
(20)
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
1.
Automatic Self-Tuning Architecture
for Batch Scheduler on Large Scale Computing System
2.
I am Sugree
Phatanapherom from Kasetsart University.
3.
This research is
a co-work with Asst. Prof. Putchong Uthayopas.
4.
Ready, steady, go.
5.
What is batch
scheduler?
6.
Batch scheduler is
responsible to schedule jobs to execute on resources at the right time.
7.
Why do we
need batch scheduler?
8.
To utilize resources
efficiently.
9.
To finish all
jobs as fast as possible.
10.
To minimize power
consumption.
11.
In general, it
is so called "resource scheduling problem".
12.
Jobs, Resources and
Time time resources
13.
In this research,
main criteria is to minimize cost to run the resources.
14.
Back to the
past, most works focused on improving algorithms.
15.
To simplify the
problem, this research limits scope job characteristics to independent sequential jobs.
16.
In short, a
job contains the one and only one task.
17.
In other words,
job = task.
18.
Scheduling Algorithms Scheduling
On-line Batch RR OLB MET MCT MinMin MaxMin Sufferage XSufferage CMinMin CMaxMin CSufferage
19.
There are on-line
and batch scheduling.
20.
The most simple
algorithm is "Round Robin".
21.
"Opportunistic Load Balancing"
assigns job to the next available machine.
22.
"Minimum Execution Time"
assigns job to the fastest machine.
23.
"Minimum Completion Time"
assigns job to the machine with minimum completion time for that job.
24.
Next are batch
scheduling algorithms.
25.
"MinMin" assigns shortest
job to the fastest machine.
26.
"MaxMin" assign longest
job to the fastest machine.
27.
"Sufferage" is reassignable
MaxMin.
28.
"XSufferage" is Sufferage
with data locality.
29.
CMinMin, CMaxMin and
CSufferage are derivative with costing.
30.
How to verify?
How to evaluate?
31.
The answer is
simulation. Why?
32.
Closed. Controllable. Reproducible.
33.
Simulation is assumption
and modeling.
34.
Grid is a
meta-scheduler and underlying cluster schedulers managing hosts.
35.
Grid Grid Scheduler
Cluster Scheduler Host Cluster Scheduler Cluster Scheduler jobs Host
36.
Interconnection between scheduler
and processors are dedicated.
37.
Network Scheduler Processor
Storage Processor Processor Processor
38.
Job consists of
inputs, outputs and executable.
39.
Job Executable Input
Output Machine
40.
Operations are 2
steps; mapping and scheduling.
41.
Mapping "job" to
"machine".
42.
Schedule "job" to
the exact time.
43.
In short, the
result is generic priority index.
44.
45.
Time ready time
execution time deadline period before deadline time
46.
Cost cumulative cost
cost cost
47.
Experimented based on
GAMESS job log in ThaiGrid to assume a small and a big system and named them, KUGrid and ThaiGrid, respectively.
48.
Makespan and cost
are observed.
49.
Makespan is the
period of time from when the first job submitted to the last job finished.
50.
Price-Performance
51.
Cost
52.
Makespan
53.
Looks great! Any
problems? Yes!
54.
Priority index contains
5 factors. What are the right values?
55.
What are the
factors of those factors?
56.
There are so
many dependencies. Job characteristics. Resource characteristics. User characteristics.
57.
This problem is
so called "Multi-variate Optimization".
58.
Plus, a bit
more complex with evaluation in simulator.
59.
How to solve?
60.
Optimization Architecture Optimizer
Simulator Simulator Simulator Simulator Batch Scheduler Monitoring System Accounting System
61.
Optimization Algorithm?
62.
Particle Swarm Optimization
is selected as the first one to try.
63.
The position of
each particle in n-dimension plane represents solution.
64.
PSO is social
influence in various scopes.
65.
Local, neighbor and
global.
66.
Usually, one trust
oneself, friends and the world, respectively. The level of trust.
67.
PSO
68.
How to fully
automate self-tuning process?
69.
Historical data are
the key.
70.
The quality of
solution depends on optimizer.
71.
Running optimizer longer
may return better solution.
72.
Precision of using
historical data depends on data period and amount of data.
73.
How to use
historical data? Log replay or estimation.
74.
How to maximize
solution quality to near optimal?
75.
Just run more
simulations using the whole grid system to optimize itself at night!
76.
Results? Please accept
my apologize. They are not published yet.
77.
Conclusion.
78.
Flexible algorithms introduce
more adjustable factors.
79.
The factors are
vary from time to time.
80.
In other view,
these algorithms are improved by external optimization periodically.
81.
Particle swarm optimization
is selected to solve multi-variate optimization.
82.
Improve scheduler by
scheduler itself.
83.
Any questions?
Télécharger maintenant