SlideShare une entreprise Scribd logo

Gaming dev ops - Eduardo Saito

•
•
D
Devopsdays
TechnologieDesign
1  sur  20
Télécharger pour lire hors ligne
Alert workflow in Gaming DevOps Eduardo Saito Director of Engineering - Server Operations GREE International November 2013
Traditional Alert workflow Ops NOC SME (Network, DBA,…) Dev
Traditional Alert workflow Ops NOC SME (Network, DBA,…) Dev
Alert workflow – previous Critical
Alert workflow – previous Ops Critical Dev
Alert workflow – previous Ops: where’s the runbook for this? Ops: app bug or system issue? Ops: who’s the devel of this game? Phone #? Ops: I can’t find the developer… who’s his manager? Critical Critical NonCritical Ops Dev
Alert workflow 2.0 Ops: where’s the runbook for this? Ops: app bug or system issue? Ops: who’s the devel of this game? Phone #? Ops: I can’t find the developer… who’s his manager? Ops Critical Dev
Alert Workflow 3.0 - current Ops Dev, Project X, Server
Alert Workflow 3.0 - current Ops Dev, Project X, Server Each alert go directly to the right team that can resolve it ! Dev, Project Y, Client, Android Dev, …
Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
Alerts go to the person that can resolve Type App-level alerts can beChecked byby issuesto page? triggered Scope Who in: ELB System-level • Load balancer ELB Server-side • health-check Client-side •  iOS Check cpu / •  Android Pingdom / disk / memory / network App-level Pingdom Ops team Nagios Application issues / bugs No one – email alert only Dev and Ops teams
Dev and Ops are responsible Team On-call Ops 8 Dev 32, from 20 games (Serverside or client-side Android or iOS) Analytics 5
Big display dashboard = quick status
Big display dashboard = quick status
IM Bot = better communication Skype Bot informs in the game channel that an alert was triggered
IM Bot = better communication Ops and Dev receive the alert, and troubleshoot
IM Bot = better communication Skype Bot detects issue is resolved and send allclear
Thank You! eduardo.saito@gree.net We’re hiring! Vancouver and San Francisco http://gree-corp.com/jobs

Recommandé

Mobile application testing
Mobile application testingMobile application testing
Mobile application testingshanu_123
 
Apps Qualification for the NAO Store
Apps Qualification for the NAO StoreApps Qualification for the NAO Store
Apps Qualification for the NAO StoreNicolas Rigaud
 
Mobile First: How To Approach Mobile Testing!
Mobile First: How To Approach Mobile Testing!Mobile First: How To Approach Mobile Testing!
Mobile First: How To Approach Mobile Testing!Usersnap
 
Presentation (software engineering)
Presentation (software engineering)Presentation (software engineering)
Presentation (software engineering)Abdullah Al Shiam
 
KrankyGeek WeRTC Conference 2014
KrankyGeek WeRTC Conference 2014KrankyGeek WeRTC Conference 2014
KrankyGeek WeRTC Conference 2014Jonathan Nolen
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testingBugRaptors
 
.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew tamir dresher.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew tamir dresherTamir Dresher
 
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...Eduardo Saito
 

Recommandé

Mobile application testing
Mobile application testingMobile application testing
Mobile application testingshanu_123
 
Apps Qualification for the NAO Store
Apps Qualification for the NAO StoreApps Qualification for the NAO Store
Apps Qualification for the NAO StoreNicolas Rigaud
 
Mobile First: How To Approach Mobile Testing!
Mobile First: How To Approach Mobile Testing!Mobile First: How To Approach Mobile Testing!
Mobile First: How To Approach Mobile Testing!Usersnap
 
Presentation (software engineering)
Presentation (software engineering)Presentation (software engineering)
Presentation (software engineering)Abdullah Al Shiam
 
KrankyGeek WeRTC Conference 2014
KrankyGeek WeRTC Conference 2014KrankyGeek WeRTC Conference 2014
KrankyGeek WeRTC Conference 2014Jonathan Nolen
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testingBugRaptors
 
.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew tamir dresher.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew tamir dresherTamir Dresher
 
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...
AWS re:Invent 2013 - MBL303 Gaming Ops - Running High-performance Ops for Mob...Eduardo Saito
 
451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint Security451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint SecurityAdrian Sanabria
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecurityTao Xie
 
Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)Giacomo Bergami
 
The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management Rundeck
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Rundeck
 
Are Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping ProgrammersAre Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping ProgrammersChris Parnin
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Rundeck
 
File hippo
File hippoFile hippo
File hippoSandeep Kamath
 
Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)Scott Wlaschin
 
Do Bugs Reside in Complex Code?
Do Bugs Reside in Complex Code?Do Bugs Reside in Complex Code?
Do Bugs Reside in Complex Code?CISPA Helmholtz Center for Information Security
 
Machine programming
Machine programmingMachine programming
Machine programmingDESMOND YUEN
 
Let's Make the PAIN Visible!
Let's Make the PAIN Visible!Let's Make the PAIN Visible!
Let's Make the PAIN Visible!Arty Starr
 
Cracking the mobile application code
Cracking the mobile application codeCracking the mobile application code
Cracking the mobile application codeSreenarayan A
 
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should HaveAppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should HaveRobert Grupe, CSSLP CISSP PE PMP
 
PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016Guillermo A. Fisher
 
SAD15 - Maintenance
SAD15 - MaintenanceSAD15 - Maintenance
SAD15 - MaintenanceMichael Heron
 
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...Papitha Velumani
 
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022lior mazor
 
How to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot EnvironmentsHow to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot EnvironmentsDevOps.com
 
Ch01-whyTest.pptx
Ch01-whyTest.pptxCh01-whyTest.pptx
Ch01-whyTest.pptxSteveIrwin25
 
Dev opsdays scriptcode
Dev opsdays scriptcodeDev opsdays scriptcode
Dev opsdays scriptcodeDevopsdays
 
Zero to hero - Geoff Webb
Zero to hero - Geoff WebbZero to hero - Geoff Webb
Zero to hero - Geoff WebbDevopsdays
 

Contenu connexe

Similaire à Gaming dev ops - Eduardo Saito

451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint Security451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint SecurityAdrian Sanabria
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecurityTao Xie
 
Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)Giacomo Bergami
 
The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management Rundeck
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Rundeck
 
Are Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping ProgrammersAre Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping ProgrammersChris Parnin
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Rundeck
 
Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)Scott Wlaschin
 
Machine programming
Machine programmingMachine programming
Machine programmingDESMOND YUEN
 
Let's Make the PAIN Visible!
Let's Make the PAIN Visible!Let's Make the PAIN Visible!
Let's Make the PAIN Visible!Arty Starr
 
Cracking the mobile application code
Cracking the mobile application codeCracking the mobile application code
Cracking the mobile application codeSreenarayan A
 
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should HaveAppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should HaveRobert Grupe, CSSLP CISSP PE PMP
 
PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016Guillermo A. Fisher
 
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...Papitha Velumani
 
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022lior mazor
 
How to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot EnvironmentsHow to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot EnvironmentsDevOps.com
 

Similaire à Gaming dev ops - Eduardo Saito (20)

451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint Security451 and Cylance - The Roadmap To Better Endpoint Security
451 and Cylance - The Roadmap To Better Endpoint Security
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and Security
 
Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)Android Internals (This is not the droid you’re loking for...)
Android Internals (This is not the droid you’re loking for...)
 
The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management The Last Mile Continued: Incident Management
The Last Mile Continued: Incident Management
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE
 
Are Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping ProgrammersAre Automated Debugging Techniques Actually Helping Programmers
Are Automated Debugging Techniques Actually Helping Programmers
 
Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE Incident Management in the Age of DevOps and SRE
Incident Management in the Age of DevOps and SRE
 
File hippo
File hippoFile hippo
File hippo
 
Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)Designing with capabilities (DDD-EU 2017)
Designing with capabilities (DDD-EU 2017)
 
Do Bugs Reside in Complex Code?
Do Bugs Reside in Complex Code?Do Bugs Reside in Complex Code?
Do Bugs Reside in Complex Code?
 
Machine programming
Machine programmingMachine programming
Machine programming
 
Let's Make the PAIN Visible!
Let's Make the PAIN Visible!Let's Make the PAIN Visible!
Let's Make the PAIN Visible!
 
Cracking the mobile application code
Cracking the mobile application codeCracking the mobile application code
Cracking the mobile application code
 
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should HaveAppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
 
PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016PHP, AWS, and Sleep - Hampton Roads DevFest 2016
PHP, AWS, and Sleep - Hampton Roads DevFest 2016
 
SAD15 - Maintenance
SAD15 - MaintenanceSAD15 - Maintenance
SAD15 - Maintenance
 
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
Catch Me If You Can- Evaluating Android Anti-Malware Against Transformation A...
 
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
 
How to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot EnvironmentsHow to Manage the Risk of your Polyglot Environments
How to Manage the Risk of your Polyglot Environments
 
Ch01-whyTest.pptx
Ch01-whyTest.pptxCh01-whyTest.pptx
Ch01-whyTest.pptx
 

Plus de Devopsdays

Dev opsdays scriptcode
Dev opsdays scriptcodeDev opsdays scriptcode
Dev opsdays scriptcodeDevopsdays
 
Zero to hero - Geoff Webb
Zero to hero - Geoff WebbZero to hero - Geoff Webb
Zero to hero - Geoff WebbDevopsdays
 
Letting go gavin - Mc Donald
Letting go gavin - Mc DonaldLetting go gavin - Mc Donald
Letting go gavin - Mc DonaldDevopsdays
 
Dw tpain - Gordon Klok
Dw tpain - Gordon KlokDw tpain - Gordon Klok
Dw tpain - Gordon KlokDevopsdays
 
Dev ops finishes what agile started - Manfred Moser
Dev ops finishes what agile started - Manfred MoserDev ops finishes what agile started - Manfred Moser
Dev ops finishes what agile started - Manfred MoserDevopsdays
 
Game of thrones - Jonathan Thorpe
Game of thrones - Jonathan ThorpeGame of thrones - Jonathan Thorpe
Game of thrones - Jonathan ThorpeDevopsdays
 
Heka - Rob Miller
Heka - Rob MillerHeka - Rob Miller
Heka - Rob MillerDevopsdays
 
From the classroom to the cloud a journey with node.js - Christopher Hogue
From the classroom to the cloud a journey with node.js - Christopher HogueFrom the classroom to the cloud a journey with node.js - Christopher Hogue
From the classroom to the cloud a journey with node.js - Christopher HogueDevopsdays
 
Dev ops at mobify - Kyle Young
Dev ops at mobify - Kyle YoungDev ops at mobify - Kyle Young
Dev ops at mobify - Kyle YoungDevopsdays
 
Your business needs devops, so don’t follow - Brian johnson
Your business needs devops, so don’t follow - Brian johnson Your business needs devops, so don’t follow - Brian johnson
Your business needs devops, so don’t follow - Brian johnson Devopsdays
 
Test kitchen 1.0 - Fletcher Nichol
Test kitchen 1.0 - Fletcher NicholTest kitchen 1.0 - Fletcher Nichol
Test kitchen 1.0 - Fletcher NicholDevopsdays
 
Living system or build factory - Chris Maxwell
Living system or build factory - Chris MaxwellLiving system or build factory - Chris Maxwell
Living system or build factory - Chris MaxwellDevopsdays
 
From vagrant to production - Mark Eijsermans
From vagrant to production - Mark EijsermansFrom vagrant to production - Mark Eijsermans
From vagrant to production - Mark EijsermansDevopsdays
 
Dev ops lessons learned - Michael Collins
Dev ops lessons learned - Michael CollinsDev ops lessons learned - Michael Collins
Dev ops lessons learned - Michael CollinsDevopsdays
 
Building for operations - Reinhardt Quelle
Building for operations - Reinhardt QuelleBuilding for operations - Reinhardt Quelle
Building for operations - Reinhardt QuelleDevopsdays
 
Taking devops to the Next Level - Max Martin
Taking devops to the Next Level - Max MartinTaking devops to the Next Level - Max Martin
Taking devops to the Next Level - Max MartinDevopsdays
 
Sensu intro - Sean Porter
Sensu intro - Sean PorterSensu intro - Sean Porter
Sensu intro - Sean PorterDevopsdays
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John BrittonDevopsdays
 
Effective monitoring with statsd - Alexis lê-quôc
Effective monitoring with statsd - Alexis lê-quôcEffective monitoring with statsd - Alexis lê-quôc
Effective monitoring with statsd - Alexis lê-quôcDevopsdays
 
Being healthy dev and ops in cookpad - Issei Naruta
Being healthy dev and ops in cookpad - Issei NarutaBeing healthy dev and ops in cookpad - Issei Naruta
Being healthy dev and ops in cookpad - Issei NarutaDevopsdays
 

Plus de Devopsdays (20)

Dev opsdays scriptcode
Dev opsdays scriptcodeDev opsdays scriptcode
Dev opsdays scriptcode
 
Zero to hero - Geoff Webb
Zero to hero - Geoff WebbZero to hero - Geoff Webb
Zero to hero - Geoff Webb
 
Letting go gavin - Mc Donald
Letting go gavin - Mc DonaldLetting go gavin - Mc Donald
Letting go gavin - Mc Donald
 
Dw tpain - Gordon Klok
Dw tpain - Gordon KlokDw tpain - Gordon Klok
Dw tpain - Gordon Klok
 
Dev ops finishes what agile started - Manfred Moser
Dev ops finishes what agile started - Manfred MoserDev ops finishes what agile started - Manfred Moser
Dev ops finishes what agile started - Manfred Moser
 
Game of thrones - Jonathan Thorpe
Game of thrones - Jonathan ThorpeGame of thrones - Jonathan Thorpe
Game of thrones - Jonathan Thorpe
 
Heka - Rob Miller
Heka - Rob MillerHeka - Rob Miller
Heka - Rob Miller
 
From the classroom to the cloud a journey with node.js - Christopher Hogue
From the classroom to the cloud a journey with node.js - Christopher HogueFrom the classroom to the cloud a journey with node.js - Christopher Hogue
From the classroom to the cloud a journey with node.js - Christopher Hogue
 
Dev ops at mobify - Kyle Young
Dev ops at mobify - Kyle YoungDev ops at mobify - Kyle Young
Dev ops at mobify - Kyle Young
 
Your business needs devops, so don’t follow - Brian johnson
Your business needs devops, so don’t follow - Brian johnson Your business needs devops, so don’t follow - Brian johnson
Your business needs devops, so don’t follow - Brian johnson
 
Test kitchen 1.0 - Fletcher Nichol
Test kitchen 1.0 - Fletcher NicholTest kitchen 1.0 - Fletcher Nichol
Test kitchen 1.0 - Fletcher Nichol
 
Living system or build factory - Chris Maxwell
Living system or build factory - Chris MaxwellLiving system or build factory - Chris Maxwell
Living system or build factory - Chris Maxwell
 
From vagrant to production - Mark Eijsermans
From vagrant to production - Mark EijsermansFrom vagrant to production - Mark Eijsermans
From vagrant to production - Mark Eijsermans
 
Dev ops lessons learned - Michael Collins
Dev ops lessons learned - Michael CollinsDev ops lessons learned - Michael Collins
Dev ops lessons learned - Michael Collins
 
Building for operations - Reinhardt Quelle
Building for operations - Reinhardt QuelleBuilding for operations - Reinhardt Quelle
Building for operations - Reinhardt Quelle
 
Taking devops to the Next Level - Max Martin
Taking devops to the Next Level - Max MartinTaking devops to the Next Level - Max Martin
Taking devops to the Next Level - Max Martin
 
Sensu intro - Sean Porter
Sensu intro - Sean PorterSensu intro - Sean Porter
Sensu intro - Sean Porter
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John Britton
 
Effective monitoring with statsd - Alexis lê-quôc
Effective monitoring with statsd - Alexis lê-quôcEffective monitoring with statsd - Alexis lê-quôc
Effective monitoring with statsd - Alexis lê-quôc
 
Being healthy dev and ops in cookpad - Issei Naruta
Being healthy dev and ops in cookpad - Issei NarutaBeing healthy dev and ops in cookpad - Issei Naruta
Being healthy dev and ops in cookpad - Issei Naruta
 

Dernier

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfhans926745
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdfChristopherTHyatt
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 

Dernier (20)

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 

Gaming dev ops - Eduardo Saito

  • 1. Alert workflow in Gaming DevOps Eduardo Saito Director of Engineering - Server Operations GREE International November 2013
  • 2. Traditional Alert workflow Ops NOC SME (Network, DBA,…) Dev
  • 3. Traditional Alert workflow Ops NOC SME (Network, DBA,…) Dev
  • 4. Alert workflow – previous Critical
  • 5. Alert workflow – previous Ops Critical Dev
  • 6. Alert workflow – previous Ops: where’s the runbook for this? Ops: app bug or system issue? Ops: who’s the devel of this game? Phone #? Ops: I can’t find the developer… who’s his manager? Critical Critical NonCritical Ops Dev
  • 7. Alert workflow 2.0 Ops: where’s the runbook for this? Ops: app bug or system issue? Ops: who’s the devel of this game? Phone #? Ops: I can’t find the developer… who’s his manager? Ops Critical Dev
  • 8. Alert Workflow 3.0 - current Ops Dev, Project X, Server
  • 9. Alert Workflow 3.0 - current Ops Dev, Project X, Server Each alert go directly to the right team that can resolve it ! Dev, Project Y, Client, Android Dev, …
  • 10. Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
  • 11. Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
  • 12. Alerts go to the person that can resolve Type Scope Checked by Who to page? ELB Load balancer health-check ELB No one – email alert only System-level Check cpu / disk / memory / network Pingdom / Nagios Ops team App-level Application issues / bugs Pingdom Dev and Ops teams
  • 13. Alerts go to the person that can resolve Type App-level alerts can beChecked byby issuesto page? triggered Scope Who in: ELB System-level • Load balancer ELB Server-side • health-check Client-side •  iOS Check cpu / •  Android Pingdom / disk / memory / network App-level Pingdom Ops team Nagios Application issues / bugs No one – email alert only Dev and Ops teams
  • 14. Dev and Ops are responsible Team On-call Ops 8 Dev 32, from 20 games (Serverside or client-side Android or iOS) Analytics 5
  • 15. Big display dashboard = quick status
  • 16. Big display dashboard = quick status
  • 17. IM Bot = better communication Skype Bot informs in the game channel that an alert was triggered
  • 18. IM Bot = better communication Ops and Dev receive the alert, and troubleshoot
  • 19. IM Bot = better communication Skype Bot detects issue is resolved and send allclear
  • 20. Thank You! eduardo.saito@gree.net We’re hiring! Vancouver and San Francisco http://gree-corp.com/jobs