SlideShare une entreprise Scribd logo
1  sur  12
Télécharger pour lire hors ligne
BT5	
Concurrent	Session	
11/9/17	11:30	AM	
	
	
	
	
	
Risk	Aware,	Not	Risk	Averse		
	
Presented	by:	
	
Siva	Katir	
PlayFab,	Inc.	
	
Brought	to	you	by:		
		
	
	
	
	
350	Corporate	Way,	Suite	400,	Orange	Park,	FL	32073		
888---268---8770	··	904---278---0524	-	info@techwell.com	-	https://www.techwell.com/
Siva	Katir	
PlayFab,	Inc.	
	
Siva	Katir	has	more	than	a	dozen	years	of	experience	developing	cloud,	desktop,	
and	mobile	systems.	Currently	he	works	for	PlayFab,	a	backend-as-a-service	for	
some	of	the	largest	video	games	today,	where	he	wrestles	with	maintaining	near-
perfect	uptime	on	a	massive	scale	with	a	small	team.	Siva	has	designed	and	
developed	high-availability	infrastructures—from	synchronizing	order	and	
inventory	data	between	multiple	companies	to	managing	work	permits,	worker	
statuses,	and	their	physical	locations	across	multiple	oil	refineries.	His	diverse	
work	experience	includes	deploying	CI	pipelines,	managing	servers,	running	his	
own	software	company	for	six	years,	and	serving	as	CTO	of	another.	Siva	is	
passionate	about	programming	and	sharing	his	knowledge	with	others.
Risk Aware, Not Risk Averse
November 9th, 2017 Siva Katir
Better Software East - 2017
Risk Aware Buys Flexibility & Speed
PlayFab Xbox Live
79 million players a month 55 million players a month
15 employees ~500 employees
5m players per employee 100k players per employee
Last 30 Days – Features and Fixes
It Buys Stability
PlayFab: Success over Error 30 Days99.99% Uptime
First, we fail
“Why do we fall, Bruce? So we can learn to pick ourselves up.” - Thomas Wayne
Learn To Love Failure
• Humans naturally avoid discomfort
• Failure must be about learning
Embracing learning is embracing risk
“I can accept
failure, but I
can’t accept
not trying.”
- Michael Jordan
Passing The Buck
• It is not my fault
• It is my fault
taxRate = 0.8;
subTotal = “1,070.00”;
tax = subTotal * taxRate;
total = subTotal + tax;
tax and total are both “0”
Not Our Fault…
But our responsibility. PlayFab: Not Authenticated Errors
Datacenter On Fire
• System failures occur
• Still our responsibility
Then, we mitigate
Learning to pick ourselves back up.
The “Do Nothing” Method
Preparing for Risk
• Preparation means training
• Avoid failures through past lessons
“I never stopped
getting ready.
Just in case.”
- Cmdr. Chris Hadfield
Preparation Starts at Design
• Ask hard questions
• Ask lots of questions
• Never assume answers
Red Flags
– Broad solution
– No one thinks it’s a bad idea
Green Flags
– Focused
– Detailed
Document & Code Reviews
Red Flags
– It’s huge!
– Unrelated changes
Green Flags
– It’s small and focused
– Everything is related
Monitoring
• What is shipped?
• What is nominal?
• 100’s of metrics - pick 2 or 3
– Netflix: streams started &
call center calls
Red Flags
– Too many alarms
– Healthy not defined
Green Flags
– Health metrics known
– Nominal defined
Monitors need Alarms
PlayFab Website Errors (30 Days)
Now, we know
Throwing ourselves down the well.
Playing Devil’s Advocate
Why build it?
Should we build it?
Can we build it?
– No doesn’t mean don’t.
How will we build it?
Risk Aware is Hard
Risk aware is hard.
It’s hard to not cut corners to
avoid risk.
It’s hard to get buy-in for time
consuming best practices.
The quality that being risk-aware
buys is worth every penny.
The only easy day was
yesterday.
- U.S. Navy Seals
Now, we question?
"And knowing is half the battle." - GI Joe

Contenu connexe

Plus de TechWell

Plus de TechWell (20)

Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 
Scale: The Most Hyped Term in Agile Development Today
Scale: The Most Hyped Term in Agile Development TodayScale: The Most Hyped Term in Agile Development Today
Scale: The Most Hyped Term in Agile Development Today
 
Measure DevOps for Objective Continuous Improvement Practices
Measure DevOps for Objective Continuous Improvement PracticesMeasure DevOps for Objective Continuous Improvement Practices
Measure DevOps for Objective Continuous Improvement Practices
 
Microservices and Docker at Scale: The PB&J of Modern Systems
Microservices and Docker at Scale: The PB&J of Modern SystemsMicroservices and Docker at Scale: The PB&J of Modern Systems
Microservices and Docker at Scale: The PB&J of Modern Systems
 
Automation Anti-Patterns: Deal with Them
Automation Anti-Patterns: Deal with ThemAutomation Anti-Patterns: Deal with Them
Automation Anti-Patterns: Deal with Them
 
Put Agile to the Test: A Case Study for Test Agility on a Large IT Project
Put Agile to the Test: A Case Study for Test Agility on a Large IT ProjectPut Agile to the Test: A Case Study for Test Agility on a Large IT Project
Put Agile to the Test: A Case Study for Test Agility on a Large IT Project
 
Sustaining Agility—After the Consultants Leave
Sustaining Agility—After the Consultants LeaveSustaining Agility—After the Consultants Leave
Sustaining Agility—After the Consultants Leave
 
It's All in Your Head: Use Neuroscience to Improve Performance
It's All in Your Head: Use Neuroscience to Improve PerformanceIt's All in Your Head: Use Neuroscience to Improve Performance
It's All in Your Head: Use Neuroscience to Improve Performance
 
To Estimate or Not to Estimate, is that the Question?
To Estimate or Not to Estimate, is that the Question?To Estimate or Not to Estimate, is that the Question?
To Estimate or Not to Estimate, is that the Question?
 

Dernier

The title is not connected to what is inside
The title is not connected to what is insideThe title is not connected to what is inside
The title is not connected to what is inside
shinachiaurasa2
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
VishalKumarJha10
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 

Dernier (20)

Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
 
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) SolutionIntroducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
 
The title is not connected to what is inside
The title is not connected to what is insideThe title is not connected to what is inside
The title is not connected to what is inside
 
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students
 
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
Exploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdfExploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdf
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
 
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnVTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
 
SHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions PresentationSHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions Presentation
 
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 

Risk Aware, Not Risk Averse