SlideShare a Scribd company logo
1 of 45
Download to read offline
IS IT WORTH MY TIME?
AUTOMATION
@JRHUNT
J. RANDALL HUNT
@JRHUNT
J. RANDALL HUNT
AGENDA
▸Why automate?
▸When to automate?
▸How to automate?
▸Stories from the field
WHY THE HELL AM
I DOING THIS?
EXAMPLE!
sorry, I like big fonts!
WHY?
FIND LARGEST FILE IN A DIRECTORY
WHY?
FIND LARGEST FILE IN A DIRECTORY
▸ ls -S | head -1
▸ takes about 4 seconds to type
▸ takes about 45 seconds to google
WHY?
FIND LARGEST FILE IN A DIRECTORY
▸ ls -S | head -1
▸ takes about 4 seconds to type
▸ takes about 45 seconds to google
▸ largest () { ls -S | head -1 }
▸ takes about 15 seconds to type...
▸ and then 2 seconds forever after
WHY?
FIND LARGEST FILE IN A DIRECTORY
▸ ls -S | head -1
▸ takes about 4 seconds to type
▸ takes about 45 seconds to google
▸ largest () { ls -S | head -1 }
▸ takes about 15 seconds to type...
▸ and then 2 seconds forever after
WHY?
FIND LARGEST FILE IN A DIRECTORY
WHY?
FIND LARGEST FILE IN A DIRECTORY
▸ find . -type f -print0 | xargs -0 du -h | sort -hr | head -1
WHY?
FIND LARGEST FILE IN A DIRECTORY
▸ find . -type f -print0 | xargs -0 du -h | sort -hr | head -1
WHY?
5 HOURS OF FASCINATED CLICKING LATER
https://xkcd.com/1319/
https://xkcd.com/1319/
TIAD - Is Automation Worth My Time?
https://xkcd.com/974/
AWS SIMPLE SYSTEMS
MANAGEMENT
WHEN SHOULD I
DO THIS?
https://xkcd.com/1205/
https://xkcd.com/1205/
https://xkcd.com/1205/
PERFORMANCE
QUALITY
WHEN?
CHESS GAME WINNERS
▸ ~14 million games recorded in ASCII
▸ Read in files, aggregate winners, report stats
▸ Sounds like a job for:
AMAZON EMR
AMAZING PERFORMANCE --
ENTIRE DATASET COMPUTED IN
JUST 20 MINUTES FOR <$10!!
HMM....
WHEN?
CHESS GAME WINNERS
▸ ~14 million games == ~10 gigs
▸ Read in files, aggregate winners, report stats == grep /
awk
▸ Sounds like a job for: THE COMMAND LINE!
find . -type f -name '*.pgn' -print0 | xargs -0 -n4 -P4 mawk '/
Result/ { split($0, a, "-"); res = substr(a[1], length(a[1]), 1); if (res ==
1) white++; if (res == 0) black++; if (res == 2) draw++ } END { print
white+black+draw, white, black, draw }' | mawk '{games += $1;
white += $2; black += $3; draw += $4; } END { print games, white,
black, draw }'
Thanks to Adam Drake for figuring this out!
http://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html
30 SECONDS
OK, TELL ME
HOW!
HOW?
I HAVE NO IDEA WHAT I'M DOING, BUT THIS IS WHAT I DO
1. Identify constraints
‣ Money, Business Goals, Time, Performance, Beauty
2. Pursue mitigating factors within constraints
3. Relentlessly pursue constraints until the laws of physics
tell you to stop
4. Iterate
WHAT ARE THE ACTUAL
CONSTRAINTS?
CONSTRAINTS
HARDWARE
▸ AWS EBS gp2 volumes have max ~160MiB/s*
▸ AWS EBS io2 volumes have max ~320 MiB/s*
▸ Realistic max network performance of ~800 MiB/s*
▸ CPU performance varies on instance type but up to 8
terraflops per instance
CONSTRAINTS
BUSINESS CONSTRAINTS (MONEY)
▸ How much money can we spend on this problem?
▸ How much time can we spend on this problem?
▸ Will a partially automated solution allow us to kick this
problem down the road?
▸ Is it cheaper/easier/more accurate to hire humans?
CONSTRAINTS
QUALITY CONSTRAINTS (AI VS HUMANS)
▸ How much better are humans at solving this?
TIAD - Is Automation Worth My Time?
CRAZY STUFF
WE'VE AUTOMATED
TIAD - Is Automation Worth My Time?
BRINGING IT BACK
HOME
CLOSING THOUGHTS
INFRASTRUCTURE AS CODE == AUTOMATION AS CODE
▸ Chef, Puppet, Ansible or something similar is vital
(you already know this)
▸ AWS CloudFormation lets you launch entire
FLEETS and NETWORKS in moments. This system
is free...
▸ Heterogeneity is the death of automation (cattle
not pets)
CLOSING THOUGHTS
STOP DOING THE THINGS YOU ALREADY KNOW HOW TO DO
▸ You already know how to run a reverse proxy -- write the
code once and be done with it (Amazon API Gateway).
▸ You already know how to maintain your database (AWS
RDS) set up protocols and be done with it
▸ You already know how to manage security and
permissions, don't set it up yet again
CLOSING THOUGHTS
▸ Identify the biggest automation wins
▸ Pursue your automation by pursuing constraints
▸ Add humanity to the process
▸ Don't overthink it, but definitely think about it
AWS User Groups
Lille
Paris
Rennes
Nantes
Bordeaux
Lyon
Montpellier
Toulouse
facebook.com/groups/AWSFrance/
@aws_actus
AWS Enterprise Summit – 27/10/2016, Paris
http://amzn.to/1X2yp0i

More Related Content

Similar to TIAD - Is Automation Worth My Time?

Python quickstart for programmers: Python Kung Fu
Python quickstart for programmers: Python Kung FuPython quickstart for programmers: Python Kung Fu
Python quickstart for programmers: Python Kung Fuclimatewarrior
 
Intro to Programming
Intro to ProgrammingIntro to Programming
Intro to ProgrammingVincent Sam
 
Writing Modular Command-line Apps with App::Cmd
Writing Modular Command-line Apps with App::CmdWriting Modular Command-line Apps with App::Cmd
Writing Modular Command-line Apps with App::CmdRicardo Signes
 
Get-Help: An intro to PowerShell and how to Use it for Evil
Get-Help: An intro to PowerShell and how to Use it for EvilGet-Help: An intro to PowerShell and how to Use it for Evil
Get-Help: An intro to PowerShell and how to Use it for Eviljaredhaight
 
Get the most out of your Mac OS X
Get the most out of your Mac OS XGet the most out of your Mac OS X
Get the most out of your Mac OS XJanne Warén
 
Linux: Beyond ls and cd
Linux: Beyond ls and cdLinux: Beyond ls and cd
Linux: Beyond ls and cdjacko91
 
Úvod do programování 5
Úvod do programování 5Úvod do programování 5
Úvod do programování 5Karel Minarik
 
Developer-friendly task queues: what we learned building MRQ, Sylvain Zimmer
Developer-friendly task queues: what we learned building MRQ, Sylvain ZimmerDeveloper-friendly task queues: what we learned building MRQ, Sylvain Zimmer
Developer-friendly task queues: what we learned building MRQ, Sylvain ZimmerPôle Systematic Paris-Region
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneSylvain Zimmer
 
Effective codereview | Dave Liddament | CODEiD
Effective codereview | Dave Liddament | CODEiDEffective codereview | Dave Liddament | CODEiD
Effective codereview | Dave Liddament | CODEiDCODEiD PHP Community
 

Similar to TIAD - Is Automation Worth My Time? (12)

Shell scripting
Shell scriptingShell scripting
Shell scripting
 
Python quickstart for programmers: Python Kung Fu
Python quickstart for programmers: Python Kung FuPython quickstart for programmers: Python Kung Fu
Python quickstart for programmers: Python Kung Fu
 
Intro to Programming
Intro to ProgrammingIntro to Programming
Intro to Programming
 
Writing Modular Command-line Apps with App::Cmd
Writing Modular Command-line Apps with App::CmdWriting Modular Command-line Apps with App::Cmd
Writing Modular Command-line Apps with App::Cmd
 
C# to python
C# to pythonC# to python
C# to python
 
Get-Help: An intro to PowerShell and how to Use it for Evil
Get-Help: An intro to PowerShell and how to Use it for EvilGet-Help: An intro to PowerShell and how to Use it for Evil
Get-Help: An intro to PowerShell and how to Use it for Evil
 
Get the most out of your Mac OS X
Get the most out of your Mac OS XGet the most out of your Mac OS X
Get the most out of your Mac OS X
 
Linux: Beyond ls and cd
Linux: Beyond ls and cdLinux: Beyond ls and cd
Linux: Beyond ls and cd
 
Úvod do programování 5
Úvod do programování 5Úvod do programování 5
Úvod do programování 5
 
Developer-friendly task queues: what we learned building MRQ, Sylvain Zimmer
Developer-friendly task queues: what we learned building MRQ, Sylvain ZimmerDeveloper-friendly task queues: what we learned building MRQ, Sylvain Zimmer
Developer-friendly task queues: what we learned building MRQ, Sylvain Zimmer
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
 
Effective codereview | Dave Liddament | CODEiD
Effective codereview | Dave Liddament | CODEiDEffective codereview | Dave Liddament | CODEiD
Effective codereview | Dave Liddament | CODEiD
 

More from Randall Hunt

WhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter BotWhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter BotRandall Hunt
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019Randall Hunt
 
Randall's re:Invent Recap
Randall's re:Invent RecapRandall's re:Invent Recap
Randall's re:Invent RecapRandall Hunt
 
Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017Randall Hunt
 
Canada DevOps Conference
Canada DevOps ConferenceCanada DevOps Conference
Canada DevOps ConferenceRandall Hunt
 
A Century Of Weather Data - Midwest.io
A Century Of Weather Data - Midwest.ioA Century Of Weather Data - Midwest.io
A Century Of Weather Data - Midwest.ioRandall Hunt
 
MongoDB at LAHacks :)
MongoDB at LAHacks :)MongoDB at LAHacks :)
MongoDB at LAHacks :)Randall Hunt
 
Schema Design in MongoDB - TriMug Meetup North Carolina
Schema Design in MongoDB - TriMug Meetup North CarolinaSchema Design in MongoDB - TriMug Meetup North Carolina
Schema Design in MongoDB - TriMug Meetup North CarolinaRandall Hunt
 
Replication MongoDB Days 2013
Replication MongoDB Days 2013Replication MongoDB Days 2013
Replication MongoDB Days 2013Randall Hunt
 
Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Randall Hunt
 
Replication and replica sets
Replication and replica setsReplication and replica sets
Replication and replica setsRandall Hunt
 

More from Randall Hunt (13)

WhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter BotWhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter Bot
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019
 
Where ml ai_heavy
Where ml ai_heavyWhere ml ai_heavy
Where ml ai_heavy
 
Randall's re:Invent Recap
Randall's re:Invent RecapRandall's re:Invent Recap
Randall's re:Invent Recap
 
Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017
 
Canada DevOps Conference
Canada DevOps ConferenceCanada DevOps Conference
Canada DevOps Conference
 
Git
GitGit
Git
 
A Century Of Weather Data - Midwest.io
A Century Of Weather Data - Midwest.ioA Century Of Weather Data - Midwest.io
A Century Of Weather Data - Midwest.io
 
MongoDB at LAHacks :)
MongoDB at LAHacks :)MongoDB at LAHacks :)
MongoDB at LAHacks :)
 
Schema Design in MongoDB - TriMug Meetup North Carolina
Schema Design in MongoDB - TriMug Meetup North CarolinaSchema Design in MongoDB - TriMug Meetup North Carolina
Schema Design in MongoDB - TriMug Meetup North Carolina
 
Replication MongoDB Days 2013
Replication MongoDB Days 2013Replication MongoDB Days 2013
Replication MongoDB Days 2013
 
Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013
 
Replication and replica sets
Replication and replica setsReplication and replica sets
Replication and replica sets
 

Recently uploaded

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 

Recently uploaded (20)

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 

TIAD - Is Automation Worth My Time?