SlideShare une entreprise Scribd logo
1  sur  31
Télécharger pour lire hors ligne
Seattle | September 16-17, 2019
Automated production deployments
with HIPAA/HITRUST compliance
MATT LAVIN
Seattle | September 16-17, 2019
I’m an impatient developer
@mdlavin at @LifeOmic
Seattle | September 16-17, 2019
I’m envious of Netflix and Instagram
Seattle | September 16-17, 2019
I work at LifeOmic
where all data is sensitive data
Seattle | September 16-17, 2019
The Sucker’s Choice
• Don’t pick between security or speed
• Get more security and more speedSecurity
Speed
Where you think the boundary is
The actual boundary
You are here
Seattle | September 16-17, 2019
Tools we use
• AWS
• Git (Bitbucket)
• Jenkins
• Jira
• Snyk
Seattle | September 16-17, 2019
From idea to production
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
Seattle | September 16-17, 2019
Implementation
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
Implementation of change and test automation
on the developers machine
Push change to a Git branch, test automation
runs in Jenkins
Seattle | September 16-17, 2019
Review and deploy
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
After approval, changes are merged into
Git master branch and Jenkins deploys to
AWS dev account
Team members review code and click ‘Approve’
Seattle | September 16-17, 2019
Request production deployment
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
Humans review the new changes being
deployed and approve if the appropriate
processes were followed
Developer creates a Jira issue to request
deployment of specific project and build to
production
Seattle | September 16-17, 2019
Production deployment
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
Jenkins watches for the Jira issue and deploys
the new project version when approved
Seattle | September 16-17, 2019
Process requirements
• Nobody can act alone to modify production
• Changes to production are auditable
• Security scans are run
Seattle | September 16-17, 2019
Automation > Humans
CM Ticket
Bitbucket PR
Code Review / Approval
CM Approvals
Deploy to AWS dev
Deploy to AWS Prod
Local development
Automate review of process compliance
Automate detailed change description
collection for Jira issue
Seattle | September 16-17, 2019
Old Change Management Ticket Flow
Developer runs job to promote
Jira emails sent to approvers
Humans review and approve
Jira ticket moves to implementing
Jira CM issue is created
Deploy job waits for approval
Deploy job resumes
Seattle | September 16-17, 2019
Automation possibilities
Creating the ticket
• What has changed
• Has anything been removed
Reviewing changes
• Bitbucket PRs for all changes
• All PRs were reviewed by peers
• Security scans have been run
Seattle | September 16-17, 2019
New Change Management Ticket Flow
Developer runs job to promote
Jira emails sent to approvers
CM Bot Updates
Jira ticket moves to implementing
Jira CM issue is created
Deploy job waits for approval
Deploy job resumes
Humans Review
Fewer details
Seattle | September 16-17, 2019
CM Bot execution
Jira Issue Created Webhook
CM Description updated
Approval (or not)
Bitbucket / Jenkins / Jira Analysis
CM Verdict in comment
Seattle | September 16-17, 2019
Git Commits
Commit 1 Commit 2 Commit 3 Commit 4 Commit 5
Merge 1 Merge 2 Merge 3
Master Branch
• Every commit must be from a LifeOmic Bitbucket user
• All commits into master are through merge commits
• No Fast Forwards or squashes
Seattle | September 16-17, 2019
Pull requests
PR 2 PR 3PR 1
• Every merge into master must be associated with a Bitbucket pull request
• Every pull request must have approvers who are not the committer
Seattle | September 16-17, 2019
Jenkins Builds
Build 1 Build 2 Build 3
• Every merge into Master starts a Jenkins build for test and deploy
Seattle | September 16-17, 2019
Change Management Jira Issues
CM 1 CM 2 CM 3
• Production deployment requests refer to a Jenkins build
Seattle | September 16-17, 2019
Verifying a promotion request
Commit 4 Commit 5
Merge 3
PR 3
Build 3
CM 1 CM 2 CM 3
Master Branch
• Find previously approved CM issue
• Find all new artifacts associated with the
request
• Verify the processes were followed
• Leave a comment in the CM issue with
with changes detected
Seattle | September 16-17, 2019
Automation
• Good at finding and listing every
single change in the new build
• Good at filling CM tickets with
details
• Very fast
Developers
• Good at remembering what they
did recently
• Good at listing just enough details
to get the CM ticket approved
• Comparatively slow
Seattle | September 16-17, 2019
Automation audit log
Seattle | September 16-17, 2019
Rejected Changes
Seattle | September 16-17, 2019
Automation
• Good at checking every single change
for peer reviews and security scans
• Good at catching edge cases which
can be hard to detect manually
• Very fast
• Reviewing at odd hours
Human Approvers
• Vulnerable to being rushed and
sometimes spot checking changes
• Good at thinking big picture for
security and compliance
• Comparatively slow
• Sleeping and spending time with family
Seattle | September 16-17, 2019
Git merge edge cases
• Detecting any changes in the merge commits
• Handling cases for PRs with multiple committers
• Detecting removed commits
• Squash merges
Seattle | September 16-17, 2019
Changing culture
• Worst case is fallback to human review
• Teams eagerly follow review process to avoid waiting on human approvals
• Security team doesn’t have to play the bad guy, the tool is the bad guy
• Motivation to automate production changes
• Shame for squash merges is a right of passage for new hires
Seattle | September 16-17, 2019
The future
• Detecting risky Terraform changes
• Allowing automatic rollbacks
• More types of security scans
Seattle | September 16-17, 2019
Continuous delivery and HIPAA compliance is possible
• We’ve open-sourced a library to help
• Utilizes JupiterOne for data collection
• https://github.com/JupiterOne/jupiter-change-management-client
Seattle | September 16-17, 2019
Thank you!
Ask me questions
@mdlavin

Contenu connexe

Tendances

GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery
 GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery
GITPro World Apr 2015 - Continuous Innovation with Rapid Software DeliverySangeeta Narayanan
 
Oscon2014 Netflix API - Top 10 Lessons Learned
Oscon2014 Netflix API - Top 10 Lessons LearnedOscon2014 Netflix API - Top 10 Lessons Learned
Oscon2014 Netflix API - Top 10 Lessons LearnedSangeeta Narayanan
 
Move Fast;Stay Safe:Developing & Deploying the Netflix API
Move Fast;Stay Safe:Developing & Deploying the Netflix APIMove Fast;Stay Safe:Developing & Deploying the Netflix API
Move Fast;Stay Safe:Developing & Deploying the Netflix APISangeeta Narayanan
 
Containers at Netflx - An Evolving Story QConSF2015
Containers at Netflx - An Evolving Story QConSF2015Containers at Netflx - An Evolving Story QConSF2015
Containers at Netflx - An Evolving Story QConSF2015Sangeeta Narayanan
 
Agile, Cloud Computing, Open Source and what's in between
Agile, Cloud Computing, Open Source and what's in between Agile, Cloud Computing, Open Source and what's in between
Agile, Cloud Computing, Open Source and what's in between Alon Girmonsky
 
The Architect is Not Convinced
The Architect is Not ConvincedThe Architect is Not Convinced
The Architect is Not Convincedduvander
 
Serverless — it all started in Vegas
Serverless — it all started in VegasServerless — it all started in Vegas
Serverless — it all started in VegasMartin Buberl
 

Tendances (8)

GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery
 GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery
GITPro World Apr 2015 - Continuous Innovation with Rapid Software Delivery
 
Oscon2014 Netflix API - Top 10 Lessons Learned
Oscon2014 Netflix API - Top 10 Lessons LearnedOscon2014 Netflix API - Top 10 Lessons Learned
Oscon2014 Netflix API - Top 10 Lessons Learned
 
Testing APIs & Microservices
Testing APIs & MicroservicesTesting APIs & Microservices
Testing APIs & Microservices
 
Move Fast;Stay Safe:Developing & Deploying the Netflix API
Move Fast;Stay Safe:Developing & Deploying the Netflix APIMove Fast;Stay Safe:Developing & Deploying the Netflix API
Move Fast;Stay Safe:Developing & Deploying the Netflix API
 
Containers at Netflx - An Evolving Story QConSF2015
Containers at Netflx - An Evolving Story QConSF2015Containers at Netflx - An Evolving Story QConSF2015
Containers at Netflx - An Evolving Story QConSF2015
 
Agile, Cloud Computing, Open Source and what's in between
Agile, Cloud Computing, Open Source and what's in between Agile, Cloud Computing, Open Source and what's in between
Agile, Cloud Computing, Open Source and what's in between
 
The Architect is Not Convinced
The Architect is Not ConvincedThe Architect is Not Convinced
The Architect is Not Convinced
 
Serverless — it all started in Vegas
Serverless — it all started in VegasServerless — it all started in Vegas
Serverless — it all started in Vegas
 

Similaire à LifeOmic Change Management Automation -- DevSecCon Seattle 2019

DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...
DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...
DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...DevSecCon
 
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...Amazon Web Services
 
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...Erkang Zheng
 
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...Amazon Web Services
 
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...CI/CD best practices for building modern applications - MAD301 - Santa Clara ...
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...Amazon Web Services
 
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...Amazon Web Services
 
Apics 2017 samelson (final) blockchain
Apics 2017 samelson (final)   blockchain Apics 2017 samelson (final)   blockchain
Apics 2017 samelson (final) blockchain Quentin Samelson
 
Analytics im DevOps Lebenszyklus
Analytics im DevOps LebenszyklusAnalytics im DevOps Lebenszyklus
Analytics im DevOps LebenszyklusSplunk
 
Serverless: What happens next will blow your mind!
Serverless: What happens next will blow your mind!Serverless: What happens next will blow your mind!
Serverless: What happens next will blow your mind!Chris Williams
 
Attacking and defending GraphQL applications: a hands-on approach
 Attacking and defending GraphQL applications: a hands-on approach Attacking and defending GraphQL applications: a hands-on approach
Attacking and defending GraphQL applications: a hands-on approachDavide Cioccia
 
GitOps is the best modern practice for CD with Kubernetes
GitOps is the best modern practice for CD with KubernetesGitOps is the best modern practice for CD with Kubernetes
GitOps is the best modern practice for CD with KubernetesVolodymyr Shynkar
 
CI/CD best practices for building modern applications - MAD310 - New York AWS...
CI/CD best practices for building modern applications - MAD310 - New York AWS...CI/CD best practices for building modern applications - MAD310 - New York AWS...
CI/CD best practices for building modern applications - MAD310 - New York AWS...Amazon Web Services
 
COMIT community day summer 2018 - main slides
COMIT community day summer 2018 - main slidesCOMIT community day summer 2018 - main slides
COMIT community day summer 2018 - main slidesComit Projects Ltd
 
Scaling Security Operations and Automating Governance: Which AWS Services Sho...
Scaling Security Operations and Automating Governance: Which AWS Services Sho...Scaling Security Operations and Automating Governance: Which AWS Services Sho...
Scaling Security Operations and Automating Governance: Which AWS Services Sho...Amazon Web Services
 
#spsclt18 vincent biret #spfx #devops
#spsclt18 vincent biret #spfx #devops#spsclt18 vincent biret #spfx #devops
#spsclt18 vincent biret #spfx #devopsVincent Biret
 
Estimation of a micro services based estimation application bhawna thakur -...
Estimation of a micro services based estimation application   bhawna thakur -...Estimation of a micro services based estimation application   bhawna thakur -...
Estimation of a micro services based estimation application bhawna thakur -...Nesma
 
Amazon CI-CD Practices for Software Development Teams
Amazon CI-CD Practices for Software Development Teams Amazon CI-CD Practices for Software Development Teams
Amazon CI-CD Practices for Software Development Teams Amazon Web Services
 
Mandated BIM! Are you ready?
Mandated BIM! Are you ready?Mandated BIM! Are you ready?
Mandated BIM! Are you ready?CCT International
 
Software Supply Chain Management with Grafeas and Kritis
Software Supply Chain Management with Grafeas and KritisSoftware Supply Chain Management with Grafeas and Kritis
Software Supply Chain Management with Grafeas and KritisAysylu Greenberg
 
Scribe online 01 best practices for sol performance
Scribe online 01   best practices for sol performanceScribe online 01   best practices for sol performance
Scribe online 01 best practices for sol performanceScribe Software Corp.
 

Similaire à LifeOmic Change Management Automation -- DevSecCon Seattle 2019 (20)

DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...
DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...
DevSecCon Seattle 2019: Liquid Software as the real solution for the Sec in D...
 
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...
CI/CD best practices for building modern applications - MAD302 - Atlanta AWS ...
 
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...
Overcoming the old ways of working with DevSecOps - Culture, Data, Graph, and...
 
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...
CI/CD best practices for building modern applications - MAD304 - Chicago AWS ...
 
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...CI/CD best practices for building modern applications - MAD301 - Santa Clara ...
CI/CD best practices for building modern applications - MAD301 - Santa Clara ...
 
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...
CI/CD Best Practices for Building Modern Applications - MAD302 - Anaheim AWS ...
 
Apics 2017 samelson (final) blockchain
Apics 2017 samelson (final)   blockchain Apics 2017 samelson (final)   blockchain
Apics 2017 samelson (final) blockchain
 
Analytics im DevOps Lebenszyklus
Analytics im DevOps LebenszyklusAnalytics im DevOps Lebenszyklus
Analytics im DevOps Lebenszyklus
 
Serverless: What happens next will blow your mind!
Serverless: What happens next will blow your mind!Serverless: What happens next will blow your mind!
Serverless: What happens next will blow your mind!
 
Attacking and defending GraphQL applications: a hands-on approach
 Attacking and defending GraphQL applications: a hands-on approach Attacking and defending GraphQL applications: a hands-on approach
Attacking and defending GraphQL applications: a hands-on approach
 
GitOps is the best modern practice for CD with Kubernetes
GitOps is the best modern practice for CD with KubernetesGitOps is the best modern practice for CD with Kubernetes
GitOps is the best modern practice for CD with Kubernetes
 
CI/CD best practices for building modern applications - MAD310 - New York AWS...
CI/CD best practices for building modern applications - MAD310 - New York AWS...CI/CD best practices for building modern applications - MAD310 - New York AWS...
CI/CD best practices for building modern applications - MAD310 - New York AWS...
 
COMIT community day summer 2018 - main slides
COMIT community day summer 2018 - main slidesCOMIT community day summer 2018 - main slides
COMIT community day summer 2018 - main slides
 
Scaling Security Operations and Automating Governance: Which AWS Services Sho...
Scaling Security Operations and Automating Governance: Which AWS Services Sho...Scaling Security Operations and Automating Governance: Which AWS Services Sho...
Scaling Security Operations and Automating Governance: Which AWS Services Sho...
 
#spsclt18 vincent biret #spfx #devops
#spsclt18 vincent biret #spfx #devops#spsclt18 vincent biret #spfx #devops
#spsclt18 vincent biret #spfx #devops
 
Estimation of a micro services based estimation application bhawna thakur -...
Estimation of a micro services based estimation application   bhawna thakur -...Estimation of a micro services based estimation application   bhawna thakur -...
Estimation of a micro services based estimation application bhawna thakur -...
 
Amazon CI-CD Practices for Software Development Teams
Amazon CI-CD Practices for Software Development Teams Amazon CI-CD Practices for Software Development Teams
Amazon CI-CD Practices for Software Development Teams
 
Mandated BIM! Are you ready?
Mandated BIM! Are you ready?Mandated BIM! Are you ready?
Mandated BIM! Are you ready?
 
Software Supply Chain Management with Grafeas and Kritis
Software Supply Chain Management with Grafeas and KritisSoftware Supply Chain Management with Grafeas and Kritis
Software Supply Chain Management with Grafeas and Kritis
 
Scribe online 01 best practices for sol performance
Scribe online 01   best practices for sol performanceScribe online 01   best practices for sol performance
Scribe online 01 best practices for sol performance
 

Dernier

Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 

Dernier (20)

Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 

LifeOmic Change Management Automation -- DevSecCon Seattle 2019

  • 1. Seattle | September 16-17, 2019 Automated production deployments with HIPAA/HITRUST compliance MATT LAVIN
  • 2. Seattle | September 16-17, 2019 I’m an impatient developer @mdlavin at @LifeOmic
  • 3. Seattle | September 16-17, 2019 I’m envious of Netflix and Instagram
  • 4. Seattle | September 16-17, 2019 I work at LifeOmic where all data is sensitive data
  • 5. Seattle | September 16-17, 2019 The Sucker’s Choice • Don’t pick between security or speed • Get more security and more speedSecurity Speed Where you think the boundary is The actual boundary You are here
  • 6. Seattle | September 16-17, 2019 Tools we use • AWS • Git (Bitbucket) • Jenkins • Jira • Snyk
  • 7. Seattle | September 16-17, 2019 From idea to production CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development
  • 8. Seattle | September 16-17, 2019 Implementation CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development Implementation of change and test automation on the developers machine Push change to a Git branch, test automation runs in Jenkins
  • 9. Seattle | September 16-17, 2019 Review and deploy CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development After approval, changes are merged into Git master branch and Jenkins deploys to AWS dev account Team members review code and click ‘Approve’
  • 10. Seattle | September 16-17, 2019 Request production deployment CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development Humans review the new changes being deployed and approve if the appropriate processes were followed Developer creates a Jira issue to request deployment of specific project and build to production
  • 11. Seattle | September 16-17, 2019 Production deployment CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development Jenkins watches for the Jira issue and deploys the new project version when approved
  • 12. Seattle | September 16-17, 2019 Process requirements • Nobody can act alone to modify production • Changes to production are auditable • Security scans are run
  • 13. Seattle | September 16-17, 2019 Automation > Humans CM Ticket Bitbucket PR Code Review / Approval CM Approvals Deploy to AWS dev Deploy to AWS Prod Local development Automate review of process compliance Automate detailed change description collection for Jira issue
  • 14. Seattle | September 16-17, 2019 Old Change Management Ticket Flow Developer runs job to promote Jira emails sent to approvers Humans review and approve Jira ticket moves to implementing Jira CM issue is created Deploy job waits for approval Deploy job resumes
  • 15. Seattle | September 16-17, 2019 Automation possibilities Creating the ticket • What has changed • Has anything been removed Reviewing changes • Bitbucket PRs for all changes • All PRs were reviewed by peers • Security scans have been run
  • 16. Seattle | September 16-17, 2019 New Change Management Ticket Flow Developer runs job to promote Jira emails sent to approvers CM Bot Updates Jira ticket moves to implementing Jira CM issue is created Deploy job waits for approval Deploy job resumes Humans Review Fewer details
  • 17. Seattle | September 16-17, 2019 CM Bot execution Jira Issue Created Webhook CM Description updated Approval (or not) Bitbucket / Jenkins / Jira Analysis CM Verdict in comment
  • 18. Seattle | September 16-17, 2019 Git Commits Commit 1 Commit 2 Commit 3 Commit 4 Commit 5 Merge 1 Merge 2 Merge 3 Master Branch • Every commit must be from a LifeOmic Bitbucket user • All commits into master are through merge commits • No Fast Forwards or squashes
  • 19. Seattle | September 16-17, 2019 Pull requests PR 2 PR 3PR 1 • Every merge into master must be associated with a Bitbucket pull request • Every pull request must have approvers who are not the committer
  • 20. Seattle | September 16-17, 2019 Jenkins Builds Build 1 Build 2 Build 3 • Every merge into Master starts a Jenkins build for test and deploy
  • 21. Seattle | September 16-17, 2019 Change Management Jira Issues CM 1 CM 2 CM 3 • Production deployment requests refer to a Jenkins build
  • 22. Seattle | September 16-17, 2019 Verifying a promotion request Commit 4 Commit 5 Merge 3 PR 3 Build 3 CM 1 CM 2 CM 3 Master Branch • Find previously approved CM issue • Find all new artifacts associated with the request • Verify the processes were followed • Leave a comment in the CM issue with with changes detected
  • 23. Seattle | September 16-17, 2019 Automation • Good at finding and listing every single change in the new build • Good at filling CM tickets with details • Very fast Developers • Good at remembering what they did recently • Good at listing just enough details to get the CM ticket approved • Comparatively slow
  • 24. Seattle | September 16-17, 2019 Automation audit log
  • 25. Seattle | September 16-17, 2019 Rejected Changes
  • 26. Seattle | September 16-17, 2019 Automation • Good at checking every single change for peer reviews and security scans • Good at catching edge cases which can be hard to detect manually • Very fast • Reviewing at odd hours Human Approvers • Vulnerable to being rushed and sometimes spot checking changes • Good at thinking big picture for security and compliance • Comparatively slow • Sleeping and spending time with family
  • 27. Seattle | September 16-17, 2019 Git merge edge cases • Detecting any changes in the merge commits • Handling cases for PRs with multiple committers • Detecting removed commits • Squash merges
  • 28. Seattle | September 16-17, 2019 Changing culture • Worst case is fallback to human review • Teams eagerly follow review process to avoid waiting on human approvals • Security team doesn’t have to play the bad guy, the tool is the bad guy • Motivation to automate production changes • Shame for squash merges is a right of passage for new hires
  • 29. Seattle | September 16-17, 2019 The future • Detecting risky Terraform changes • Allowing automatic rollbacks • More types of security scans
  • 30. Seattle | September 16-17, 2019 Continuous delivery and HIPAA compliance is possible • We’ve open-sourced a library to help • Utilizes JupiterOne for data collection • https://github.com/JupiterOne/jupiter-change-management-client
  • 31. Seattle | September 16-17, 2019 Thank you! Ask me questions @mdlavin

Notes de l'éditeur

  1. I'm Matt Lavin I work at LifeOmic I hate waiting. I hate delays to getting bug fixes and features to customers. - want get feedback quickly - want happy customers - want security fixes out - Please take my word that I'm better at building software than making slides or presenting
  2. been to conferences and seen the cool companies moving quickly amazing continuous delivery pipelines constantly churn out changes their speed allows experimentation They have big teams
  3. I work at LifeOmic All our data is sensitive data because we manage patient data for doctors and researchers. We have to maintain HIPAA / HITRUST compliance tempting to say that our data is so sensitive that fast production change is not possible or realistic. We take security very seriously and yet we want to develop new software as fast as possible to help people We are a small company
  4. If you haven't read read the book, Crucial Conversations, you should. I first heard the idea of the “Sucker’s Choice” from that book. Knowing about the sucker’s choice can avoid disagreements between dev and sec by asking “how can we have it all”. We don't need to pick between security or speed. We can have both security and speed. Side note, I think this same graphic could be applied to quality and speed too. If you start asking “how can we have two things at once” instead of either/or then good ideas come to the surface !! So I built a tool that gives us both !!
  5. Pretty standard workflow We expect test automation for all changes, that's key to continuous delivery with or without security involved
  6. Here are the two steps that caused me the most pain. Can't automate code writing or code reviews (until AI takes over) Deployments are automated already So the biggest slowdown was writing the details of the CM ticket and waiting on a human to review the details and click approve. We could do better!
  7. Suffix on commit message
  8. Jenkins API for each build includes the git commit hashes that were built -- to detect new changes Also includes the logs -- security scans
  9. The automation really can be both more secure and faster at the same time