Development, Deployment & Collaboration at Etsy

•

5 j'aime•1,633 vues

Daniel Schauenberg

Technologie

http://www.ﬂickr.com/photos/brianglanz/1095706242

How comfortable
are you deploying
a change right
now?

@mrtazzhttp://www.ﬂickr.com/photos/renaissancechambara/2349811492
small change

“If this is your first
day at Etsy, you
deploy the site”

@mrtazz
Developer VMs
• KVM
• Every engineer has one
• Fully Chef’d with the Etsy Stack
• Different sizes and Chef roles

@mrtazz
Continuous Integration
• Run set of tests before each deploy
• Full QA suite
• Princess/Production smoker tests
• Try (yup, there is one)

http://www.ﬂickr.com/photos/egfocus/6962179321

@mrtazz
The Bobs
• LXC virtualized hosts
• 14/physical hosts
• Spread over 3 SSDs
• Most of them attached to try

@mrtazz
Deployinator
• 2 Buttons, no ambiguity
• Overview of current state of deploy
• Links to Logwatcher and Dashboards
• Easy to add stacks for new tools to deploy

http://www.ﬂickr.com/photos/jbgeronimi/6363087361

@mrtazz
Monitoring
• Devs do their feature monitoring
• Everybody can access all the graphs
• Dashboard All The Things!
• Stream All The Logs!

If you are writing
code, you are
on-call

@mrtazz
On-Call Schedules
• ops on-call
• dev on-call
• payments on-call
• support on-call

@mrtazz
Dev On-Call
• On-call for 3 days
• All developers who are not in another
rotation
• L1 and L2 escalations
• L1 if it’s your first time

@mrtazz
Incident Response
• “This graph looks funny”
• “Hey I just got paged for elevated error rate
after deploys”
• “Supergrep is going crazy!!”

@mrtazz
#warroom
• only outage related conversations
• coordinate investigation, communication,
countermeasures and monitoring
• good place to lurk for new engineers

@mrtazz
Summary
• These are things that work for *us*
• Culture is an on-going effort
• Share everything
• Encourage learning/teaching

@mrtazz
Summary
• Lunch ’n learns
• DC visits
• On-call for a day
• Bootcamps/Senior rotations

codeascraft.com

etsy.com/codeascraft/talks

etsy.github.com

etsy.com/careers

Development,
Deployment and
Collaboration at Etsy
Daniel Schauenberg
dschauenberg@etsy.com

Contenu connexe

Tendances

Concourse and DatabaseGwenn Etourneau

Practical Continuous Deployment - Atlassian - London AUG 18 Feb 2014Matthew Cobby

VersaPay's Tools for Happyfficient DevelopersPhilippe Creux

Monitoring in the cloud with PuppetKris Buytaert

My Top 5 Favorite GemsJimmy Ngu

PuppetConf 2017: Using Puppet Enterprise APIs with PowerShell- Jason Johnson,...Puppet

Data + Visualization Toronto Meetup 4: The Google Maps APImattholtom

Railsconf 2014 - Deploying Rails is Easier Thank It Lookstalkingquickly

Update Smart: Use a Staging Site!Meagan Hanes

PuppetConf 2017: No Server Left Behind - Miguel Di Ciurcio Filho, InstructPuppet

Dev-Friendly OpsJosh Schramm

What's New in JHipsterLand - DevNexus 2017Matt Raible

The Ruby workflowClemens Helm

Stripe con 2021 UI stackMaximeRainville1

From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015Corley S.r.l.

Modern Front End Tools & WorkflowAdir Amsalem

Background processes and tasks in an async worldparticlebanana

Anton Boyko "The future of serverless computing"Fwdays

React for .net developersmacsdickinson

Anton Boyko, "The evolution of microservices platform or marketing gibberish"Sigma Software

Tendances (20)

Concourse and Database

Practical Continuous Deployment - Atlassian - London AUG 18 Feb 2014

VersaPay's Tools for Happyfficient Developers

Monitoring in the cloud with Puppet

My Top 5 Favorite Gems

PuppetConf 2017: Using Puppet Enterprise APIs with PowerShell- Jason Johnson,...

Data + Visualization Toronto Meetup 4: The Google Maps API

Railsconf 2014 - Deploying Rails is Easier Thank It Looks

Update Smart: Use a Staging Site!

PuppetConf 2017: No Server Left Behind - Miguel Di Ciurcio Filho, Instruct

Dev-Friendly Ops

What's New in JHipsterLand - DevNexus 2017

The Ruby workflow

Stripe con 2021 UI stack

From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015

Modern Front End Tools & Workflow

Background processes and tasks in an async world

Anton Boyko "The future of serverless computing"

React for .net developers

Anton Boyko, "The evolution of microservices platform or marketing gibberish"

Similaire à Development, Deployment & Collaboration at Etsy

Continuous Integration, the minimum viable productJulian Simpson

Continuous TestingCrevise Technologies

Continuous DeploymentBrian Moon

Pragmatic Continuous Delivery - ReaktorDevDay 2012Neeme Praks

Untangling spring week10Derek Jacoby

Devops For DrupalKris Buytaert

Maintaining Retainers as a WordPress DeveloperDevinVinson

Great Tools Heavily Used In Japan, You Don't Know.Junichi Ishida

Production Ready WordPress #WPLDNEdmund Turbin

Devops Devops Devops, at FrosconKris Buytaert

Take control of your Jenkins jobs via job DSL.Łukasz Proszek

Continuous Delivery of Puppet ManifestsKris Buytaert

Frontend Engineer ToolboxYnon Perek

Essential Plone development tools - Plone conf 2012Kim Chee Leong

Vagrant for Effective DevOps CultureVaidik Kapoor

Infrastructure as Code PatternsKris Buytaert

stackconf 2022: Infrastructure Automation (anti) patternsNETWAYS

Task runners + theming automating your workflowJoshua Gilmer

Ship It ! with Ruby/ Rails EcosystemYi-Ting Cheng

Lessons Learnt in 2009pratiknaik

Similaire à Development, Deployment & Collaboration at Etsy (20)

Continuous Integration, the minimum viable product

Continuous Testing

Continuous Deployment

Pragmatic Continuous Delivery - ReaktorDevDay 2012

Untangling spring week10

Devops For Drupal

Maintaining Retainers as a WordPress Developer

Great Tools Heavily Used In Japan, You Don't Know.

Production Ready WordPress #WPLDN

Devops Devops Devops, at Froscon

Take control of your Jenkins jobs via job DSL.

Continuous Delivery of Puppet Manifests

Frontend Engineer Toolbox

Essential Plone development tools - Plone conf 2012

Vagrant for Effective DevOps Culture

Infrastructure as Code Patterns

stackconf 2022: Infrastructure Automation (anti) patterns

Task runners + theming automating your workflow

Ship It ! with Ruby/ Rails Ecosystem

Lessons Learnt in 2009

Plus de Daniel Schauenberg

Human Factors and PostMortemsDaniel Schauenberg

Deploy, Collaborate and ListenDaniel Schauenberg

Scaling Deployment at EtsyDaniel Schauenberg

Feature Flagging your Infrastructure for Fun and ProfitDaniel Schauenberg

Scaling Deployment at EtsyDaniel Schauenberg

StatsD Workshop Monitorama 2013Daniel Schauenberg

Etsy chef-workflowDaniel Schauenberg

Plus de Daniel Schauenberg (7)

Human Factors and PostMortems

Deploy, Collaborate and Listen

Scaling Deployment at Etsy

Feature Flagging your Infrastructure for Fun and Profit

Scaling Deployment at Etsy

StatsD Workshop Monitorama 2013

Etsy chef-workflow

Dernier

Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix

The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad

Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard

04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG

Salesforce Community Group Quito, Salesforce 101Paola De la Torre

Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700

Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung

Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC

Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge

08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls

A Call to Action for Generative AI in 2024Results

Slack Application Development 101 Slidespraypatel2

The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los

Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik

Scaling API-first – The story of a global engineering organizationRadu Cotescu

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada

How to convert PDF to text with Nanonetsnaman860154

From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software

Dernier (20)

Swan(sea) Song – personal research during my six years at Swansea ... and bey...

The Codex of Business Writing Software for Real-World Solutions 2.pptx

Maximizing Board Effectiveness 2024 Webinar.pptx

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

Salesforce Community Group Quito, Salesforce 101

Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...

Unblocking The Main Thread Solving ANRs and Frozen Frames

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Breaking the Kubernetes Kill Chain: Host Path Mount

Handwritten Text Recognition for manuscripts and early printed texts

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

08448380779 Call Girls In Civil Lines Women Seeking Men

A Call to Action for Generative AI in 2024

Slack Application Development 101 Slides

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Injustice - Developers Among Us (SciFiDevCon 2024)

Scaling API-first – The story of a global engineering organization

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024

How to convert PDF to text with Nanonets

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Development, Deployment & Collaboration at Etsy

1. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com @mrtazz

3. @mrtazz Etsy Stats

4. @mrtazz Etsy Stats

5. @mrtazzItem by TheBackPackShoppe

6. http://www.ﬂickr.com/photos/brianglanz/1095706242

7. avg 50 deploys/ day

8. avg n > m deploys/ day

9. How comfortable are you deploying a change right now?

10. @mrtazzhttp://www.ﬂickr.com/photos/renaissancechambara/2349811492 small change

11. Config Flags Item by RocajoStudio

12.

13. “If this is your first day at Etsy, you deploy the site”

14. Developer VMs

15. @mrtazz Developer VMs • KVM • Every engineer has one • Fully Chef’d with the Etsy Stack • Different sizes and Chef roles

16.

17. Continuous Integration

18.

19. @mrtazz Continuous Integration • Run set of tests before each deploy • Full QA suite • Princess/Production smoker tests • Try (yup, there is one)

20. http://www.ﬂickr.com/photos/egfocus/6962179321

21. @mrtazz The Bobs • LXC virtualized hosts • 14/physical hosts • Spread over 3 SSDs • Most of them attached to try

22.

23. Item by decomodwalls

24. Deployinator

25. @mrtazz Deployinator • 2 Buttons, no ambiguity • Overview of current state of deploy • Links to Logwatcher and Dashboards • Easy to add stacks for new tools to deploy

26. http://www.ﬂickr.com/photos/jbgeronimi/6363087361

27.

28. Monitoring

29. @mrtazz shouldigraphit.com

30. @mrtazz Monitoring • Devs do their feature monitoring • Everybody can access all the graphs • Dashboard All The Things! • Stream All The Logs!

31.

32.

33.

34. On Call

35. If you are writing code, you are on-call

36. @mrtazz On-Call Schedules • ops on-call • dev on-call • payments on-call • support on-call

37.

38. @mrtazz Dev On-Call • On-call for 3 days • All developers who are not in another rotation • L1 and L2 escalations • L1 if it’s your first time

39. Incident Response

40. @mrtazz Incident Response • “This graph looks funny” • “Hey I just got paged for elevated error rate after deploys” • “Supergrep is going crazy!!”

41. Is the site down?

42.

43. #warroom

44. @mrtazz #warroom • only outage related conversations • coordinate investigation, communication, countermeasures and monitoring • good place to lurk for new engineers

45. Post Mortems

46. blameless

47. Everybody’s invited

48. Learning Opportunity

49. Summary

50. @mrtazz Summary • These are things that work for *us* • Culture is an on-going effort • Share everything • Encourage learning/teaching

51. @mrtazz Summary • Lunch ’n learns • DC visits • On-call for a day • Bootcamps/Senior rotations

52. codeascraft.com etsy.com/codeascraft/talks etsy.github.com etsy.com/careers

53. Questions?

54. Development, Deployment and Collaboration at Etsy Daniel Schauenberg dschauenberg@etsy.com