How do you migrate over 65,000 of the most demanding software engineers from infrastructure built up over decades of high-intensity work into a common engineering system based on modern software development technologies and best practices?
7. Visual Studio Team Services
Continuous
Delivery for
Every Team,
Every App,
Every Platform Continuous
Delivery
Agile Planning
Dashboards
Kanban Boards
Taskboards
Build and Test
Git Source Control
Modern Code Workflow
Continuous Integration
Continuous Testing
Package Management
Open Source Compliance
Plan +
Track
Monitor +
Learn
Release
Develop +
Test
Delivery
Deployment of app and
infrastructure
PaaS, IaaS and
Containers
Monitoring
Telemetry
Diagnostics
Analysis
8.
9.
10. Sprint 1
August 2010
Sprint 29
June 2012
VSTS Preview
Sprint 64
April 2014
VSTS GA
Sprint 67
June 2014
1ES
Sprint 92
Nov 2015
Extensions
Sprint 102
June 2016
GVFS
Sprint 124
Sep 2016
17. • Original estimate
• Completed hours
• Lines of Code
• Team capacity
• Team burndown
• Team velocity
• # of bugs found
Things we don’t watch• Acquisition
• Engagement
• Satisfaction
• Churn
• Feature Usage
Usage
• Time to Detect
• Time to Communicate
• Time to Mitigate
• Customer Impact
• Incident Prevention Items
• Aging Live Site Problems
• SLA per Customer
• Customer Support Metrics
Live Site Health
• Time to Build
• Time to Self Test
• Time to Deploy
• Time to Learn
Velocity
18. Found one of the top customers with low availability.
Proactively reached out and resolved their issue.
25. Before
• Redundant alerts for same the issue
• Needed to set right thresholds and
tune often
• Stateless alerts contributed to
further noise
After
• Every alert must be actionable and
represent a real issue with the
system.
• Alerts should create a sense of
urgency – false alerts dilutes that
26. Health model in action
• 3 errors for memory and
performance
• All 3 related to same
code defect
• APM component mapped to feature team
• Auto-dialer engaged Global DRI
Eliminated alert noise
~928 alerts per week to
~22 and reduced DRI
escalations by ~56%
28. “One time” deployment commands in OneNote, email
Set-Options “-p 0”
Imagine a dozen more steps like that…
And then…someone misses a step half way through
We once broke pre-production for a day
33. Double blind test
Full disclosure at or near end
vs.
Share tactics & lessons learned
Continued evolution
Assume Breach - Use War Games to the learn attacks and practice response
36. Phone VS/PS
Phone VS/PS
Phone VS/PS
Phone VS/PS
Phone VS/PSUnique PS accounts and
workflows
1 year migration plan
WDG Product Studio Account
WDG VSTS Account
37. Can you add all of my
custom workflow fields
to the bug template
that everyone uses?
Can you add a new
work item type that is
only used by my feature
team?
Is it okay if I
track my team’s
hardware
purchases in
VSTS?
Can we store every
Watson report as a bug
even if we’re not sure it
needs to be fixed?
Is it okay if I create new
rules that everyone has
to follow for how to
track dependencies?
Is it cool if I add a
few hundred new
default values?
I prefer
Tags, let’s
use those!
I prefer
Keywords,
let’s use
those!
Yes!
38.
39.
40. Work Management
Information Tools
Coding Tools
Testing Tools
Build Tools
Debugging Tools
Developer
Time
Source: Internal Microsoft Telemetry analysis of 4700 engineers in WDG
41.
42.
43.
44. Teams
Physical team rooms
Cross discipline
10-12 people
Self managing
Clear charter and goals
Intact for 12-18 months
Own features in production
Own deployment of features
47. Employee choice, not
manager driven
Typically <20%
change, but 100% get
to make a choice
Cross-pollinate talent
and micro-culture
Sticky Note Exercise - Self Forming Teams
69. Rule: If your bug count exceeds your bug cap… stop working
on new features until you’re back under the cap.
5 50x =10
70.
71. Code Test & Stabilize Code Test & Stabilize
Code
Complete
Planning
72.
73. • Tests that anyone can run anywhere (inc production)
• Shifted to unit tests from automated functional tests
• Core tests run before pull request
• Fast and 100% reliable build and test is critical
• Rolling tests run after commit
74.
75. PR Validations
(Impacted Test runs
and tests)
PR Validations
Build and static tools
PR Creation
Create PR
VSO.PR
Build+L0
VSO.L1
OnPrem.SelfTest
RM.SelfTest
Manual
Tfs.SelfHost
VSO.CredScan
Cred Scanner
Policheck
CI – Required
Runs
CI - Test
(Release Defs)
CI - Build
(Build Defs)
Branch
Update
Push
VSO.CI
SPS.SelfTest
Tfs.SelfTest
Onprem.SelfTest Report
RM.SelfTest Report
Spsext.SelfTest
Tfs.SelfHost Report
VSO.Packaging.CI
VSO.SDKSample.
CI
78. Shared abstraction layer
Single master branch, multiple release branches
Update 1
Update 2
Update N
Visual Studio Team Services
Team Foundation Server
One code base with multiple delivery streams
79. Shared Platform Services (SPS)
North Central
TFS SU1
North Central
AT
AT
AT
JA
JA
JA
Blob
TFS SU7
Australia
TFS SU0
West Central
Containerized Services
80. •
•
•
•
•
• If you are starting out today and cloud native, consider
PaaS, Service Fabric + Azure Functions
• We need to ship same code to on-prem & cloud
81.
82. After
• 3-week sprints
• Vertical teams
• Team rooms
• Continual Planning & Learning
• PM & Engineering
• Continual customer engagement
• Everyone in master
• 8-12 person teams
• Publicly shared roadmap
• Zero debt
• Mockups in PPT
• Inner source
• Flattened organization hierarchy
• User satisfaction determines success
• Features shipped every sprint
Before
• 4-6 month milestones
• Horizontal teams
• Personal offices
• Long planning cycles
• PM, Dev, Test
• Yearly customer engagement
• Feature branches
• 20+ person teams
• Secret roadmap
• Bug debt
• 100 page spec documents
• Private repositories
• Deep organizational hierarchy
• Success is a measure of install numbers
• Features shipped once a year
