So you’ve put your databases in source control and you’re figuring out how to deploy them with VSTS or Octopus Deploy etc. That’s great, but DevOps is about more than just the continuous delivery of software updates. If your new code is broken all you’ve managed to do is ship bugs to your users more quickly than before.
When writing applications we use test frameworks like xUnit or JUnit to create unit tests that can be run on a developers machine or as part of a CI process. That helps us to catch regressions. We should be applying the same diligence to our stored procedures and functions to help us to spot when we accidentally break the database too.
In this session I’ll use tSQLt to create a suite of automated tests and I’ll run them as part of a VSTS build. I’ll share all my scripts on GitHub so you can recreate my demo afterwards on your own machine.
20. @_AlexYates_
#WinOps
Image sources
Author Source Information
Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share
Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”
Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”
Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.
Ctrl.Alt.Design ctrla.lt Social Media share icons
Notes de l'éditeur
Show of hands… Who is a developer? Project manager?
Do we have any DBAs?
DBAs care about… NEXT SLIDE
What about databases? The persistent data the our businesses our built on?
Schema changes vs existing data
There is an odd relationship here between how you want the code to look and how you update prod. You can’t simply drop and replace the DB because you’ll lose your data. You have to create some sort of upgrade script. Twice as much code normally means twice as many errors and if the model you want and your way to get there don’t match, which is right? What is your source of truth? Without a clear idea of your source of truth – how is DevOps even possible?
Reference data vs production data
But while the database has existing data, not all of it is ‘production’ data in that sense. What about country codes? Reference data, lookup data, static data, the data that makes your system work. This data needs to get deployed with your schema changes. And you also need to think about how to transfer data in the other direction for testing. How do you test the latest dev build with production (or-production-like) data?
Teamwork and testing
The word DevOps refers to the problems associated with siloed Dev and Ops teams. Nowhere is this more apparent than in the land of the database? Is there anyone here who has never heard of any problems between Dev and DBA teams?
But it goes further than that. With application source code we use source control, we invented distributed source control systems and we debate about the optimal branching strategies and strategies for implementing continuous integration. We’ve barely begun to have these conversations about databases. We barely have strategies for how to provision individual developers with their own sandboxes. Sometimes different developers work in different ways, some working off scripts and others working directly on the database. We need a better way of working together.
And finally testing. Our problems with database source control and acquiring suitable test data make it hard for us to provision test database environments manually, let alone automatically.
Database drift
And all these problems result in this final point. In DevOps and Continuous Delivery we often talk about cycle time. How long would it take you to make a one line change, run it through your normal testing process and get it to production? If the cycle time for your database is measured in days, weeks or months (or years?) then when you hit a production issue you don’t have time to go back to your source code. The business is haemorrhaging money and the DBA will make a decision: What is more expensive, the cost of delaying the fix, or the risk of making that low risk change now and fixing the problem right away. It’s all well and good wagging your finger at people who choose to make hot-fixes directly on production, but until we sort out the cycle time problem, it won’t go away. Production drift is a symptom of poor DevOps or DLM strategy.
And drift causes more problems: Environment inconsistencies undermine your tests and can cause failed deployments either because code clashes or because important fixes are accidentally rolled back. Drift and poor DevOps processes are a vicious circle that needs to be broken.
What about databases? The persistent data the our businesses our built on?
Schema changes vs existing data
There is an odd relationship here between how you want the code to look and how you update prod. You can’t simply drop and replace the DB because you’ll lose your data. You have to create some sort of upgrade script. Twice as much code normally means twice as many errors and if the model you want and your way to get there don’t match, which is right? What is your source of truth? Without a clear idea of your source of truth – how is DevOps even possible?
Reference data vs production data
But while the database has existing data, not all of it is ‘production’ data in that sense. What about country codes? Reference data, lookup data, static data, the data that makes your system work. This data needs to get deployed with your schema changes. And you also need to think about how to transfer data in the other direction for testing. How do you test the latest dev build with production (or-production-like) data?
Teamwork and testing
The word DevOps refers to the problems associated with siloed Dev and Ops teams. Nowhere is this more apparent than in the land of the database? Is there anyone here who has never heard of any problems between Dev and DBA teams?
But it goes further than that. With application source code we use source control, we invented distributed source control systems and we debate about the optimal branching strategies and strategies for implementing continuous integration. We’ve barely begun to have these conversations about databases. We barely have strategies for how to provision individual developers with their own sandboxes. Sometimes different developers work in different ways, some working off scripts and others working directly on the database. We need a better way of working together.
And finally testing. Our problems with database source control and acquiring suitable test data make it hard for us to provision test database environments manually, let alone automatically.
Database drift
And all these problems result in this final point. In DevOps and Continuous Delivery we often talk about cycle time. How long would it take you to make a one line change, run it through your normal testing process and get it to production? If the cycle time for your database is measured in days, weeks or months (or years?) then when you hit a production issue you don’t have time to go back to your source code. The business is haemorrhaging money and the DBA will make a decision: What is more expensive, the cost of delaying the fix, or the risk of making that low risk change now and fixing the problem right away. It’s all well and good wagging your finger at people who choose to make hot-fixes directly on production, but until we sort out the cycle time problem, it won’t go away. Production drift is a symptom of poor DevOps or DLM strategy.
And drift causes more problems: Environment inconsistencies undermine your tests and can cause failed deployments either because code clashes or because important fixes are accidentally rolled back. Drift and poor DevOps processes are a vicious circle that needs to be broken.