This talk covers the importance of automated testing in the proper Continuous Delivery process. Different types of testing are covered in the talk along with how to apply them.
3. @PavelChunyayev
About me
• 12 years of IT experience
• Lived and worked in Ukraine and Estonia
• Moved a year and half ago to the Netherlands
• Love cycling
• Love Dutch language
• Love software development processes
• Love working with people
5. @PavelChunyayev
Continuous Delivery
Incept
• Business idea
• Is needed
immediately
• Should be validated
Plan
• Refine
• Estimate
• Prioritize
Develop
• Put into sprint
• Develop in a branch
• Conduct a code
review
• Merge into master
Build
• Trigger pipeline
• Build
• Unit testing
• Integration testing
• Static code analysis
Test
• Contract testing
• E2E testing
• Security testing
• Resilience testing
Release
• Zero-downtime
• Canary testing
• Rolling deployment
• Blue / green
deployment
Operate
• Monitoring
• Validation of the
idea
• Money generation
• Disposal
9. @PavelChunyayev
Continuous Delivery
Incept
• Business idea
• Is needed
immediately
• Should be validated
Plan
• Refine
• Estimate
• Prioritize
Develop
• Put into sprint
• Develop in a branch
• Conduct a code
review
• Merge into master
Build
• Trigger pipeline
• Build
• Unit testing
• Integration testing
• Static code analysis
Test
• Contract testing
• E2E testing
• Security testing
• Resilience testing
Release
• Zero-downtime
• Canary testing
• Rolling deployment
• Blue / green
deployment
Operate
• Monitoring
• Validation of the
idea
• Money generation
• Disposal
36. @PavelChunyayev
Build quality in
• Testing is not just presence or absence of defects
• Testing is not a separate process
• Test should not just raise the cost of maintenance
• Stop thinking about functional testing only
• Quality goal need to be established early in the development process
• Automated testing – part of Definition of Done
• Test early, move tests to the left
• TDD
39. @PavelChunyayev
To practice
• Deploy software at the end of every iteration (done = released to
production).
• And then start doing it quicker.
• Optimize flow - don’t make ineffective more efficient.
40. @PavelChunyayev
Optimize the flow
• Honour existing processes, records,
and controls.
• They arose for a reason.
• Streamline and simplify all
processes, records, and controls.
• Challenge the level of ceremony.
42. @PavelChunyayev
Lean principles
• Optimize the whole
• Eliminate waste
• Deliver fast by managing flow
• Build quality into the system
• Create knowledge
• Defer commitment
• Respect people
44. @PavelChunyayev
“Developing people and the system
so that together they are capable of
achieving successful results is the
point.”
Mary and Tom Poppendieck
45. @PavelChunyayev
Continuous Delivery
Incept
• Business idea
• Is needed
immediately
• Should be validated
Plan
• Refine
• Estimate
• Prioritize
Develop
• Put into sprint
• Develop in a branch
• Conduct a code
review
• Merge into master
Build
• Trigger pipeline
• Build
• Unit testing
• Integration testing
• Static code analysis
Test
• Contract testing
• E2E testing
• Security testing
• Resilience testing
Release
• Zero-downtime
• Canary testing
• Rolling deployment
• Blue / green
deployment
Operate
• Monitoring
• Validation of the
idea
• Money generation
• Disposal
Keep the product releasable
Build quality in
Improve continuously
Build the right culture
pavel@levi9.com
Notes de l'éditeur
Ideas come from all the sources.
Questions are welcome.
Disagreement is welcome, but after the talk.
Yes, but; what if.
There's always a big gap between a piece of general advice and how it applies in a particular circumstance (which is why consultants are teased for saying "it depends").
There's no way I can know all the particular issues and constraints in your project, so you have to take my half-baked comments and finish them off yourself by deciding how they apply to your circumstances.
.
.
Not every feature is needed.
Shorter and shorter release cycles =>
Release more software in less time.
TPS
Can’t churn out.
No manual testing.
.
.
.
Reproducible and repeatable process (including testing).
Potentially shippable product -> Keeping the product releasable
.
Testing criteria.
Run over and over again.
.
No DTAP.
Immutable (testing framework?)
TTL
Docker
Framework to allow restart - select arbitrary test
Building the thing right
Building the right thing
Unit + Integration – level of microservices.
.
100% coverage?
Test your unit tests.
Know exactly what is broken.
Feature toggling.
Connect units together
How parts of application work together
Feature toggling.
Mock or not to mock
Black box testing.
How service works as a whole.
Different integrations – pub/sub, request/reply.
Feature toggling.
Mock or not to mock
Proof we are still on track
Not only selenium
Focus on:
business flows
personas, user journeys
BDD + DSL?
Data independency
Feature toggling, page objects
Mock or not to mock
Communication – via contracts
* Semantic or not semantic
* API versioning - public or also internal?
* Transition between versions – testing different versions
* Running several versions together
* Semantic or not semantic
* API versioning - public or also internal?
* Transition between versioning – testing different versions
* Running several versions together
* Semantic or not semantic
* API versioning - public or also internal?
* Transition between versioning – testing different versions
* Running several versions together
* Semantic or not semantic
* API versioning - public or also internal?
* Transition between versioning – testing different versions
* Running several versions together
* Semantic or not semantic
* API versioning - public or also internal?
* Transition between versioning – testing different versions
* Running several versions together
Testing in production
Canary
Dark launch
Real users, not synthetic transactions
Security testing
Availability testing
Resilience testing
.
.
.
.
.
.
Unit + Integration – level of microservices.
Unit + Integration – inside
Contract + E2E - outside
The same testing is needed for monolith, but crucial for microservices.
Shared responsibility.
Repetition makes mastery.
Feedback from users.
Feedback to devs, to bussiness
Think of end-user throughout the whole application lifecycle.
A story isn't done until it is being used by real users in production and has been validated to be a useful part of a product.
.
In conflict on purpose.
This is reality.
Evolution, not revolution.
.
Kaizen, Japanese for "improvement."
The most important thing you can build is culture. Don’t imitate, that won’t bring the culture you want.
Stamp on problems, not on people! Solve problems, not people.