This is a draft of a presentation for a course on Visual Studio 2010 Unit Testing, I've uploaded mainly because I tried to create a Metro Style presentation, so if everyone like it, he can use as base for own presentation.
1. Unit testing in Visual Studio 2010
Because it is a matter of quality!!!
2. Different Type of Tests
Automated tests aimed to verify the functionality of a little
piece of code, run very often by developers and from
automated build.
Unit Tests
Test Suites managed with Microsoft Test Manager, manually
executed from testers and associated with User Stories. Can
be automated with Action Recording
Manual Tests
Web Request record and replay to simulate the interaction
with web sites. Can be used to do load or stress test to
verify the behavior of software under heavy usage
Web Tests
Record and replay user interaction with UI, plugin-based
recorder to maximize reproducibility, generation of UiMap
that contains all the data used to reproduce the automation.
Coded UI Tests
3. Test and Lifecycle - Waterfall
Requirements Design Develop Test Deploy
In waterfall processes testing has a well-defined phase, located between
Develop and Deploy. With such a process developers tend to do manual test
to verify that requirements are satisfied
Modern waterfall approaches tend to distribute testing in all the phases of
the process.
4. Test and Lifecycle - Waterfall
Requirement are software artifacts that can be tested
following basic principle of testing
The purpose of testing a program is to find problems
in it
The purpose of finding problem is to get them fixed
Requirements Test (Testing Computer Software – Cem Kaner et Al.)
Requirements should be verified with stakeholders and users to
verify that we are going to create the right software
A mismatch between the program and its specification is an error
in the program if and only if the specification exists and is correct
A program that follows a terrible specification is terrible not
perfect
(Testing Computer Software – Cem Kaner et Al.)
5. Test and Lifecycle - Waterfall
Even with Emerging Architecture you need to
validate the design of the software.
The design phase should produce a «proof of
concept» for each «typical» part of the software
Early testing on POC should be used to validate the
infrastructure
Design Test
Technique used: Unit Test, Performance Test
Testing of the Design phase should take place when:
No existing architecture, start from scratch
First project with new technology
Strict performance requirements
External service
Verify testability of the architecture
6. Test and Lifecycle - Waterfall
During developement all form of testing are used
Unit test: used by developers to validate new code
Manual test: Used by test team to validate code for
«done» requirements
Coded UI test: Used by developers and test team to
automate testing through the interface
Develop Test
The main purpose of testing during development is guarantee quality of
the codebase.
When code is out of the developing phase, it should be ready to be
“beta tested” to find bug that are not caught during development.
7. Test and Lifecycle - Waterfall
Testing phase is where you find bug during real-
usage of the software
The software is usually in «beta» stage, meaning that
Real-usage Formal bugfix
it can be used by real user.
This is the moment where you mainly use
Acceptance test with Microsoft Test Manager
Develop
Quality
All defect found during this phase should be filled up in issue/bug tracker
to be investigated by the team
No developer should be allowed to «pick a bug» to fix it without a formal
review.
The aim of this phase is to fix all most important bugs of the software, as
well as gather user impression of the overall product, to make little
modification before the real shipping.
All steps should be reported.
8. Test and Lifecycle - Agile
Requirements Design Develop Test Deploy
Each agile process has its own development cycle, but quite all of them
have an interactive cycle
Each interaction value is added to the software and this value should be
immediately available to Stakeholders
Acceptance and Unit Testing are the skeleton of this lifecycle, without
tests it is difficult to be really Agile. Testing permits to
Establish Definition Of Done (Acceptance)
Refactor and do incremental development (Unit testing)
Clarify User Stories (Acceptance)
Assure internal quality of the codebase (Unit testing in CI)
10. Unit test principle
Automated: Test can be executed without user interaction
Integrated: Written in the same language of the code to test, available
in the same environment
Develop
Standard: Many frameworks available
Develop
Agile: Support for refactoring and agile lifecycle
Develop
Pattern support: www.xunitpattern.org estabilish a ground language
and series of pattern for unit testing in all language/environments
11. xUnit framework pattern
Setup: Initial conditions are set to maximize
reproducibility of the test
Setup
Exercise: Code to be tested is executed, usually is a
call to a single function or a series of calls.
Exercise
Verify: An expectation is checked, if the expectation
fails, the whole test is considered to be failed
Verify
Teardown: If the test modify the system/environment
everything is restored to a good / clean state
Teardown
12. Quality of good tests
Automated: Test can be executed without user interaction
Indipendent: The test does not depend on any other test
Develop
Focused: The test should verify a single condition
Develop
Repetable: Test depends only on precondition contained in the setup
phase and it should give the same result at each execution
Develop
Fast: Slow tests are not executed frequently
13. Test is First-Class code
Part of the project: testing code has the same
importance of tested code
Part of the project
Refactored: Time is spent refactoring and adapting
test code to the modifications of tested code.
Refactored
Time: writing good tests takes time but having
good Unit Test suite dramatically raise quality
Hard to write
Long run ROI: Sadly enough, Unit Testing usually
does not pay in the short time, so it is difficult to take
the path to unit testing
Plan for the future
15. Test smells
Symptom: too much time to maintain tests
Impact: test are not executed and removed from suite
Cause: Code duplication, low quality test code
High maintenance
Symptom: test fails but the code under test is correct
Impact: time wasted to look for inexistent bugs
Cause: Test is too complex
Buggy tests
Symptom: test has conditional logic inside
Impact: it is not clear what it is testing
Cause: Dependence from environment, wrong logic
Conditional test logic
Symptom: it is difficult to understand what is tested
Impact: it is difficult to understand why it failed
Cause: Eager test, mystery guest, irrelevant
Obscure test information, hard coded test data
16. Avoid test smells
Users does not execute the test often
Continuous integration automatically executes tests
Develop
Test are slow, they needs several minutes to be executed
Develop
Categorized test permits execution of only a little part of a test,
continuous test tools (Mighty Moose, Test Impact) can automate this
Develop
Manual intervention before test run
Develop
Write unit test first, write test that is testable
18. The fixture
Everything is needed to create preconditions for the
execution of test, it can be data in database, setting
global variables, cleaning up file on disks, etc.
Setup Exercise
This is the most complex part of the whole Unit-
Testing stuff, because it is the source of many smell
Verify Teardown
Fresh Fixture – transient: The fixture is recreated at each test, at the end of
the test the system is restored to the original state
Fresh Fixture – persistent: Same as transient, but the system is not restored
at the end of the test
Shared Fixture: The fixture is shared for many test, it is created, then a series
of test is executed
19. Fresh transient
Fixture is created during the setup phase, take track of every
Setup modification that is done to the system.
It is useful to create a mechanism that permits to schedule
automatic cleanup at the end of the test.
Exercise
Verify
Fixture is removed during the teardown phase, everything is
Teardown restored.
Garbage collector should not be used because it is not-
deterministic.
If multiple restore operations should be executed, use try-
catch clause to be sure to cleanup the most of them
20. Fresh persistent
Fixture is created during the setup phase, no need to track
Setup operations
Next test will create another fixure, but every
Exercise Setup modification done by the previous test is still
there
This can augment the risk of interacting-test
Verify Exercise
smell.
This technique is useful only if we are sure that
Teardown Verify the fresh persistent fixture will not impact
other tests.
It is useful mainly if the restore operation is
Teardown
slow and you do not want to lose time.
21. Shared fixture
Fixture is created during the setup phase, modification are
Setup tracked
First test is executed, the test is composed only by Exercise
Exercise and Verify steps
Subsequent tests execution will use the same
Verify Exercise
Setup fixture
Verify Exercise
Setup
Verify Exercise
Setup
After the latest verification the fixture is
removed. Teardown Verify
Teardown
22. Shared fixture
Fixture Setup Setup Setup Setup
Exercise Exercise Exercise
Verify Verify Verify
Teardown Teardown Teardown Teardown
A shared fixture is created before the execution of each test
Each test can create another specific fixture and remove it in
Setup/Teardown
After the latest test of the suite is executed the final Teardown is run to
remove the shared fixture.
23. Assertion
How to write good verification code
that maximize the advantage of unit
testing
24. Assertion
Single assertion: A test should verify only one fact of the code under
test
Clear assertion: Reading verification code should immediately clarify
what fact we are testing
Develop
Test named assertion: Name of the test should clarify the purpose
and what is verified by the test.
25. Writing good assertions
Too many assertions in the test
Expected Object, create an object that reflect the status you want to
check, name that object in clear way.
Develop
Complex assertion code, that is not readable.
Develop
Use FluentInterface based assertion frameworks (SharpTestEx)
Develop
Magic values to verify return values or object properties
Develop
Use good names for variables and expected values
26. Test doubles
Creates fake component to be sure to
exercise only a single part of the system
27. Depend On Component
The System Under Test dialogates with external
Setup component and the verification phase has not access
to it.
Exercise
Verify
SUT DOC
Teardown If the test fails, we are not sure if the reason is in bug
in the SUT or for a bug or different behavior of the
DOC.
It leads to erratic test and frequent debugging test
smell.
28. Test Double
Setup
Test Double
Exercise
Verify
SUT
Teardown
We need to design the SUT to loose coupling with the DOC, with this
technique you can create a double of the DOC in the test.
The Test Double simulates the real component, but it has a well
defined and reproducible behavior that is defined in the Setup phase
29. Verifiable Test Double
Setup
Test Double
Exercise
Verify
SUT
Teardown
With a verifiable Test Double the Verify step can ask to Test Double to
verify how the SUT interacted with him during Exercise phase.
As an example, if you create a Test Double of an E-Mail sender
component, you can verify that during Exercise the SUT asked to
send an E-Mail.
30. Types of TestDoubles
Stub: its only purpose is to answer to the requests of
the SUT with well defined and known answers, or with
a default one
Stub
Dummy: It does nothing, simply avoid the SUT to
crash for missing components.
Dummy
Mock and Spy: Behave like a Stub, but also permits to
make assertion on how the SUT interacted with them.
Mock Object and Test Spy
Fake Object: It implements a DOC with minimal
functionality.
Fake Object
32. Database Testing Basic
Sandbox: Each tester/environment should have a
dedicated database to execute testing. Minimize
dependency, maximize execution time avoid conflicts
Database Sandbox
Database Fixture: Shared fixture to minimize setup
time, use of Back Door Manipulation and
backup/restore attach/detach method. Heavy use of
Fixture Setup
Shared Fixture.
Smart teardown: Use preload at each test and make
every test transactional so you can rollback after each
verification phase
Teardown
Database Projects: Specific project type introduced
with VS2008 that contains a dedicated section to unit
testing.
Database Project