Testing with Visual Studio 2012 Agile TMap

1. Module 02
Test Specification
and Execution

2. Topics

3. During the Specification phase, the required tests and starting points are specified. The aim is to have as much as
possible prepared, in order to be able to run the test as quickly as possiblewhen the developers deliver the test object.
http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/specification-phase
The Specification Phase

4. Tests in the top quadrants
validate business needs.
Tests in the bottom quadrants
validate the implementation.
Agile Testing Quadrants
Performance &
Load Tests
Security Tests
‘ility’ Tests

5. Tests in the right-hand
quadrants help the product.
Tests in the left-hand quadrants
help the team.
Agile Testing Quadrants
Performance &
Load Tests
Security Tests
‘ility’ Tests

6. To obtain insight into the quality of the test object through the execution of the agreed tests.
http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/execution-phase
The Execution Phase

7. Test Types

8. Acceptance Tests
Business-facing
Critique Product
A test script that checks
if a feature behaves as
expected.
Defined during backlog
grooming and early in
the sprint.

9. Requirements and bugs link
directly to related test cases
Integrated Storyboarding with
PowerPoint keeps stakeholders and
all team members in sync
Measurable acceptance
criteria, visible to all team
members, aligns with tests

10. Result
User Story Test Status Report

11. Demo
Creating a Test Plan with
Logical Acceptance Tests

12. Unit Tests
Supporting
the team
Technology
facing
Isolate each part of the
program and show that
the individual parts are
correct.
Created in the sprint,
Executed continuously.

13. Unit testing is a method by which individual units of source code, sets of one or more computer program modules together
with associated control data, usage procedures, and operating procedures, are tested to determine if they are fit for use.
Intuitively, one can view a unit as the smallest testable part of an application.
Definition of Done
All unit tests are written and have been successfully executed.
Code Coverage
Code coverage records for each package are available and whatever the team has decided as the minimum benchmark
has been achieved.
Unit Tests

14. Use the testing framework
you’re familiar with to write
your unit tests
Quickly run all unit tests, or
target specific tests or
groups of tests
Quickly examine results of each
test during the previous test run
with the Test Explorer

15. Developer-focused
Testing
Red Green Refactor
Pluggable test
frameworks
Run tests after build
Unit Test Explorer

16. Shims can be used to fake almost
any method call, rendering your
legacy code testable
Create a “fake” reference to any .dll
referenced by your testing project

17. The Fakes Framework allows you
to shim SharePoint web parts,
improving testability
Explore and execute tests using
the same first-class Test Explorer
Manage the unit test
project directly alongside
your SharePoint projects
Use familiar SharePoint code
to create your unit tests

18. Group unit tests by project,
traits, duration, results and more
Code coverage analysis determines
the portion of your code covered by
unit tests, down to the method level
Results can be exported and
saved for future analysis

19. Code Coverage
Code Clone Analysis
Static Code Analysis
Code Reviews
Test Impact Analysis*
Developer-focused Code Quality Tools

20. Demo
Code Quality Tools

21. Functional Testing
Supporting
the team
Technology
facing
Validates the
functionality quality
characteristic, with the
functional design as the
test basis.
Specified and executed
during the sprint.

22. Specified and executed during the sprint
All functionality created during a sprint is tested during the same sprint.
Test coverage
Linked work items are used for test coverage.
Definition of Done
All priority 1 test cases are executed.
All high risk backlog items have a test coverage of 80%.
Test Case promotion to regression sets
Some test cases will be part of regression testing efforts.
Functional Tests

23. Test Steps
Shared Steps
Test Data Parameters
Specify Test Cases

24. Organize testing in multiple
configurations
Playback a prior recording to save
time with frequently-repeated
setup steps and prerequisites
Bind data to controls, to
support multiple iterations
and future automation
Pause, save, close, return, and
pick up where you left off

25. Test Runner
Action Recording
Execute Test Cases

26. IntelliTrace Logs
System logs & configuration
Detailed status and history
visible to all team members
Audio & Video recordings

27. Get an overview of the test run
with summary pie charts, or drill
into the details for each test.
Analyze code coverage for manual
tests to determine which specific
parts of your code were tested .
Associate manual test runs
with builds and environments.

28. Recommended Tests
Shows passed Tests with
changed source code
Test Impact Analysis

29. Exploratory Testing
Supporting
the product
Supports the
Implementation
Simultaneous test
design and execution.
Tests working software.

30. Create rich text comments
with screenshots while
exploring
Exploratory sessions can
be recorded to increase
reproducibility
Explore your application to
uncover new defects
The Test Runner docks on the left
side of the screen and automatically
records your steps as you explore

31. Time spent during
exploratory testing
Courtesy of Jon Bach
Test Setup
Bug Filing
Testing
The 3 Components of Testing

32. Comments and steps are
automatically added
The exploratory test session
is paused while you create a
bug report
File new bugs without ending the
exploratory test session
Attach the video recorded during
the exploratory test session for
maximum reproducibility
Quickly create an
automatically linked test
case for this bug

33. Add verification values to
indicate the expected result
Session is paused while creating
test cases
Test steps are added
automatically

34. Comment block reflects
the added bugs and tests
Continue exploratory testing
right from where you left off
Quick access to bugs and test
cases created during the session

35. Demo
Exploratory Testing

36. Performance and Load Testing
Supporting
the product
Supports the
Implementation
Performance Test
To determine or validate speed, scalability, and/or stability.
Load Test
To verify application behavior under normal andpeak
load conditions.

37. Web Performance Test

38. Load Test

39. Demo
Web Performance Test
Load Test

40. Test Automation

41. Test Automation
Performance &
Load Tests
Security Tests
‘ility’ Tests
Automated
and Manual
Manual
Automated
Tools
Automated

42. Visual Studio Support
Performance &
Load Tests
Security Tests
‘ility’ Tests

43. Convert the test case to
code for enhanced
flexibility
Generate your Coded UI test from
an existing recording, or record a
new test case on the fly

44. Test Automation with CodedUI Tests

45. CodedUI tests interact
directly with your app’s UI
Tests can be run and monitored
on the local machine, or further
automated on a virtual machine
Cross-browser testing support
allows automated testing on
multiple browsers

46. Coded UI tests run as part of your
unit testing suite, enabling
developers to detect defects early
Even if the look & feel changes, the
automated test still executes properly
Coded UI tests are part of
source control for
maximum visibility

47. Demo
End-to-end Test Workflow
with Coded UI Automation