Plan, manage, and execute tests with Microsoft Test Manager and Lab Management in Visual Studio 2013 which will make it easier to conduct manual and automated testing across a variety of environments. This presentation covers the new exploratory testing approach offered by Microsoft Test Manager; the simplified setup and administration of Lab Management environments; and some of the other fit-and-finish features across the testing scenario.
3. Modern app lifecycle
Continuous value delivery with modern business applications
Define
Ideation
Develop
Idea to working
software
Implement Monitor
SHORTENED
CYCLE TIMES
4. Conventional QA in the Modern app lifecycle
QA
Testing post
implementation. Mostly
manual with limited
automation.
UAT
Operations
readiness
verification
UAT post
implementation
and systems
testing
Pre-deployment
verification
Document requirements
as BRDs and
functional specs
Write code to
implement
requirements
6. Quality enablement practices
Continuous quality for continuous value delivery
Continuous quality
Shortened cycle times
7. Acceptance test planning
Business goals
Actionable
requirements
Acceptance
criteria
Acceptance test
plan
Acceptance test
driven
development
8. Continuous acceptance testing
User
Interface
Services
Business Processes
Business Rules and Logic
Service integrations
Data access
Identity
Data
Manual testing
Automated testing
Automated testing
12. The Microsoft Solution
Traceability with work item tracking
Requirements
management
Test lab
management
Source code
management
Extensibility interfaces
Team
development
services
Testing services
Build and
deployment
automation
Reporting and
Business
Intelligence
Lab infrastructure Test recorders and
runners
Build engines
• Microsoft Office
• Feedback client
• Microsoft Office and
SharePoint
• Microsoft Project
• Browser apps
• Visual Studio IDE
• Eclipse
• Cross-platform
command line tools
• Microsoft Test
Manger
• Microsoft Visual
Studio Premium or
Ultimate
Microsoft System Center Microsoft Office and
SharePoint
Project
planning and
management
Git
13. Quality Enablement Solution Architecture
Work item tracking Build automation Test lab management
Extensibility interfaces
SCM
Test execution
services
Deployment
automation
Reporting and BI
Lab infrastructure
Test recorders and
runners
Build engines
• Actionable feedback
• Exploratory testing
• Requirements
management
• Acceptance criteria
definition
• Developer testing
• Build automation
• Actionable
diagnostics
• Test planning and
management
• Test execution and
defect reporting
• Lab management
• Production monitoring
• Integrated incident
management
Quality metrics and
reporting
Role tailored tools
16. Business
challenges
Long, opaque manual testing
cycles
“No-repro” defect reports
Repetitive and error prone
Unable to track tested vs.
untested code
Solutions
Actionable bugs containing
data to reproduce defects
Code coverage reports show
untested features
Automated fast-forward
Connect dev and test with
tools to instrument and repro
Business value
Reduce dev-test “ping pong”
to cut waste
Speed delivery of high
quality software
Delivered software meets
user and value expectations
19. Organize testing in multiple
configurations
Playback a prior recording to save
time with frequently-repeated setup
steps and prerequisites
Pause, save, close, return, and
pick up where you left off
Bind data to controls, to
support multiple iterations
and future automation
20. System logs & configuration
IntelliTrace Logs
Detailed status and history visible
to all team members
Audio & Video recordings
21. Get an overview of the test run
with summary pie charts, or drill
into the details for each test.
Analyze code coverage for manual
tests to determine which specific
parts of your code were tested .
Associate manual test runs with
builds and environments.
24. Business
challenges
Prescribed test cases can’t
anticipate all possible bug
scenarios
Ad-hoc testing often results
in “no-repro” defects
Unable to validate original
bug has been fixed
Solutions
Leverage inherent tester
creativity with ad-hoc testing
Record ad-hoc test steps
Create test cases from bugs
to reproduce complex
scenarios or to verify bug
fixes later
Business value
Discover unknown scenarios
and hidden defects
Eliminate manual capturing
of test steps to cut waste
Increase quality in delivered
software
27. Create rich text comments
with screenshots while
exploring
Exploratory sessions can be
recorded to increase
reproducibility
The Test Runner docks on the left side
of the screen and automatically records
your steps as you explore
Explore your application to
uncover new defects
28. Comments and steps are
automatically added
File new bugs without ending the
exploratory test session
The exploratory test session is
paused while you create a bug
report
Attach the video recorded during
the exploratory test session for
maximum reproducibility
Quickly create an
automatically linked test
case for this bug
29. Add verification values to indicate
the expected result
Session is paused while creating test
cases
Test steps are added
automatically
30. Comment block reflects the
added bugs and tests
Continue exploratory testing right
from where you left off
Quick access to bugs and test cases
created during the session
33. Business
challenges
Modern apps run in diverse
environments
Difficult to repro issues found
Replicating environments for
test is time-consuming, costly
Solutions
Virtualization reduces cost to
provision environments
Templates allow a wide range
of baselines to be spun up
quickly and on demand
Snapshots capture states for
reproducing test conditions
and verifying defects
Business value
Dramatically reduced
hardware and maintenance
expenses
Improved quality and
reliability of apps across
platforms
Operations engaged in
earlier phases of testing
36. Create a new environment using
virtual machine templates in
SCVMM
Create an environment
using machines running on
SCVMM
Create new environments
on existing machines
37. Quickly choose multiple test
suites to run after
deployment
Choose your environment, and build,
deploy and test in that environment
from a single dialog
38. Spin up virtual test environments from
within Test Manager, or attach a virtual
environment to a manual testing session
Snapshot environments during
manual testing to increase
reproducibility
39. Test run results are attached to
specific builds and environments
Collect all the data required to
recreate bugs from virtual
environments
40. The Microsoft differentiation
Application
Lifecycle
Continuous quality practices Role tailored tools Hybrid IT capable
New value
Continuous value delivery
41. Call to Action
• Take Microsoft Virtual Academy (MVA) courses
• http://aka.ms/mvasoftwaretesting
• http://aka.ms/mvaswtestingvs
• Use TechNet Virtual Lab (http://aka.ms/virt-labs)
• http://aka.ms/VLTestPlanning
• http://aka.ms/VLTestCase
• http://aka.ms/VLExploratoryTest
• Use Azure Trial (http://azure.microsoft.com/en-us/
pricing/free-trial/)
• Download Visual Studio ‘14 CTP
• http://aka.ms/vsctp14
Notes de l'éditeur
Microsoft Virtual Academy (MVA) => www.microsoftvirtualacademy.com
TechNet Virtual Lab => http://aka.ms/virt-labs