In this session we will provide an introduction of agile testing practices and how they can be supported with the Microsoft Visual Studio ALM solution. This introduction will focus on introducing the concepts behind agile testing practices. The different types of test and how they integrate within a Scrum agile process will be covered. It will also provide a brief overview of the test tools available in the Microsoft Visual Studio ALM solution and how they relate and support these different types of test. Author: Ryan Riehle. More information on Agile with Visual Studio: www.incyclesoftware.com
3. Work Flow with Team Foundation Server
Features
Tasks
Tests
Check-in
Build
Bug
4. Impediments to Continuous Quality
Delivery REQUIREMENTS
WORKING SOFTWARE
Misunderstood
requirements
Conflicting
priorities
Unmet user
expectations
Can’t get
actionable feedback
Disparate management tools
Operations readiness
requirements are not met
Production incidents
New learnings
Loss of focus
Integrating heterogeneous
development teams
Quality after thought:
dev and test hand-offs
5. Requirements don’t reflect customer intent
Rework, long delivery cycles and frustration
Storyboarding tool
Graphical mockups in new PowerPoint add-on reduce misunderstandings and
clarify intent
Problem
Solution
Misunderstood Requirements
UML tools
UML Diagrams provide a standard way to confer intent to development
Solution
6. Misunderstood Requirements
Developers don’t know when
requirements are complete
Development churn, rework, long delivery cycles, and
misunderstandings between stakeholders and development team
360 acceptance criteria
Complete requirements provide functional and non-functional acceptance
criteria ensuring developers know when a requirement is complete
Problem
Solution
Microsoft Test Manager
Manual test cases provide developer-executable acceptance criteria
Solution
7. Team unsure what is most important
Building the wrong thing, rework, unmet customer expectations
Product Backlog
Online backlog interface allows team to prioritize most important work based on
customer interaction
Problem
Solution
Conflicting Priorities
Sprint Planning, Task and Kanban Boards
Sprint planning view manages iterations, task and kanban boards provide graphical
views into individual and team workSolution
8. Quality Afterthought
Quality not build in
Failure to meet user needs, rework, long delivery cycles and customer
dissatisfaction
Code Metrics and Code Analysis
Rich Code Metrics ensure you are building quality, manageable code. Code
Analysis tools help you identify dangerous code early.
Code Review
Integrated Code Review tools make it easy to do pier or manager review of code
before shipping
Problem
Solution
Solution
9. Quality Afterthought
Quality not build in
Failure to meet user needs, rework, long delivery cycles and customer
dissatisfaction
Unit Testing
Extensible unit test framework target specific needs or technologies to ensure
code correctness
Fakes framework
Powerful framework to simplify unit testing and isolate logic
Problem
Solution
Solution
10. Unmet User Expectations
Delivered code fails to satisfy users
Failure to meet user needs, rework, long delivery cycles and customer
dissatisfaction
Manual Testing
Formalized manual tests help both developers and testers ensure that what has
been built meets the requirements defined for the product
Problem
Solution
11. Unmet User Expectations
Delivered code fails to satisfy users
Failure to meet user needs, rework, long delivery cycles and customer
dissatisfaction
Exploratory testing
Ad-hoc exploratory tests help uncover usability and consistency issues, and
result in actionable bugs and reusable test cases
Coded UI Tests
Convert exploratory tests into Coded UI tests to automate the re use of test
cases
Problem
Solution
Solution
12. Ops readiness requirements not met
Delayed feedback on code changes
Slows fixes of critical bugs, increases context switching, rework and long delivery
cycles
Performance and Load Testing
Ensure application performance and scalability with comprehensive web
performance and load testing tools
Lab Management
Self service test lab provisioning ensures you can deploy and test against a
mirror of your real operational environment and speeds both manual and
automated build-deploy-test feedback cycles
Problem
Solution
Solution
The business analyst starts by adding user stories. CLICK Once the user stories has been entered the developer creates tasks for implementing each user storyCLICK Meanwhile the tester authors tests against those user stories CLICK Now the developer writes code that implements a task and checks it into TFSCLICK The checking are materialized to a buildCLICK The tester examines the build, notes the delivered changes and deploys the build to test environment (not shown)CLICK The tester begins testing the build by choosing a test and running it using Microsoft Test Manager CLICK The tester identifies a bug and files it with one click – the bug is automatically associated with the test and the user storyCLICK The cycle can continue as the developer fixes the bug, associates a check-in, and then creates a build which the tester then pulls into test (and so on) CLICK
Point out the highlighted impediment: Misunderstood Requirements.
Instructor Note: There is another slide following this one, also with a Misunderstood Requirements impediment.Requirements don’t always reflect the customer’s intent. We’ve all been in the situation where we’ve delivered software that the users just didn’t feel was what they asked for. Or worse, they say “Hmm. That’s EXACTLY what we asked for, but not at all what we wanted. I guess we just needed to see an example before we really understood.” What makes this so bad, is that if the users could have expressed their desires in something graphical, rather than in a long text document, they could have provided that feedback far earlier, resulting in delivering the right thing, the first time through! Speaking of feedback, if users and other stakeholders were able to rapidly provide high fidelity feedback directly to development teams, devs would be able to respond with a lot more accuracy. Whether this be by simply implementing the changes as requested, or by responding with further questions or time estimates, developers and stakeholders can up their level of discussion. In the process, they can each understand more clearly the objective – features enabled by code.
Even when provided with concrete, understandable requirements, it’s common for developers to not know when the feature is truly complete. Maybe there are dozens of ways the feature can be improved, or maybe it’s not clear how certain functionality should behave. More likely, developers may not have visibility into non-functional requirements around the feature. For instance, maybe users are expecting a sub-second response on a page, yet the developer decided to add some very powerful functionality that they believe the users want, yet it pushes response time to over 3 seconds. To prevent this from happening, business analysts, users, and others providing requirements should identify 360 degree Acceptance Criteria. These acceptance criteria can be provided in User Stories, Requirements, Product Backlog Items, and other work items, to let the developers know exactly when they are done writing code, and when they have met the users expectations. This can reduce “gold plating”, speed development, and generally results in a better working relationship between business analysts and developers!For extra credit, developers or testers can leverage Microsoft Test Manager to create a track test cases that map to the acceptance criteria!
Instructor Note: There is another slide following this one, also with a Misunderstood Requirements impediment.Requirements don’t always reflect the customer’s intent. We’ve all been in the situation where we’ve delivered software that the users just didn’t feel was what they asked for. Or worse, they say “Hmm. That’s EXACTLY what we asked for, but not at all what we wanted. I guess we just needed to see an example before we really understood.” What makes this so bad, is that if the users could have expressed their desires in something graphical, rather than in a long text document, they could have provided that feedback far earlier, resulting in delivering the right thing, the first time through! Speaking of feedback, if users and other stakeholders were able to rapidly provide high fidelity feedback directly to development teams, devs would be able to respond with a lot more accuracy. Whether this be by simply implementing the changes as requested, or by responding with further questions or time estimates, developers and stakeholders can up their level of discussion. In the process, they can each understand more clearly the objective – features enabled by code.
Quality is not something that can be tested into a product. It has to be built in. Modern development practices focus on ensuring that quality is in place early, and that the team bakes in quality throughout the entire software delivery process. For developers, this starts with good unit testing. Unit testing, especially with a test first practice, ensures that code methods deliver correct responses. This is the first step in good code. But unit testing isn’t always easy! With VS 2012, the unit testing framework has been made extensible, allowing developers to plug in their own unit test frameworks. This isn’t just important because developers can choose to use something other than MSTest to test their .NET code. No, the true importance is that developers can now work in a range of languages and technologies, and leverage the unit test frameworks that are specific to those languages. For instance, JavaScript testing is now fully integrated into VS 2012, and it comes from a JavaScript unit testing framework not written by Microsoft. This flexibility helps ensure that all code is unit tested, not just C# or VB.NET code.One of the major impediments to unit testing is the difficulty of writing unit tests that isolate functionality. Traditionally this has been done with a unit test mocking framework. With Visual Studio 2012, a new powerful Fakes framework was introduced by Microsoft to dramatically simplify the isolation of business logic to be tested from other code.
Quality is not something that can be tested into a product. It has to be built in. Modern development practices focus on ensuring that quality is in place early, and that the team bakes in quality throughout the entire software delivery process. For developers, this starts with good unit testing. Unit testing, especially with a test first practice, ensures that code methods deliver correct responses. This is the first step in good code. But unit testing isn’t always easy! With VS 2012, the unit testing framework has been made extensible, allowing developers to plug in their own unit test frameworks. This isn’t just important because developers can choose to use something other than MSTest to test their .NET code. No, the true importance is that developers can now work in a range of languages and technologies, and leverage the unit test frameworks that are specific to those languages. For instance, JavaScript testing is now fully integrated into VS 2012, and it comes from a JavaScript unit testing framework not written by Microsoft. This flexibility helps ensure that all code is unit tested, not just C# or VB.NET code.One of the major impediments to unit testing is the difficulty of writing unit tests that isolate functionality. Traditionally this has been done with a unit test mocking framework. With Visual Studio 2012, a new powerful Fakes framework was introduced by Microsoft to dramatically simplify the isolation of business logic to be tested from other code.
Point out the highlighted impediment: Unmet user expectations.
Point out the highlighted impediment: Unmet user expectations.
Instructor Notes:Explain the benefits of perf and load testing and lab management. This subject will be covered in more detail in the session Close the Loop: Bridging the Gap Between Development and Operations
Logoincycle imageGold ALM partner Since 2002ServicesMVPLocations