2. Who Am I?
Ady Stokes
Nearly 2 decades in Software Testing,
Quality Assurance and Risk management and still learning
Currently manage QA and Risk at HML in Skipton for their
Business Intelligence Division
Contacts:
Twitter: @CricketRulz
Email work: adrian.stokes@hml.co.uk
Email home: adystokes@sky.com
3. Where I Work
HML (part of the Skipton Group)
Has over 42 billion in assets under
management
Offices in Skipton, Derry, Glasgow
Highly regulated financial organisation
Agile adoption since 2008
5. Everyone Tests!
It’s a fact of life everyone tests all the time!
Think of your first coffee this morning. You
had a sip to see if it wouldn’t burn your mouth!
That’s testing!
When you had your generic satellite system installed you
paused, rewound and fast forwarded live TV! That’s testing!!!
Everyone tests! It’s nothing to be frightened of
However, getting from undisciplined to structured,
now that takes skills and knowledge!
6. Triangle Test
Triangle Test Example
By Glenford J Myers 1979
Scope – Software defines what type of triangle
based on 3 possible outcomes,
Equilateral (all sides same length)
Isosceles (two sides of equal length)
Scalene (all sides are unequal in length)
List as many tests as you can in 10 minutes
7. Triangle Test
Side One:
Side Two:
Output Grid
Side Three:
DRAW
8. Possible Tests
It is impossible to execute every possible test, but here are some things that I’d
expect you to have covered:
• Valid entry for all three types of triangle
• Permutations for isosceles with two equal sides? (3,3,4; 3,4,3; 4,3,3)
• Incorrect or mathematically impossible triangles (1,2,3; 1,1,9999 ; 2, 5, 10)
• Negative and zero inputs for at least one side (0, -1,)
• Blanks (blank, 1, 1) for each field?
• Spaces (space, 1, 1) for each field?
• Boundaries and decimals (9999, 9999, 9999, 0.99 etc.)
• Field size, what is the maximum entries or is it unlimited?
• Special Characters ($%^&), Alpha’s, Alphanumeric?
For each of the above do you know what the result should be?
Can you think of any others?
9. It Is Not Fair Because…
The triangle test is not really fair because…
• You have no requirements so everything you did was
based on an assumption about the program
• I didn’t offer any help with what valid and invalid data was
• I didn’t tell you how to start the program, what version it
was, if it was live or not.
• There was no information on dependency or performance
But hopefully it does make you think about testing and
how there are a great many things that can affect and
influence your software?
11. So, What is the
‘Testing Mindset’
It’s not…
Developers Make, Testers Break
(Sorry, but it was like that when we got there!)
Or
Testers Tell Tales To Teacher!
(We want to tell YOU directly because we are all in it together)
Its more like…
What would happen given every possible practical
scenario?
12. The Testing Mindset
‘We can’t solve problems by
using the same kind of thinking
we used when we created them’
Albert Einstein
12
13. Types of Tests!
When it comes to software testing there are many methods
techniques and considerations which are useful to be aware of.
• Boundary Value Analysis – at the edge of value entry
• Equivalence Partitioning – grouping value entry
• Unit testing – focus on individual components
• Functional testing – focus on outputs
• Load / Stress / Performance / Deployment and Rollback
• Happy path / Sad path – does as it should, handles the
unexpected
• Usability – flow / understandable / navigation
• Security – access / safe / secure
14. Top tips for testing
Be curious
Don’t make
assumptions
14
15. The Testing of Requirements
New World
Old World
• Singular
• Singular
• Complete
• Complete
• Consistent
• Consistent
• Correct
• Correct
• Current
• Current
• Unambiguous
• Unambiguous
• Prioritised
• Prioritised
• Traceable
• Traceable
• Testable
17. Who Am I
Ady Stokes
Contacts:
Twitter: @CricketRulz
Email work: adrian.stokes@hml.co.uk
Email home: adystokes@sky.com
Notes de l'éditeur
Part of project to create a new data warehouse using agile principles. We now have over 10,000 automated tests in our regression suite giving continuous validation for new and existing code. Each time something changes we know the impacts and can adjust. These tests didn’t just appear. I helped create them and helped get everyone involved to add to the suite which the developers automated as part of their definition of done.
Played with a new phone, iPad, laptop etc.Everyone tests but it is undisciplined. What are you testing for, why. What is most important? What is the most essential part?
Played with a new phone, iPad, laptop etc.Everyone tests but it is undisciplined. What are you testing for, why. What is most important? What is the most essential part?
From ‘The Art of Software Testing’ still an excellent book of principles 30 plus years on. It has been hijacked by some test managers as a test to check new testers knowledge and that’s not really fair but I’ll explain why after… you have a go!Do this in pairs or on your own but please join in and have a go.Over to you guys! Exercise those brains!
Visual representation of programme to help while trying the exercise.
Put your hand up if you wrote a test. 3 tests, 5 tests, 7 tests, 10 tests.Do you know what the results should be? They shouldn’t because – next slide…The book says there are 13 fundamental tests which I have summarised in the slide.Professional testers on average still ‘score’ 7 or 8 even though this is 30 years old.
I said it wasn’t fair and here’s why. (see slide) You will have heard the expression, you can’t test everything and you can’t. But to understand why you can’t test everything you first need to have the skills to identify everything then reduce it using techniques like priority, risk, likelihood of occurrence and so on.Can anyone give me an example of a very simple software game? Does it work at every resolution? All browsers? All flash/font versions? Operating systems? Independently? With other tabs? Other programmes? The list goes on forever and we haven’t begun on hardware yet!
Testers don’t write the software so actually they can’t break it. They can only expose behaviour.Unless your organisation puts developers and testers in direct conflict by rewarding both for bugs, least for devs, most for test (and if so leave immediately!) then you are after the exact same goal. Working software that is acceptable by the customer. Testers want to talk to you more than they want to spend hours filling in forms, honest.As before ‘every’ is reduced based on lots of things, including time constraints. This is massively helped if testers are involved from the beginning and testing requirements and writing tests from the off.
Testers don’t write the software so actually they can’t break it. They can only expose behaviour.Unless your organisation puts developers and testers in direct conflict by rewarding both for bugs, least for devs, most for test (and if so leave immediately!) then you are after the exact same goal. Working software that is acceptable by the customer. Testers want to talk to you more than they want to spend hours filling in forms, honest.As before ‘every’ is reduced based on lots of things, including time constraints. This is massively helped if testers are involved from the beginning and testing requirements and writing tests from the off.
I think we would agree this guy knows his onions!
Lots of variations of tests that a good specialist tester will apply given the situation.
Top two tips
There were a few questions to Chris Moran about testing requirements last time so here’s a brief overview of it.When we did BUFD (Big Up Front Design) and had 400 page documents, it was ‘possible’ to (try to) ensure requirements were complete but today we accept there are things we don’t know we don’t know and try to reduce what we don’t know through discovery.I never liked ‘correct’ because it is in itself and ambiguous word that needs definition.So if a requirement or User Story is not singular, decompose it until it is. Etc… and at the end you should be left with something that is a good basis for conversation, manageable, estimable and most importantly testable.Techniques like INVEST help user story decomposition. (Independent; Not Too Specific; Valuable; Estimable; Small; Testable)
Chris Moran said last month, don’t agitate, collaborate. It doesn’t just apply to testers! It applies to everyone!