NWEWT_Slides_Ash_Winter_04_2016

•Download as PPTX, PDF•

0 likes•134 views

Ash Winter

“Don’t make it
any worse.”
A tale of regression testing,
misguided expectations and
logical fallacies…

Instability – it’s a thing
• Projects to
stabilise
• Update/upgrade
underlying
technology
• While
maintaining
“normal”
service

Context…
• Wobbly
• PCI
• POC as a £££
revenue stream
• New for some,
old for others
• New database/
binaries/
hardware.

Two statements…
• “Don’t make it any worse”
• “Customers shouldn’t be
able to tell the
difference”

Immediate Questions…
• “To determine if
something is
worse, I’ll need
to know how bad it
is?”
• “What happens if
its good for some
too?”
• “For customers to
notice, I’ll need
to know what they
pay attention to…”

Deeper Questions…
• “We’re going to change everything,
how can I test that nothing has
changed?”
• “If you change the responsiveness
and capacity of a system, that’s
still noticeable?”

Orders of Ignorance…
• Little
codified
knowledge.
• We wouldn’t
know how to
know if
something had
changed…

#1 Risk Modelling
• FIBLOTS again!
• 4 people &
whiteboard x n
• People were
the primary
oracle
• With a bit of
existing
system…

#2 Light the path…
• Map the system
• Explore the thing
• Decide what sheds light

#3 Pyramid Scheme
•Filthy UI
Checks
•Less
grimy API
checks

#4 Check yourself
• Filled in
the unit
level
• Big?
definitely
• Clever?
Hmm

Results – Rumsfeldian
Problems
“Don’t touch
that, its
borked in a
known way to
us, but
unknown to
our clients.”

Results - Everything
changes but you…
Client A - “This
thingy responds
much faster than
before…”
Client B – “Before
we only do X jobs
a minute, now its
increased to Y
jobs”

Results - Move any
mountain?
• When does one stop
exposing a previously
unseen mountain of
knowledge?

Conclusions? #1
• Shallow
statements,
taken
literal
• People fix
stuff,
sub/un/consc
iously

Conclusions #2
•Deterministic
> random…
•Software >
Hardware…
•Mission
impossible?

Similar to NWEWT_Slides_Ash_Winter_04_2016

Next generation pentest your company cannot buy

Vlad Styran

Владимир Стыран - Пентест следующего поколения, который ваша компания не може...

UISGCON

Mapping your network

Maria H

Webinar by Clarke Ching Agile and ToC expert. Agile: the Good, the Bad and the Ugly. If your Agile is broken then this is how to fix it! Your Agile teams are busy. Busy delivering. Busy improving. Your quality is amazing. Rework is low. The product looks great. Your users love it. You are a high performing team! But your internal customers say your teams are slow. This session will teach you how to use the Theory of Constraints to figure out how to speed up, by finding the one thing that’s slowing them down. This webinar will cover how, in an Agile environment: - to better control scope creep, - to reinforce your relationship with the I.T. Development team’s client, - to be able to make commitments and honour them and - to decide where your bottleneck should be. About the speaker Clarke Ching is a computer scientist with an MBA who discovered Goldratt’s Theory of Constraints (ToC) in 2003 and has been using it ever since to accelerate Agile initiatives. He is fascinated by Agile and obsessed with ToC. He wrote the amazon best-sellers Rolling Rocks Downhill and The Bottleneck Rules. Rolling Rocks Downhill teaches 3 things: the fundamentals of Agile combined with ToC; how to use those fundamentals to deliver big projects faster and on time; and how to deliver quietly huge transformations. It’s been featured in The Guardian newspaper and The Spectator magazine. It was one of Barbara Oakley’s top 10 books of 2019. It was the #2 best-selling Leadership book on amazon.com, just behind Steven Covey’s 7-habits book. He has been Agile / Lean / ToC expert in: GE Energy, Dell, Royal London (life insurance & pensions), Gazprom and Standard Life Aberdeen among other organizations. He is the past Chairperson of Agile Scotland. He is a lecturer at Victoria University School Of Management in New Zealand where he now lives. Today he is the founder and Chief Productivity Officer of Odd Socks Consulting

Agile: the Good, the Bad and the Ugly - Webinar by Clarke Ching Agile - Septe...

MARRIS Consulting

Customer Feedback: the missing piece of the Agile puzzle

skierkowski

Pdf analytics-and-witch-doctoring -why-executives-succumb-to-the-black-box-me...

OrateTeam

An Introduction to a Systems Thinking Approach-2

Andrew Stangroom

Life occurs in real-time, and not surprisingly, more solutions are being built using streaming technologies. Event-based architectures are becoming the norm, and customers are expecting immediate access to their data. This new world offers many exciting opportunities, but also some new challenges. What do you do when your streaming data is not complete? What if it relies on another data source? Does the dependent data exist yet, and does it come from a 3rd party? How do we merge a complete picture of data when data is sourcing from multiple places at the same time? A new norm in the world of distributed services. Join us as we dive deep into the technical details around these scenarios and more. Expect to learn about stream-stream joins, enriching stream data using local or remote data, and ways to anticipate and correct errors within the stream. Leave with a better understanding of managing data dependencies within a Spark Structured Streaming application.

Life is but a Stream

Databricks

Material for the 26 Oct 2015 lecture I held for Aalto University business students. The lecture focuses on the high level topics in analytics and Big Data that are either central to the subject or just highly visible in the media. The main messages of the lecture are: - The purpose of analytics and of the data analyst is to solve business problems - Big Data brings over some very special traits to doing analytics that don't exist when working working with smaller datasets. Understanding these traits is a must for successful analytics. - Deploying analytics is more dependent on humans than on technology - Data and analytics are nowadays significant assets to many companies. Therefore they need their own strategy and need to be managed just like any other business critical assets.

Analytics in business

Niko Vuokko

Max Shron, Thinking with Data at the NYC Data Science Meetup

mortardata

MSCPA Update 2017

Allen Lloyd

Your Data Scientist Hates You

Bradford Stephens

Retrospective and different ways to run it

Liandra Bassiane

Data Con LA 2022 - Customer-Driven Data Engineering

Data Con LA

Welcome to the data-driven world. 39% of executives say their companies are already highly data-driven, but mentioning data can often result in concerned faces. Why is this? Is it the toxic nature that 'metrics' have become synonymous with or is it because we view using data as a dangerous flirtation with placing more value in tools and processes? This session will showcase how you can bring to life the scientific method in your coaching arsenal. I will share stories of data-based coaching in PwC and how we leverage it to have open, transparent conversations and more informed decision making.

LKCE18 Nicolas Brown - Coaching in a data-driven world

Lean Kanban Central Europe

To meet expectations and optimize flow, managing risk is an important part of Kanban. Anticipating and adapting to things that "go wrong" and the uncertainty they cause is topic of this session. We look at techniques for quantifying what risks should be considered important to deal with. Although discouraging, forecasting size, effort, staff and cost is sometimes necessary. Of course we have to do as little of this as possible, but when we do, we have to do it well with the data we have available. Forecasting is made difficult by un-reliable information as inputs to our process – the amount of work is uncertain, the historical data we are basing our forecasts on is biased and tainted, the situation seems hopeless. But it isn't. Good decisions can be made on imperfect data, and this session discusses how. This session shows immediately usable and simple techniques to capture, analyze, cleanse and assess data, and then use that data for reliable forecasting. Second and hopefully draft of LKCE 2014 talk.

Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...

Troy Magennis

This session presents a simple, human-based approach to create test suites targeting multiple points of contact in a data solution. Commonly, an enterprise will pick a data processing solution with heavy GUIs because it can make an easy to understand workflow around data. However, those solutions still are not able to verify the simplest use case, i.e. “If I put data into a solution to process data, then I should get a desired result.” FIS will demonstrate and teach you how to build a unique testing solution on top of Apache Spark. Under its solution, FIS can actually prove to users in their organization that when they put data in, they get the correct result out. They can also enlist their entire team from product owner to developer to write complete unit tests. The type of flexibility Spark enables allows you to take unique paths in building robust, understandable data flows. The transformational element is the ability to do this in milliseconds, and not wait till the entire pipeline finishes.

Behavior-Driven Development (BDD) Testing with Apache Spark with Aaron Colcor...

Databricks

Market Sounding Brief: ACT Government Data Management

Christopher Norman

Every year, software companies spend a huge amount of time and effort estimating large projects, and still end up regularly missing the mark - often by huge amounts. What the heck is going on? With all of the planning poker, and PI planning, and #noestimates, why isn't this figured out yet? In this talk, we'll dive into probability theory and psychology to discover some of the common underlying causes for a lack of predictability. Once we understand why the world is so uncertain, we'll talk about how we can live with our estimation failures, while still thrilling our customers and maintaining enough predictability to succeed as an organization.

Software estimation is crap

Ian Garrison

Stop SharePoint Project Failure

Matthew J. Bailey , MCT

Similar to NWEWT_Slides_Ash_Winter_04_2016 (20)

Next generation pentest your company cannot buy

Владимир Стыран - Пентест следующего поколения, который ваша компания не може...

Mapping your network

Agile: the Good, the Bad and the Ugly - Webinar by Clarke Ching Agile - Septe...

Customer Feedback: the missing piece of the Agile puzzle

Pdf analytics-and-witch-doctoring -why-executives-succumb-to-the-black-box-me...

An Introduction to a Systems Thinking Approach-2

Life is but a Stream

Analytics in business

Max Shron, Thinking with Data at the NYC Data Science Meetup

MSCPA Update 2017

Your Data Scientist Hates You

Retrospective and different ways to run it

Data Con LA 2022 - Customer-Driven Data Engineering

LKCE18 Nicolas Brown - Coaching in a data-driven world

Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...

Behavior-Driven Development (BDD) Testing with Apache Spark with Aaron Colcor...

Market Sounding Brief: ACT Government Data Management

Software estimation is crap

Stop SharePoint Project Failure

More from Ash Winter

Testability Advocacy Canvas

Ash Winter

Testability Sales Pitch

Ash Winter

Architectural Testability Workshop for Test Academy Barcelona

Ash Winter

Testability Squad Health Check

Ash Winter

Testability is a first class concern for all disciplines within software development. There, I said it. No hedging, no nebulous phrasing, maybes or it depends. Too often we labour under systems that are hard to test, manifesting themselves with frantic searches for more testers, lengthy acceptance test runs, fearful testing for regressions with a hopeful release at the end. Worst of all, it usually ends up with a project manager sat on the testers desk asking 'when will testing be done.' It's never done, it can only stop, just so you know. Throughout my career, often the testability of a system has been deemed to be the testers concern. If something was hard to test, then it was the testers problem. However, the causes of low testability effect the activities of all disciplines, whether it be speed of feedback to developers or flow of value generating features for product managers. During the talk, we will cover: * How testability is a key advantage in building systems of ever increasing complexity. * Why it's important for developers and operational stakeholders to build inherently testable systems. * What testers can do to be catalysts for testability improvements. The activity of testing is rarely the bottleneck, how testable your system is is your problem. Poor testability cannot be remedied by one discipline alone. It's for all of us to care about.

Testability is Everyone's Responsibility

Ash Winter

One area that testers might be able to enhance their contributions to software development teams is how we perceive and contribute to unit testing. Being able to influence this type of testing in a positive manner is a skill that testers will need to get to grips with, as more companies start to embrace a model of lone testers in cross functional teams. The shift of focus from primarily the testing that testers do, to the testing that the team does, is a key shift in thinking and behaviour. To facilitate this shift, I believe testers busting their own illusions about this aspect of building something good would bring us much closer to developers and help us realise what other layers of testing can cover most effectively. The last point is pertinent here, as knowing and guiding unit testing brings the role of integration, acceptance and exploratory testing into sharp focus. This is a topic that has always intrigued me, having predominantly worked as a single tester on a team for the last five or so years. I reached out to the community with the question “What do testers believe about unit testing?” and received a lot of engagement. The good users of Twitter added another 50 or so illusions that testers might have about this layer of testing. I figured that based on that level of engagement, maybe this would make an interesting talk! It wasn’t only testers who responded too, suggesting that there might be some shared illusions about unit testing that are cross disciplinary. The list alone is interesting but now I would like to share my analysis of it with you, focusing on: * Recurring themes within the list and how to address them as a tester or developer. * Particular illusions to look out for with examples from my recent past. * A guide for developers to engage with testers on unit testing, and testers with developers.

Testers Guide to the Illusions of Unit Testing

Ash Winter

10 P's of Testability

Ash Winter

One of my recent endeavours has been to create a "career model" for the testers within my organisation. I sat in my office at home and designed "the testing wheel". I wanted it to be simple, inclusive and offer questions but no answers. Careers are long and winding. My own career has not been a linear progression, so building a linear model seemed wrong to me. But, this brought me into conflict with the ideas of others. My experience report will show the highs and lows of this model after introducing it into the wild. People still tried to measure and rank with it, got mad people got when I refused to answer the questions it posed. Finally how it spread and entered battle with wider organisations career models.

The Wheel of Testing

Ash Winter

This is a call to action. On our cross functional teams and during our devops transformations we talk about how testing is for the whole team. Quality is everyone's responsibility. How much are we really doing to make this happen? Often we are working on systems that are hard to test for many reasons, but if we simply do more testing, write more automation we are neglecting what should be our main mission, advocating for increasing levels of testability, to truly get everyone involved in testing. We all have stories about how something is difficult to test, often never being tested or certainly left with the tester to figure it out. It doesn't have to be this way. During my talk, I want to introduce a set of principles for testability engineering. A new way to approach our work as testers. These principles will tackle how we make our systems more observable, controllable, how we share knowledge across teams and improve the testability of our dependencies. I believe it is time to create a new focus on testability, as it affects everything we do, what our teams do and beyond into how value is delivered for customers. I want you to take away from the talk: * Why a focus on testability can multiply your effectiveness as a tester * What the principles of testability engineering are and how to advocate for them * How you can make iterative changes to what you do in order to embrace testability New technology and complexity is rendering many software development techniques and paradigms obsolete at an increasing rate. We already exist in a space where an infinite number of tests of an array of different types can be performed. A new mission is needed, one that leverages the varied talents of all kinds of testers and culminates in a new focus on the exponential benefits that testability brings.

Making testability our mission

Ash Winter

A Testers Guide to the Myths, Legends and Tales of Unit Testing

Ash Winter

How often have you found a problem with your application which is directly related to the infrastructure it is deployed upon? How often have you found a problem and not known? Testing is beginning to reach new depths. In my experience, saying that infrastructure needed to be tested used to trigger disbelieving looks in Ops teams faces, but less so now. The lines between infrastructure and code are blurring, so lets update our skills and outlook accordingly. * What I mean by infrastructure and why it is important to apply a testing mindset to this area * Questions to determine what to test, how to organise those thoughts and techniques that might be applied * A selection of tools to both explore and programatically check your infrastructure pre-deployment

Testing Below the Application

Ash Winter

Shift Testability

Ash Winter

Nobody /really/ likes change, its human nature. Testers have a special relationship with changing tools and techniques, they change and we tend to flounder a little and end up very nervous about our place in the new world. Continuous delivery is one such circumstance, I see and speak to many testers really struggling. However, with a significant shift in outlook and a chunk of personal development, testers can excel in environments such as these. It’s time to start to get out in front of a changing world, rather than always battling to catch up. I want to share my experience of adding value as a tester in a continuous delivery environment, what new technologies and techniques I've learned, using your Production environment as an oracle, advocating testability and most crucially, not overestimating what our testing can achieve. Testing is not the only form of feedback, it’s time to let go of some the aspects of testing we cling to. Continuous delivery adds richness and variety to our role as testers. To me, it is a facilitator for the autonomy and respect that testers have craved for a long time, so let’s get involved...

Part of the Pipeline

Ash Winter

Scroll Based Testing Strategy

Ash Winter

Coaching Model for Unrecognised Internal Models

Ash Winter

Critical Thinking for Consultants-External

Ash Winter

Ash Versus Consulting V1 - External

Ash Winter

More from Ash Winter (17)

Testability Advocacy Canvas

Testability Sales Pitch

Architectural Testability Workshop for Test Academy Barcelona

Testability Squad Health Check

Testability is Everyone's Responsibility

Testers Guide to the Illusions of Unit Testing

10 P's of Testability

The Wheel of Testing

Making testability our mission

A Testers Guide to the Myths, Legends and Tales of Unit Testing

Testing Below the Application

Shift Testability

Part of the Pipeline

Scroll Based Testing Strategy

Coaching Model for Unrecognised Internal Models

Critical Thinking for Consultants-External

Ash Versus Consulting V1 - External

NWEWT_Slides_Ash_Winter_04_2016

1. Ash Winter Short stuff here: @northern_tester Long stuff here:

2. “Don’t make it any worse.” A tale of regression testing, misguided expectations and logical fallacies…

3. Instability – it’s a thing • Projects to stabilise • Update/upgrade underlying technology • While maintaining “normal” service

4. Context… • Wobbly • PCI • POC as a £££ revenue stream • New for some, old for others • New database/ binaries/ hardware.

5. Two statements… • “Don’t make it any worse” • “Customers shouldn’t be able to tell the difference”

6. Immediate Questions… • “To determine if something is worse, I’ll need to know how bad it is?” • “What happens if its good for some too?” • “For customers to notice, I’ll need to know what they pay attention to…”

7. Deeper Questions… • “We’re going to change everything, how can I test that nothing has changed?” • “If you change the responsiveness and capacity of a system, that’s still noticeable?”

8. Orders of Ignorance… • Little codified knowledge. • We wouldn’t know how to know if something had changed…

9. #1 Risk Modelling • FIBLOTS again! • 4 people & whiteboard x n • People were the primary oracle • With a bit of existing system…

10. #2 Light the path… • Map the system • Explore the thing • Decide what sheds light

11. #3 Pyramid Scheme •Filthy UI Checks •Less grimy API checks

12. #4 Check yourself • Filled in the unit level • Big? definitely • Clever? Hmm

13. Results – Rumsfeldian Problems “Don’t touch that, its borked in a known way to us, but unknown to our clients.”

14. Results - Everything changes but you… Client A - “This thingy responds much faster than before…” Client B – “Before we only do X jobs a minute, now its increased to Y jobs”

15. Results - Move any mountain? • When does one stop exposing a previously unseen mountain of knowledge?

16. Conclusions? #1 • Shallow statements, taken literal • People fix stuff, sub/un/consc iously

17. Conclusions #2 •Deterministic > random… •Software > Hardware… •Mission impossible?

18. End!

NWEWT_Slides_Ash_Winter_04_2016

Recommended

Recommended

More Related Content

Similar to NWEWT_Slides_Ash_Winter_04_2016

Similar to NWEWT_Slides_Ash_Winter_04_2016 (20)

More from Ash Winter

More from Ash Winter (17)

NWEWT_Slides_Ash_Winter_04_2016