Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Holy "Test Automation
Pyramid"
How it ruins your life
About
Yaroslav Pernerovskyy
@ysparrow
https://qaguild-slack.herokuapp.com/
About
http://automation-remarks.com/podcast/
About
Unit
Integration
E2E
Unit
Integration
E2E
Manual
Unit
Integration
E2E
White-box
Independent units
Mocks/stubs
White/Black-box
Some deployment
External systems
Black-box
Fu...
Integration
ü uses database
ü uses network to call another
component/application
ü external system (MQ, mail server etc.)
...
What is wrong with
the Pyramid?
It is a theoretical
metaphor
It is almost
unachievable in
real world
Unit
Integration
E2E
Created by
Developers
Sometimes by
Developers
Sometimes by Testers
Usually by Testers
Contexts differences
Communication issues
Industry stereotypes
Developers
should create all
the tests to build
the canonical
test pyramid
https://seleniumcamp.com/talk/good-tests-change-your-application/
Become a "developer"
Talk to each other
Brake stereotypes
What is wrong with
the Tests?
Having "wrong" kind of tests
Unit
Integration
Unit
Integration
Having "wrong" kind of tests
Unit
Integration
GUI
Having "wrong" kind of tests
Testing wrong functionality
Core
Critical
Other
20%
Code coverage
•http://blog.codepipes.com
Codepipes Testing Metrics
(CTM)
Metric Name Description Ideal value Usual value
Problematic
value
PDWT
% of
Developers
wri...
Codepipes Testing Metrics
(CTM)
Metric Name Description Ideal value Usual value
Problematic
value
PDWT
% of
Developers
wri...
Codepipes Testing Metrics
(CTM)
Metric Name Description Ideal value Usual value
Problematic
value
PDWT
% of
Developers
wri...
Codepipes Testing Metrics
(CTM)
Metric Name Description Ideal value Usual value
Problematic
value
PDWT
% of
Developers
wri...
Codepipes Testing Metrics
(CTM)
Metric Name Description Ideal value Usual value
Problematic
value
PDWT
% of
Developers
wri...
0% 100%Code coverage
effortneeded
0% 100%Code coverage
Valuefromtests
Treating ATDD as a religion
ü you can write tests before
ü you can write tests at the same time
ü you can write tests afte...
What else is wrong
with the Pyramid?
Context-free geometric
analogy
It is not about Testing
http://www.satisfice.com/blog/archives/4947
ü The Round Earth model shows
testing problems at multiple levels
ü The Round Earth model reminds us
about data
http://www.satisfice.com/blog/archives/4947
ü The Round Earth model shows
testing problems at multiple levels
ü The Round Earth model reminds us
about data
ü The Roun...
Quality above requires
quality below
Quality above reduces
dependence on expensive
high-level testing
Inexpensive low-level
testing reduces dependence
on expensive high-level
testing
Risk grows toward the user
Itogi podvedyom
ü Triangle is a mental abstraction
ü It should be treated as abstraction
ü Think before act
ü Don't blame ...
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life
Prochain SlideShare
Chargement dans…5
×

QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life

205 vues

Publié le

Практически любую проблему человечества можно описать в виде "пирамиды". Про пирамиду Маслоу и финансовые пирамиды слышали все. Тестировщики тоже люди, и они придумали свою священную пирамиду - пирамиду автоматизации тестирования. Эта концепция доминирует на рынке уже многие годы и приносит свои позитивные плоды. Но есть и темная сторона этого процесса, выгорание, уныние и забвение в погоне за пирамидой тоже не редкость. В своем докладе хочу немного покритиковать устоявшиеся устои и разобраться как выжить в мире разработки и автоматизации с пирамидой или без нее, а также пройтись по антипатернам в автоматизации, которые так или иначе касаются пирамиды.

Publié dans : Formation
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

QA Fest 2018. Ярослав Пернеровский. Test Automation Pyramid, how it ruins your life

  1. 1. Holy "Test Automation Pyramid" How it ruins your life
  2. 2. About Yaroslav Pernerovskyy @ysparrow
  3. 3. https://qaguild-slack.herokuapp.com/ About
  4. 4. http://automation-remarks.com/podcast/ About
  5. 5. Unit Integration E2E
  6. 6. Unit Integration E2E Manual
  7. 7. Unit Integration E2E White-box Independent units Mocks/stubs White/Black-box Some deployment External systems Black-box Fully deployed system Real life scenarios
  8. 8. Integration ü uses database ü uses network to call another component/application ü external system (MQ, mail server etc.) ü reads/writes files or performs other I/O ü does not rely on source code, use deployed binary instead http://blog.codepipes.com/testing/software-testing-antipatterns.html
  9. 9. What is wrong with the Pyramid?
  10. 10. It is a theoretical metaphor
  11. 11. It is almost unachievable in real world
  12. 12. Unit Integration E2E Created by Developers Sometimes by Developers Sometimes by Testers Usually by Testers
  13. 13. Contexts differences Communication issues Industry stereotypes
  14. 14. Developers should create all the tests to build the canonical test pyramid
  15. 15. https://seleniumcamp.com/talk/good-tests-change-your-application/
  16. 16. Become a "developer" Talk to each other Brake stereotypes
  17. 17. What is wrong with the Tests?
  18. 18. Having "wrong" kind of tests Unit Integration
  19. 19. Unit Integration Having "wrong" kind of tests
  20. 20. Unit Integration GUI Having "wrong" kind of tests
  21. 21. Testing wrong functionality Core Critical Other
  22. 22. 20%
  23. 23. Code coverage
  24. 24. •http://blog.codepipes.com
  25. 25. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  26. 26. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  27. 27. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  28. 28. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  29. 29. Codepipes Testing Metrics (CTM) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70% Anything less than 100% PBCNT % of bugs that create new tests 100% 0%-5% Anything less than 100% PTVB % of tests that verify behavior 100% 10% Anything less than 100% PTD % of tests that are deterministic 100% 50%-80% Anything less than 100% http://blog.codepipes.com/testing/software-testing-antipatterns.html
  30. 30. 0% 100%Code coverage effortneeded
  31. 31. 0% 100%Code coverage Valuefromtests
  32. 32. Treating ATDD as a religion ü you can write tests before ü you can write tests at the same time ü you can write tests after ü you can never write test
  33. 33. What else is wrong with the Pyramid?
  34. 34. Context-free geometric analogy It is not about Testing
  35. 35. http://www.satisfice.com/blog/archives/4947
  36. 36. ü The Round Earth model shows testing problems at multiple levels ü The Round Earth model reminds us about data
  37. 37. http://www.satisfice.com/blog/archives/4947
  38. 38. ü The Round Earth model shows testing problems at multiple levels ü The Round Earth model reminds us about data ü The Round Earth model reminds us about testability
  39. 39. Quality above requires quality below
  40. 40. Quality above reduces dependence on expensive high-level testing
  41. 41. Inexpensive low-level testing reduces dependence on expensive high-level testing
  42. 42. Risk grows toward the user
  43. 43. Itogi podvedyom ü Triangle is a mental abstraction ü It should be treated as abstraction ü Think before act ü Don't blame yourself ü Use right metrics ü Improve your programming skills ü Right tests over making triangles ü Do not hesitate to kill shitty tests ü Think about other metaphors

×