What you can, can’t and must test in Magento 
! 
presented by joshua warren 
! 
@joshuaSWarren
Don't worry about finding the perfect tools 
to test Magento. Just start testing. 
#MM14NY
creatuity.com/mmny 
! 
@joshuaswarren
75% of Magento developers surveyed are not 
conducting any automated testing of their 
Magento projects 
#MM14NY
Less than 20% of developers not testing 
Magento aren't testing because they think 
Magento is too complex to test 
#MM14N...
lack of budget is the #1 reason developers 
aren’t testing their magento projects 
#MM14NY
Lack of knowledge is the 2nd most common 
reason for not performing automated testing 
of magento 
#MM14NY
Lack of time is the 3rd most common reason 
for not performing automated testing of 
magento 
#MM14NY
testing is important
automated testing allows magento 
developers to focus on building new features 
#MM14NY
with automated testing, bugs are found and 
fixed as part of development. 
#MM14NY
one interruption to fix a bug that made it past 
QA costs the best developers 15 minutes of 
lost productivity 
#MM14NY
SOURCE: ‘PROGRAMMER INTERRUPTED’ - http://blog.ninlabs.com/2013/01/programmer-interrupted/
a well designed automated testing system 
reduces interruptions and increases 
productivity and happiness. 
#MM14NY
types of tests used to test magento
unit testing 
logical 
analytical 
great for ideal projects
unit testing 
magento’s flexibility greatly 
complicates unit testing
unit testing 
great for extensions 
github.com/EcomDev/EcomDev_PHPUnit 
@IvanChepurnyi
it’s not that you can’t test magento - you 
can’t unit test magento, but there’s more to 
testing than just unit tests 
#M...
100% of developers surveyed that are testing 
magento are using functional tests 
#MM14NY
functional testing 
focuses on the entire site as a system 
doesn’t care what function is being called 
tests that the sit...
with full test coverage, functional tests 
confirm that your magento site still works 
as you add new features 
#MM14NY
one more thing… 
only 50% of developers surveyed are 
conducting performance testing 
! 
performance testing is a must
performance testing 
not just for sysadmins 
not just for front-end developers 
bad performance = bad sales
performance testing 
backend developers don’t always remember 
that extensions, integrations and new 
features can slow th...
when testing your magento projects, include 
performance testing - poor performance is a 
bug 
#MM14NY
slow code should be optimized or removed 
just like any other bug would be 
#MM14NY
how to test magento
testing servers 
bamboo - commercial, general purpose 
! 
jenkins - open source, general purpose 
! 
phpci - open source, ...
only 1 developer reported using phpci for 
testing magento; 50% are using bamboo, 50% 
are using jenkins 
#MM14NY
jenkins vs bamboo 
jenkins is free 
Bamboo $10 starter license 
! 
Try both, pick whichever you like better
which testing server you use isn’t important - 
the fact that you have a testing server and 
are testing is. 
#MM14NY
100% of developers who said they are 
performing functional tests are using 
selenium 
#MM14NY
selenium test cases 
selenese vs phpunit 
test standard magento features 
test your custom features
gatling test cases 
scripted in scala 
normal visitor workflow 
randomized delays
gatling test cases 
specify number of simultaneous visitors and 
overall test length 
! 
for example…
gatling test cases should be set for the 
maximum simultaneous users you expect and a 
timeframe of 60-120 seconds 
#MM14N...
automated tests of magento should be fast - 
test each commit and receive pass/fail 
before you move to your next task 
#M...
gatling test cases 
develop baseline at start of project 
! 
if tests are 20% slower than baseline, 
consider the test fai...
what’s Next?
testing magento 1 sites 
can’t - unit testing 
can - functional testing 
must - performance testing
testing magento 
automated testing saves time, money & sanity
let’s keep talking 
what did I get wrong? 
TDD vs BDD? 
BeHat? 
@joshuaswarren / joshuawarren.com
behat, bdd vs tdd, unit testing in magento 1 - 
tweet at @Joshuaswarren with what you 
disagree with 
#MM14NY
but wait, there’s more! 
in-depth talk at phpworld 
november, washington dc 
world.phparch.com
spread the word 
educate our peers, managers and clients 
! 
summer 2015 - automated testing is the norm
spread the word 
Creatuity will be open sourcing our testing 
tools and test cases 
! 
foundation for fast, flexible, reus...
coming soon to github - @joshuaswarren and 
@creatuity are releasing a foundation for 
performance and functional testing ...
thank you! 
creatuity.com/mmny 
! 
@joshuaswarren
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Automated Testing Talk from Meet Magento New York 2014
Prochain SlideShare
Chargement dans…5
×

Automated Testing Talk from Meet Magento New York 2014

1 440 vues

Publié le

Opinions differ on test driven development, and some very smart developers are debating TDD and test-first vs system tests at a very high level. However, down in the trenches when building and supporting complex eCommerce websites built around Magento we live in a very different world with a number of constraints about what we can, can’t and absolutely must test. Magento itself, clients and budgets put constraints on what we are able to test, but testing is a critical part of keeping a site stable. I will briefly discuss the options available for testing Magento sites and how you could be testing your work with Magento.

Publié dans : Technologie
0 commentaire
3 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Nombre de vues
1 440
Sur SlideShare
0
Issues des intégrations
0
Intégrations
172
Actions
Partages
0
Téléchargements
2
Commentaires
0
J’aime
3
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

Automated Testing Talk from Meet Magento New York 2014

  1. 1. What you can, can’t and must test in Magento ! presented by joshua warren ! @joshuaSWarren
  2. 2. Don't worry about finding the perfect tools to test Magento. Just start testing. #MM14NY
  3. 3. creatuity.com/mmny ! @joshuaswarren
  4. 4. 75% of Magento developers surveyed are not conducting any automated testing of their Magento projects #MM14NY
  5. 5. Less than 20% of developers not testing Magento aren't testing because they think Magento is too complex to test #MM14NY
  6. 6. lack of budget is the #1 reason developers aren’t testing their magento projects #MM14NY
  7. 7. Lack of knowledge is the 2nd most common reason for not performing automated testing of magento #MM14NY
  8. 8. Lack of time is the 3rd most common reason for not performing automated testing of magento #MM14NY
  9. 9. testing is important
  10. 10. automated testing allows magento developers to focus on building new features #MM14NY
  11. 11. with automated testing, bugs are found and fixed as part of development. #MM14NY
  12. 12. one interruption to fix a bug that made it past QA costs the best developers 15 minutes of lost productivity #MM14NY
  13. 13. SOURCE: ‘PROGRAMMER INTERRUPTED’ - http://blog.ninlabs.com/2013/01/programmer-interrupted/
  14. 14. a well designed automated testing system reduces interruptions and increases productivity and happiness. #MM14NY
  15. 15. types of tests used to test magento
  16. 16. unit testing logical analytical great for ideal projects
  17. 17. unit testing magento’s flexibility greatly complicates unit testing
  18. 18. unit testing great for extensions github.com/EcomDev/EcomDev_PHPUnit @IvanChepurnyi
  19. 19. it’s not that you can’t test magento - you can’t unit test magento, but there’s more to testing than just unit tests #MM14NY
  20. 20. 100% of developers surveyed that are testing magento are using functional tests #MM14NY
  21. 21. functional testing focuses on the entire site as a system doesn’t care what function is being called tests that the site works the way client wants
  22. 22. with full test coverage, functional tests confirm that your magento site still works as you add new features #MM14NY
  23. 23. one more thing… only 50% of developers surveyed are conducting performance testing ! performance testing is a must
  24. 24. performance testing not just for sysadmins not just for front-end developers bad performance = bad sales
  25. 25. performance testing backend developers don’t always remember that extensions, integrations and new features can slow the site down
  26. 26. when testing your magento projects, include performance testing - poor performance is a bug #MM14NY
  27. 27. slow code should be optimized or removed just like any other bug would be #MM14NY
  28. 28. how to test magento
  29. 29. testing servers bamboo - commercial, general purpose ! jenkins - open source, general purpose ! phpci - open source, PHP-focused
  30. 30. only 1 developer reported using phpci for testing magento; 50% are using bamboo, 50% are using jenkins #MM14NY
  31. 31. jenkins vs bamboo jenkins is free Bamboo $10 starter license ! Try both, pick whichever you like better
  32. 32. which testing server you use isn’t important - the fact that you have a testing server and are testing is. #MM14NY
  33. 33. 100% of developers who said they are performing functional tests are using selenium #MM14NY
  34. 34. selenium test cases selenese vs phpunit test standard magento features test your custom features
  35. 35. gatling test cases scripted in scala normal visitor workflow randomized delays
  36. 36. gatling test cases specify number of simultaneous visitors and overall test length ! for example…
  37. 37. gatling test cases should be set for the maximum simultaneous users you expect and a timeframe of 60-120 seconds #MM14NY
  38. 38. automated tests of magento should be fast - test each commit and receive pass/fail before you move to your next task #MM14NY
  39. 39. gatling test cases develop baseline at start of project ! if tests are 20% slower than baseline, consider the test failed
  40. 40. what’s Next?
  41. 41. testing magento 1 sites can’t - unit testing can - functional testing must - performance testing
  42. 42. testing magento automated testing saves time, money & sanity
  43. 43. let’s keep talking what did I get wrong? TDD vs BDD? BeHat? @joshuaswarren / joshuawarren.com
  44. 44. behat, bdd vs tdd, unit testing in magento 1 - tweet at @Joshuaswarren with what you disagree with #MM14NY
  45. 45. but wait, there’s more! in-depth talk at phpworld november, washington dc world.phparch.com
  46. 46. spread the word educate our peers, managers and clients ! summer 2015 - automated testing is the norm
  47. 47. spread the word Creatuity will be open sourcing our testing tools and test cases ! foundation for fast, flexible, reusable set of functional and performance tests
  48. 48. coming soon to github - @joshuaswarren and @creatuity are releasing a foundation for performance and functional testing of magento #MM14NY
  49. 49. thank you! creatuity.com/mmny ! @joshuaswarren

×