Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (11)
Similaire à High-flying Cloud Testing Techniques
Similaire à High-flying Cloud Testing Techniques (20)
Plus de TechWell
Plus de TechWell (20)
Dernier
Dernier (20)
High-flying Cloud Testing Techniques
- 1. TM PM Tutorial 4/30/13 1:00PM High-flying Cloud Testing Techniques Presented by: Ruud Teunissen Polteq Test Services BV Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Ruud Teunissen An international test consultant at Polteq Test Services BV, Ruud Teunissen has performed several test functions in a number of IT projects: tester, test specialist, test consultant, and test manager. Ruud participated in the development of the structured testing methodology TMap®—Test Management Approach. Together with Martin Pol and Erik van Veenendaal, Ruud is coauthor of several books on structured testing, including Software Testing: A Guide to the TMap® Approach.
- 3. High Flying Cloud Testing Techniques Ruud Teunissen Polteq Test Services BV The Netherlands 1
- 4. searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, denying, talking, designing, making, verifying, measuring 3 Develop and Test Operate and Manage Email Surf Transfer Store 4 2
- 5. Develop and Test Email Surf Transfer redundancy, proliferation limitations 80% unused storage claim environmentally unfriendly Operate and Manage Store 5 Develop and Test Email Surf Transfer SOA internet technology virtualization standard software Operate and Manage bandwidth Store 6 3
- 6. searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, Develop and transmitting, learning, controlling, Email analyzing, Test Surf purchasing, testing, alarming, changing, updating, Transfer deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, Operate and Manage accelerating, enhancing, creating, growing,Store checking in, checking out, finding out, reaching, denying, talking, designing, making, verifying, measuring 7 4
- 7. Essential characteristics On-demand service Self service provisioning, pay-per-use No human interaction US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Standard mechanisms over networks “Any” client US: National Institute of Standards and Technology http://www.nist.gov 5
- 8. Essential characteristics On-demand service Broad network access Resource pooling Multi-tenant Storage, processing, memory, virtual machines, … Location independent US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Rapid scale in and out “Any quantity” at any time US: National Institute of Standards and Technology http://www.nist.gov 6
- 9. Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service Controlled resource use Transparency, pay-per-use US: National Institute of Standards and Technology http://www.nist.gov Essential characteristics On-demand service Broad network access Resource pooling Rapid elasticity Measured service Deployment models – private cloud – community cloud – public cloud – hybrid cloud Service Models Software as a Service Platform as a Service Infrastructure as a Service US: National Institute of Standards and Technology http://www.nist.gov 7
- 10. Question What is being moved into the cloud? 15 Pack List 1 2 3 4 5 16 8
- 11. What is “done” in the cloud? <500, SME >500 Consumer Public Private Hybride Community Public *aaS IaaS, PaaS, DaaS, SaaS SaaS Taas *aaS Mail Storage Infrastructure CRM Finance Business processes Data Centre Data Management Business processes 17 Surf and mail Apps Social media Dropbox Google services Spotify Picasa Games …………… Standards Cyber crime Continuity Privacy Legislation Multi platform Impact organisation 143 9
- 12. Performance Security Availability & Continuity Functionality Risks Manageability 19 Legislation & Regulations Suppliers & Outsourcing Performance Security YOUR Operational Profile Availability & Continuity Functionality Other customers Risks Manageability 20 Legislation & Regulations Suppliers & Outsourcing YOUR Operational Profile PLUS 10
- 13. Performance Security The idea: “it’s safe” Availability & Continuity Functionality Everything over the web Risks Manageability 21 Legislation & Regulations Home gound for hackers Suppliers & Outsourcing Performance Security No free choice of device. Availability & Continuity Functionality Bring Your Own Device Risks Manageability 22 Legislation & Regulations Endless possibilities. Suppliers & Outsourcing 11
- 14. Performance Taken care of. Security Availability & Continuity Functionality Backup and recovery Risks Manageability 23 Legislation & Regulations Who will support me? Suppliers & Outsourcing Performance Security Planned and controlled Availability & Continuity Functionality Updates, patches, fixes, Risks Manageability 24 Legislation & Regulations Do I have a choice? Suppliers & Outsourcing 12
- 15. Performance In house. Security Availability & Continuity Functionality Where is my data? And is that OK? Risks Manageability 25 Legislation & Regulations Somewhere Suppliers & Outsourcing Performance Security Availability & Continuity Functionality Risks Manageability 26 Legislation & Regulations Suppliers & Outsourcing 13
- 16. Question – Group Session Let’s look at the clustered risks What test measures would you consider applying? 27 Test Measures 1 2 3 4 5 28 14
- 17. Testing? Interview Check Trial Proof of concept Intake 29 Performance Testing Security Testing Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Interview Proof ofProef concept Testen Intake Functional Testing Migration Testing Testing caused by Legislation & Regulations 30 Testing in Production 15
- 18. Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 31 Legislation & Regulations Suppliers & Outsourcing Testing in Production Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Testing caused by Legislation & Regulations Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 32 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production 16
- 19. Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 33 Legislation & Regulations Suppliers & Outsourcing Testing in Production Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Architecture Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Testing caused by Legislation & Regulations Availability & Continuity Testing Risks From “individual” risks Functional Testing to Manageability “individual” test measures Migration Testing 34 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production 17
- 20. Performance Testing Selection Security Testing Manageability Testing Implementation Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Production Migration Testing Testing caused by Legislation & Regulations Testing in Production 35 Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 36 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production 18
- 21. Performance Testing Security Testing Manageability Testing Selection Criteria Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Testing in Production 37 Performance Testing Security Testing Manageability Testing Proof of Concept Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations 38 Testing in Production 19
- 22. Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 39 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production Performance Testing Security Testing Known measures tuned and tweaked Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing New measures developed Migration Testing Testing caused by Legislation & Regulations 40 Testing in Production 20
- 23. Question – Group Session Suppose your CRM appliction is moved into the Cloud. What would you test? 41 Test Measures When CRM Moves into the Cloud 1 2 3 4 5 42 21
- 24. YOUR Operational Profile Performance Testing Security Testing Manageability Testing Load Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing YOUR Operational Profile PLUS ACTUAL MOMENT 43 Testing caused by Legislation & Regulations Testing in Production Yes, you can! Performance Testing Security Testing Manageability Testing Stress Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Definitly NOT! 44 Testing caused by Legislation & Regulations Testing in Production 22
- 25. Load and stress. Performance Testing Security Testing Manageability Testing Elasticity Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Load and elasticity. 45 Testing in Production Question How would you test “Elasticity”? 46 23
- 26. Test Elasticity 47 load test – ‘down’ load test – ‘up’ 101 100 boundary values 99 load path test extend? ‘up’ tc 1: use=99, pay 100 tc 2: use=100, pay 100 tc 3: use=101, pay 200 no yes 200 charged boundary values 100 charged ‘down’ tc1: use=101, pay 200 tc2: use=100, pay 100 tc3: use=99, pay 100 48 24
- 27. load test – ‘down’ load test – ‘up’ 101 100 boundary values • (Automatic) scaling up or down 99 does not perform as required load • At scaling moments functional problems test boundary values path emerge extend? yes no • Insight in use not sufficient 200 charged 100 charged ‘up’ tc 1: use=99, pay based costspay100 is 100 tc 2: use=100, tc 3: use=101, pay 200 ‘down’ tc1: use=101, pay 200 tc2: use=100, pay 100 tc3: use=99, pay 100 49 • Security at: – Network – Supplier – User Performance Testing Security Testing Manageability Testing • Encryption • Authentication and autorisation • Test logs and audit trails • Security Audits 50 Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Testing in Production 25
- 28. • Completeness and correctness of specifications – Supplier – User • Availibilty of test environments • Management of: – Defects – Changes • Maintainability of the software 51 Performance Testing Security Testing Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Testing in Production Manageablity of test environments • Everything in the cloud 52 26
- 29. Manageablity of test environments • Link all current environments to the service 53 Manageablity of test environments • Link Production to the real service • Link other environmnets to a MOCK SERVICE 54 27
- 30. Question Who will solve the defects? And how? 55 Defect resolution 56 28
- 31. Defect Management 57 Performance Testing • Role of system architecture Security Testing • Monitoring and Logging Manageability Testing • Guarantees and SLA’s Availability & Continuity Testing • Test fail-over mechanism Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Functional Testing • Test online/offline Migration Testing Testing caused by Legislation & Regulations 58 Testing in Production 29
- 32. Performance Testing Security Testing Manageability Testing Online – Offline Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Use case testing. Global testing. 59 Testing caused by Legislation & Regulations Testing in Production Fail-over testing 60 30
- 33. Performance Testing Security Testing Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations 61 Testing in Production Question So what’s new here? 62 31
- 34. Functional Testing 63 Multiplatform testing. Performance Testing Security Testing Manageability Testing Any device – any platform Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Multiplatform testing. 64 Testing caused by Legislation & Regulations Testing in Production 32
- 35. Internet Explorer 6 Windows XP Internet Explorer 7 Windows Vista Internet Explorer 8 Windows 7 Firefox 3.5 Windows 2003 server Firefox 3.6 Browsers Windows 8 Firefox 4 Safari 4 Windows CE Safari 5 Linux Operating Systems Chrome11 Unix Opera11 Multiplatform Mac OS Lion PC Mac OS Snowleopard SUN Computer iOS Macintosh Android iPhone .. Windows Mobile Samsung Devices NOKIA Mobile Xxx ASUS.. Blackberry Tablet MOTOROLA Xxx 65 Multiplatform testing. Performance Testing Security Testing Manageability Testing Any device – any platform Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Multiplatform testing. 66 Testing caused by Legislation & Regulations Testing in Production 33
- 36. Performance Testing Security Testing Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection • Interruption of business processes is minimal • All migrated data can be tracked (audit trail) • All data is converted correctly • All pending transactions are successfully finished after the migration • Defects in data before migration do not lead to problems during migration • Defects in data are solved and not migrated defective • No more data than necessary is migrated to the service Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Testing in Production 67 Incidental testing. Performance Testing Security Testing Legislation + Regulations = Test basis Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functional Testing Migration Testing Testing caused by Legislation & Regulations Compliancy testing. 68 Testing in Production 34
- 37. Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Functional Testing Manageability Risks Migration Testing 69 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production Performance Testing Continuous End-to-End Test Security Testing Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Testing during Selection Availability & Continuity Testing Functionals and non-functionals Functional Testing Migration Testing Testing caused by Legislation & Regulations 70 Testing in Production 35
- 38. Standards Cyber crime Check Continuity Interview Privacy Legislation Proof of concept Trial Multi platform Intake Impact organisation Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability Test starts earlier & Continuity Testing Test scope is widened Test will never Functional Testing stop Manageability Functionality Risks Migration Testing 72 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production 36
- 39. Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Availability & Continuity Testing Questions? Functional Testing Manageability Risks Migration Testing 73 Legislation & Regulations Suppliers & Outsourcing Testing in Production Testing during Selection Performance Testing Security Security Testing Availability & Continuity Manageability Testing Functionality Te s t M e a s u r e s Te s t M e a s u r e s Performance Testing caused by Legislation & Regulations Availability & Continuity Testing Thank you! Functional Testing Manageability Risks Migration Testing 74 Legislation & Regulations Suppliers & Outsourcing Testing caused by Legislation & Regulations Testing in Production 37
- 40. About the speaker Ruud Teunissen Polteq Test Services, The Netherlands ruud.teunissen@polteq.com - http://www.polteq.com In the testing world since 1989, Ruud Teunissen has held numerous test functions in different organizations and projects: tester, test specialist, test consultant, test manager, etcetera. Ruud is co-author of several books on software testing and is a frequent speaker at (inter)national conferences and workshops. He was a member of the program committee for Quality Week Europe and EuroSTAR. Ruud is currently Senior Test Consultant at Polteq Test Services BV and responsible for the quality of Polteq services and assignments. 75 38