Alfresco Day Roma 2015: Full Stack Load Testing
•
0 j'aime•904 vues
Alfresco Day Roma 2015: Full Stack Load Testing on Alfresco, Derek Hulley, Founding Engineer, Alfresco
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Alfresco Day Roma 2015: Full Stack Load Testing
Similaire à Alfresco Day Roma 2015: Full Stack Load Testing (20)
Plus de Alfresco Software
Plus de Alfresco Software (20)
Dernier
Dernier (20)
Alfresco Day Roma 2015: Full Stack Load Testing
- 1. Full Stack Load Testing Load Testing On Alfresco 5.1
- 2. Derek Hulley Alfresco Founding Engineer
- 3. Introduction • Resource Limits • Load Test Accuracy • Load Test Software Introduction
- 5. Limiting Factors • Some Basic Server Limitations – Memory consumption – CPU usage – IOPS / Bandwidth / IO • Other Server Limitations – Monitoring (getting at logs, charts, monitors, etc) – Backups – CRON jobs • Features – Supported file types – Auditing – Jodconverter – Clustering Resources
- 7. Start with a guess and try it out in anger Alfresco and Share 20% Transforma4on s 10% SOLR 30% Database 30% Content 5% Load Balancer 5% Resource Guess 0 Alfresco and Share Transforma2ons SOLR Database Content Load Balancer Resources Monitor all Run for days Resolve Issues Repeat Adjust resources
- 8. Focus e.g. SOLR Resources Alfresco and Share 20% Transforma4on s 10% SOLR 30% Database 30% Content 5% Load Balancer 5% Resource Guess 0 Alfresco and Share Transforma2ons SOLR Database Content Load Balancer
- 9. Resource Focus Example: SOLR • CPU: underused • Memory: underused • IO: underused • Network: Keep and eye on this! Resources
- 10. Reduce spend and iterate … Resources Alfresco and Share 23% Transforma4on s 12% SOLR 18% Database 35% Content 6% Load Balancer 6% Resource Guess 2 Alfresco and Share Transforma2ons SOLR Database Content Load Balancer
- 12. Load Test Risk Factors: Scale • Increases Cost – Your Time – Getting the data in does not normally produce results • Bulk Loading – Different hardware for this phase? – Simulate bulk loading as accurately as necessary • Number of Documents – Easier to account for with experience – Data storage costs – Backup/Restore procedures • Users and Groups – LDAP sync – Complex Group Hierarchy – Site Group Permissions – Mass-authentication LoadTestAccuracy
- 13. Load Test Risk Factors: Documents • Document Types – Image thumbnail generation – Text to PDF for preview – Document library listings (many transformations a.s.a.p) • Document Size – Network and IO costs and limits – CPU used for streaming – PDF to text memory consumption – External converters • More CPU or more infrastructure LoadTestAccuracy
- 14. Load Test Risk Factors: API Usage Profile • Load Profile – Burst load – Trickle load • Data Structure – Paging – Sorting – Permissions • The Internet – CMIS across the Atlantic – Southern hemisphere client apps LoadTestAccuracy
- 15. Load Test Risk Factors: UI Complexity • Share APIs – /private • Upload File – Stream binary – Then all of these at once: • Register user activity • Generate thumbnail • Generate PDF preview and display • Get and display document metadata • Get and display document version history • Replay Doom (You Only Live Once) LoadTestAccuracy
- 16. Load Test Risk Factors: UI Complexity LoadTestAccuracy
- 17. Load Test Risk Factors: UI Complexity • Page-Object Pattern: Browsers do what they do Login Event Processor Login Page Dashboard Page Browser processEvent loginAs Send keystrokes Login button click DashboardPage Timed render Wait For: Activities Wait For: My Sites Wait For: My Sites Wait For: My Sites Wait For: My Docs Wait For: etc Increasing Complexity Server x100 LoadTestAccuracy
- 18. Load Test Software Alfresco Benchmark Server and Tests
- 19. References • Martin Bergljung’s walkthrough • Alfresco Wiki: Benchmark_Testing_with_Alfresco • GitHub: AlfrescoBenchmark TestSoftware
- 20. Alfresco Benchmark Suite TestSoftware Benchmark Server Tomcat 7 Rest API MongoDB Config Data Services MongoDB Test Data UI Benchmark Driver (xN) Benchmark Driver (xN) Benchmark Driver Tomcat 7 Extras (Selenium) Servers / APIs Servers / APIs Load Balancer Servers / APIs Test Services Rest API
- 21. Load Tests: /tests/ent-signup • Create users • User base reused – Prerequisite for other tests TestSoftware
- 22. Load Tests: /tests/dataload • Reuse user base • Create – Sites – Site Members • Site data – Folders – Files uploaded using CMIS or TestSoftware
- 23. Load Tests: /tests/dataload • AlfrescoBenchmark/alfresco-text-gen • Text data generated in server • Option to push data into ContentStore TestSoftware
- 24. Load Tests: GitHub benchmark-cmis • Burst load • Easy to extend TestSoftware
- 25. Load Tests: FTP sample files vs spoofing • FTP server – A few thousand samples – Use for Share tests • Transformations • Text extractions • Renditions • Spoofing – Stress SOLR – document library size • Search: searchterms.txt TestSoftware
- 26. Share Load Testing: /tests/share • Concurrent Users: – A new user logs in every N seconds – The user logs out after spending Y seconds – Number of concurrent users C = Y/N • Selenium Driver Overload – Use Windows – 5s to start a browser on EC2 m3.2xlarge – Up to 40 browsers per load driver on EC2 m3.2xlarge – Drivers required D = max( 5s/N, C/40) – m3.2xlarge Windows à spot price of $0.3/hour – 500 concurrent user à20 drivers à $6/hour TestSoftware
- 27. Summary
- 28. Summary • Build up load testing software and skills • Design tests around existing tools – Understand risks and where to extend or introduce new tests • Iterate Summary
- 29. Questions