Contenu connexe

Plus de GoQA(20)


АНТОН СЕРПУТЬКО « Якісний перфоманс сценарій в Jmeter, це як?» Online QADay 2023

  1. Jmeter Scripting Essentials Common best practices & approaches to keep your jmeter script in shape
  2. Anton Serputko Staff Performance Engineer at ServiceNow Performance testing: - community: - events: - trainings, outsourcing, consulting: Linkedin:
  3. Performance testing and Jmeter in 40 mins?
  4. No, it’s tips for you to start using in your work with Jmeter
  5. New to Jmeter? Join Jmeter Bootcamp
  6. Source of knowledge 1. Jmeter documentation 2. Blazemeter knowledgebase 3. Stackoverflow 4. Past experience 5. Performance community
  7. What is good Jmeter scenario 1. realistic production-like user flow and load profiles 2. correlation and parameterization 3. handling static data 4. maintainability 5. performance 6. test metrics results gathering
  8. Test results gathering
  9. Instead of
  10. Get this
  11. Test results in influxdb and grafana 1. Get real-time data 2. Extensive monitoring and analysis 3. Test results comparison 4. Configure backend listener props to get valid results! 5. Contact me to get dashboards
  12. Backend Listener configuration 1. application: ${__TestPlanName}@${__time(yyyy-MM-dd'T'hh:mm:ss,)} 2. test title: ${__P(CUSTOM_INFO,default value)} in application and 3. eventTags: users=1,rampup=,... 4. percentiles: 90;95;99;50 5. summaryOnly: false
  13. Get valid results quick fix without going too deep in details edit file 1. backend_metrics_window_mode=fixed -> timed 2. increase window sizes to 100000
  14. Scenario structure 1. keep scripting style a. variable naming convention b. elements location c. elements naming convention 2. be careful while recording 3. keep scenario logical using Logic controllers 4. control load profile(% of actions execution and think times) 5. debug and validate 6. re-use elements if needed
  15. Elements naming 1. name all elements(pre/post processors) 2. do not parameterize name of sampler with ${} 3. replace parameterized value with static value in name 4. add method/details to recorded request url if they have similar naming 5. name requests for API testing like GET /v1/auth/login
  16. Elements scope Be sure element is located where you expect
  17. Scenario recording
  18. Static data Be default: 1. exclude “All suggested excludes” in Http test script recorder 2. remove all hardcoded static data from the scenario 3. enable retrieve all embedded resources and parallel downloads a. use cache manager if retrieving static data b. use 6 parallel downloads to c. disable when you understand possible affect
  19. 3-d party resources 1. remove from scenario 2. when 3-d party could be left a. not sure about their performance and it’s not async b. use sandboxes or contact service providers 3. mock them if 3d party response is required for the flow 3d party examples: Google analytics, Google AdSense, Paypal, UPS shipping, Browser internal tracking
  20. Using recorder 1. filtering a. better to leave extra then leave less b. ask devs/check api docs if not sure 2. requests grouping with transaction controllers 3. remove toggled redirect requests 4. always think if what you see is logical: request duplication, extra requests, etc 5. keep recording action as realistic as possible, pay attention to actions which triggers ajax requests
  21. Auto replace while recording a. auto replace variables while recording if User defined variables is in scope of recorder i. use with caution as more data could be replaced then you expect b. auto replace with http request defaults to change to blank
  22. Before scripting Before scripting: 1. analyze logs, analytics, APM statistics from prod 2. define load profile a. think times between actions b. requests, actions percent distribution c. scenario logic: loops, ifs, etc
  23. While scripting 1. keep scenario logic realistic with logic controllers a. use built-in jmeter components if possible b. avoid overcomplicating scenario c. avoid using custom JSR223 scripting when possible 2. add think time to the scenario: a. Timers in Flow control action between transaction controllers 3. control % of action execution a. Throughput controller b. Weighted switch controller c. ALWAYS validate percentage distribution when using throughput/weighted switch controllers on dummy samples d. Do not use multiple thread groups 4. cover client logic. Extra requests for logged in user, etc 5. use assertions only in key places to reduce overhead
  24. After scripting 1. validate your scenario 2. compare you test result profile with production profile, and adjust if needed
  25. Non-linear/Custom logic 1. If/While controller: a. use groovy functions for best performance b. do NOT use JS in any case c. do not forget about .toInteger() for numbers d. check positive and negative flow 2. Parallel controller a. new connection and internal thread is created in for each request 3. Weighted switch controller a. save values by switching focus to another field
  26. JSR223 Elements 1. Use groovy as a must! a. a lot of solutions on stackoverflow using beanshell beanshell is compatible with groovy b. avoid using JS in any case! 2. Use .setIgnore() for JSR223 sampler 3. Use“{}”,var), out.println to debug groovy scripts 4. Do not use ${} in groovy, use vars.get() 5. Do not use ${VAR}, ${__JmeterFunction()} as they will be cached 6. do not forget about .toInteger() in comparison 7. choose “cache compiled script if available” option
  27. Scenario parameterization 1. what if scenario is not parameterized a. will not work on different env b. data is hardcoded so hitting cache c. auth will expire after some time 2. types of parameterization a. static values parameterization- variables b. requests correlation- post processors c. user input parameterization- jmeter functions 3. prefer dynamic data over reading from csv files
  28. Variables 1. naming conventions a. a, var, ProductID, productid - bad b. PRODUCT_ID, productId - good 2. use User Defined Variables for variables creation a. use Jmeter properties in variable value b. variable could be created in Test Plan, but don’t do that 3. use jmeter properties to pass parameters from command line 4. CSV file to get data a. use csv only for static data like user credentials b. avoid using dynamic data which frequently changes, use post processors to get data from server responses c. use relative path using get base directory + separator groovy function d. use just file name if csv is located near jmx
  29. Post processors/extractors 1. Correlating transactions a. usually requests are dependent from each other, data sent by the client was previously received from server b. get data from responses using regex, jsonpath, css selector, boundary and xpath c. usually no need to parameterize 100% of parameters i. for example no need to parameterize amount of products on add to cart request 2. use multiple variables using regular expression groups from single extractor 3. use regex, jsonpath tester in view results tree to debug queries 4. set NOT_FOUND default value in extractors 5. do not place regex extractor on transaction controller level as it applies to multiple requests
  30. Jmeter functions ● install Jmeter custom plugins ● use function helper ● re-use jmeter function result as a variable ● if function is used in user defined variables it will be initialized only 1 time and then not changed when you call the variable
  31. Debugging 1. add 1 debug post-processor on test plan level a. avoid adding many debug samplers/post-processors 2. validate jmeter errors in jmeter.log 3. common issues with scope: wrong placement, not in element 4. if you put function call in user defined variables it will be initialized inly 1 time and then not changed when you call the variable 5. validate positive and negative cases for if controllers 6. use“{}”,var), out.println to debug groovy scripts 7. debug issues/validate logic with dummy sampler 8. common issues working with variables: a. misspelled ${} with $(), $[], {} without $, etc. b. misspelled variable name on create or use c. [ ] space before/after VARIABLE_NAME
  32. Re-using elements 1. module controllers. re-use ONLY really frequently used actions 2. do not move everything to module controllers thoughtlessly a. harder to navigate through scenario 3. if re-usable element is moved the link in module controller will be broken
  33. Non gui mode ● startup command and parameters ● generate html report ● write results to csv ● set jmeter properties to parameterize scenario ● write to log ● console status logger ● validate jmeter errors in jmeter.log ● run tests in non-gui mode ● parameterize non-gui run with jmeter properties
  34. Jmeter plugins ● plugins in plugin manager might be outdated ● webdriver support plugin was not supporting headless but github master had this changes ● check for bugs and read documentation(sometimes several times) ● weighted switch ● parallel controller
  35. Undo/Redo
  36. Modify undo.history.size=0 in
  37. Use backups
  38. Thank You Let’s keep in touch: Join our performance testing: - community: - events: - trainings, outsourcing, consulting: