Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Run tests at scale with on-demand Selenium Grid using AWS Fargate

303 vues

Publié le

My talk from DSTC 2019 edition based on a cloud-hosted container solution to run 1000s of tests in an on-demand, scalable Selenium Grid

Publié dans : Technologie
  • Soyez le premier à commenter

Run tests at scale with on-demand Selenium Grid using AWS Fargate

  1. 1. RUN TESTS AT SCALE WITH ON-DEMAND SELENIUM GRID USING AWS FARGATE Presenter: Megha Mehta
  2. 2. Megha Mehta Senior Automation Engineer S&P Global Market Intelligence, Gurgaon https://www.linkedin.com/in/m-mehta/ SPEAKER mmehta.ten@gmail.com @meghamehtatech
  3. 3. PROBLEM DESCRIPTION • Slower feedback • Hardware provisioning challenges • Additional maintenance effort • Challenges with cross-browser testing
  4. 4. • UI tests take long to run • More expensive • Longer waits for feedback without parallelization SLOWER FEEDBACK
  5. 5. HARDWARE PROVISIONING CHALLENGES • Performance of under-provisioned hardware suffers under peak loads • Avoid wastage due to over-provisioning by optimizing capacity utilization Application stops responding Infrastructure slow under load Tests failed
  6. 6. • Frequent software upgrades and patches for system efficiency -Typical maintenance activities – browser upgrades, OS hotfixes, security patches, system restart, disk cleanup etc. • This takes focus away from testing ADDITIONAL MAINTENANCE EFFORT
  7. 7. CHALLENGES WITH CROSS- BROWSER TESTING CHROME 75 FIREFOX 60 FIREFOX 65 On different versions of same browser On different browsers
  8. 8. SOLUTION Auto- provisioned with minimal maintenance requirement s • Auto- scalable in response to varying load • Reliable and cost- effective Run tests in hyper-parallel mode for faster feedback Dynamic and quick to change
  9. 9. OUR SOLUTION A dockerized Selenium Grid running on AWS
  10. 10. TAKEAWAYS • Designing your own high-capacity Selenium Grid using Docker and AWS • A scalable yet disposable grid • Faster feedback – reduce execution time from hours to minutes • Enable Continuous Integration • Major cost savings compared to traditional on-prem infrastructure
  11. 11. ARCHITECTURE DEEP DIVE
  12. 12. |WHY DOCKER CONTAINERS? VMs heavyweight Take up lot of memory resources Startup time in minutes Containers lighweight Require lower memory Startup time in seconds
  13. 13. SELENIUM GRID – HUB SETUP • Hub should have High Bandwidth to support large number of requests from test clients and nodes • High CPU needed for 1000’s of tests but lower memory is fine • In our case, hub is single container of 16GB RAM, 4 vCPU
  14. 14. SELENIUM GRID – NODE SETUP • Containers of 16 GB RAM, 4 vCPU • Each container can run up to 20 browsers without any performance hit
  15. 15. Docker Swarm AWS EKS (Kubernetes) AWS ECS/Fargate Do we need to provision instances in advance? Yes Yes No, automatically done by AWS Is it suitable for short workloads? Yes No, master node always up Yes How cost-effective is the solution? * Charged for provisioned EC2’s Additional charges for cluster management Charged only for duration that containers are running *Considering the container cluster only runs for a few hours HOW TO MANAGE SO MANY CONTAINERS? Deciding factors ↓ Orchestration solutions 
  16. 16. CONTAINER ORCHESTRATION USING FARGATE • No need to provision, configure, or scale clusters of machines to run containers – unlike other orchestration solutions • Focus on designing and building Selenium Grid instead of managing the infrastructure that runs it AWS Fargate
  17. 17. LOAD BALANCER • Selenium Hub is behind an AWS LB which listens to incoming requests from clients and responses from nodes on port 4444 • Domain Name System (DNS) name received by the LB can be quite complex and lengthy - Use a friendly DNS name for load balancer using AWS Route53 AWS Application Load Balancer AWS Route53
  18. 18. • ECR – AWS Elastic Container Registry to store Docker images • Official selenium-node images modified to expose container IP to hub for registration CONTAINER_IP=$(curl -s http://169.254.170.2/v2/metadata | jq -r '.Containers[1].Networks[0].IPv4Addresses[0]') ECR – REPOSITORY FOR DOCKER IMAGES AWS ECR
  19. 19. VPC Application Load Balancer Node containers - Chrome Node containers - Firefox Autoscaling Group Amazon ECR Jenkins Node Jenkins node Jenkins Node AWS Fargate Selenium Hub Selenium Grid
  20. 20. • Monitor container metrics like CPU and memory utilization during tests • Scale out – add 2 node containers whenever CPU Utilization > 80% for 2 min • Scale down to ZERO if idle > 5 mins AUTO-SCALING USING CLOUDWATCH
  21. 21. VPC Application Load Balancer Node containers - Chrome Node containers - Firefox Autoscaling Group AWS Fargate Selenium Hub Selenium Grid Amazon ECR Jenkins Node Jenkins node Jenkins Node CloudWatch logs Cloudwatch Alarm [Triggered on CPU utilization > 80%]
  22. 22. CLOUDFORMATION – BRINGING IT ALL TOGETHER • Describe and provision all the infrastructure resources using CloudFormation template • Version control/replicate/update the templates like application code AWS CloudFormation
  23. 23. *Calculations shown for US East (N. Virginia) Region. Also, costs of other services like ECR, AWS ALB etc. are not included. COST BENEFITS 41 Fargate Containers (1 Hub + 40 Nodes, each of 4vCPU and 16GB configuration) x 10 minutes to complete 800 tests x 5 times a day x 30 days in a month = 1025 hours ~ $238.59 per month* 41 VMs x ~ $80 per VM per month ~ $3280 per month
  24. 24. COST BENEFITS $39360 $2856 Annual costs Fargate containers On-prem VM That’s a saving of ~93%!
  25. 25. SUMMARY • Designing your own high-capacity Selenium Grid using Docker and AWS • A scalable yet disposable grid • Faster feedback – reduce execution time from hours to minutes • Enable Continuous Integration • Major cost savings compared to traditional on-prem infrastructure
  26. 26. • Task networking in Fargate https://aws.amazon.com/blogs/compute/task-networking-in-aws-fargate/ • Comparison between clusters https://www.trinimbus.com/blog/fresh-reinvent-2017-aws-introduces- elastic-container-service-kubernetes-eks-fargate • FARGATE pricing calculator http://fargate-pricing-calculator.site.s3-website-us-east-1.amazonaws.com/ REFERENCES
  27. 27. CLOUDFORMATION TEMPLATE https://github.com/mmehta- 10/selenium-grid-fargate
  28. 28. THANK YOU!

×