How we Auto Scale applications based on CPU with Kubernetes at M6Web?

1. Kubernetes Auto scaling On production at M6Web

2. Photo by frank mckenna on Unsplash

3. Kubernetes Container orchestrator Maximize capacity Adapt to demand Keep applications healthy

4. Kubernetes ● Handles Network ● Container Autoscaling ● Containers High Availability ● Given available resources: CPU, RAM, etc. ● YAML config

5. Kubernetes Objects 1. Pod 2. Node 3. Cluster 4. Horizontal Pod Autoscaler 5. Service

13. Example with our geolocation API

14. Pod 2 containers: ● Nginx ○ Rewrite rules ○ HTTP headers sanitation ○ HTTP enhancements (gzip, etc.) ● PHP-FPM ○ Executes PHP code

15. Deployment This Pod is executed inside a cluster

16. Production Cluster ● Master nodes ● Worker nodes

17. Pod in production ● 2 replicas of pod ● scheduled by Kubernetes on different worker nodes

18. Horizontal Pod Autoscaler ● Controls pod replicas ● Dynamically changed according to CPU usage

19. Service ● Unchanged over time ● No matter how many pods exist ● Round Robin over pods

20. It works !!

21. Auto scales depending on the load

22. THIS IS FUN !! Yeah really !!

23. Photo by Lightscape on Unsplash

24. It’s only YAML

25. Configure HPA inside your project

26. To scale on 80% of Requests

27. Requests Containers can request ressources ● CPU ● RAM Those ressources are guaranteed Helps Kubernetes to optimize the scheduling of pods Photo by Kaizen Nguyễn on Unsplash

28. Requests are the highest resources an app will normally consume They must not be considered as minimum or maximum usable resources Requests are not Limits

29. Autoscaling 1. Horizontal Pod Autoscaler 2. Requests

30. Scalability based on CPU

31. How it works ?

32. Horizontal Pod Autoscaler compares mean utilization of pods to Requests values

33. Example: geolocation at 16h30 ● Pod Requests: 300m (millicores) ● Horizontal Pod Autoscaler target: 80% ● HPA scales from 240m

34. Example: geolocation at 16h30 ● Pod Requests: 300m (millicores) ● Horizontal Pod Autoscaler target: 80% ● HPA scales from 240m CPU: 215m CPU: 205m Mean CPU: 210m 70% of Requests No need to scale

35. Example: geolocation at 18h30 ● Pod Requests: 300m (millicores) ● Horizontal Pod Autoscaler target: 80% ● HPA scales from 240m CPU: 282m CPU: 264m Mean CPU: 273m 91% of Requests need to scale up

36. Photo by Roman Schurte on Unsplash

37. Of how much it scales ?

38. desiredReplicas = ceil[ currentReplicas * ( currentMetricValue / desiredMetricValue )]

39. Example A: 18h30 2 * ( 0.708 / 0.480 ) = 2.95 +1 pod

40. Example B: 20h47 10 * ( 3.361 / 2.4 ) = 14 +4 pods

41. Photo by Willian Justen de Vasconcellos on Unsplash

42. How can a developer define requests?

43. Kubernetes metrics With Prometheus ● Pod usage ● Use by container ● Observation over time ● Define pod limits ● Load test

44. Example of a dashboard that shows CPU/RAM utilization of a pod

45. It will evolve over time

46. Metrics are per container? Yes, and so are Requests I lied earlier. Metrics are not by pod.

47. Containers Requests inside Pods

48. Containers Requests inside Pods

49. By container metrics ● Horizontal Pod Autoscaler compares containers metrics overall pods ● It does the same calculation as above, but for each different container ● We can therefore scale up because of Nginx or PHP independently ● It always takes the highest value to define the number of replicas

50. Photo by rawpixel on Unsplash

51. cluster- autoscaler Node scaling: + Unschedulable pods - Underutilized nodes

52. autoscaling based on Metrics Custom metrics: ● Redis queue ● PHP-FPM listen queue ● etc.

53. Thanks Photo by Benjamin Voros on Unsplash

54. Photo by Kalle Kortelainen on Unsplash questions and answers available in comments

How we Auto Scale applications based on CPU with Kubernetes at M6Web?

Vincent Gallissot

How we Auto Scale applications based on CPU with Kubernetes at M6Web?