As the need for more computing resources has accelerated, so too have the ways in which computing have evolved. The advent of the cloud has allowed us to easily scale to suit our needs, but if we want to keep pace, we need an even more automated way to scale our infrastructure. In this session, we look at automatic scaling using Kubernetes, including how to set it up and, most important, what you should monitor in order to drive your scaling. This session is brought to you by AWS partner, Datadog.

1. Automatically Scaling Your
Kubernetes Workloads

2. Automatically Scaling Your
Kubernetes Workloads
Sarah Schaab
Solutions Engineer
Datadog

3. Solutions Engineer
Former barber
Gleams the cube

4. @datadoghq
SaaS-based monitoring
250+ integrations
Trillions of data points per day
https://datadoghq.com

5. “I think there is a world market for maybe
five computers.”
– Thomas Watson, IBM

6. “I think there is a world market for maybe
five computers.”
– Thomas Watson, IBM, Wildly
misquoted

21. Why the HPA Is a Big Deal

22. Why the HPA Is a Big Deal

23. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g., CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscale based on ANY metric, including
external

24. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g. CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscale based on ANY metric, including
external

25. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g. CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscaling based on ANY metric,
including external

26. Choosing the
Autoscaling Metric

28. Recurse until you find the cause

29. Recurse until you find the cause

34. Implementing Autoscaling with a Custom
Metric

35. Implementing Autoscaling with a Custom
Metric

37. Prerequisites
❏ Kubernetes running v1.10 or higher
❏ Enable the aggregation layer: https://kubernetes.io/docs/tasks/access-
kubernetes-api/configure-aggregation-layer/
❏ Datadog account (free trials available)
❏ K8S nodes running Datadog Agent (ideally as a DaemonSet), with
Autodiscovery enabled
❏ Agents are configured to securely communicate with Cluster Agent (not
mandatory, but has more enriched data).
https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/cluster-
agent/README.md#security-premise

38. Spin up the Datadog Cluster Agent (RBAC)
kubectl apply -f manifests/cluster-agent/rbac/rbac-cluster-
agent.yaml
clusterrole.rbac.authorization.k8s.io "dca" created
clusterrolebinding.rbac.authorization.k8s.io "dca" created
serviceaccount "dca" created

39. Spin up the Datadog Cluster Agent (RBAC) Output
kubectl apply -f manifests/cluster-agent/rbac/rbac-cluster-
agent.yaml
clusterrole.rbac.authorization.k8s.io "dca" created
clusterrolebinding.rbac.authorization.k8s.io "dca" created
serviceaccount "dca" created

40. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

42. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

43. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

44. Services in Kubernetes
Services == Load Balancer
Service == VIP

45. Verifying Cluster Agent
kubectl get pods, svc -l app=datadog-cluster-agent
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 28m
SVCS:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
default datadog-custom-metrics-server ClusterIP 192.168.254.87 <none>
443/TCP 28m
default datadog-cluster-agent ClusterIP 192.168.254.197 <none>
5005/TCP 28m

46. Verifying Cluster Agent Output
kubectl get pods, svc -l app=datadog-cluster-agent
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 28m
SVCS:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
default datadog-custom-metrics-server ClusterIP 192.168.254.87 <none>
443/TCP 28m
default datadog-cluster-agent ClusterIP 192.168.254.197 <none>
5005/TCP 28m

47. Register the External Metrics Provider
kubectl apply -f manifests/cluster-agent/hpa-example/rbac-hpa.yaml
clusterrolebinding.rbac.authorization.k8s.io "system:auth-delegator" created
rolebinding.rbac.authorization.k8s.io "dca" created
apiservice.apiregistration.k8s.io "v1beta1.external.metrics.k8s.io" created
clusterrole.rbac.authorization.k8s.io "external-metrics-reader" created
clusterrolebinding.rbac.authorization.k8s.io "external-metrics-reader" created

48. Register the External Metrics Provider Output
kubectl apply -f manifests/cluster-agent/hpa-example/rbac-hpa.yaml
clusterrolebinding.rbac.authorization.k8s.io "system:auth-delegator" created
rolebinding.rbac.authorization.k8s.io "dca" created
apiservice.apiregistration.k8s.io "v1beta1.external.metrics.k8s.io" created
clusterrole.rbac.authorization.k8s.io "external-metrics-reader" created
clusterrolebinding.rbac.authorization.k8s.io "external-metrics-reader" created

49. Datadog Cluster Agent Running
kubectl get pods,svc
PODS
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-agent-4c5pp 1/1 Running 0 14m
default datadog-agent-ww2da 1/1 Running 0 14m
default datadog-agent-2qqd3 1/1 Running 0 14m
[...]
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 16m

50. Example HPA Manifest
● Is configured to scale a deployment called 'nginx'
● Set a maximum number of replicas to create to 3, with a minimum of 1
● HPA triggers autoscaling off of the Datadog metric
'nginx.net.request_per_sec' over the scope
'kube_container_name: nginx'

51. Example HPA Manifest
● Is configured to scale a deployment called 'nginx'
● Set a maximum number of replicas to create to 3, with a minimum of 1
● HPA triggers autoscaling off of the Datadog metric
'nginx.net.request_per_sec' over the scope
'kube_container_name: nginx'

54. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

56. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

57. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

58. POD:
default nginx-6757dd8769-5xzp2 1/1 Running 0 3m
SVC:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default nginx ClusterIP 192.168.251.36 none 8090/TCP 3m
HPAS:
NAMESPACE NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS
AGE
default nginxext Deployment/nginx 0/9 (avg) 1 3 1
3m

59. curl <NGINX_SVC>:8090/nginx_status
while true; do curl <NGINX_SVC>:8090/nginx_status; sleep 0.1;
done
Let’s Stress it Out

60. +

61. +

62. kubectl get pods,svc
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 9m
default nginx-6757dd8769-5xzp2 1/1 Running 0 2m
default nginx-6757dd8769-k6h6x 1/1 Running 0 2m
default nginx-6757dd8769-vzd5b 1/1 Running 0 29m
HPAS:
NAMESPACE NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
default nginxext Deployment/nginx 30/9 (avg) 1 3 3 29m

65. function yamllint() {
for i in $(find . -name '*.yml' -o -name '*.yaml');
do echo $i; ruby -e "require
'yaml';YAML.load_file("$i")"; done
}

66. In Conclusion
● Walked through a brief history of scaling and deploying
software
● Described some of the key components and history of the
HPA
● How to choose a metric on which to base autoscaling
● Think of autoscaling as an output for your monitoring
● Walkthrough implementing autoscaling with a custom
provider
● How a simple linting script can save you a lot of time,
effort, and embarrassment

67. In Conclusion
● Walked through a brief history of scaling and deploying
software
● Described some of the key components and history of the
HPA
● How to choose a metric on which to base autoscaling
● Think of autoscaling as an output for your monitoring
● Walkthrough implementing autoscaling with a custom
provider
● How a simple linting script can save you a lot of time,
effort, and embarrassment

68. Datadog Cluster Agent: https://github.com/DataDog/datadog-agent/tree/master/Dockerfiles/manifests/cluster-agent (includes an
example)
HPA blog: https://www.datadoghq.com/blog/autoscale-kubernetes-datadog/
DCA blog: https://www.datadoghq.com/blog/datadog-cluster-agent/
HPA Docs: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
Enable the Kubernetes Aggregation Layer: https://kubernetes.io/docs/tasks/access-kubernetes-api/configure-aggregation-layer/
Configure Agents to communicate securely with Cluster Agent: https://github.com/DataDog/datadog-
agent/blob/master/Dockerfiles/cluster-agent/README.md#security-premise
Datadog Cluster Agent Deployment Manifest: https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/manifests/cluster-
agent/cluster-agent.yaml
Datadog Example HPA: https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
Resources

69. Automatically Scaling Your
Kubernetes Workloads

70. Visit Our Booth