As our need for more computing resources has accelerated, so too have the ways in which computing has evolved. The cloud has enabled us to easily scale to suit our needs. To keep pace, we need more automated way to scale our infrastructure. In this session, we discuss automatic scaling with Kubernetes, how to set it up, and—most importantly—what to monitor in order to drive your automatic scaling. This session is brought to you by AWS partner, Datadog.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Automatically Scaling Your
Kubernetes Workloads
H. “Waldo” Grunenwald
Tech Evangelist
Datadog
S V C 2 0 9 - S

2. Automatically Scaling Your
Kubernetes Workloads

3. Automatically Scaling Your
Kubernetes Workloads
H. “Waldo” Grunenwald
Technical Evangelist
Datadog

4. @gwaldo
Tech Evangelist & Writer
“Docs & Talks”
Recovering Ops Engineer

5. @datadoghq
SaaS-based monitoring
250+ integrations
Trillions of data points per day
https://datadoghq.com

6. “I think there is a world market for maybe
five computers.”
– Thomas Watson, IBM

7. “I think there is a world market for maybe
five computers.”
– Thomas Watson, IBM, Wildly misquoted

22. Why the HPA is a Big Deal

23. Why the HPA is a Big Deal

24. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g. CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscale based on ANY metric, including
external

25. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g. CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscale based on ANY metric, including
external

26. History of Kubernetes HPA
● K8S v1.2: autoscaling based off basic metrics within
cluster (e.g. CPU util.)
● K8S v1.6: autoscaling on custom metrics within
cluster
● K8S v1.10: autoscale based on ANY metric, including
external

27. Choosing the
Autoscaling Metric

29. Recurse until you find the cause

30. Recurse until you find the cause

35. Implementing Autoscaling with a Custom
Metric

36. Implementing Autoscaling with a Custom
Metric

38. Prerequisites
❏ Kubernetes running v1.10 or higher
❏ Enable the aggregation layer: https://kubernetes.io/docs/tasks/access-
kubernetes-api/configure-aggregation-layer/
❏ Datadog account (free trials available)
❏ K8S nodes running Datadog Agent (ideally as a DaemonSet), with
Autodiscovery enabled
❏ Agents are configured to securely communicate with Cluster Agent (not
mandatory, but has more enriched data).
https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/cluster-
agent/README.md#security-premise

39. Spin up the Datadog Cluster Agent (RBAC)
kubectl apply -f manifests/cluster-agent/rbac/rbac-cluster-
agent.yaml
clusterrole.rbac.authorization.k8s.io "dca" created
clusterrolebinding.rbac.authorization.k8s.io "dca" created
serviceaccount "dca" created

40. Spin up the Datadog Cluster Agent (RBAC) output
kubectl apply -f manifests/cluster-agent/rbac/rbac-cluster-
agent.yaml
clusterrole.rbac.authorization.k8s.io "dca" created
clusterrolebinding.rbac.authorization.k8s.io "dca" created
serviceaccount "dca" created

41. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

43. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

44. Create Datadog Cluster Agent & Services
In the Cluster Agent’s deployment manifest (cluster-agent.yaml)
add Datadog <API_KEY> and <APP_KEY>,
and set DD_EXTERNAL_METRICS_PROVIDER_ENABLED to true.
kubectl apply -f manifests/cluster-agent/cluster-agent.yaml
kubectl apply -f manifests/cluster-agent/datadog-cluster-
agent_service.yaml
kubectl apply -f manifests/cluster-agent/hpa-example/cluster-
agent-hpa-svc.yaml

45. Services in Kubernetes
Services == Load Balancer
Service == VIP

46. Verifying Cluster Agent
kubectl get pods, svc -l app=datadog-cluster-agent
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 28m
SVCS:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
default datadog-custom-metrics-server ClusterIP 192.168.254.87 <none>
443/TCP 28m
default datadog-cluster-agent ClusterIP 192.168.254.197 <none>
5005/TCP 28m

47. Verifying Cluster Agent Output
kubectl get pods, svc -l app=datadog-cluster-agent
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 28m
SVCS:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
default datadog-custom-metrics-server ClusterIP 192.168.254.87 <none>
443/TCP 28m
default datadog-cluster-agent ClusterIP 192.168.254.197 <none>
5005/TCP 28m

48. Register the External Metrics Provider
kubectl apply -f manifests/cluster-agent/hpa-example/rbac-hpa.yaml
clusterrolebinding.rbac.authorization.k8s.io "system:auth-delegator" created
rolebinding.rbac.authorization.k8s.io "dca" created
apiservice.apiregistration.k8s.io "v1beta1.external.metrics.k8s.io" created
clusterrole.rbac.authorization.k8s.io "external-metrics-reader" created
clusterrolebinding.rbac.authorization.k8s.io "external-metrics-reader" created

49. Register the External Metrics Provider Output
kubectl apply -f manifests/cluster-agent/hpa-example/rbac-hpa.yaml
clusterrolebinding.rbac.authorization.k8s.io "system:auth-delegator" created
rolebinding.rbac.authorization.k8s.io "dca" created
apiservice.apiregistration.k8s.io "v1beta1.external.metrics.k8s.io" created
clusterrole.rbac.authorization.k8s.io "external-metrics-reader" created
clusterrolebinding.rbac.authorization.k8s.io "external-metrics-reader" created

50. Datadog Cluster Agent running
kubectl get pods,svc
PODS
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-agent-4c5pp 1/1 Running 0 14m
default datadog-agent-ww2da 1/1 Running 0 14m
default datadog-agent-2qqd3 1/1 Running 0 14m
[...]
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 16m

51. Example HPA Manifest
● is configured to scale a deployment called 'nginx'
● set a maximum number of replicas to create to 3, with a minimum of 1
● HPA triggers autoscaling off of the Datadog metric
'nginx.net.request_per_sec' over the scope
'kube_container_name: nginx'

52. Example HPA Manifest
● is configured to scale a deployment called 'nginx'
● set a maximum number of replicas to create to 3, with a minimum of 1
● HPA triggers autoscaling off of the Datadog metric
'nginx.net.request_per_sec' over the scope
'kube_container_name: nginx'

55. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

57. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

58. Creating an Autoscaling Deployment
# Nginx deployment
kubectl apply -f manifests/cluster-agent/hpa-
example/nginx.yaml
# HPA manifest
kubectl apply -f manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
# Verify
kubectl get pods,svc

59. POD:
default nginx-6757dd8769-5xzp2 1/1 Running 0 3m
SVC:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default nginx ClusterIP 192.168.251.36 none 8090/TCP 3m
HPAS:
NAMESPACE NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS
AGE
default nginxext Deployment/nginx 0/9 (avg) 1 3 1
3m

60. curl <NGINX_SVC>:8090/nginx_status
while true; do curl <NGINX_SVC>:8090/nginx_status; sleep 0.1;
done
Let’s Stress it out

61. +

62. +

63. kubectl get pods,svc
PODS:
NAMESPACE NAME READY STATUS RESTARTS AGE
default datadog-cluster-agent-7b7f6d5547-cmdtc 1/1 Running 0 9m
default nginx-6757dd8769-5xzp2 1/1 Running 0 2m
default nginx-6757dd8769-k6h6x 1/1 Running 0 2m
default nginx-6757dd8769-vzd5b 1/1 Running 0 29m
HPAS:
NAMESPACE NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
default nginxext Deployment/nginx 30/9 (avg) 1 3 3 29m

66. function yamllint() {
for i in $(find . -name '*.yml' -o -name '*.yaml');
do echo $i; ruby -e "require
'yaml';YAML.load_file("$i")"; done
}

67. In Conclusion
● Walked through a brief history of scaling and deploying
software
● Described some of the key components and history of the
HPA
● How to choose a metric on which to base autoscaling
● Think of autoscaling as an output for your monitoring
● Walk-through implementing Autoscaling with a custom
provider
● How a simple linting script can save you a lot of time,
effort, and embarrassment.

68. In Conclusion
● Walked through a brief history of scaling and deploying
software
● Described some of the key components and history of the
HPA
● How to choose a metric on which to base autoscaling
● Think of autoscaling as an output for your monitoring
● Walk-through implementing Autoscaling with a custom
provider
● How a simple linting script can save you a lot of time,
effort, and embarrassment.

69. Datadog Cluster Agent: https://github.com/DataDog/datadog-agent/tree/master/Dockerfiles/manifests/cluster-agent (includes an
example)
HPA blog: https://www.datadoghq.com/blog/autoscale-kubernetes-datadog/
DCA blog: https://www.datadoghq.com/blog/datadog-cluster-agent/
HPA Docs: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
Enable the Kubernetes Aggregation Layer: https://kubernetes.io/docs/tasks/access-kubernetes-api/configure-aggregation-layer/
Configure Agents to communicate securely with Cluster Agent: https://github.com/DataDog/datadog-
agent/blob/master/Dockerfiles/cluster-agent/README.md#security-premise
Datadog Cluster Agent Deployment Manifest: https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/manifests/cluster-
agent/cluster-agent.yaml
Datadog Example HPA: https://github.com/DataDog/datadog-agent/blob/master/Dockerfiles/manifests/cluster-agent/hpa-example/hpa-
manifest.yaml
Resources

70. Automatically Scaling Your
Kubernetes Workloads

71. Visit Our Booth

72. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
H. “Waldo” Grunenwald
waldo@datadoghq.com
@gwaldo