2. 19/11/2020 - Emmanuelle Demompion
Understand, verify, and act on
the security of your clusters

3. What’s the plan?
Introduction
Secure your k8s cluster at its root
Security as a coding philosophy
Managing third parties security
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
1
2
3
4
5
6
7
Lock your data
Security cheat sheet
Conclusion
3

4. From the Cloud to your code
Introduction
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

5. Scaleway
Private
Infrastructure
High-end
dedicated Servers
Public Cloud
Ecosystem
3 specialized brands to cover all cloud and infrastructure needs
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

6. 19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
6
Compute
Storage
NetworkIoT
AI
Steering
Scaleway
ELEMENTS

7. Kubernetes Kapsule
Slack #k8s - @ben - @jth - @Sh4d1 - @ema
Just a high availability, fully managed, up-to-date
Kubernetes environment
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

8. TL;DR; of the webinar
Security issues
Everybody should care about
security
Very few really do
Be aware of the most
common issues
Solutions
Certificates
Admission controllers
Pod security policies
Security by design
Responsibilities
The boss
The analyst
The architect
The developer
The user
On a clock?
You don’t have time to go
through the presentation or
you fell asleep during it, go
to the end, there is a
Kubernetes Cheat Sheet
waiting for you… ;)
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion8

9. How will this webinar go?
On each slide, I will use these images to illustrate who should feel the most concerned about each topic. You are all concerned,
you just might not be the one dealing with it directly.
First Strike: A creature with first strike will deal its
fight damage before a creature it doesn’t.
Trample: Static ability. After attacking, the creature
with Trample can assign the rest of the damage not
taken by the first target to target other creatures.
Creature: Boss
“No one can defeat the decision maker.”
“Scaleway’s mascot rules” - Marc-Etienne Peintre
5 / *
Landwalk: Static ability. The affected creature may
not be obstructed providing the protecting player
settings a land associated with the specified type.
Shroud: Static ability. The impacted permanent or
player cannot be the goal of every means or abilities,
together with your own.
Creature: Analyst
“Data is everything and more.”
“Connections and correlations” - Marc-Etienne Peintre
3 / 2
Lifelink: Fixed capability. Whenever a card with
lifelink deals damage, the controller of this card
additionally gains life corresponding to the amount of
damage dealt.
Banding: When several creatures with banding,
temporally 'band' together, it can become one
creature to either assault or block.
Creature: Developer
“It’s not a bug, it’s a feature!”
“Scaleway’s mascot space cowboy” - Marc-Etienne Peintre
2 / 2
Indestructible: Any permanent which has
indestructible can't be damaged, and cannot perish
as a consequence of deadly harm.
Intimidate: Fixed capability. A creature with intimidate
cannot be blocked, except by artefact creatures and
also by creatures that share a colour with it.
Creature: User
“I’m a leaf on the wind, watch how I soar…”
“The magnificent” - Marc-Etienne Peintre
* / *
Creature: Architect
“You need strong foundations for a building to last.”
“The builder” - Marc-Etienne Peintre
1 / 4
Cover: Static ability. The affected permanent is
granted with the inability becoming dealt damage,
enchanted, blocked, or targeted by any card aided by
the reported high quality.
Vigilance: Static ability. The affected creature does
not touch as soon as declared as an attacker.
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion9
The takeaways I hope you’ll get from this webinar: You know solutions to address security issues, you feel responsible about
security, whatever your job is, and you are willing to accept that cloud-native apps should include security by design.

10. Terraform Helm CI/CD
K8s
Third parties Your Service
Code & Services
Deployment
Cluster
Your Data
Everything we should take a look at…
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion10

11. How bad can it really be?
My cluster: 63 issues
A bunch of tutorials, an old Github
project of mine and on-the-shelf
application installed using Helm.
My old unmaintained Github project:
1054 vulnerabilities
Security issues spread
3 %
57 %
40 %
Cluster
Images
Exposures
13 pods without security policies
1 service running without data encryption
6 namespaces without network policies associated
6 docker images with vulnerable packages 1 image including a private key
3 docker images built with ROOT user
1 service exposing software information
1 service using a self signed certificate
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
ok if trusted, mine was not
11

12. Securing the infrastructure of your cluster itself
Secure your k8s cluster
at its root
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

13. How a Kubernetes cluster looks
Creature:DeveloperCreature:ArchitectCreature:Boss
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Worker nodes
Namespaces
Pods
Control-Plane
ETCD
13

14. Certificates
Managing your Kubernetes cluster
yourself
Use automatically generated certificates
(stored on the API-Server) or generate them
yourself.
Protect the location of your certificates.
Think about a process if you need to revoke
your certificates and regenerate new ones.
Using a managed Kubernetes
engine
Certificates are generated and managed by
the cloud provider.
Depending on the provider, the revoking
policy of certificates can be different and
should be part of your choice of provider.
Creature:ArchitectCreature:Boss
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion14

15. Admission controllers
19/11/2020 - Understand, verify, and act on the
security of your clusters - Emmanuelle Demompion
Control every request reaching the API-Server1
2
3
Creature:DeveloperCreature:Architect
Limit creation, deletion, modification, or
connection rights
Define the behaviour of every Kubernetes object
running on your cluster
Set security strategies on pods or containers
4
15

16. Admission controllers
LimitRanger
• Any incoming request should not violate
the Namespace limit ranges constraints.
• Limit Ranges objects enforce min and max
resources, storage, requests, for k8s objects
in a Namespace (Pod, Container,
PersistentVolumeClaim…)
ServiceAccount
• Automation implementation for
ServiceAccounts.
• Mandatory if you are using ServiceAccount
objects in your cluster (to manage different
users accesses to your cluster)
NamespaceLifeCycle
• New objects cannot be created in a
terminating namespace.
• Requests on a non-existing namespace are
rejected.
• Prevents deletion of system reserved
namespaces (default, kube-system, kube-
public).
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
DefaultStorageClass
• Observes the creation of
PersistentVolumeClaim objects which does
not explicitly provide the storage class to
use.
• This is usually set by default on managed
Kubernetes engine to use the provider's
Block storage volumes (bssd)
Creature:DeveloperCreature:Architect
16

17. Admission controllers
AlwaysPullImages
Every new Pod will have its pull policy set to
“Always”.
• Valid credentials are always required.
• Images will always be pulled before
starting a container.
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
NodeRestriction
• Limits the Node and Pod objects a kubelet
can modify.
• Kubelets must use credentials in the
system:nodes group with a specific
username.
• Ensures kubelets have the minimal set of
permissions required to operate correctly
Creature:DeveloperCreature:Architect
ResourceQuota
• Ensures that incoming requests do not
violate constraints listed in a namespace’s
ResourceQuota object.
• Mandatory if you use ResourceQuota
objects on your namespaces.
PodSecurityPolicy
• Determines if the creation or modification
of a pod should be admitted.
• This admission controller should be set to
allow usage of Pod Security Policies (more
details later).
17

18. Network policies
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:Architect
Worker nodes
Namespaces
Pods
By default, pods are non-isolated and accept
traffic from any source.
By setting a NetworkPolicy that selects pods,
they become isolated.
Network policies are additive. The network
policy applied on a pod is the intersection of
all applicable policies egress/ingress rules.
Network policies require a CNI supporting
them. Worker nodes
Namespace
Pods Pods
NetworkPolicy1
NetworkPolicy2
NetworkPolicy1
Namespace
18

19. Pod security policies
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:ArchitectCreature:Analyst
19
Pod security policies allow you to define rules in order to accept or reject the creation
of pods.
This enables the cluster administrators to decide what should run, and what shouldn't,
in the cluster.
For instance, they could decide to deny all pods that are running as root or that runs in
privileged mode.

20. Role-Based Access Control
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:ArchitectCreature:Analyst
A standard authorization mode that can easily be used within a Kubernetes cluster is
RBAC, by defining a Role or a ClusterRole.
Once Roles are defined, they can be bound by defining a RoleBinding (or
ClusterRoleBinding) and assigned specific ServiceAccounts.
Accesses can be defined by user or by groups of users.
20

21. Include security concerns in your cloud native
application by design
Security as a coding
philosophy
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

22. Can we code now?
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:Architect
No!
Your cluster is safer now that you defined restrictions and rules within the cluster itself,
but what about the accesses from outside that you definitely need?
I am talking here about your CI/CD, which should have dedicated accesses to the
Container Registry, and to your namespaces.
One pipeline does not need access to the entire cluster, but only to a dedicated part of
it, with a dedicated user defined.
22

23. Now that your CI/CD is set, you may code :)
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:Architect
It may feel like there are a lot of restrictions going on here, but restriction means
security.
And by designing a production environment with all the necessary security constraints,
the value you provide your customers and end users with your services and
applications will only be greater.
On the development side, there is also the topic of packages vulnerabilities. It may be
useful to ask ourselves, on a regular basis, a few simple questions:
• Are our lists of required packages/libraries up-to-date?
• Do we force a specific version of each package, and how often do we upgrade to the
newest version?
• If one of our dependancies becomes deprecated, should we invest development time
to replacing it so we can keep upgrading our packages? Or should we keep it, no
matter the security risk, and freeze our package version from this point?
23

24. Security as a coding philosophy
Security should not be seen as a
constraint, but as an opportunity to
learn, improve, and we should be
willing to do it by default.
It will never be a waste of time.
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion24

25. Managing third parties
security
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Tl;dr; Being on the shelf does not make it safe

26. Third parties: do you really know what you use?
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:ArchitectCreature:AnalystCreature:Boss
We need third parties because we cannot nor do we want to reinvent the
wheel.
On-the-shelf services and applications allows us to go faster, open source or
on-premise, businesses use third parties for the purpose they serve:
monitoring, logging, database, caching, analytics, and much more.
Those services, like all others, have been coded by humans, and they are not
immune to vulnerabilities.
26

27. Third parties: do you really know what you use?
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:DeveloperCreature:ArchitectCreature:AnalystCreature:Boss
Using a container on the shelf might have been built by ROOT user, making it
accessible for anyone, including an attacker who got into your production
system if it is not secured enough.
Those thirds parties must not be forgotten while defining security policies
(pods and network), namespace isolation, and user access rights.
Is the security level of an “on-the-shelf service" a criteria when designing your
architecture and software?
27

28. How to put your data in a safe
Lock your data
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

29. Your data
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:UserCreature:DeveloperCreature:ArchitectCreature:AnalystCreature:Boss
Users data
• Can be stored the same way as your software data, but
contain valuable, critical and personal information
about your customers.
• A leak of your users' data can kill your business.
Software data
• Often stored in a managed database where its security,
redundancy, and availability is ensured by the cloud
provider.
• Sometimes managed on a dedicated server(s),
implying managing the security of the server(s), as well
as the redundancy.
Analytics data
• Statistics, metrics, sensors, valuable data that can be
treated and in multiple ways, from simple statistics to
transformation pipelines and even machine learning
algorithms.
• These data can be the added value you have
compared to your competitors, and they often require
dedicated storage because of their volume, format,
and specific querying requirements.
29

30. Access restriction, encryption
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:UserCreature:DeveloperCreature:ArchitectCreature:Analyst
Kubernetes and data storage
• Kubernetes is often described as stateless, meaning
that it is not meant to host persistent data directly on
the nodes, which is logical once we consider the node
auto-healing and node autoscaling features.
• Though, data storage is possible in a Kubernetes
cluster, using persistent volumes, such as Block Storage.
• Even if running a database in a Kubernetes cluster is
not recommended, it is possible and quite easy to do
so.
• Persistent volumes can be protected by restricting their
access right solely to the pod to which they are
connected.
• Data stored in persistent volumes can also be
encrypted if it is supported by the CSI of the cloud
provider you use.
30

31. Your customer’s data: laws are coming
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Creature:ArchitectCreature:AnalystCreature:Boss
Data under protection
• GDPR: General Data Protection Regulation (Europe)
• LGPD: Lei Geral de Proteção de Dados Pessoais (Brazil)
Customers, users and people are now increasingly protected by laws like the GDPR.
These laws state that personal data, such as web browsing information, social information, actions tracked on a website
or software... are the property of the user, and that, at any time, they have to be deleted upon the user’s demand.
This request has a huge effect on Software and data pipelines, because not only is it a specific feature, but if your
software is communicating this data to another software, the deletion order should follow.
This also asks the question of anonymised data for machine learning or statistics. For now, a machine learning model
based on anonymised data does not have to be re-trained without the removed data, but if the law evolves, it could be
the case and have a drastic impact on some software and businesses.
The best thing to do is to ensure that this data is safe, anonymous, and legitimate. Only keep what you need, and justify
what it is used for.
31

32. Tl;dr; if you did not listen nor read, at least take
this home with you
Security cheat sheet
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

33. Security Cheat Sheet
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
Choose wisely
• Your cloud provider: what is its revoke certificate strategy?
• Your third parties: are they secured, vulnerable, mandatory?
Design before code
• The production infrastructure of your software will define the
roles, users, and best practices to code with serenity.
• Learn and be curious about vulnerabilities.
• Know your priorities when it comes to security.
• Who can access what, and should they be able to?
Start fresh
• Set your admission controllers (at least):
NamespaceLifeCycle, LimitRanger, ServiceAccount,
DefaultStorageClass, PodSecurityPolicy, ResourceQuota.
• Define Pod security policies to define rules for pods' creation.
• Define Network security policies to isolate pods within your
cluster.
Be responsible
• A production environment is everyone's responsibility
• You are in the same boat, don't let other people row when
things get complicated
33

34. An engaged team makes for a protected customer
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
FirstStrike:Acreaturewithfirststrikewilldealits
fightdamagebeforeacreatureitdoesn’t.
Trample:Staticability.Afterattacking,thecreature
withTramplecanassigntherestofthedamagenot
takenbythefirsttargettotargetothercreatures.
Creature:Boss
“Noonecandefeatthedecisionmaker.”
“Scaleway’smascotrules”-Marc-EtiennePeintre
5/*
Landwalk:Staticability.Theaffectedcreaturemay
notbeobstructedprovidingtheprotectingplayer
settingsalandassociatedwiththespecifiedtype.
Shroud:Staticability.Theimpactedpermanentor
playercannotbethegoalofeverymeansorabilities,
togetherwithyourown.
Creature:Analyst
“Dataiseverythingandmore.”
“Connectionsandcorrelations”-Marc-EtiennePeintre
3/2
Lifelink:Fixedcapability.Wheneveracardwith
lifelinkdealsdamage,thecontrollerofthiscard
additionallygainslifecorrespondingtotheamountof
damagedealt.
Banding:Whenseveralcreatureswithbanding,
temporally'band'together,itcanbecomeone
creaturetoeitherassaultorblock.
Creature:Developer
“It’snotabug,it’safeature!”
“Scaleway’smascotspacecowboy”-Marc-EtiennePeintre
2/2
Indestructible: Any permanent which has
indestructible can't be damaged, and cannot perish
as a consequence of deadly harm.
Intimidate: Fixed capability. A creature with intimidate
cannot be blocked, except by artefact creatures and
also by creatures that share a colour with it.
Creature: User
“I’m a leaf on the wind, watch how I soar…”
“The magnificent” - Marc-Etienne Peintre
* / *
Creature:Architect
“Youneedstrongfoundationsforabuildingtolast.”
“Thebuilder”-Marc-EtiennePeintre
1/4
Cover:Staticability.Theaffectedpermanentis
grantedwiththeinabilitybecomingdealtdamage,
enchanted,blocked,ortargetedbyanycardaidedby
thereportedhighquality.
Vigilance:Staticability.Theaffectedcreaturedoes
nottouchassoonasdeclaredasanattacker.
34

35. Better safe than sorry
KAPSLOCKED
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion

36. Conclusion
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion
You should be safer than 50% of other production systems.
Good enough? That’s for you to decide.
Have a look at the official Kubernetes documentation about the different topics related to this presentation
• Certificates
• Admission Controllers
• Service Account
• Network Policies
• Pod Security Policy
Feel free to join us on slack (https://slack.scaleway.com/) #k8s.
Our Kubernetes community now has more than 650 members.
36

37. Conclusion
“Care, but don’t overthink it”
19/11/2020 - Understand, verify, and act on the security of your clusters - Emmanuelle Demompion37

38. #k8s slack channel
@ema - @ben - @jth - @Sh4d1
Merci !