One obvious side effect of migrating to a microservices architecture is the need for infrastructure automation. Unfortunately, most automation systems do not take security into consideration, making production deployments orders of magnitude more complex than the initial testbed deployment. The perfect example of this steep increase in deployment difficulty is the creation and management of Public-Key-Infrastructures (PKI). Even though the use of TLS Certificates for service to service communication is known as a best-practice, very few companies actually deploy their systems using mutually-authenticated TLS connections. In this talk I will go over why TLS is the right solution for service to service communication, describe ways to automate the creation and management of your PKI, and present in detail how Docker's swarm orchestration system bootstraps and manages individual node certificates.

1. MTLS in a Microservices 🌍
@diogomonica

2. - Mobile payments company
- Moving $48 billion annually
- Infra: Java & Ruby, some Go
- Infrastructure Company
- Moving 1+ billion containers annually
- Infra: Go, some python

3. Microservice Security Goals
- Provide common security infrastructure
- Follow the principle of least-privilege
- Better security monitoring
- Automated end-to-end secure service
deployment

4. The Security boundary is
the service

5. Internet
DatabasesServicesFront-end
VS
Internet

6. Every service call should be
authorized and authenticated

7. One node, one ID

10. Your provisioning system is
your Registration Authority

11. Isolated Network
Provisioning
CSR
Cert

12. Trust on first use

13. MTLS

14. Key-material
stays secret
Supported by
everything
MTLS
Confidentiality
Authentication
Integrity

15. Key-material
stays secret
Supported by
everything
MTLS
A LOT of Certs!
Confusing
for Engineers Unforgiving
Revocation?
Running a PKI
Confidentiality
Authentication
Integrity

16. Key-material
stays secret
Supported by
everything
MTLS
A LOT of Certs!
Confusing
for Engineers Unforgiving
Revocation?
Running a PKI
Encryption
Authentication

17. SWARM

18. Mutual TLS by default
• First node generates a new
self-signed CA.

19. Mutual TLS by default
• First node generates a new
self-signed CA.
• New nodes can get a
certificate issued w/ a
token.

20. Mutual TLS by default
• First node generates a new
self-signed CA.
• New nodes can get a
certificate issued w/ a
token.
• Workers and managers
identified by their
certificate.

21. Mutual TLS by default
• First node generates a new
self-signed CA.
• New nodes can get a
certificate issued w/ a token.
• Workers and managers
identified by their certificate.
• Communications secured
with Mutual TLS.

22. The Token
SWMTKN-1-mx8susrv1etsmc8omaom825bet6-cm6zts22rl4hly2
Prefix to allow VCS
searches for leaked
Tokens
Token Version
Cryptographic Hash
of the CA Root Certificate
for bootstrap
Randomly generated
Secret

23. Bootstrap
1. Retrieve and validate Root CA
Public key material.
2. Submit new CSR along with
secret token.
3. Retrieve the signed
certificate.

24. Automatic Certificate Rotation
1. Submit new CSR using old
key-pair.
2. Retrieve the new signed
certificate.

25. Support for External CAs
• Managers support BYO CA.
• Forwards CSRs to external
CA.

26. Demo

27. One app, one ID

28. Your Orchestration System is
your Registration Authority

29. Orchestration
System
Registration
Authority
CSR Cert

30. MTLS for service
authentication

31. CN=api01 CN=db01

32. MTLS for service
authorization

33. CN=api01
OU=web-api
O=production
CN=db01
OU=credit-card-db
O=production

34. [ { "permission":
{ "method": "GET", "resource": "/user" },
"allow": ["web", "fulfillment", "payments"] },
{ "permission":
{ "method": "POST", "resource": "/user" },
"allow": ["signup", "web"] },
{ "permission":
{ "method": "DELETE", "resource": "/user/.*" },
"allow": ["web"]
}]

35. Sane access to raw secrets

36. Demo

39. Thank you
@diogomonica