"The majority of the container security discussion revolves around containers on Linux while the security of containers in Windows is left as a mystical black box. In this talk we'll peel back the curtain and dive in to how Windows containers are secured. Does Windows have namespaces? How does it compose the layers of a container's filesystem? How does it limit resource usage of containers? I heard there's a Hyper-V isolation thing, what's that about? We'll answer all these questions and more!"

1. v
Windows Container Security

2. Security Engineer, Microsoft
Saruhan Karademir
Product Manager, Docker Inc.
Yuvraj Mehta

3. Agenda
• Docker EE Architecture
• Secure Software Supply Chain
• Secure Application Zones
• Docker Windows Container Architecture
• Process Isolation
• Hyper-V Isolation
• Windows Internals – Resource Limitations
• Security Best Practices
• Microsoft Security Approach

4. Operating
Systems
Platform Architecture
Config Mgt Monitoring LoggingCI/CD ..more..Images Networking Volumes
PhysicalVirtualizationPublic Cloud
Platform
Security
Developer
Services
Registry
Services
Access
Policies
App Lifecycle
Management
Automation &
Extensibility
Networking Orchestration Storage
Container Engine
ENTERPRISE EDITION PLATFORM
Operating
Systems
Platform Architecture
Config Mgt Monitoring LoggingCI/CD ..more..Images Networking Volumes
PhysicalVirtualizationPublic Cloud
Platform
Security
Developer
Services
Registry
Services
Access
Policies
App Lifecycle
Management
Automation &
Extensibility
Networking Orchestration Storage
Container Engine
ENTERPRISE EDITION PLATFORM

5. Production Environments
Docker Trusted Registry
Docker UCP
Production Environments
Version Control
Docker UCP
Non-Production EnvironmentsDeveloper Machine
Development CI/CD Operations
Datacenter 1
Datacenter 2
Docker Trusted Registry
Docker for
Docker EE Secure Supply Chain

6. Secure Application Zones
Node
Worker
Node
Worker
Node
Worker
Node
Worker
swarm mode cluster
Node
Worker
Node
Worker
.NET Dev Team
Using Swarm
Java Dev Team
using K8s
Java Dev Team
Using Swarm
Ops Team
DOCKER ENTERPRISE
EDITION

7. Runtime Architecture
Docker Client Docker Compose
Docker Engine
containerd + runc
Operating System
REST interface
Namespaces
(Silos)
Control
Groups
(Job Objects)
Union file-
systems
(WCIFS)
…
Network Virtualization
(Compartments, Vswitch,
WinNAT, ..)
Host Compute Service (HCS)
libcontainerd libnetwork plugins
Host Compute Service Shim (hcsshim)
Host Network Service (HNS)

8. System
Processes
Application
Process(es)
Windows Server Containers
Host User Mode
Container
Management
System
Processes
Application
Process(es)
System
Processes

9. Hyper-V Isolation
Hyper-V isolation
Host User Mode
Container
Management
System
Processes
Application
Process(es)
System
Processes
Hyper-V VM
System
Processes
Application
Process(es)

10. Windows Containers
Kernel Isolation

11. Linux
• Namespaces
• Resource Control (cgroups)
• System call whitelisting (seccomp)
• Linux Security Modules
• Scanning
• Image Signing
Namespaces
Resource Control (cgroups)
System call whitelisting
(seccomp)
Linux Security Modules
Scanning
Image Signing
Windows
• Namespaces
• Resource Control (cgroups)
• System call whitelisting (seccomp)
• Linux Security Modules
• Scanning
• Image Signing
Silos
Job Objects
Per Process limited blacklisting
Opt-in Sandboxing API
(AppContainer)
Scanning
Image Signing

12. Job ObjectsSilos
Kernel Isolation in Windows
Assigned to process and all
children
Pristine NT Object Table
Separate volume mount
No namedpipes, LPC, Sockets
from root
Unless the root wants to alias
one in…
API accessible from root
Can be assigned to child
process or a Silo
Resource constraints
[CPU, Memory, IO]
API accessible from Medium
Integrity for child processes

13. SandboxingSyscall Filtering
Kernel Isolation in Windows
Per-Process
Limited Blacklist
Targets Win32k
Per-Process
AppContainer
Capability-based Access
Control

14. Sandbox.vhd
Windows
Container
File System
Filter

15. Host User Mode
System Processes
Container
Management
Bob’s Apps

16. Host User Mode
System Processes
Container
Management
Bob’s Apps
New Job Object
Resource Restrictions
• CPU
• Memory
• I/O

17. Host User Mode
System Processes
Container
Management
System
Processes
Bob’s Apps
Silo
• New pristine Namespace
• Object table
• Registry
• Network Compartment
• Starts in UnionFS
directory
• Assigned Job Object
New Job Object
Resource Restrictions
• CPU
• Memory
• I/O

18. Host User Mode
System Processes
Container
Management
System
Processes
Bob’s Apps

19. Host User Mode
System Processes
Container
Management
System
Processes
Bob’s Apps
Application
Processes

20. Host User Mode
System Processes
Container
Management
System
Processes
Bob’s Apps
Application
Processes

21. Host User Mode
System Processes
Container
Management
System
Processes
Bob’s Apps
Application
Processes

22. Windows Containers
Hyper-V Isolation

23. Hyper-V Isolation
Host User Mode
Container
Management
System
Processes
Application
Process(es)
System
Processes
Hyper-V VM
System
Processes
Application
Process(es)

24. Host User Mode
VM Worker
Process
Container’s ‘Physical’ Memory
Container
Management
VM Memory
Process
0x00000000
0xf7ffffff
Guest
Memory
Guest Memory

25. Host User Mode
VM Worker
Process
Hyper-V VM
Container
Management
System
Processes
Application
Processes
Container
Management

26. Host User Mode
VM Worker
Process
Hyper-V VM
StorVSP
Container
Storage
C:
Container
Management
Sandbox.vhd
System
Processes
Application
Process(es)

27. VM Worker
Process
OS
App
OS
OS
Hyper-V VM
System
Processes
Application
Process(es)

28. Host User Mode
VM Worker
Process
Hyper-V VM
StorVSP
Container
Storage
C:
Container
Management
Target
Folder
System
Processes
Application
Process(es)

29. Host User Mode
VM Worker
Process
Container’s ‘Physical’ Memory
Container
Management
VM Memory
Process
0x00000000
0xf7ffffff
Guest
Memory
Guest Memory
File 1
File 1

30. v
cgroups -> Job Objects
Windows Internals – Resource Limitations

31. Restricting Memory
--memory
Kernel Isolation Hyper-V Isolation

32. Restricting Compute
--cpu, --cpu-percent, --cpu-shares
Kernel Isolation Hyper-V Isolation

33. Restricting IO
--io-maxbandwidth, --io-maxiops
Kernel Isolation Hyper-V Isolation

34. Restricting Storage
--storage-opt size
Kernel Isolation Hyper-V Isolation

38. Best Practices

39. Network Isolation
always matters
Image Scanning
Image Signing
Process
Isolation/Restriction
Set up automated
builds
Hyper-V Isolation
OR
Process
Isolation/Restriction
Use Docker EE
secrets
N e t w o r kC o n t a i n e r H o s t
Think Defense-in-Depth

40. QUESTIONS?