Load Balancing for Containers and Cloud Native Architecture

LOAD BALANCING FOR
CONTAINERS
August 2016
Copyright: Citrix Systems, Inc

About me
¨ Distinguished Engineer
at Citrix
¨ Apache CloudStack
PMC
¨ Work on Citrix
Netscaler and
containers

Load balancers are important (again)
¨ Containers and Microservices
¨ Teaching old load balancers new tricks
¨ Emerging patterns for Load Balancing
¨ Future directions

What’s Changed?
Time
Web
Digital
Social
Mobile
Things
Smart Things
RateofChange

What’s changed?
¨ DevOps + Automation
¨ Microservices /
Containers
Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 deCopyright: Citrix Systems, Inc

Containers or Microservices?
¨ Microservices ==
architecture
¨ Containers ==
implementation
¨ Containers win over
VMs for Microservices

Monoliths vs. Microservices
LB
Web
Data

Load Balancing: Traditional vs. Cloud Native
Static Applications, well
defined topology
Dynamic Microservices,
changing topology
W W W
A A A A A
W
M
M
TrafficismostlyN-S
M
M
M
M
M
M
M
M
M
Traffic is mix
of N-S and E-W

Cloud-Native
Patterns of
architecture
and
organization
that deliver
software with
speed and
reliability
¨ Auto Scale
¨ Continuous Delivery
¨ Baked-in Resilience
¨ Deep Monitoring
¨ Collaboration

Cloud Native + Containers is Network
Intensive
¨ Implications on
¤ Performance
¤ Reliability
¤ Security
¤ Routing
¤ Naming and discovery
¤ Monitoring

Cloud Native Landscape - Microservices
Microservices
APIs
Performance
Resilience
Security Visibility
Continuous
Delivery
AutoScal
e
Circuit
Breaker
Load
Balancing
Throttling
Discovery
Audit Segmentation
E2E
Encryption
Routing
Chaos
Monkey
Distributed
Debug
Back-off
Lifecycle
Management
Auth

Containers
Container
Networking
Container
Security
Integrity Isolation AuthN/Z
Container
Orchestration
Infrastructure
Orchestration
Network
Orchestration
Cloud
Orchestration
Multi-cloud
Orchestration
Private
Cloud
OrchestrationHypervisor
Orchestration
Network
Orchestration
IPAM / DNS
Overlay /
Underlay
Microservices
Lifecycle
Management
Container
Performance
Network
Perf
scheduler
Container
Storage
Storage
Orchestration
Storage
Orchestration
Cloud Native Landscape - Infrastructure

Container Managers to the rescue?
Docker Swarm Kubernetes
(from Google)
Mesosphere DCOS
(based on Apache Mesos)

Container Cluster Managers
¨ Manage / orchestrate multi-
host clusters of containers
¨ Enable DevOps automation
with APIs
¨ Manage network and
storage as well as compute
α1
β1
δ1
β2
α2
δ3
α3
α4
δ2
α5
α6
HostH1
HostH2
HostH3
HostH4
δ4
ClusterManager
Cluster API

Cluster Managers – Sweet Spots
¨ Infrastructure independence
¨ Scheduling
¨ Discovery
¨ Scaling (partially)
¨ Security (a little bit)
¨ Load balancing (limited)

Load Balancing for container clusters –
Ingress / Edge
α1
β1
δ1
HostH1
β2
α2
δ3
HostH2
α3
α4
δ2
HostH3
Public Endpoint
https://alpha:443
LB
α1
β1
δ1
HostH1
β2
α2
δ3
HostH2
α3
α4
δ2
HostH3
α5
α6
HostH4
Public Endpoint
https://alpha:443
LB
Scale out

Load Balancing – intra-cluster
α1
β1
HostH1
α2
δ3
HostH2
α4
δ2
HostH3
LB
α1
β1
HostH1
α2
δ3
HostH2
α4
δ2
HostH3
LBLB LB
LB per endpoint
LB per container host

Load Balancing for Container Clusters
¨ High performance
¨ Few instances
¨ Somewhat frequent
reconfiguration
¨ Hardware or virtual
¨ Advanced: TLS, WAF, content
switching
¨ Stateful
¨ Frequent reconfiguration
¨ Short lifespans
¨ Large number of instances
¨ Lower performance
¨ L4 common, L7 desirable
¨ Stateless
Ingress (edge) Intra-cluster

Reconfiguration of Load Balancer
α1
β1
HostH1
β2
α2
HostH2
α3
α4
HostH3
Ingress LB
α5
α6
HostH4
ClusterManager
LB
Controller
Cluster API
Container
Events Reconfigur
ation
Container
State
Container
State Query

Orchestration vs. Choreography
¨ Cluster Manager / Orchestrator drives
predefined (often hardcoded) process
¨ Points in the process can be “plugged in”.
¨ E.g., Load Balancer controller can be
plugged in.
¨ Brittle, hard to change
¨ Typical of IaaS stacks (e.g., OpenStack,
CloudStack)
¨ Cluster Manager emits events
¨ Controllers react to events asynchronously
¨ Easy to swap controllers
¨ Loosely coupled
¨ Harder to debug.
¨ No “god view” of state of the system.
¨ Typical of Cluster Managers (Kubernetes)
Orchestration Choreography

Typical Form factors
¨ Virtual, hardware (F5,
Netscaler), or service
(ELB)
¨ Usually Proprietary
¨ Containerized OSS
(HAProxy, Nginx)
¨ Written from scratch
(kube-proxy, Uber
hyperbahn, linkerd,
traefik)
Ingress LB Intra-cluster

VPX
Hypervisor
Virtual
Run Anywhere
=
NetScaler CPX: NetScaler in a Container
MPX
Physical
Price-Performance
CPX (new)
Container
SDX
Platform
Multi-Tenant
=

Netscaler CPX
Containerized
Netscaler for
Developers /
Cloud native
deployments
¨ Proprietary
¨ Same hardened code, same features as
bigger form factors
¨ Unified control plane with Ingress and
other LB (Netscaler MAS)
¨ Unified monitoring, logging and
analytics

Netscaler Family
Automation
Network and
Load Balancer
Full Featured
Un-compromised
Network/ Load
Balancer
functionality
All Appliance/
Workloads
Containers, VM,
Physical form
factors
Investment
Protection
Keep what you
already have
Single Management
Platform
One manager
for all your
appliances

Packaged as Docker Container
¨ Investment protection
¤ Same code bits à container form factor
¤ Managed like any other NetScaler platform
¤ Seamless transition from Development to Production
¨ Functionality:
¤ Load balancing
¤ Content Switching
¤ All traffic types and protocols
¤ SSL offloading
¤ DNS
¤ Monitoring and logging
¤ Nitro API
You Can Deploy In Seconds!
Server
Linux OS
Docker Engine
AppA
bin/libs
App
B
bin/libs
App
C
bin/libs
C
P
X
bin/libs

Netscaler CPX Express
¨ “Developer” Edition*
¨ No license required
¨ Available for download from Docker Hub in Q3
¨ Drop-in replaceable with licensed version
*Not for production use

MAS Turns NetScalers into a Pool – Control One to Thousands
NetScaler
MAS
CPX VPX MPX
Container
Management:
Mesos & Marathon
Docker Swarm
Kubernetes
Service
Discovery
Orchestrator: Self
Service Portal
NetScaler MAS Functions
App-centric Life
Cycle
Configuration at
Scale
Visibility and
Insights

What do We Mean by App-Centricity
App configs vs.
network configs
Provide role-based,
partitioned access to
application owners
Provide tools to map
app config to
NetScaler:
Stylebooks

¨ Template-driven configuration for Citrix Netscaler.
¨ Intent-driven
¨ Sharing, collaboration and re-use
¨ Automate via an API
App-centric: Netscaler Stylebooks

Configuration at Scale

¨ The ADC has visibility to users on
one side and apps/infrastructure
on the other
¨ Outbound: Data provides insights
on customer usage and behavior
¨ Inbound: Insights on app health and
performance
¨ Security: Recognizing and
mitigating increasingly
sophisticated attacks
Analytics at Scale:
Insight from Network Data
Users Apps
Devices

Cluster Managers: Native LB support
¨ Built-in LB based
on IPVS
¨ No ingress LB
¨ Native Service
abstraction
¨ Kube-proxy for
intra-cluster L4 load
balancing (uses
iptables)
¨ “Ingress” object for
edge routing, but
not built-in
Docker Swarm Kubernetes
¨ Native ‘Task’
(service)
abstraction
¨ Iptables-based
L4 LB
¨ No built-in Ingress
LB
Mesos / Marathon

HostH1
Ingress Challenge (e.g., Kubernetes)
• Hardware / Virtual LB
has to “participate” in
overlay (usually
VxLAN)
• Interact with overlay
manager / SDN
controller
• Kubernetes: Kube-
Proxy introduces extra
hop
Intra Cluster Network (Overlay / Routing)
External Network
LB
Ingress
α1
β1
β2
α2
HostH2
α5
α6
HostHn

Emerging LB patterns
¨ Client-side LB
¨ Sophisticated routing
¨ Resilience patterns
¨ Visibility / Insights

Client-side LB
¤ Embedded into calling application
n e.g., Netflix Ribbon (with Eureka)
¤ Run as side-cars (alongside each
application or one-per-host)
n Netflix Prana
n Twitter Finagle
n Linkerd from Buoyant.io (based on
Finagle)
n AirBnB Smartstack/Synapse (uses
HAProxy)
n Uber Hyperbahn (like Finagle,
switched from HAProxy)
n Kube Proxy
Eureka
M M
M
M
M
M

Advanced Resilience
¨ LB implements / assists
resilience patterns such
as:
¤ Circuit Breaker (e.g.,
Netflix Hystrix)
¤ Anti-DDOS
¤ Throttling
¤ Chaos Monkey
Credit: http://martinfowler.com/bliki/CircuitBreaker.html

Routing
¨ Red-black deploys
¨ Content routing
M M M M
Version N
M M M M M
Version N+1
10%
90%
CD
Pipeline

Visibility
¨ Compliance
¨ Debug
¨ Topology
¨ Protocol Insights
M
M
M M M
M M M M M
M
Analytics

Wrap-up
¨ Load balancing is
different for containers /
microservices
¨ Integration with container
cluster managers is
needed
¨ Consider using the same
LB technology for ingress
and intra-cluster
¨ Emerging patterns
solidify the importance
of the load balancer
¨ LB in the wire brings
¤ Simplicity
¤ Resilience
¤ Future proofing

Load Balancing for Containers and Cloud Native Architecture

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Load Balancing for Containers and Cloud Native Architecture

Similaire à Load Balancing for Containers and Cloud Native Architecture (20)

Plus de Chiradeep Vittal

Plus de Chiradeep Vittal (13)

Dernier

Dernier (20)

Load Balancing for Containers and Cloud Native Architecture