2. Load balancer
• Distribute load among a set of nodes
• Static
(fixed set of nodes)
• Dynamic
(add/remove nodes at runtime)
• Load balancing algorithm
(Round robin / weighted RR / weighted RR least connection)
3. Session affinity / sticky sessions
• All subsequent connections within a
session is served by the worker node
which processed the first request of
the session
• Alternate solution to session
replicated backend services
4. Session replication in AppServer
• AppServer clustering – Apache Tribes
• Session data storage,
1. ServiceContext
(data that should only be available to the service)
2. ServiceGroupContext
(common data for all the services in a service group)
3. ConfigurationContext
(common data for all service groups)
• AppServer replicates session data in
all 3 levels (costly compared to sticky sessions)
5. LB – Membership mgt. schemes
• Static
• Dynamic
Join by specifying the group name
• Hybrid (WKA)
Set of well known members – static
Join by specifying the group name
6. Evolution of WSO2 ELB
• Started as a solution to scale PaaS –
(WSO2 Stratos)
• Stratos is a multi-tenanted PaaS
• Tenant – An organization (a domain)
• Isolated from other tenants
• Support multiple PaaS services
• A service = WSO2 product (DSS, ESB
etc…) offered as a service
7. Evolution of WSO2 ELB cont.
• A tenant can deploy their own
services, mediations, workflows etc…
• Multi-tenant services (ESB, DS,
AppServer etc…)
• Stratos can provision resources on
demand
• Problem: making a scalable PaaS
transparent to the user
8. Solution 1
• Stratos have multi-tenant services
(ESB, AppServer, DS etc…)
• User talks to Identity Server, gets an
SAML token, logs into any service
• Store tenant data in Registry
• Each server loads all tenant data at
startup
• Can support any tenant
10. Solution 1 - Problems
• Doesn’t scale with increasing number
of tenants
11. Solution 2
• Started running multiple service
instances
• Put a LB in front
• When the load is high LB will spin up
a new instance and shuts it off when
the load decreases (autoscale)
13. Solution 2 - Problems
• When we had a large number of
tenants with tens of services, took a
long time to load at startup
• Some tenants stayed inactive
(consumed resources)
14. Solution 3
• Add lazy loading
• Tenant info in the central registry
• Tenant info loaded only when needed
• More info - http://blog.afkham.org/2011/11/lazy-loading-
deployment-artifacts-in.html
15. Solution 3 - Problems
• When a tenant has many artifacts
loading those takes time
• Initial requests for an artifact of a
tenant timed out
16. Solution 4
• Introduce the GhostDeployer
• Loads only the metadata of a tenant
• Actual artifacts are loaded on demand
• More info - http://blog.afkham.org/2011/11/lazy-loading-
deployment-artifacts-in.html
17. Solution 4 - Problems
• LB didn’t scale to handle large
number of requests
20. Solution 5 - Problems
• Many to many relationship between
LBs for synchronizing metadata
• LB is not aware of tenants
• A single node might have too many
tenants
23. WSO2 ELB
• Autoscaling component keeps track of
traffic for each cloud service cluster
• Calls EC2 API
• Implementation: 2 mediators and a
task (autoscale-in and autoscale-out)
• An ESB endpoint routes requests to
the correct node
24. WSO2 ELB
• Service cluster identification through
Host HTTP header
• According to LB algo. select a node
• If the node is down it’ll fail over to
another member
• Keeps track of sticky sessions
(identified by JSESSIONID)
25. WSO2 ELB
• BinaryRelay is used as a message pass
through mechanism
(no building or processing the message)
• Membership discovery & mgt. – Axis2
clustering. Membership handler for
each clustering domain
• Use Apache Tribes Group Mgt.
Framework
28. WSO2 ELB – Port mapping
• Advertise the mapped ports when
joining the cluster
• LB retrieves these as properties of the
member
• When dispatching requests, get
mapped real port from the member
(as advertised)
30. WSO2 ELB – Autoscaling
• AutoscaleInMediator creates a unique
token when a request comes, put into a
list
• Removes this when a response is sent
• LoadAnalyzerTaks periodically checks
the list. Autoscale according to config
params
