SOA, Low latency, highly scalable, High volume

1. Building Highly Scalable and High Scalable
Applications using Java/JEE Technologies
Shivnarayan Varma
Sr. Architect
Tech COE, Fiserv

2. © 2010 Fiserv, Inc. or its affiliates.
Agenda :
• Defining and understanding scalability.
• Scalability variables & Factors
• Building our own Scalable Application
• (Scale seamlessly
• High Performance
• Unlimited storage expansion
• Reduce Network Latency
• Direct the user to nearest server
• Scale out, no Scale in
• Sweet spot: Low Cost, High Performance, Scalable
)
• Scalability Real Time Issue

3. © 2012 Fiserv, Inc. or its affiliates.•3
• How do I know if I have a scalability problem?
• Why we need scalability?
Defining and understanding scalability.

4. © 2012 Fiserv, Inc. or its affiliates.
Scalability Variables
• Scalability - Number of users / sessions / transactions / operations
the entire system can perform
• Performance – Optimal utilization of resources
• Responsiveness – Time taken per operation
• Availability - Probability of the application or a portion of the
application being available at any given point in time
• Downtime Impact - The impact of a downtime of a
server/service/resource - number of users, type of impact etc
• Cost
• Maintenance Effort
4

5. © 2012 Fiserv, Inc. or its affiliates.
Factors for Building Robust Scalable JEE
Application
• Platform selection
• Hardware
• Network limits
• Garbage Collection
• Application Design/ Architecture
• Database Structure and Architecture
• Deployment Architecture
• Storage Architecture
• Monitoring mechanisms
5

6. 6
Typical JEE Reference Architecture

7. 7
Building our own Scalable Application.
Platform selection
Hardware
Application Design
Database/Data store Structure and Architecture
Deployment Architecture
Storage Architecture
Monitoring Mechanism

8. © 2012 Fiserv, Inc. or its affiliates.
Identify areas of bottlenecks
• Database red/Write
• Memory Clustering
• Failover Caching
8

9. © 2012 Fiserv, Inc. or its affiliates.•9
Typical JEE Reference Architecture

10. © 2012 Fiserv, Inc. or its affiliates.

11. © 2012 Fiserv, Inc. or its affiliates.
JEE Scalability Points
• We will now build an example architecture for an example app using
the following iterative incremental steps –
• Inspect current Architecture
• Identify Scalability Bottlenecks
• Identify SPOFs and Availability Issues
• Identify Downtime Impact Risk Zones
• Apply one of -
• Vertical Scaling
• Vertical Partitioning
• Horizontal Scaling
• Horizontal Partitioning
• Repeat process
11

12. © 2012 Fiserv, Inc. or its affiliates.
JEE Scalability Points
• We will now build an example architecture for an example app using
the following iterative incremental steps –
• Inspect current Architecture
• Identify Scalability Bottlenecks
• Identify SPOFs and Availability Issues
• Identify Downtime Impact Risk Zones
• Apply one of -
• Vertical Scaling
• Vertical Partitioning
• Horizontal Scaling
• Horizontal Partitioning
• Repeat process
12

13. © 2012 Fiserv, Inc. or its affiliates.
Clustering factors
• High Availability
• Load balancing
• Fault Tolerance
• Failover
• Idempotent methods
13

14. © 2012 Fiserv, Inc. or its affiliates.
Best Practices
• Use scalable session replication mechanisms
Use collocated deployment instead of distributed one
Shared resources and services
Distributed cache
• Memcached/Terracotta
• Paralleled processing
• MapReduce
• MPI
• Using unorthodox approach to achieve high scalability
•
•
14

15. © 2012 Fiserv, Inc. or its affiliates.
How can I integrate multiple applications so that they work
together and can exchange information?
Remote Procedure Invocation
File Transfer Shared Database
Messaging- MOM

16. © 2012 Fiserv, Inc. or its affiliates.
Asynchronous MOM Element
Channels are separate from applications
Channels are asynchronous & reliable

17. © 2012 Fiserv, Inc. or its affiliates.
How does one application communicate with another
using messaging?
Messaging Channel -A virtual pipe that connects a sender to a
receiver application.

18. © 2012 Fiserv, Inc. or its affiliates.
Point To Point Channel
How can the sender be sure that exactly one
receiver will receive the message?
In point-to-point Java Message Service (JMS) , messages are
routed to an individual
consumer which maintains a queue of "incoming" messages.
This messaging type is built on the concept of message queues,
senders, and receivers

19. © 2012 Fiserv, Inc. or its affiliates.
Publish – Subscribe Channel
How can the sender broadcast a message to all
interested receivers?
In JMS publish – subscribe ,subscribers may register interest in
receiving messages on a particular message topic. In this model,
neither the publisher nor the subscriber knows about each other.

20. 20
Message Routing

21. © 2012 Fiserv, Inc. or its affiliates.
Web Banking Page
21
• Partitioning
•HTTP Caching
•RDBMS Sharding
•NOSQL
• Distributed Caching
•Data Grids
•Concurrency
Scalability Patterns: State

22. © 2012 Fiserv, Inc. or its affiliates.
Advance Confirmation Page
22
• HTTP Caching
• Reverse Proxy
• • Varnish
• • Squid
• • rack-cache
• • Pound
• • Nginx
• • Apache mod_proxy
• • Traffic Server

23. © 2012 Fiserv, Inc. or its affiliates.
Successful Creation of RA
23
• Generate Static Content
• Precompute content
• • Homegrown + cron or Quartz
• • Spring Batch
• • Gearman
• • Hadoop
• • Google Data Protocol
• • Amazon Elastic MapReduce

24. © 2012 Fiserv, Inc. or its affiliates.24

25. © 2012 Fiserv, Inc. or its affiliates.25
HTTP Caching
Subsequent request

26. © 2012 Fiserv, Inc. or its affiliates.
How to scale out RDBMS?
26
• Sharding
• •Partitioning
• •Replication

27. © 2012 Fiserv, Inc. or its affiliates.
Sharding: Partitioning
27

28. © 2012 Fiserv, Inc. or its affiliates.
NOSQL in the wild
28
•Google: Bigtable
• Amazon: Dynamo
• Amazon: SimpleDB
• Yahoo: HBase
• Facebook: Cassandra
• LinkedIn: Voldemort
NOSQL in the wild

29. © 2012 Fiserv, Inc. or its affiliates.
Sharding: Partitioning
29
• Relational DBs (MySQL, Oracle,
Postgres)
• Object DBs (Gemstone, db4o)
• Clustering products (Coherence,
Terracotta)
• Most caching products (ehcache)

30. © 2012 Fiserv, Inc. or its affiliates.
Sharding: Partitioning
30
• Types of NOSQL stores
• • Key-Value databases (Voldemort, Dynomite)
• • Column databases (Cassandra, Vertica, Sybase IQ)
• • Document databases (MongoDB, CouchDB)
• • Graph databases (Neo4J, AllegroGraph)
• • Datastructure databases (Redis, Hazelcast)

31. © 2012 Fiserv, Inc. or its affiliates.
Distributed Caching
31
• •Write-through
• •Write-behind
• • Eviction Policies
• • Replication
• • Peer-To-Peer (P2P)
• Distributed Caching

32. © 2012 Fiserv, Inc. or its affiliates.
Distributed Caching
32
• Pain of Duplication
• • How do I get failover capability while avoiding
• excessive duplication of data?

33. © 2012 Fiserv, Inc. or its affiliates.
Distributed Caching
33

34. © 2012 Fiserv, Inc. or its affiliates.
References
34
• http://www.theserverside.com/news/1320914/Scaling-Your-Java-EE-
Applications-Part-2