1.
The Next Generation Application Server – How Event Based Processing yields scalability Guy Korland R&D Team Leader

2.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

3.
About me… <ul><li>Core Team Leader – GigaSpaces since 2005 </li></ul><ul><li>MSc – Technion (Prof. Roy Friedman) </li></ul><ul><li>PhD candidate – Tel Aviv University (Prof. Nir Shavit) </li></ul><ul><li>Lead Deuce STM – ( www.deucestm.org ) </li></ul><ul><li>Java Software Transactional Memory </li></ul>

4.
GigaSpaces XAP – Designed For: Performance Scalability Latency

5.
GigaSpaces Evolution Single space Load Balance Partition & Replication SLA container Event Container NG Application Server PaaS Cloud 2000 2003 2005 2006 2007 2008 2009

6.
Not going to talk about… <ul><li>Jini (Java SOA) </li></ul><ul><li>Data Grid implementation. </li></ul><ul><li>Map-Reduce. </li></ul><ul><li>JDBC/JMS/JPA. </li></ul><ul><li>Cloud computing. </li></ul><ul><li>Batch processing. </li></ul><ul><li>Mule ESB. </li></ul><ul><li>WAN vs LAN </li></ul><ul><li>Different languages interoperability. </li></ul><ul><li>TupleSpace model extension. </li></ul><ul><li>JDK improvements (RMI, Reflection, Serialization, Classloading…) </li></ul>

7.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

8.
Today’s Reality – Tier Based Architecture Separate technology implementation Bottlenecks in all areas where state is stored, architecture can’t scale linearly! Separate technology implementation Separate technology implementation bottlenecks bottlenecks

9.
Traditional Architecture - path to complexity… (marktplaats.nl) Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service Auction Service Bid Service Trade Service Info Service Timer Service T Bidder Validate Result Process Bid Bid Accepted Bid Result Process Trade Get Bid Result Place bid

10.
Traditional Architecture - path to complexity… Business tier Back-up Back-up <ul><li>Redundancy doubles network traffic </li></ul><ul><li>Bottlenecks are created </li></ul><ul><li>Latency is increased </li></ul><ul><li>Separate failover strategy and implementation for each tier </li></ul>Bidder Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service

11.
Do you see the Problem? Business tier <ul><li>Scalability is not linear </li></ul><ul><li>Scalability management nightmare </li></ul>Back-up Back-up Back-up Back-up Bidder Auction Owner

12.
There is a huge gap between peak and average loads

13.
Bottlenecks, Performance, Scalability and High availability headaches Bad Publicity Revenue Loss Customer Dissatisfaction Regulatory Penalties

15.
TBA – Summary <ul><li>• Historically the following has been done… </li></ul><ul><li>– Tune, tune and tune configuration and code </li></ul><ul><li>• Once a bottleneck has been resolved, the next one glooms </li></ul><ul><li>– Hardware over provision </li></ul><ul><li>• To make sure that at peak times the response times were still acceptable </li></ul><ul><li>– Hardware upgrades </li></ul><ul><li>• To get rid of bottlenecks, whose origin was impossible to track down </li></ul><ul><li>– Alternative patterns </li></ul><ul><li>• Avoiding 2-phase-commit, using patterns like ‘compensating transactions’ </li></ul><ul><li>• Using Active/Passive failover, to make the response times faster, risking and </li></ul><ul><li>in fact accepting potential data-loss </li></ul><ul><li>• Partition the database, but not for size-reasons </li></ul>

16.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

17.
Event Containers

18.
Based on JavaSpaces C++

19.
The Master Worker Pattern

20.
GigaSpaces - based on Shared Transactional Memory Read Write <ul><ul><li>Write – writes a data object </li></ul></ul><ul><ul><li>Read – reads a copy of a data object </li></ul></ul><ul><ul><li>Take – reads a data object and deletes it </li></ul></ul><ul><ul><li>Notify – generates an event on data updates </li></ul></ul>Write Take Notify <ul><ul><li>Write + Read  Data Caching </li></ul></ul><ul><ul><li>Write + Notify  Messaging - Pub/Sub </li></ul></ul><ul><ul><li>Write + Take  Master Worker </li></ul></ul>

21.
Event Containers

22.
Step 1 – Create a Processing Unit Business tier Processing Unit <ul><li>Single model for design, deployment and management </li></ul><ul><li>No integration effort </li></ul><ul><li>Manage data in memory </li></ul><ul><li>Collapse the tiers </li></ul><ul><li>Collocate the services </li></ul>Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner

23.
Step 2 – Async Persistency Processing Unit Validate Process Bid Process Trade Process Results Persist for Compliance & Reporting purposes: <ul><ul><ul><ul><li>- Storing State - Register Orders - etc. </li></ul></ul></ul></ul><ul><li>Collocation of data, messaging and services in memory: </li></ul><ul><ul><li>Minimum Latency (no network hops) </li></ul></ul><ul><ul><li>Maximum Throughput </li></ul></ul>Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner Place Bid Get Bid Results

24.
Step 3 – Resiliency Processing Unit <ul><li>Single, built-in failover/redundancy investment strategy </li></ul><ul><li>Fewer points of failure </li></ul><ul><li>Automated SLA driven failover/redundancy mechanism </li></ul><ul><li>Continuous High Availability </li></ul>SLA Driven Container Backup

25.
Step 3 – Resiliency Processing Unit <ul><li>Automated SLA driven failover/redundancy mechanism </li></ul><ul><li>Continuous Availability </li></ul><ul><li>Self Healing Capability </li></ul>SLA Driven Container Backup <ul><li>Single, built-in failover/redundancy investment strategy </li></ul><ul><li>Fewer integration points mean fewer chances for failure </li></ul>Primary Backup

26.
Step 4 – Scale Processing Unit <ul><li>Write Once Scale Anywhere: </li></ul><ul><ul><li>Linear scalability </li></ul></ul><ul><ul><li>Single monitoring and management engine </li></ul></ul><ul><ul><li>Automated , SLA-Driven deployment and management </li></ul></ul><ul><ul><ul><li>Scaling policy, System requirements, Space cluster topology </li></ul></ul></ul>Backup Backup

27.
Event Containers

28.
Step 5 – Auto Scale Out

29.
Processing Unit – Scalability Unit Single Processing Unit Processing Unit - Scaled Involves Config Change No code changes!

30.
Processing Unit – High-Availability Unit Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode

31.
Database Integration - Async persistency Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode Mirror Process ORM Initial Load Async Replication Async Replication

32.
XAP = Enterprise Grade Middleware <ul><li>Scale-out application server </li></ul><ul><ul><li>End 2 End scale-out middleware for: Web, Data, Messaging, Business logic </li></ul></ul><ul><ul><li>Space Based Architecture – designed for scaling stateful applications In-memory </li></ul></ul><ul><li>Proven performance, Scalability, Low latency, Reliability </li></ul><ul><li>SLA Driven </li></ul><ul><li>Unique database scaling solution that fits cloud environment </li></ul><ul><ul><li>In Memory Data Grid </li></ul></ul><ul><ul><li>O/R mapping support </li></ul></ul><ul><li>Support major Enterprise languages </li></ul><ul><ul><li>Java, .Net, C++ </li></ul></ul>

33.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

34.
Built-in Event Containers <ul><li>Polling Container </li></ul><ul><li>Notify Container </li></ul>

35.
Polling Container <ul><li>Used for point-to-point messaging </li></ul><ul><li>Container polls the Space </li></ul><ul><li>for events </li></ul><ul><li>Comparable with the </li></ul><ul><li>way Ajax works </li></ul>

36.
Notify Container <ul><li>Used for publish-subscribe messaging </li></ul><ul><li>Space notifies the container </li></ul>

37.
Typical Application

38.
Service Grid Summary <ul><li>Powerful Universal Container </li></ul><ul><ul><li>Java/Net/C++ </li></ul></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>Fault Tolerant </li></ul></ul><ul><ul><li>Object based </li></ul></ul><ul><ul><li>Transactional </li></ul></ul><ul><ul><li>Publish/Subscribe </li></ul></ul>

39.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

40.
Event Containers

41.
The POJO Based Data Domain Model @SpaceClass(fifo=true) public class Data { … @SpaceId ( autoGenerate = true ) public String getId () { return id; } public String setId ( String id ) { this .id = id; } public void setProcessed ( boolean processed ) { this . processed = processed; } public boolean isProcessed ( boolean processed ) { return this . processed; } } SpaceClass indicate that this is a SpaceEntry – SpaceClass includes classlevel attributes such as FIFO,Persistent… SpaceId used to define the key for that entry.

42.
Data Processor Service Bean @SpaceDataEvent to be called when an event is triggered. public class DataProcessor{ @SpaceDataEvent public Data processData ( Data data ) { … data . setProcessed ( true ) ; //updates the space return data; } } Updates the data in the Space.

43.
Wiring Order Processor Service Bean through Spring <bean id=&quot; dataProcessor “ class=&quot;com.gigaspaces.pu.example1.processor.DataProcessor&quot; /> <os-events:polling-container id=&quot;dataProcessorPollingEventContainer&quot; giga-space=&quot;gigaSpace&quot;> <os-events:tx-support tx-manager=&quot;transactionManager&quot;/> <os-core:template> <bean class=&quot;org.openspaces.example.data.common.Data&quot;> <property name=&quot;processed&quot; value=&quot;false&quot;/> </bean> </os-core:template> <os-events:listener> <os-events:annotation-adapter> <os-events:delegate ref=&quot; dataProcessor &quot;/> </os-events:annotation-adapter> </os-events:listener> </os-events:polling-container> The event Template The event Listener

44.
Data Feeder public class DataFeeder { public void feed(){ Data data = new Data(counter++); data.setProcessed( false ); //feed data gigaSpace.write(data); } } Feed Data

45.
Remoting – Taking one step forward Event

46.
Remoting – Taking one step forward Reduce

47.
Remoting – IDataProcessor Service API public interface IDataProcessor { // Process a given Data object Data processData(Data data); }

48.
Remoting - DataProcessor Service @RemotingService public class DataProcessor implements IDataProcessor { public Data processData(Data data) { … data.setProcessed( true ); return data; } }

49.
Remoting - Order Feeder public class DataFeeder { private IDataProcessor dataProcessor; public void setDataProcessor(…) { this .dataProcessor = dataProcessor; } public Data feed(){ Data data = new Data(counter++); // remoting call return dataProcessor.process (data) } }

50.
Summary

51.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

52.
Scale up Throughput Benchmark – Physical Deployment Topology <ul><li>Remote (multiple machines , multiple processes) </li></ul>white box Client X4450 GigaSpaces 4 spaces , one per GSC X4450 GigaSpaces 4 spaces , one per GSC Switched Ethernet LAN Embedded (one machine , one process) X4450 Client GigaSpaces 8 spaces

53.
Scale up Throughput Benchmark – Embedded mode 1.8 Million read sec! 1.1 Million write/take sec!

54.
Scale up Throughput Benchmark – Remote mode 90,00 read sec! 45,00 write/take sec!

55.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

56.
Event Containers

57.
<ul><li>Web Container </li></ul><ul><li>Grid </li></ul>

58.
Web application – Pet Clinic

59.
Classic Architecture – Step 1- Request Submission Get request and invoke Service Data Grid

60.
Classic Architecture – Step 2- Retrieve Results Page Generation Data Grid

61.
Web Application Benchmark Results - Capacity

62.
Web Application Benchmark Results - Capacity

63.
<ul><li>Game Server </li></ul>

64.
Query Notify Intercepts update events Game Servers Table Feeder <ul><li>Loading Game Tables into the partitioned spaces </li></ul><ul><li>Randomly updates the game tables </li></ul>Publisher (lobby) <ul><li>Game Table Directory </li></ul><ul><li>Game Table search </li></ul><ul><li>Player search </li></ul>Partitioned Space Pub/Sub messaging Scaling out GameTable Space Based Architecture – Game Server Publisher (II)

65.
Publisher Servers Game Servers GigaSpaces Service Grid <ul><li>Uploading 30,000 players for 6000 tables </li></ul><ul><li>Randomly updates game tables </li></ul>Java runtime Physical backup <ul><li>Running continues query per user </li></ul>Space Based Architecture – Game Server Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable Notify / Query Notify / Query Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable

66.
Dynamic repartitioning and load sharing I SLA Driven Container Indexed Notify / Query template Notify / Query template Partitioned Space Partitioned Space Partitioned Space

67.
Dynamic repartitioning and load sharing II SLA Driven Container SLA Driven Container Partitioned Space Partitioned Space Partitioned Space

68.
Scaling Throughput: ~6K/sec Throughput: ~12K/sec Throughput: ~18K/sec Partitioned Space Backup Space SLA Driven Container <ul><li>6000 tables </li></ul><ul><li>30,000 players </li></ul>SLA Driven Container Partitioned Space Backup Space <ul><li>4000 tables </li></ul><ul><li>20,000 players </li></ul>SLA Driven Container Partitioned Space Backup Space <ul><li>2000 tables </li></ul><ul><li>10,000 players </li></ul>

69.
Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>

70.
Thank You! Q&A

71.
Appendix

72.
SLA Driven Deployment <ul><li>SLA: </li></ul><ul><li>Failover policy </li></ul><ul><li>Scaling policy </li></ul><ul><li>Ststem requirements </li></ul><ul><li>Space cluster topology </li></ul><ul><li>PU Services beans definition </li></ul>

73.
Continuous High Availability Fail-Over Failure

74.
Dynamic Partitioning = Dynamic Capacity Growth VM 1 ,2G GSC VM 3 , 2G GSC VM 2 ,2G GSC Max Capacity=2G Max Capacity=4G Max Capacity=6G In some point VM 1 free memory is below 20 % - it about the time to increase the capacity – lets move Partitions 1 to another GSC and recover the data from the running backup! Later .. Partition 2 needs to move… After the move , data is recovered from the backup VM 5 , 4G GSC VM 4 ,4G GSC P - Primary B - Backup P P P B B B E F Partition 1 A B Partition 2 C D Partition 3 A B Partition 2 E F Partition 1 C D Partition 3

75.
<ul><li>Executors </li></ul>

76.
Task Executors – Task Execution <ul><li>Executing a task is done using the execute method </li></ul>AsyncFuture<Integer> future = gigaSpace .execute( new MyTask(2) ); int result = future.get(); 1 2 4 3

77.
Task Executors – Task Routing <ul><li>Routing a task can be done in three ways </li></ul><ul><ul><li>1. Using the task itself </li></ul></ul><ul><ul><li>2. Passing a POJO to the execute method </li></ul></ul><ul><ul><li>3. Specifying a routing-parameter in the execute method </li></ul></ul>

78.
Task Executors – DistributedTask Execution <ul><li>Executing a distributed task is done using the execute method </li></ul>AsyncFuture<Integer> future = gigaSpace .execute( new MyDistTask() ); int result = future.get();

79.
Task Executors – DistributedTask Routing <ul><li>Routing a distributed task can be done </li></ul><ul><ul><li>1. In the same ways as with the plain Task interface </li></ul></ul><ul><ul><li>2. By broadcasting </li></ul></ul><ul><ul><li>3. Specifying a number of routing-parameters in the execute method </li></ul></ul>

80.
Service Executors

81.
Service Executors

82.
<ul><li>IMDG </li></ul><ul><li>Operations </li></ul>

83.
IMDG Basic Operations

84.
IMDG Access – Space Operations – Write Operation <ul><li>write -operation writes a new object to a space </li></ul><ul><ul><li>Instantiate an object </li></ul></ul><ul><ul><li>Set fields as necessary </li></ul></ul><ul><ul><li>Write the object to the space </li></ul></ul>Auction auction = new Auction(); auction.setType( &quot;Bicycle&quot; ); gigaSpace.write(auction);

85.
IMDG Access – Space Operations – Read Operation <ul><li>read -operation reads an object from a space </li></ul><ul><li>A copy of the object is returned </li></ul><ul><li>The original copy remains in the space </li></ul><ul><ul><li>Build a template/query (more on this later) </li></ul></ul><ul><ul><li>Read a matching object from the space </li></ul></ul>Auction template = new Auction(); Auction returnedAuction = gigaSpace.read(template);

86.
Object SQL Query Support <ul><li>Supported Options and Queries </li></ul><ul><ul><li>Opeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN. </li></ul></ul><ul><ul><li>GROUP BY – performs DISTINCT on the POJO properties </li></ul></ul><ul><ul><li>Order By (ASC | DESC) </li></ul></ul><ul><li>SQLQuery rquery = new SQLQuery(MyPojo.class,&quot;firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'&quot;); </li></ul><ul><li>Object[] result = space.readMultiple(rquery); </li></ul><ul><li>Dynamic Query Support </li></ul><ul><ul><li>SQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?&quot;); </li></ul></ul><ul><ul><li>query.setParameters(“david”,”lee”,50); </li></ul></ul><ul><li>Supported Options via JDBC API </li></ul><ul><ul><li>COUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases </li></ul></ul><ul><ul><li>Join with 2 tables </li></ul></ul><ul><li>Non Supported </li></ul><ul><ul><li>HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON. </li></ul></ul><ul><ul><li>NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK. </li></ul></ul><ul><ul><li>Union, Minus, Union All. </li></ul></ul><ul><ul><li>STDEV, STDEVP, VAR, VARP, FIRST, LAST. </li></ul></ul><ul><ul><li># LEFT , RIGHT [INNER] or [OUTER] JOIN </li></ul></ul>

87.
IMDG Access – Space Operations – Take Operation <ul><li>take -operation takes an object from a space </li></ul><ul><li>The matched object is removed from the space </li></ul><ul><ul><li>Build a template/query (more on this later) </li></ul></ul><ul><ul><li>Take a matching object from the space </li></ul></ul>Auction template = new Auction(); Auction removedAuction = gigaSpace.take(template);

88.
IMDG Access – Space Operations – Update Operation <ul><li>update is equivalent to performing take and write </li></ul><ul><li>Executed in a single atomic call </li></ul>AuctionItem item = new AuctionItem(); item.setType( &quot;Bicycle&quot; ); gigaSpace.write(item); item = gigaSpace.read(item); item.setType( &quot;Motorbike&quot; ); Object returnedObject = space.update(item, null , Lease.Forever, 2000L, UpdateModifiers.UPDATE_OR_WRITE);

89.
IMDG Access – Space Operations – Batch API <ul><li>Apart from the single methods GigaSpaces also provides batch methods </li></ul><ul><li>The methods are: </li></ul><ul><ul><li>writeMultiple : writes multiple objects </li></ul></ul><ul><ul><li>readMultiple : reads multiple objects </li></ul></ul><ul><ul><li>updateMultiple : updates multiple objects </li></ul></ul><ul><ul><li>takeMultiple : reads multiple objects and deletes them </li></ul></ul><ul><li>Notes: </li></ul><ul><ul><li>Performance of the batch operations is generally higher </li></ul></ul><ul><ul><ul><li>Requires one call to the space </li></ul></ul></ul><ul><ul><li>Can be used with Template matching or SQLQuery </li></ul></ul>

90.
IMDG Access – Space Operations – Batch API <ul><li>writeMultiple writes the specified objects to the space. </li></ul>Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.writeMultiple(auctions, 100);

91.
IMDG Access – Space Operations – Batch API <ul><li>readMultiple reads all the objects matching the specified template from the space. </li></ul>Auction auction = new Auction(); Auction[] auctions = gigaSpace.readMultiple(auction, 100);

92.
IMDG Access – Space Operations – Batch API <ul><li>takeMultiple takes all the objects matching the specified template from the space. </li></ul>Auction auction = new Auction(); Auction[] auctions = gigaSpace.takeMultiple(auction, 100);

93.
IMDG Access – Space Operations – Batch API <ul><li>updateMultiple updates a group of specified objects. </li></ul>Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.updateMultiple(auctions, 100);

94.
IMDG Summary <ul><li>Powerful shared memory Service </li></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>Fault Tolerant </li></ul></ul><ul><ul><li>Object based </li></ul></ul><ul><ul><li>Single and Batch Operations </li></ul></ul><ul><ul><li>Transactional </li></ul></ul>