Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
The Next Generation  Application Server –  How Event Based Processing yields scalability  Guy Korland R&D Team Leader
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
About me… <ul><li>Core Team Leader – GigaSpaces since 2005  </li></ul><ul><li>MSc – Technion (Prof. Roy Friedman)  </li></...
GigaSpaces XAP – Designed For: Performance Scalability Latency
GigaSpaces Evolution  Single space Load  Balance Partition & Replication SLA container Event Container NG Application Serv...
Not going to talk about… <ul><li>Jini (Java SOA) </li></ul><ul><li>Data Grid implementation. </li></ul><ul><li>Map-Reduce....
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Today’s Reality – Tier Based Architecture Separate technology  implementation  Bottlenecks in all areas where state is sto...
Traditional Architecture - path to complexity… (marktplaats.nl) Auction Owner Auction Service Bid Service Trade Service In...
Traditional Architecture - path to complexity… Business tier Back-up Back-up <ul><li>Redundancy doubles network traffic </...
Do you see the Problem? Business tier <ul><li>Scalability is not linear </li></ul><ul><li>Scalability management nightmare...
There is a huge gap between peak and average loads
Bottlenecks, Performance, Scalability and High availability headaches Bad Publicity Revenue Loss Customer Dissatisfaction ...
 
TBA – Summary <ul><li>•  Historically the following has been done… </li></ul><ul><li>–  Tune, tune and tune configuration ...
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Event   Containers
Based on JavaSpaces C++
The Master Worker Pattern
GigaSpaces - based on Shared Transactional Memory Read Write <ul><ul><li>Write  – writes a data object </li></ul></ul><ul>...
Event   Containers
Step 1 – Create a Processing Unit Business tier Processing Unit <ul><li>Single model for design, deployment and management...
Step 2 – Async Persistency Processing Unit Validate Process Bid Process   Trade Process   Results Persist for Compliance &...
Step 3 – Resiliency Processing Unit <ul><li>Single, built-in failover/redundancy investment strategy </li></ul><ul><li>Few...
Step 3 – Resiliency  Processing Unit <ul><li>Automated SLA driven failover/redundancy mechanism </li></ul><ul><li>Continuo...
Step 4 – Scale Processing Unit <ul><li>Write Once Scale Anywhere: </li></ul><ul><ul><li>Linear scalability </li></ul></ul>...
Event   Containers
Step 5 – Auto Scale Out
Processing Unit – Scalability Unit Single Processing Unit Processing Unit - Scaled Involves  Config Change No code changes!
Processing Unit – High-Availability Unit Sync Replication Primary - Processing Unit Business logic – Active mode Backup - ...
Database Integration - Async persistency Sync Replication Primary - Processing Unit Business logic – Active mode Backup - ...
XAP = Enterprise Grade Middleware <ul><li>Scale-out application server </li></ul><ul><ul><li>End 2 End scale-out middlewar...
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Built-in Event Containers <ul><li>Polling Container </li></ul><ul><li>Notify Container </li></ul>
Polling Container <ul><li>Used for point-to-point messaging </li></ul><ul><li>Container polls the Space </li></ul><ul><li>...
Notify Container <ul><li>Used for publish-subscribe messaging </li></ul><ul><li>Space notifies the container   </li></ul>
Typical Application
Service Grid Summary <ul><li>Powerful Universal Container </li></ul><ul><ul><li>Java/Net/C++ </li></ul></ul><ul><ul><li>Di...
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Event   Containers
The POJO Based Data Domain Model @SpaceClass(fifo=true) public class  Data { … @SpaceId ( autoGenerate  =  true ) public S...
Data Processor Service Bean @SpaceDataEvent   to be called when an event is triggered.   public   class  DataProcessor{ @S...
Wiring Order Processor Service Bean through Spring <bean id=&quot; dataProcessor “ class=&quot;com.gigaspaces.pu.example1....
Data Feeder public class  DataFeeder {  public void  feed(){ Data data =  new  Data(counter++);  data.setProcessed( false ...
Remoting – Taking one step forward Event
Remoting – Taking one step forward Reduce
Remoting – IDataProcessor Service API public interface  IDataProcessor   {   // Process a given Data object Data processDa...
Remoting - DataProcessor  Service @RemotingService  public class  DataProcessor   implements  IDataProcessor {   public  D...
Remoting - Order Feeder public class  DataFeeder {  private  IDataProcessor dataProcessor;  public void  setDataProcessor(...
Summary
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Scale up Throughput Benchmark – Physical Deployment Topology <ul><li>Remote (multiple machines , multiple processes) </li>...
Scale up Throughput Benchmark – Embedded mode 1.8 Million read sec! 1.1 Million   write/take sec!
Scale up Throughput Benchmark – Remote mode 90,00 read sec! 45,00 write/take sec!
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Event   Containers
<ul><li>Web Container  </li></ul><ul><li>Grid </li></ul>
Web application – Pet Clinic
Classic Architecture – Step 1- Request Submission Get request and invoke Service Data Grid
Classic Architecture – Step 2- Retrieve Results Page Generation Data Grid
Web Application Benchmark Results - Capacity
Web Application Benchmark Results - Capacity
<ul><li>Game Server </li></ul>
Query Notify Intercepts update events Game Servers Table Feeder <ul><li>Loading Game Tables into the partitioned spaces </...
Publisher Servers Game Servers GigaSpaces Service Grid <ul><li>Uploading 30,000 players for 6000 tables </li></ul><ul><li>...
Dynamic repartitioning and load sharing I SLA Driven Container Indexed  Notify / Query template Notify / Query template Pa...
Dynamic repartitioning and load sharing II SLA Driven Container SLA Driven Container Partitioned Space Partitioned Space P...
Scaling Throughput: ~6K/sec Throughput: ~12K/sec Throughput: ~18K/sec Partitioned Space Backup Space SLA Driven Container ...
Agenda <ul><li>Preview  </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </...
Thank You! Q&A
Appendix
SLA Driven Deployment <ul><li>SLA: </li></ul><ul><li>Failover policy </li></ul><ul><li>Scaling policy </li></ul><ul><li>St...
Continuous High Availability Fail-Over Failure
Dynamic Partitioning  =  Dynamic Capacity Growth VM 1 ,2G GSC VM 3 , 2G GSC VM 2 ,2G GSC Max Capacity=2G Max Capacity=4G M...
<ul><li>Executors </li></ul>
Task Executors – Task Execution <ul><li>Executing a task is done using the  execute  method </li></ul>AsyncFuture<Integer>...
Task Executors – Task Routing <ul><li>Routing a task can be done in three ways </li></ul><ul><ul><li>1. Using the task its...
Task Executors – DistributedTask Execution <ul><li>Executing a distributed task is done using the  execute  method </li></...
Task Executors – DistributedTask Routing <ul><li>Routing a distributed task can be done </li></ul><ul><ul><li>1. In the sa...
Service Executors
Service Executors
<ul><li>IMDG </li></ul><ul><li>Operations </li></ul>
IMDG Basic Operations
IMDG Access – Space Operations – Write Operation <ul><li>write -operation writes a new object to a space </li></ul><ul><ul...
IMDG Access – Space Operations – Read Operation <ul><li>read -operation reads an object from a space </li></ul><ul><li>A  ...
Object SQL Query Support <ul><li>Supported Options and Queries </li></ul><ul><ul><li>Opeations: =, <>, <,>, >=, <=, [NOT] ...
IMDG Access – Space Operations – Take Operation <ul><li>take -operation takes an object from a space </li></ul><ul><li>The...
IMDG Access – Space Operations – Update Operation <ul><li>update  is equivalent to performing take and write </li></ul><ul...
IMDG Access – Space Operations – Batch API <ul><li>Apart from the single methods GigaSpaces also provides batch methods </...
IMDG Access – Space Operations – Batch API <ul><li>writeMultiple   writes the specified objects to the space. </li></ul>Au...
IMDG Access – Space Operations – Batch API <ul><li>readMultiple  reads all the objects matching the specified template fro...
IMDG Access – Space Operations – Batch API <ul><li>takeMultiple  takes all the objects matching the specified template fro...
IMDG Access – Space Operations – Batch API <ul><li>updateMultiple  updates a group of specified objects. </li></ul>Auction...
IMDG Summary <ul><li>Powerful shared memory Service </li></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>Fault Tol...
Prochain SlideShare
Chargement dans…5
×

The Next Generation Application Server – How Event Based Processing yields scalability

GigaSapces on CANOE'09

  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

The Next Generation Application Server – How Event Based Processing yields scalability

  1. 1. The Next Generation Application Server – How Event Based Processing yields scalability Guy Korland R&D Team Leader
  2. 2. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  3. 3. About me… <ul><li>Core Team Leader – GigaSpaces since 2005 </li></ul><ul><li>MSc – Technion (Prof. Roy Friedman) </li></ul><ul><li>PhD candidate – Tel Aviv University (Prof. Nir Shavit) </li></ul><ul><li>Lead Deuce STM – ( www.deucestm.org ) </li></ul><ul><li>Java Software Transactional Memory </li></ul>
  4. 4. GigaSpaces XAP – Designed For: Performance Scalability Latency
  5. 5. GigaSpaces Evolution Single space Load Balance Partition & Replication SLA container Event Container NG Application Server PaaS Cloud 2000 2003 2005 2006 2007 2008 2009
  6. 6. Not going to talk about… <ul><li>Jini (Java SOA) </li></ul><ul><li>Data Grid implementation. </li></ul><ul><li>Map-Reduce. </li></ul><ul><li>JDBC/JMS/JPA. </li></ul><ul><li>Cloud computing. </li></ul><ul><li>Batch processing. </li></ul><ul><li>Mule ESB. </li></ul><ul><li>WAN vs LAN </li></ul><ul><li>Different languages interoperability. </li></ul><ul><li>TupleSpace model extension. </li></ul><ul><li>JDK improvements (RMI, Reflection, Serialization, Classloading…) </li></ul>
  7. 7. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  8. 8. Today’s Reality – Tier Based Architecture Separate technology implementation Bottlenecks in all areas where state is stored, architecture can’t scale linearly! Separate technology implementation Separate technology implementation bottlenecks bottlenecks
  9. 9. Traditional Architecture - path to complexity… (marktplaats.nl) Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service Auction Service Bid Service Trade Service Info Service Timer Service T Bidder Validate Result Process Bid Bid Accepted Bid Result Process Trade Get Bid Result Place bid
  10. 10. Traditional Architecture - path to complexity… Business tier Back-up Back-up <ul><li>Redundancy doubles network traffic </li></ul><ul><li>Bottlenecks are created </li></ul><ul><li>Latency is increased </li></ul><ul><li>Separate failover strategy and implementation for each tier </li></ul>Bidder Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service
  11. 11. Do you see the Problem? Business tier <ul><li>Scalability is not linear </li></ul><ul><li>Scalability management nightmare </li></ul>Back-up Back-up Back-up Back-up Bidder Auction Owner
  12. 12. There is a huge gap between peak and average loads
  13. 13. Bottlenecks, Performance, Scalability and High availability headaches Bad Publicity Revenue Loss Customer Dissatisfaction Regulatory Penalties
  14. 15. TBA – Summary <ul><li>• Historically the following has been done… </li></ul><ul><li>– Tune, tune and tune configuration and code </li></ul><ul><li>• Once a bottleneck has been resolved, the next one glooms </li></ul><ul><li>– Hardware over provision </li></ul><ul><li>• To make sure that at peak times the response times were still acceptable </li></ul><ul><li>– Hardware upgrades </li></ul><ul><li>• To get rid of bottlenecks, whose origin was impossible to track down </li></ul><ul><li>– Alternative patterns </li></ul><ul><li>• Avoiding 2-phase-commit, using patterns like ‘compensating transactions’ </li></ul><ul><li>• Using Active/Passive failover, to make the response times faster, risking and </li></ul><ul><li>in fact accepting potential data-loss </li></ul><ul><li>• Partition the database, but not for size-reasons </li></ul>
  15. 16. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  16. 17. Event Containers
  17. 18. Based on JavaSpaces C++
  18. 19. The Master Worker Pattern
  19. 20. GigaSpaces - based on Shared Transactional Memory Read Write <ul><ul><li>Write – writes a data object </li></ul></ul><ul><ul><li>Read – reads a copy of a data object </li></ul></ul><ul><ul><li>Take – reads a data object and deletes it </li></ul></ul><ul><ul><li>Notify – generates an event on data updates </li></ul></ul>Write Take Notify <ul><ul><li>Write + Read  Data Caching </li></ul></ul><ul><ul><li>Write + Notify  Messaging - Pub/Sub </li></ul></ul><ul><ul><li>Write + Take  Master Worker </li></ul></ul>
  20. 21. Event Containers
  21. 22. Step 1 – Create a Processing Unit Business tier Processing Unit <ul><li>Single model for design, deployment and management </li></ul><ul><li>No integration effort </li></ul><ul><li>Manage data in memory </li></ul><ul><li>Collapse the tiers </li></ul><ul><li>Collocate the services </li></ul>Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner
  22. 23. Step 2 – Async Persistency Processing Unit Validate Process Bid Process Trade Process Results Persist for Compliance & Reporting purposes: <ul><ul><ul><ul><li>- Storing State - Register Orders - etc. </li></ul></ul></ul></ul><ul><li>Collocation of data, messaging and services in memory: </li></ul><ul><ul><li>Minimum Latency (no network hops) </li></ul></ul><ul><ul><li>Maximum Throughput </li></ul></ul>Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner Place Bid Get Bid Results
  23. 24. Step 3 – Resiliency Processing Unit <ul><li>Single, built-in failover/redundancy investment strategy </li></ul><ul><li>Fewer points of failure </li></ul><ul><li>Automated SLA driven failover/redundancy mechanism </li></ul><ul><li>Continuous High Availability </li></ul>SLA Driven Container Backup
  24. 25. Step 3 – Resiliency Processing Unit <ul><li>Automated SLA driven failover/redundancy mechanism </li></ul><ul><li>Continuous Availability </li></ul><ul><li>Self Healing Capability </li></ul>SLA Driven Container Backup <ul><li>Single, built-in failover/redundancy investment strategy </li></ul><ul><li>Fewer integration points mean fewer chances for failure </li></ul>Primary Backup
  25. 26. Step 4 – Scale Processing Unit <ul><li>Write Once Scale Anywhere: </li></ul><ul><ul><li>Linear scalability </li></ul></ul><ul><ul><li>Single monitoring and management engine </li></ul></ul><ul><ul><li>Automated , SLA-Driven deployment and management </li></ul></ul><ul><ul><ul><li>Scaling policy, System requirements, Space cluster topology </li></ul></ul></ul>Backup Backup
  26. 27. Event Containers
  27. 28. Step 5 – Auto Scale Out
  28. 29. Processing Unit – Scalability Unit Single Processing Unit Processing Unit - Scaled Involves Config Change No code changes!
  29. 30. Processing Unit – High-Availability Unit Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode
  30. 31. Database Integration - Async persistency Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode Mirror Process ORM Initial Load Async Replication Async Replication
  31. 32. XAP = Enterprise Grade Middleware <ul><li>Scale-out application server </li></ul><ul><ul><li>End 2 End scale-out middleware for: Web, Data, Messaging, Business logic </li></ul></ul><ul><ul><li>Space Based Architecture – designed for scaling stateful applications In-memory </li></ul></ul><ul><li>Proven performance, Scalability, Low latency, Reliability </li></ul><ul><li>SLA Driven </li></ul><ul><li>Unique database scaling solution that fits cloud environment </li></ul><ul><ul><li>In Memory Data Grid </li></ul></ul><ul><ul><li>O/R mapping support </li></ul></ul><ul><li>Support major Enterprise languages </li></ul><ul><ul><li>Java, .Net, C++ </li></ul></ul>
  32. 33. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  33. 34. Built-in Event Containers <ul><li>Polling Container </li></ul><ul><li>Notify Container </li></ul>
  34. 35. Polling Container <ul><li>Used for point-to-point messaging </li></ul><ul><li>Container polls the Space </li></ul><ul><li>for events </li></ul><ul><li>Comparable with the </li></ul><ul><li>way Ajax works </li></ul>
  35. 36. Notify Container <ul><li>Used for publish-subscribe messaging </li></ul><ul><li>Space notifies the container </li></ul>
  36. 37. Typical Application
  37. 38. Service Grid Summary <ul><li>Powerful Universal Container </li></ul><ul><ul><li>Java/Net/C++ </li></ul></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>Fault Tolerant </li></ul></ul><ul><ul><li>Object based </li></ul></ul><ul><ul><li>Transactional </li></ul></ul><ul><ul><li>Publish/Subscribe </li></ul></ul>
  38. 39. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  39. 40. Event Containers
  40. 41. The POJO Based Data Domain Model @SpaceClass(fifo=true) public class Data { … @SpaceId ( autoGenerate = true ) public String getId () { return id; } public String setId ( String id ) { this .id = id; } public void setProcessed ( boolean processed ) { this . processed = processed; } public boolean isProcessed ( boolean processed ) { return this . processed; } } SpaceClass indicate that this is a SpaceEntry – SpaceClass includes classlevel attributes such as FIFO,Persistent… SpaceId used to define the key for that entry.
  41. 42. Data Processor Service Bean @SpaceDataEvent to be called when an event is triggered. public class DataProcessor{ @SpaceDataEvent public Data processData ( Data data ) { … data . setProcessed ( true ) ; //updates the space return data; } } Updates the data in the Space.
  42. 43. Wiring Order Processor Service Bean through Spring <bean id=&quot; dataProcessor “ class=&quot;com.gigaspaces.pu.example1.processor.DataProcessor&quot; /> <os-events:polling-container id=&quot;dataProcessorPollingEventContainer&quot; giga-space=&quot;gigaSpace&quot;> <os-events:tx-support tx-manager=&quot;transactionManager&quot;/> <os-core:template> <bean class=&quot;org.openspaces.example.data.common.Data&quot;> <property name=&quot;processed&quot; value=&quot;false&quot;/> </bean> </os-core:template> <os-events:listener> <os-events:annotation-adapter> <os-events:delegate ref=&quot; dataProcessor &quot;/> </os-events:annotation-adapter> </os-events:listener> </os-events:polling-container> The event Template The event Listener
  43. 44. Data Feeder public class DataFeeder { public void feed(){ Data data = new Data(counter++); data.setProcessed( false ); //feed data gigaSpace.write(data); } } Feed Data
  44. 45. Remoting – Taking one step forward Event
  45. 46. Remoting – Taking one step forward Reduce
  46. 47. Remoting – IDataProcessor Service API public interface IDataProcessor { // Process a given Data object Data processData(Data data); }
  47. 48. Remoting - DataProcessor Service @RemotingService public class DataProcessor implements IDataProcessor { public Data processData(Data data) { … data.setProcessed( true ); return data; } }
  48. 49. Remoting - Order Feeder public class DataFeeder { private IDataProcessor dataProcessor; public void setDataProcessor(…) { this .dataProcessor = dataProcessor; } public Data feed(){ Data data = new Data(counter++); // remoting call return dataProcessor.process (data) } }
  49. 50. Summary
  50. 51. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  51. 52. Scale up Throughput Benchmark – Physical Deployment Topology <ul><li>Remote (multiple machines , multiple processes) </li></ul>white box Client X4450 GigaSpaces 4 spaces , one per GSC X4450 GigaSpaces 4 spaces , one per GSC Switched Ethernet LAN Embedded (one machine , one process) X4450 Client GigaSpaces 8 spaces
  52. 53. Scale up Throughput Benchmark – Embedded mode 1.8 Million read sec! 1.1 Million write/take sec!
  53. 54. Scale up Throughput Benchmark – Remote mode 90,00 read sec! 45,00 write/take sec!
  54. 55. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  55. 56. Event Containers
  56. 57. <ul><li>Web Container </li></ul><ul><li>Grid </li></ul>
  57. 58. Web application – Pet Clinic
  58. 59. Classic Architecture – Step 1- Request Submission Get request and invoke Service Data Grid
  59. 60. Classic Architecture – Step 2- Retrieve Results Page Generation Data Grid
  60. 61. Web Application Benchmark Results - Capacity
  61. 62. Web Application Benchmark Results - Capacity
  62. 63. <ul><li>Game Server </li></ul>
  63. 64. Query Notify Intercepts update events Game Servers Table Feeder <ul><li>Loading Game Tables into the partitioned spaces </li></ul><ul><li>Randomly updates the game tables </li></ul>Publisher (lobby) <ul><li>Game Table Directory </li></ul><ul><li>Game Table search </li></ul><ul><li>Player search </li></ul>Partitioned Space Pub/Sub messaging Scaling out GameTable Space Based Architecture – Game Server Publisher (II)
  64. 65. Publisher Servers Game Servers GigaSpaces Service Grid <ul><li>Uploading 30,000 players for 6000 tables </li></ul><ul><li>Randomly updates game tables </li></ul>Java runtime Physical backup <ul><li>Running continues query per user </li></ul>Space Based Architecture – Game Server Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable Notify / Query Notify / Query Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable
  65. 66. Dynamic repartitioning and load sharing I SLA Driven Container Indexed Notify / Query template Notify / Query template Partitioned Space Partitioned Space Partitioned Space
  66. 67. Dynamic repartitioning and load sharing II SLA Driven Container SLA Driven Container Partitioned Space Partitioned Space Partitioned Space
  67. 68. Scaling Throughput: ~6K/sec Throughput: ~12K/sec Throughput: ~18K/sec Partitioned Space Backup Space SLA Driven Container <ul><li>6000 tables </li></ul><ul><li>30,000 players </li></ul>SLA Driven Container Partitioned Space Backup Space <ul><li>4000 tables </li></ul><ul><li>20,000 players </li></ul>SLA Driven Container Partitioned Space Backup Space <ul><li>2000 tables </li></ul><ul><li>10,000 players </li></ul>
  68. 69. Agenda <ul><li>Preview </li></ul><ul><li>The Problem </li></ul><ul><li>The Solution </li></ul><ul><li>Event Containers </li></ul><ul><li>Example </li></ul><ul><li>Benchmarks </li></ul><ul><li>Customer Use Cases </li></ul><ul><li>Summary Q&A </li></ul>
  69. 70. Thank You! Q&A
  70. 71. Appendix
  71. 72. SLA Driven Deployment <ul><li>SLA: </li></ul><ul><li>Failover policy </li></ul><ul><li>Scaling policy </li></ul><ul><li>Ststem requirements </li></ul><ul><li>Space cluster topology </li></ul><ul><li>PU Services beans definition </li></ul>
  72. 73. Continuous High Availability Fail-Over Failure
  73. 74. Dynamic Partitioning = Dynamic Capacity Growth VM 1 ,2G GSC VM 3 , 2G GSC VM 2 ,2G GSC Max Capacity=2G Max Capacity=4G Max Capacity=6G In some point VM 1 free memory is below 20 % - it about the time to increase the capacity – lets move Partitions 1 to another GSC and recover the data from the running backup! Later .. Partition 2 needs to move… After the move , data is recovered from the backup VM 5 , 4G GSC VM 4 ,4G GSC P - Primary B - Backup P P P B B B E F Partition 1 A B Partition 2 C D Partition 3 A B Partition 2 E F Partition 1 C D Partition 3
  74. 75. <ul><li>Executors </li></ul>
  75. 76. Task Executors – Task Execution <ul><li>Executing a task is done using the execute method </li></ul>AsyncFuture<Integer> future = gigaSpace .execute( new MyTask(2) ); int result = future.get(); 1 2 4 3
  76. 77. Task Executors – Task Routing <ul><li>Routing a task can be done in three ways </li></ul><ul><ul><li>1. Using the task itself </li></ul></ul><ul><ul><li>2. Passing a POJO to the execute method </li></ul></ul><ul><ul><li>3. Specifying a routing-parameter in the execute method </li></ul></ul>
  77. 78. Task Executors – DistributedTask Execution <ul><li>Executing a distributed task is done using the execute method </li></ul>AsyncFuture<Integer> future = gigaSpace .execute( new MyDistTask() ); int result = future.get();
  78. 79. Task Executors – DistributedTask Routing <ul><li>Routing a distributed task can be done </li></ul><ul><ul><li>1. In the same ways as with the plain Task interface </li></ul></ul><ul><ul><li>2. By broadcasting </li></ul></ul><ul><ul><li>3. Specifying a number of routing-parameters in the execute method </li></ul></ul>
  79. 80. Service Executors
  80. 81. Service Executors
  81. 82. <ul><li>IMDG </li></ul><ul><li>Operations </li></ul>
  82. 83. IMDG Basic Operations
  83. 84. IMDG Access – Space Operations – Write Operation <ul><li>write -operation writes a new object to a space </li></ul><ul><ul><li>Instantiate an object </li></ul></ul><ul><ul><li>Set fields as necessary </li></ul></ul><ul><ul><li>Write the object to the space </li></ul></ul>Auction auction = new Auction(); auction.setType( &quot;Bicycle&quot; ); gigaSpace.write(auction);
  84. 85. IMDG Access – Space Operations – Read Operation <ul><li>read -operation reads an object from a space </li></ul><ul><li>A copy of the object is returned </li></ul><ul><li>The original copy remains in the space </li></ul><ul><ul><li>Build a template/query (more on this later) </li></ul></ul><ul><ul><li>Read a matching object from the space </li></ul></ul>Auction template = new Auction(); Auction returnedAuction = gigaSpace.read(template);
  85. 86. Object SQL Query Support <ul><li>Supported Options and Queries </li></ul><ul><ul><li>Opeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN. </li></ul></ul><ul><ul><li>GROUP BY – performs DISTINCT on the POJO properties </li></ul></ul><ul><ul><li>Order By (ASC | DESC) </li></ul></ul><ul><li>SQLQuery rquery = new SQLQuery(MyPojo.class,&quot;firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'&quot;); </li></ul><ul><li>Object[] result = space.readMultiple(rquery); </li></ul><ul><li>Dynamic Query Support </li></ul><ul><ul><li>SQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?&quot;); </li></ul></ul><ul><ul><li>query.setParameters(“david”,”lee”,50); </li></ul></ul><ul><li>Supported Options via JDBC API </li></ul><ul><ul><li>COUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases </li></ul></ul><ul><ul><li>Join with 2 tables </li></ul></ul><ul><li>Non Supported </li></ul><ul><ul><li>HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON. </li></ul></ul><ul><ul><li>NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK. </li></ul></ul><ul><ul><li>Union, Minus, Union All. </li></ul></ul><ul><ul><li>STDEV, STDEVP, VAR, VARP, FIRST, LAST. </li></ul></ul><ul><ul><li># LEFT , RIGHT [INNER] or [OUTER] JOIN </li></ul></ul>
  86. 87. IMDG Access – Space Operations – Take Operation <ul><li>take -operation takes an object from a space </li></ul><ul><li>The matched object is removed from the space </li></ul><ul><ul><li>Build a template/query (more on this later) </li></ul></ul><ul><ul><li>Take a matching object from the space </li></ul></ul>Auction template = new Auction(); Auction removedAuction = gigaSpace.take(template);
  87. 88. IMDG Access – Space Operations – Update Operation <ul><li>update is equivalent to performing take and write </li></ul><ul><li>Executed in a single atomic call </li></ul>AuctionItem item = new AuctionItem(); item.setType( &quot;Bicycle&quot; ); gigaSpace.write(item); item = gigaSpace.read(item); item.setType( &quot;Motorbike&quot; ); Object returnedObject = space.update(item, null , Lease.Forever, 2000L, UpdateModifiers.UPDATE_OR_WRITE);
  88. 89. IMDG Access – Space Operations – Batch API <ul><li>Apart from the single methods GigaSpaces also provides batch methods </li></ul><ul><li>The methods are: </li></ul><ul><ul><li>writeMultiple : writes multiple objects </li></ul></ul><ul><ul><li>readMultiple : reads multiple objects </li></ul></ul><ul><ul><li>updateMultiple : updates multiple objects </li></ul></ul><ul><ul><li>takeMultiple : reads multiple objects and deletes them </li></ul></ul><ul><li>Notes: </li></ul><ul><ul><li>Performance of the batch operations is generally higher </li></ul></ul><ul><ul><ul><li>Requires one call to the space </li></ul></ul></ul><ul><ul><li>Can be used with Template matching or SQLQuery </li></ul></ul>
  89. 90. IMDG Access – Space Operations – Batch API <ul><li>writeMultiple writes the specified objects to the space. </li></ul>Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.writeMultiple(auctions, 100);
  90. 91. IMDG Access – Space Operations – Batch API <ul><li>readMultiple reads all the objects matching the specified template from the space. </li></ul>Auction auction = new Auction(); Auction[] auctions = gigaSpace.readMultiple(auction, 100);
  91. 92. IMDG Access – Space Operations – Batch API <ul><li>takeMultiple takes all the objects matching the specified template from the space. </li></ul>Auction auction = new Auction(); Auction[] auctions = gigaSpace.takeMultiple(auction, 100);
  92. 93. IMDG Access – Space Operations – Batch API <ul><li>updateMultiple updates a group of specified objects. </li></ul>Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.updateMultiple(auctions, 100);
  93. 94. IMDG Summary <ul><li>Powerful shared memory Service </li></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>Fault Tolerant </li></ul></ul><ul><ul><li>Object based </li></ul></ul><ul><ul><li>Single and Batch Operations </li></ul></ul><ul><ul><li>Transactional </li></ul></ul>

×