Learn how VMware vFabric GemFire helps build

1. Building High Performance, Data Intensive Resilient, Distributed Applications

2. The Challenge Data Explosion Decision Time Compression Critical to act time frame that matters Milliseconds Seconds Minutes or even hours

3. Today’s Modern Architecture’s Web Applications Load Balancer Web Stateless Application Database Tier Stateful Storage Tier

4. Today’s Modern Architecture’s Data Ingest Applications

7. AvailabilitySolutions

8. Sources of Latency Disk access Serial access Network time Sockets (open/close) Marshalling and unmarshalling Security overhead

9. Sources of Latency - Disk Access Mitre Public Release: 10-0861. Distribution Unlimited

11. Marshalling/Unmarshalling Lazy deserialization Index serialized data Shared compact data format

12. Security overhead Mutual authentication at socket time Process/user level (optional)

15. AvailabilitySolutions

16. Architecting Infinitely Scalable Systems A seminal paper on the architecture of elastic applications by Pat Helland(Tandem Computing, Amazon.com, Microsoft) “Life Beyond Distributed Transactions: an Apostate’s Opinion” http://www.cidrdb.org/cidr2007/papers/cidr07p15.pdf http://blogs.msdn.com/b/pathelland/ Application architectures need to change to achieve infinite scalability and elasticity without using large hardware

17. Scale - Layered Code Common layered architecture in largest scale applications Top layer Scale Agnostic Code Programming Abstraction Abstraction layer Scale Aware Code Bottom layer understands application is distributed

18. Scale Shared nothing Partition/Sharding Collocated relations Replicated reference 14

21. AvailabilitySolutions

22. Reliability No data loss No data corruption Consistency Race condition Synchronous vs Asynchronous

23. No Data Loss, No Data corruption, Consistency Distributed semaphore - lightweight Primary copy Distributed transaction(s) – heavy weight MVCC – Acronyms are annoying

24. Race Conditions, Consistency Eventually Consistent Stateless ? Application Data Tier Stateful Controllably Consistent

27. AvailabilitySolutions

28. Availability - on Server Protect data Extra copies Disk? Data Center crashes Network Splits Split Brain detection

29. Availability – Between Client/Server Slow Consumers HA Queues Client Network drops Durable subscribers

32. AvailabilitySolutions

33. Latency & Reliability - Memory-based Performance Memory on a peer machine to make data updates durable, Writes return 10x to 100x faster than disk, 10s to 100s of Microseconds vs 10s to 100s Milliseconds Perform Customers Orders Product Keep redundant copies of data Update thru primary 0 data loss Optionally write updates to disk, Optional write todata warehouse asynchronously and reliably. Protect

34. Memory-based Performance Perform In Situ data processing Real-time controls Calculate: current total fuel left

35. Latency - Data-Aware Access Perform Application Client Java, C++, .Net, SQL

36. Latency & Reliability - HA Data-Aware Function Data Aware Function Execute Client Move behavior to data

37. Parallel Queries Compute Client Scatter-Gather Queries & Functions

38. Data Distribution Distribute Keep clusters synchronized in real-time Operate reliably Disconnected, Intermittent and Low-Bandwidth network environments.

39. Distributed Events Notify Targeted, Guaranteed delivery. Event notification & Continuous Queries Disconnected, Intermittent and Low-Bandwidth network environments

40. Cloud Ready Soar Load Balancer Web Tier Application Tier GemFire Jar 11MB (or less) Optional reliable, asynchronous feed to Data Warehouse or Archival Database

41. Thank you