The next-phase-of-distributed-systems-with-apache-ignite
•
3 j'aime•1,391 vues
My Talk at OSCON 2017: https://conferences.oreilly.com/oscon/oscon-tx/public/schedule/detail/60801
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à The next-phase-of-distributed-systems-with-apache-ignite
Similaire à The next-phase-of-distributed-systems-with-apache-ignite (20)
Plus de Dani Traphagen
Plus de Dani Traphagen (8)
Dernier
Dernier (20)
The next-phase-of-distributed-systems-with-apache-ignite
- 1. APACHE IGNITE THE NEXT PHASE OF DISTRIBUTED SYSTEMS WITH
- 2. OMG IT’S LIKE TOTALLY ME…. DANI TRAPHAGEN - @DTRAPEZOID ▸ The person in front of the room ▸ This doesn’t make me more special than you ▸ This just means I spent an afternoon of my life submitting an abstract ▸ No, but seriously, I did that ▸ But here’s a picture of me looking totes professional ▸ Oh also, I used to consult on NoSQL databases and now I don’t…because I sort of question………….
- 3. IS MEMORY…
- 4. WELL WHAT ARE THE TRENDS?
- 5. SOURCE HERE
- 6. SO THE TREND IS MEMORY IS DECREASING IN COST. COOL.
- 7. SO WHAT IN THE H-E- IS APACHE IGNITE?
- 8. APACHE IGNITE A LITTLE HISTORY ON
- 9. A WEE BIT OF HISTORY APACHE IGNITE’S HISTORY ▸ Apache Ignite hits the streets in Oct. 2014, released by GridGain ▸ Aug. 2015 Ignite is the 2nd fastest project to graduate after Spark ▸ Today, over 100 Contributors internationally & rapidly growing out the community ▸ Over 1m lines of code
- 11. SPEEDS UP 140 23 1 5067 7898 45
- 12. ‣ HPC ‣ Machine learning ‣ Risk analysis ‣ Grid computing ‣ HA API Services ‣ Scalable Middleware ‣ Distributed caching ‣ KeyValue Store ‣ In-Memory SQL ‣ APIs for Kafka, etc.,as well as Ignite’s ‣ Lambda Architectures ‣ Big Data ‣ Realtime Analytics ‣ Batch processing ‣ Distributed In- Memory File System ‣ Node2Node & Topic-based Messaging ‣ Fault Tolerance ‣ Multiple backups ‣ Cluster groups ‣ Auto ‣ CEP,Streaming Windows & Cont.Queries ‣ Event driven design ‣ Dist.queues ‣ Atomic variables ‣ Dist. Semaphore APACHE IGNITE IS MORE THAN A BASIC CACHE, IT’S A PLATFORM
- 13. DATA GRID This is the foundation of Ignite - rebalances the data in a load balanced and fault tolerant fashion
- 14. KEY VALUE STORE & HIGH AVAILABILITY WITH APACHE IGNITE ▸ Ignite’s Data Grid is a Distributed Key Value Store, Caching Distributed In-Memory Data Throughout the Cluster ▸ Two Flavors of Cache: Partitioned or Replicated ▸ Consistency ▸ Data Locality
- 15. SQL GRID
- 16. MICROSERVICES DEPLOYMENT WITH APACHE IGNITE ▸ The Service Grid is the foundation for a Microservices- based Architecture ▸ Aspects of the Microservices Solution ▸ Data Nodes, Service Nodes ▸ Communication ▸ Internal and External Apps ▸ Persistence
- 18. ▸ Deployment - Node Filter - a class that allows you to specify nodes to run your service ▸ Service Grid APIs - Pass Service Service Proxy ▸ Load Balancing - Sticky vs. Non-Sticky ▸ Manageable Deployment - Not necessary everywhere in your cluster, only where you need services deployment
- 19. RECAP ▸ Manage the Lifecycle of your Services ▸ Initiate, Maintain, Kill ▸ Load Balancing ▸ Requests should be going to the services you have ▸ Fault Tolerance ▸ If a service goes down, it should be addressed immediately ▸ Scalability ▸ If volume demands change, you need to be able to handle it
- 20. Persistent Storage EXTERNAL MICROSERVICES APPLICATIONS DATA NODE DATA NODE DATA NODE DATA NODE Service Node Service Node Service Node MS1MS1 MS2MS3MS2
- 21. DATA NODES - ▸ Server nodes that store the data & accepts queries (& or computations) in a distributed fashion ▸ Provides distributed storage, don’t need provide data model classes, services classes.
- 22. SERVICE NODES ▸ Server or client node that may be used as a target for a service deployment ▸ Can deploy 1 or more services ▸ Separately updated and maintained
- 23. CLUSTER COMMUNICATION - 3 TYPES ▸ In-Cluster Communication w/Service Grid API ▸ Define interface & methods supported by your service ▸ Service Interface - *must be in class path of all your service nodes* ▸ Service Nodes & Applications ▸ External Communication Protocols ▸ Doesn’t know about Apache Ignite ▸ REST, Sockets…
- 24. INTERNAL AND EXTERNAL APPS ▸ Internal Apps ▸ Connect via Ignite API ▸ Use Service Grid API ▸ External Apps ▸ Ignite “agnostic” ▸ Open endpoints - REST, sockets…whatever.
- 25. PERSISTENT STORAGE ▸ Data Nodes can connect to: ▸ Relational DBs ▸ NoSQL Das like Cassandra & MongoDB (ew) ▸ Hadoop ▸ Etc. ▸ Read-Through ▸ Write-Through ▸ Transactional ▸ Dependent on what type of persistent store you have elected
- 26. WRITE-THROUGH
- 27. BEST PLACE TO GET STARTED? https://console.gridgain.com/
- 28. REPEAT AFTER ME…
- 29. CACHE RULES EVERYTHING AROUND ME WHAT WE REALLY MEANT IN THE SONG
- 30. THANKS!!! DANI TRAPHAGEN - @DTRAPEZOID