Ecosysteme mesos university - devoxx france - 8 avril2015
•
0 j'aime•462 vues
Présentation de Mesos à Devoxx 2015
Recommandé
Contenu connexe
Tendances
Tendances (19)
Similaire à Ecosysteme mesos university - devoxx france - 8 avril2015
Similaire à Ecosysteme mesos university - devoxx france - 8 avril2015 (20)
Plus de Jean-Baptiste Claramonte
Dernier
Dernier (20)
Ecosysteme mesos university - devoxx france - 8 avril2015
- 2. @XebiaFr @YesWeScale#MesosUniversity Agenda • Mesos • Frameworks • Mesosphere • Marathon
- 4. @XebiaFr @YesWeScale#MesosUniversity Ressources partagées Master Slave Slave Slave Slave cpu: 2 mem: 2048 cpu: 4 mem: 16384 cpu: 2 mem: 4096 cpu: 32 mem: 131072
- 5. @XebiaFr @YesWeScale#MesosUniversity Resources • 2 méthodes permettent de décrire les slaves du cluster • Resources • Tags (ou « Attributes »)
- 6. @XebiaFr @YesWeScale#MesosUniversity Resources • Certaines resources types sont pré définies par Mesos • cpu • mem • disk • port
- 7. @XebiaFr @YesWeScale#MesosUniversity Resources • Certaines resources types sont pré définies par Mesos • cpu • mem • disk • port OBLIGATOIRE !
- 8. @XebiaFr @YesWeScale#MesosUniversity Resources • Trois différentes types de resources • scalars • ranges • sets
- 12. @XebiaFr @YesWeScale#MesosUniversity Resources cpu:24;mem:24576;disk:409600;ports:[21000-24000];disks:{1,2,3,4,5,6,7,8,9} Set par exemple /mnt/disk/{diskNum}
- 19. @XebiaFr @YesWeScale#MesosUniversity Framework Framework = Application
- 23. @XebiaFr @YesWeScale#MesosUniversity API Scheduler • Est notifié par le master de la mise à disposition de ressource • Demande l’exécution de tâches au master Executor • Réalise les taches soumises par le master
- 24. @XebiaFr @YesWeScale#MesosUniversity API Scheduler Mesos (master) register time Executor
- 25. @XebiaFr @YesWeScale#MesosUniversity API Scheduler Mesos (master) register registered time Executor
- 26. @XebiaFr @YesWeScale#MesosUniversity API Scheduler Mesos (master) register registered offer time Executor
- 27. @XebiaFr @YesWeScale#MesosUniversity API Scheduler Mesos (master) register registered offer launch time launch Executor
- 28. @XebiaFr @YesWeScale#MesosUniversity API Scheduler Mesos (master) register registered offer launch update time launch Executor update
- 29. @XebiaFr @YesWeScale#MesosUniversity Framework : API API du Scheduler
- 30. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Lifecycle management Executor Master Slave Scheduler registered(…) void registered(SchedulerDriver driver, FrameworkID fwkId, MasterInfo masterInfo); Master Standby
- 31. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Lifecycle management Executor Master Slave Scheduler Master Standby disconnected(…) void disconnected(SchedulerDriver driver);
- 32. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Lifecycle management Executor Slave Scheduler reregistered(…) Master Master void reregistered(SchedulerDriver driver, MasterInfo masterInfo);
- 33. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Resource Allocation Executor Master Slave Hadoop Scheduler resourceOffers(…) Master Standby Cassandra Scheduler void resourceOffers(SchedulerDriver driver, List<Offer> offers); Offre de ressources
- 34. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Resource Allocation Cassandra Executor Master Slave Hadoop Scheduler offerRescinded(…) Master Standby Cassandra Scheduler Task Le scheduler Cassandra exécute le premier la tâche void offerRescinded(SchedulerDriver driver, OfferID offerId);
- 35. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Communication Cassandra Executor Master Slave Hadoop Scheduler statusUpdate(…) Master Standby Cassandra Scheduler Task void statusUpdate(SchedulerDriver driver, TaskStatus taskStatus);
- 36. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Communication Cassandra Executor Master Slave Hadoop Scheduler frameworkMessage(…) Master Standby Cassandra Scheduler Task void frameworkMessage(SchedulerDriver driver, ExecutorID execId, SlaveID slaveId, byte[] message);
- 37. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Error events Cassandra Executor Master Slave Hadoop Scheduler slaveLost(…) Master Standby Cassandra Scheduler Task void slaveLost(SchedulerDriver driver, SlaveID slaveId);
- 38. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Error events Cassandra Executor Master Slave Hadoop Scheduler executorLost(…) Master Standby Cassandra Scheduler Task void executorLost(SchedulerDriver driver,ExecutorID execId, SlaveID slaveId, int status);
- 39. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Error events Executor Master Slave Scheduler error(…) Master Standby void error(SchedulerDriver driver, String message);
- 40. @XebiaFr @YesWeScale#MesosUniversity Framework : API API de l’Executor
- 41. @XebiaFr @YesWeScale#MesosUniversity Executor : Lifecycle management Executor Master Slave Scheduler registered(…) Master Standby void registered (ExecutorDriver driver, ExecutorInfo execInfo, FrameworkInfo fwkInfo, SlaveInfo slaveInfo);
- 42. @XebiaFr @YesWeScale#MesosUniversity Executor : Lifecycle management Executor Master Slave Scheduler Master Standby disconnected(…) void disconnected(ExecutorDriver driver);
- 43. @XebiaFr @YesWeScale#MesosUniversity Executor : Lifecycle management Executor Master Slave Scheduler Master Standby reregistered(…) void reregistered(ExecutorDriver driver, SlaveInfo slaveInfo);
- 44. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Task management Cassandra Executor Master Slave launchTask(…) Master Standby Cassandra Scheduler Task void launchTask(ExecutorDriver driver, TaskInfo taskInfo);
- 45. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Task management Cassandra Executor Master Slave killTask(…) Master Standby Cassandra Scheduler Task void killTask(ExecutorDriver driver, TaskID taskId); schedulerDriver.killTask(…)
- 46. @XebiaFr @YesWeScale#MesosUniversity Scheduler : Communication Cassandra Executor Master Slave frameworkMessage(…) Master Standby Cassandra Scheduler Task void frameworkMessage(ExecutorDriver driver, byte[] msg);
- 47. @YourTwitterHandle@YourTwitterHandle@XebiaFr @YesWeScale#MesosUniversity Mesosphere : Chronos & Marathon
- 48. @XebiaFr @YesWeScale#MesosUniversity Mesosphere Florian Leibert Benjamin Hindman Tobi Knaup
- 49. @XebiaFr @YesWeScale#MesosUniversity Mesosphere Our mission is to make building and running distributed systems as easy as building or running an app on your smartphone.
- 52. @XebiaFr @YesWeScale#MesosUniversity Quoi ? “Init Daemon” pour Data Center Private PaaS
- 53. @XebiaFr @YesWeScale#MesosUniversity Pourquoi ? Simplifier Automatiser API pour les développeurs Améliorer l’efficacité
- 55. @XebiaFr @YesWeScale#MesosUniversity UI : liste des applications
- 56. @XebiaFr @YesWeScale#MesosUniversity UI : créer une application
- 57. @XebiaFr @YesWeScale#MesosUniversity Cycle de vie Master Slave Slave Slave Slave Marathon Tâches Évènements / Offres
- 59. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "myrestapi", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT0" } Démarrer une “Self Contained App”
- 60. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "myrestapi", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT0" } Démarrer une “Self Contained App” Slave myrestapi $PORT0 ports:[0]
- 61. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "ubuntu", "container": { "type": "DOCKER", "docker": { "image": "libmesos/ubuntu" } }, "instances": 1, "cpus": 0.5, "mem": 512, "uris": [], "cmd": "while sleep 10; do date -u +%T; done" } Démarrer une application Dockerisée
- 62. @XebiaFr @YesWeScale#MesosUniversity Health check • Par défaut, information provenant de Mesos TASK_RUNNING => Healthy • Configurable via l’API REST de Marathon
- 63. @XebiaFr @YesWeScale#MesosUniversity { "id": "my-restapi-example-app", ... "healthChecks": [ { "protocol": "COMMAND", "command": { "value": "curl -f -X GET http://$HOST:$PORT0/simpleapi/info" }, "gracePeriodSeconds": 300, "intervalSeconds": 5, "timeoutSeconds": 3, "maxConsecutiveFailures": 3 } ] } Health check
- 64. @XebiaFr @YesWeScale#MesosUniversity { "id": "my-restapi-example-app", ... "healthChecks": [ { "protocol": "HTTP", "path": "/simpleapi/info", "gracePeriodSeconds": 300, "intervalSeconds": 5, "timeoutSeconds": 3, "maxConsecutiveFailures": 3 } ] } Health check
- 65. @XebiaFr @YesWeScale#MesosUniversity Les contraintes Contrôler le “où” lors du déploiement
- 66. @XebiaFr @YesWeScale#MesosUniversity Les contraintes : tags Identifier les ressources par des tags : • type de hardware • emplacement physique (datacenter , rack) • … Positionnées au démarrage du slave $ mesos-slave … --attributes="rack_id:2"
- 67. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["hostname","UNIQUE"] } Contrainte : UNIQUE
- 68. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["hostname","UNIQUE"] } Contrainte : UNIQUE Slave 1 UserMgt UserMgt Slave 2 UserMgt CreditApp Slave 3 Slave 4 UserMgt CreditApp
- 69. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["harddrive","CLUSTER", "ssd"] } Contrainte : CLUSTER Slave 1 UserMgt UserMgt Slave 2 UserMgt CreditApp Slave 3 Slave 4 UserMgt UserMgt ssd hdd
- 70. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints":[["hostname","CLUSTER", “slave-3”] } Contrainte : CLUSTER Slave 1 UserMgt Slave 2 UserMgt CreditApp Slave 3 Slave 4 UserMgt UserMgt CreditApp
- 71. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["rack_id","GROUP_BY"] } Contrainte : GROUP_BY Slave 1 UserMgt Slave 2 CreditApp Slave 3 Slave 4 Rack 1 Rack 2 UserMgt CreditApp UserMgt UserMgt
- 72. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["hostname","UNLIKE", "slave-[2-3]"] } Contrainte : UNLIKE Slave 1 UserMgt UserMgt Slave 2 UserMgt CreditApp Slave 3 Slave 4 UserMgt UserMgt
- 73. @XebiaFr @YesWeScale#MesosUniversity POST /v2/apps { "id": "my-restapi-example-app", "instances": 2, "cpus": 0.3, "mem": 128, "ports": [0], "uris": ["http://storage.googleapis.com/restapi-jb/ RestApi.jar"], "cmd": "java -jar RestApi.jar --server.port=$PORT", "constraints": [["hostname","LIKE", "slave-[1-3]"] } Contrainte : LIKE Slave 1 UserMgt UserMgt Slave 2 UserMgt CreditApp Slave 3 Slave 4 UserMgt UserMgt
- 74. @XebiaFr @YesWeScale#MesosUniversity Application groups • Définir des liens de dépendance entre les groupes • Scaler tout un groupe à la fois
- 75. @XebiaFr @YesWeScale#MesosUniversity Marathon : Application groups /myapp/bdd postgresql instance=3 redis instance=4 /myapp/service apache instance=5 tomcat instance=8 myapp Dépendance
- 76. @XebiaFr @YesWeScale#MesosUniversity { "id": "/myapp", "groups": [ { "id": "/myapp/bdd", "apps": [ { "id": "/myapp/postgresql", ... }, { "id": "/myapp/redis", ... } ] },{ "id": "/myapp/service", "dependencies": ["/myapp/bdd"], "apps": [ { "id": "/myapp/apache", ... }, { "id": "/myapp/tomcat", ... } ] } ] } Application groups
- 77. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Où est mon application ?
- 78. @XebiaFr @YesWeScale#MesosUniversity HaProxy : reload configuration • HaProxy - Marathon bridge (script) • Bamboo
- 79. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Slave 1 Slave 2 Marathon HA Proxy HA Proxy Marathon Bridge HA Proxy HA Proxy Marathon Bridge
- 80. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Slave 1 Slave 2 UserMgt Port 31100 UserMgt Port 31200 UserMgt Port 31100 CreditApp Port 31200 UserMgt Port : 10000 CreditApp Port : 20000Marathon HA Proxy HA Proxy Marathon Bridge HA Proxy HA Proxy Marathon Bridge
- 81. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Slave 1 Slave 2 UserMgt Port 31100 UserMgt Port 31200 UserMgt Port 31100 CreditApp Port 31200 UserMgt Port : 10000 CreditApp Port : 20000Marathon HA Proxy HA Proxy Marathon Bridge HA Proxy HA Proxy Marathon Bridge
- 82. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Slave 1 Marathon HA Proxy HA Proxy Marathon Bridge Slave 2 HA Proxy HA Proxy Marathon Bridge UserMgt Port 31100 UserMgt Port 31200 UserMgt Port 31100 CreditApp Port 31200 localhost:10000 { slave1:31100 slave1:31200 slave2:31100 } localhost:20000 { slave2:31200 } UserMgt Port : 10000 CreditApp Port : 20000
- 83. @XebiaFr @YesWeScale#MesosUniversity Service Discovery Slave 1 Marathon HA Proxy HA Proxy Marathon Bridge Slave 2 HA Proxy HA Proxy Marathon Bridge UserMgt Port 31100 UserMgt Port 31200 UserMgt Port 31100 CreditApp Port 31200 localhost:10000 { slave1:31100 slave1:31200 slave2:31100 } localhost:20000 { slave2:31200 } UserMgt Port : 10000 CreditApp Port : 20000