Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

MariaDB on Docker

1 545 vues

Publié le

Getting started with MariaDB? Whether it is on your laptop or server, containers are great ephemeral vessels for your applications. But what about the data that drives your business? It must survive containers coming and going, maintain its availability and reliability, and grow when you need it.

Publié dans : Technologie
  • Soyez le premier à commenter

MariaDB on Docker

  1. 1. MariaDB + Docker Scaling MariaDB
  2. 2. Conclusion Containers + Databases = Happy Developers Ephemeral Containers + Databases = DevOps headaches 4 Things you must use to evaluate • Data Redundancy • Dynamic Self Discovery & Cluster formation • Self Healing (as containers enter and leave) • Application Tier discovery of Database Cluster
  3. 3. Part One “Here’s another nice mess you have got me into” – Laurel & Hardy circa 1929
  4. 4. Existing Deployment Models Are Broken Version control 1. Development 2. Test 3. Stage / Production Developer QA / QE Sysadmin
  5. 5. Architectures Are Complex & Static Challenges • Orchestration • Velocity • Maintainability • Durability • Consistency • Scalability • Cost ($) • (Hybrid) CloudEnterprise Environment Legacy Mainframe Operational Database Caching Layer Pricing / Inventory / Billing Real-time Decisioning Real-time Consumer facing Streaming Data Data Warehouse Data Lake RDBMS Transactional Systems
  6. 6. Part Two Containers
  7. 7. What do Containers give me? Encapsulation of Dependencies • O/S packages & Patches • Execution environment (e.g. Python 2.7) • Application Code & Dependencies Process Isolation • Isolate the process from anything else running Faster, Lightweight virtualization
  8. 8. Containers vs. Virtual Machines App 1 App 2 App 3 Bins/Libs Bins/Libs Bins/Libs Guest OS Guest OS Guest OS Hypervisor Host Operating System Infrastructure Docker Engine Operating System Infrastructure App 1 App 2 App 3 Bins/Libs Bins/Libs Bins/Libs
  9. 9. Deployment Simplicity Build Ship Run Open Standards Plumbing Platform Clustering Distribution Image spec Container run-time spec Runtime Trust
  10. 10. Dockerfile - Example FROM python:2.7 ADD . /code WORKDIR /code RUN apt-get update && apt-get -y install python-dev libssl-dev RUN pip install Flask MySQL-python EXPOSE 5000 CMD python app.py
  11. 11. Open Container Initiative (OCI) – Polyglot Vendors Coalition of industry leaders join forces to eliminate fragmentation • Form a vendor-neutral, open source governance model under the Linux Foundation • Establish common standards for container format and runtime • Docker donated its container format, runtime and associated specifications • Appoint maintainers for the libcontainer project
  12. 12. Docker Toolchain in pictures Machine provisions Docker Engines Swarm clusters Docker Engines Compose orchestrates Container deployment Containers are run by Docker Engine Docker Machine Docker Compose Docker Swarm Docker Engine Container Containers encapsulates your code, dependencies…
  13. 13. But… Docker evolves: 1.13 / 17.3 GA features Docker Engine in “Swarm” mode • Engine based clustering versus Container based clustering • Master based, RAFT for consensus Docker Stacks • Bundles of Services • “Sort of” compatible with docker-compose Docker Services • Image + Configuration • Replicable across the cluster • Scale Up & Down
  14. 14. Part Three Databases & Docker
  15. 15. Requirements Data Redundancy • Containers are Ephemeral – Need more than one copy of the data Dynamic Self Discovery & Cluster formation • Need to start and stop Containers when needed • Clusters needs to grow and shrink dynamically Self Healing • Loss of nodes must not be fatal to the cluster integrity • Addition of nodes must scale capacity Application Tier discovery of Database Cluster • Automatic discovery of nodes • Automatic routing of requests to the correct nodes
  16. 16. Part Four MariaDB
  17. 17. MARIADB SERVER Enterprise-grade secure, highly available and scalable relational database with a modern, extensible architecture MARIADB MAXSCALE MARIADB CLUSTER Next-generation database proxy that manages security, scalability and high availability in scale-out deployments Multi-Master, synchronous replication - improves availability and scales reads and writes MariaDB Portfolio
  18. 18. MariaDB MaxScale High Availability Ensure uptime with no single point of failure and minimize downtime during upgrade Data Streaming Stream transactional data to data lake for real-time analytics Scalability Manage your scaled-out infrastructure without changing application code Security Secure database firewall to prevent cyber attacks like SQL injection and DDoS MariaDB MaxScale is a next-generation database proxy that manages security, scalability and high availability in a scale out deployment.
  19. 19. MariaDB Cluster Multi-Master • Synchronous replication Faster Failover • All nodes synchronized, therefore equal Scale reads and writes MariaDB MariaDB MariaDB Load Balancing and Failover Application / App Server
  20. 20. MaxScale + Galera Use Case Each application server uses only 1 connection MaxScale selects one node as “master” and the other nodes as “slaves” If the “master” node fails, a new one can be elected immediately Galera Cluster + R/W split routing Max Scale
  21. 21. Part Five Demo
  22. 22. Demo: Development Through Production Development • Build & Run an App in Development – Python + MariaDB Production • Deploy to a Swarm cluster in Production • Scale Web nodes – Add more Web containers behind HAProxy • Database High Availability – Deploy 3 nodes Galera cluster – Deploy 2 node MaxScale
  23. 23. Python / Flask Let’s Build An App! Development app
  24. 24. Production Virtual IP Virtual IP MaxScale 1 MaxScale 2 HAProxy 1 2 3 Then Scale in Production... Development app app1 app2 app3 app4 appN
  25. 25. Demo 1 Build an Application
  26. 26. Dockerfile FROM python:2.7 RUN apt-get update && apt-get -y install python-dev libssl-dev WORKDIR /app RUN pip install Flask MySQL-python ADD . /app EXPOSE 5000 CMD ["python", "app.py"]
  27. 27. docker-compose.yml services: web: build: . ports: - "5000:5000" links: - mariadb hostname: dev.myapp.com environment: APP_MARIADB_HOST: dev_mariadb_1 APP_PASSWORD: foo mariadb: image: mariadb:10.1 environment: MYSQL_ROOT_PASSWORD: foo
  28. 28. Roll The Application Behind Haproxy Development app Production Virtual IP Virtual IP MaxScale 1 HAProxy 1 app1
  29. 29. Scale the Application Tier Development app Production Virtual IP Virtual IP MaxScale 1 MaxScale 2 HAProxy 1 2 3 app1 app2 app3 app4 appN
  30. 30. Docker Networking Docker Host (swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint
  31. 31. $ docker network create -d overlay --attachable --opt encrypted myapp_back $ cat docker-compose.stack.yml ... networks: front: back: external: name: myapp_back Docker Networking
  32. 32. Demo 2 Scale Web Tier
  33. 33. Secrets… Opppsss services: web: build: . ports: - "5000:5000" links: - mariadb hostname: dev.myapp.com environment: APP_MARIADB_HOST: dev_mariadb_1 APP_PASSWORD: foo mariadb: image: mariadb:10.1 environment: MYSQL_ROOT_PASSWORD: foo
  34. 34. Docker secrets $ more ./app_password.txt appfoo $ cat docker-compose.stack.yml ... secrets: app_password: file: ./app_password.txt mariadb_root_password: file: ./mariadb_password.txt xtrabackup_password: file: ./xtrabackup_password.txt
  35. 35. haproxy & web services services: haproxy: image: dockercloud/haproxy networks: - front - back volumes: - /var/run/docker.sock:/var/run/docker.sock ports: - 80:80 deploy: placement: constraints: [node.role == manager] web: image: alvinr/demo-webapp-vote:mariadb environment: SERVICE_PORTS: "5000" VIRTUAL_HOST: "prod.myapp.com" APP_MARIADB_HOST: "maxscale" APP_USER: "app" APP_PASSWORD_FILE: "/run/secrets/app_password" APP_DATABASE: "test" networks: - back deploy: placement: constraints: [node.role != manger] secrets: - app_password
  36. 36. Demo 3 Hardened Database Tier
  37. 37. Container Placement Docker Host (swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint MariaDB Container Endpoint
  38. 38. Container Placement Docker Host (swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint MariaDB Container Endpoint
  39. 39. Container Placement mariadb_cluster: image: alvinr/mariadb-galera-swarm ... labels: com.mariadb.cluster: "myapp-prod-cluster" ... deploy: replicas: 1 placement: constraints: [engine.labels.com.mariadb.cluster != myapp-prod-cluster]
  40. 40. Restarting on Failure maxscale: image: alvinr/maxscale-swarm ... labels: com.mariadb.cluster: "myapp-maxscale" networks: - back deploy: replicas: 1 restart_policy: condition: on-failure delay: 10s placement: constraints: [engine.labels.com.mariadb.cluster != myapp-maxscale] secrets: - app_password
  41. 41. Part Six Considerations & Conclusions
  42. 42. DNS RESOLUTION • Docker assigns VIP to Service, each Task has own IP • nslookup, dig, getent etc. 3rd PARTY • consul, etcd, zookeeper etc. DOCKER EVENTS • https://docs.docker.com/engine/ reference/api/docker_remote_api/ • Interlock - ttps://github.com/ehazlett/interlock Service Discovery - How to mesh nodes?
  43. 43. Storage: Inside or Outside the Container? Inside • Encapsulation of Concerns Outside • Separation of Concerns • Storage features (e.g. Snapshots) • 3rd Party options – NetApp, Google Compute Engine, Rancher Convoy – Flocker Host Docker Daemon Container Docker Daemon Container /dev/xvdb /mnt/xx:/var/lib/mysql Networked e.g. EBS Volume Local Disk e.g. SSD / NVMe
  44. 44. Storage: Data Container? Inside • Managed like other containers • Special rule for Destruction • TBD: Performance Host Docker Daemon Container Docker Daemon Container --volumes-from {container name} Host
  45. 45. And... • Swarm locking • Image verification (trusted Images) • AppArmor / Seccomp profiles • Monitoring • Heathchecks • Rolling Upgrades
  46. 46. Summary One Solution Development -> Production • Define Images & Orchestration once • Reuse when needed, inject required behaviours MariaDB in Production with Docker • Ops define the whitelisted images, security policies • Dev approve images to build upon • Eliminate complexity (and cost) of Deployment • Scale easily, maintain SLA requirements of component
  47. 47. Thanks and Q&A Code • https://github.com/alvinr/docker-demo/tree/master/mariadb/vote Docker Images • https://hub.docker.com/_/mariadb/ MariaDB & Docker deployment guide • https://mariadb.com/kb/en/mariadb/installing-and-using-mariadb-via-docker/ Contact me! • alvin@mariadb.com • @jonnyeight

×