Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Logging & Metrics with Docker

1 946 vues

Publié le

In this deck, I quickly summarize how people have dealt with logging in Docker historically and then describe a comprehensive approach to logging and monitoring in Docker, based on research and customer interviews.

While Docker adds a welcome layer of abstraction to the deployment of applications, it also challenges assumptions on how those applications should be managed. I discuss a comprehensive approach for collecting logs and metrics into a centralized platform and dissect the latest additions to Docker itself (log drivers, stats).

Publié dans : Technologie
  • Soyez le premier à commenter

Logging & Metrics with Docker

  1. 1. Logging & Metrics With Docker A Comprehensive Monitoring Solution Stefan Zier June 13th, 2015
  2. 2. whoami Infrastructure, Backend Dev/Architect Chief Architect, Sumo Logic, since 2010 Server & Infrastructure, ArcSight (HP), 2001-2010
  3. 3. Mandatory Slide Showing Shipping Containers
  4. 4. Docker – What’s making debugging hard? One more layer of abstraction Container per app = File system per process File systems short lived, transient Resource schedulers = no container affinity to host
  5. 5. What Our Customers Are Telling Us We have one process per container We like to log to stdout We have multiple processes per container We run the Sumo Logic collector on the Docker host We are looking into using Beanstalk with Docker We are using Amazon ECS Everyone here loves Docker We are logging straight from the application We are using /dev/log for Syslog We want immutable infrastructure
  6. 6. Goal Get logs from our containerized applications to a centralized logging platform.
  7. 7. How do apps emit logs Append to a file Use syslog() Use log4j, log4net, slf4, etc. printf() to stdout
  8. 8. Getting logs out of the container - Files Use VOLUME to mount a host directory Collect files from the host Collect files from another container sharing the VOLUME Need to manage disk space, i.e. rotate logs App (where supported) Host Yet another container with logrotate
  9. 9. docker run -v /tmp/clogs:/tmp/clogs -d --name="sumo-logic-collector" sumologic/collector:latest-file [Access ID] [Access key]
  10. 10. Getting logs out - Syslog VOLUME /dev/log from host and use host syslogd Run a syslogd inside the container Emit TCP/UDP Write to a file using VOLUME Emit syslog TCP/UDP directly from the app
  11. 11. docker run -d -p 514:514 -p 514:514/udp --name="sumo-logic-collector” sumologic/collector:latest-syslog [Access ID] [Access key]
  12. 12. Getting logs out – Logging frameworks Sumo Logic blog on official collector images http://www.sumologic.com/blog/company/an-official-docker-image-for- the-sumo-logic-collector https://github.com/SumoLogic/sumologic-collector-docker Rainer Gerhards on Rsyslog’s file input module http://www.slideshare.net/rainergerhards1/using-wildcards-with- rsyslogs-file-monitor-imfile OWASP Log Injection https://www.owasp.org/index.php/Log_injection
  13. 13. Getting logs out – Logging frameworks Directly to network destinations HTTP/HTTPS Also support files, stdout, etc.
  14. 14. Getting logs out – Logging frameworks Various application stacks http://help.papertrailapp.com/ Log4J https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html Apache Web Server http://httpd.apache.org/docs/trunk/mod/mod_syslog.html https://raymii.org/s/snippets/Apache_access_and_error_log_to_syslog.html Nginx http://nginx.org/en/docs/syslog.html Postgres http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html Sumo Logic blog on official syslog collector image http://www.sumologic.com/blog/company/an-official-docker-image-for-the-sumo-logic-collector https://github.com/SumoLogic/sumologic-collector-docker
  15. 15. Getting logs out – stdout Simply printf() Logging framework to console Symlink to /dev/stdout or /dev/stderr Configure paths to /dev/stdout or /dev/stderr RUN ln -sf /dev/stdout /var/log/nginx/access.log RUN ln -sf /dev/stderr /var/log/nginx/error.log
  16. 16. Docker Logging Drivers
  17. 17. What Docker provides Captures stdout/stderr Feeds it to logging drivers docker logs command Returns the entire log every time Works with json-file driver only Can tail logs docker logs –tf –-tail 0 [ID]
  18. 18. Docker Logging Drivers Configured on docker run stdout and stderr dispatched to drivers json-file (default pre 1.6) syslog journald No stats, no events json-file driver Output unbounded, can fill up the host disk Requires logrotate on the Docker host https://github.com/docker/docker/issues/7333
  19. 19. Stats
  20. 20. Docker Stats Per-container cgroups metrics (like docker stats) Memory CPU Block I/O CONTAINER CPU % MEM USAGE/LIMIT MEM % NET I/O collector 2.23% 232.6 MiB/2 GiB 11.36% 191.9 KiB/636.3 KiB
  21. 21. Requirements How would we want it to work?
  22. 22. What information do we want to collect? Timestamp Log message Docker host info Container ID Image ID Process ID
  23. 23. How should it work? Use docker logging infrastructure Minimal moving parts Containerized - don’t touch the host Complete – pick up all available data Automatically discover new containers
  24. 24. Docker API The solution maybe?
  25. 25. Docker API Docker daemon has a REST API TCP or unix socket Streaming APIs Docker Events (container lifecycle updates) Container Stats (CPU, memory used, …) App Logs (container stdout/stderr)
  26. 26. Collecting via Docker API Discover new containers via events Start streaming their logs and stats When they go away, stop Do all of this via the API Send all of it to centralized log management
  27. 27. Collecting via Docker API, continued Single component to do it Zero footprint on the host Follows Docker standard way of logging
  28. 28. One more thing…
  29. 29. Introducing: Sumo Logic Docker Source
  30. 30. Sumo Logic Docker Source Active development Early access expected later this year
  31. 31. Demo Time
  32. 32. fin. Questions? @stefanzier