Contenu connexe Similaire à The State of Logging on Docker (20) The State of Logging on Docker2. Housekeeping
© 2015 Logentries. All Rights Reserved. Confidential. 2
• Please ask questions via the chat pane
• All registrants will receive a recorded copy of the webinar following
the presentation.
• Follow us @Logentries
3. Meet The Presenters
© 2015 Logentries. All Rights Reserved. Confidential. 3
Trevor Parsons, PhD
Logentries Co-founder
& Chief Scientist
Peter Elger
NearForm Director of
Engineering
Bright Fulton
Swipely Director of
Engineering Operations
4. Webinar Outline
© 2015 Logentries. All Rights Reserved. Confidential. 4
• Challenges with monitoring micro-services
• What should you be logging?
• The State of Logging on Docker
• Inside the container
• Outside the container
• Specialized containers
• Useful APIs
• Common Questions
• What about CoreOS?
7. What to Log?
© 2015 Logentries. All Rights Reserved. Confidential. 7
Simple best practices for what to log:
• When the event occurred: the timestamp
• Structured Logs: JSON, KVP
• Context: if someone bought a book… what book was it…
• Unique Identifiers: request ID, User ID, Session ID, thread ID
• Host Info: hostname & IP
• Container Info: Image ID, Container ID, Process ID
8. Logging on Docker: No Standard Approach
© 2015 Logentries. All Rights Reserved. Confidential. 8
Docker Logging Support includes:
• Capturing container stdout logs
• Access via Docker_remote_api
• Access via Docker CLI
• $ docker logs [ID]
• Will fetch the logs from a container
• can be used to tail also
• $ docker logs –tf –tail [ID]
9. Logging on Docker: No Standard Approach
© 2015 Logentries. All Rights Reserved. Confidential. 9
• Inside the Container
• Log from the application
• Run a logging Daemon/collector within the container
• Outside the Container (on the host)
• Containers write logs to the host
• Collector runs on the host
• Logging Container
• Collector runs in a specialized container
10. Inside the Container
© 2015 Logentries. All Rights Reserved. Confidential. 10
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORMHOST OS
DOCKER
LOGSLOGS
APP APP
CONTAINER 1 CONTAINER 2
Daemon Daemon
11. Inside the Container
© 2015 Logentries. All Rights Reserved. Confidential. 11
• Logging libs (log from your app code)
• Install a collector inside a container
• Use syslog inside a container
12. Outside the Container (on host)
© 2015 Logentries. All Rights Reserved. Confidential. 12
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORM
HOST OS
DOCKER
CONTAINER 1 CONTAINER 2
APP APP
LOGS LOGS
LOGSAGENT
13. Outside the container (on host)
© 2015 Logentries. All Rights Reserved. Confidential. 13
• Configure a container to bind to a volume & write logs to the volume
• Write logs to stdout
• Collector runs on the host
14. Logging Containers
© 2015 Logentries. All Rights Reserved. Confidential. 14
SERVER
LOG MANAGEMENT
& ANALYTICS
PLATFORMHOST OS
DOCKER
CONTAINER 1 CONTAINER 2 LOGGING CONTAINER
APP APP Agent
LOGS LOGS
LOGS
15. Logging Containers
© 2015 Logentries. All Rights Reserved. Confidential. 15
• Syslog in a container
• Collector in a container
• E.g. Logentries container will collect Stats and Logs
• https://github.com/nearform/docker-logentries
• Logspout: https://github.com/gliderlabs/logspout
16. Common Questions
© 2015 Logentries. All Rights Reserved. Confidential. 16
• Do I need to rotate any logs?
• Is there a big performance hit?
• E.g. collecting metrics from every container
• Any security concerns?
17. What about CoreOS?
© 2015 Logentries. All Rights Reserved. Confidential. 17
• Enables warehouse-scale computing
• Used for managing clusters of containers
• Distributed process manager, Fleet (based on systemd)
• etcd for service discovery and global storage of your config settings
• How you can log from your CoreOS Clusters
• Journal - a modern logging system (e.g. JSON export, forward sealing)
• Journal2Logentries: https://github.com/kelseyhightower/journal-2-logentries
18. Useful Resources:
© 2015 Logentries. All Rights Reserved. Confidential. 18
• Docker CLI
• https://docs.docker.com/reference/commandline/cli/
• Docker Remote API
• https://docs.docker.com/reference/api/docker_remote_api/
• Stats API (From Docker 1.5)
• https://github.com/docker/docker/pull/9984 (e.g. used by Logentries container)
• Docker RunTime Metrics
• Uses the native cgroup accounting metrics
• https://docs.docker.com/articles/runmetrics/ (e.g. used by DataDog)
• Docker Ruby Gem
• https://github.com/swipely/docker-api
19. Useful Resources: Blogs
© 2015 Logentries. All Rights Reserved. Confidential. 19
• The State of Logging on Docker
• https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/
• What is the Docker Stats API
• https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/
• Docker Log Management Using Fluentd
• http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/
• How to Centralize Logs from CoreOS Clusters
• https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
20. Useful Resources: Blogs
© 2015 Logentries. All Rights Reserved. Confidential. 20
• The State of Logging on Docker
• https://blog.logentries.com/2014/03/the-state-of-logging-on-docker/
• What is the Docker Stats API
• https://blog.logentries.com/2015/02/what-is-the-docker-stats-api/
• Docker Log Management Using Fluentd
• http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/
• How to Centralize Logs from CoreOS Clusters
• https://blog.logentries.com/2015/03/how-to-centralize-logs-from-coreos-clusters/
22. Getting Started
© 2015 Logentries. All Rights Reserved. Confidential. 22
• Don’t have a Logentries Free Account Yet?
• Sign-up for a free 30-day trial: http://logentries.com/quick-start
• Docker & Node.Js Development:
• http://www.nearform.com/
• http://nscale.nearform.com/
• Contact us directly!
• Trevor.parsons@logentries.com
• support@logentries.com
• peter.elger@nearform.com
• Let’s take a few questions!