xkozn blog

Software Engineer // Runner // Always on the move

Docker Logging

Posted at — Jun 12, 2019

The docker daemon manages STDIN, STDOUT and STDERR on behalf of the daemon. This post will outline how to view STDOUT and STDERR of containers on a linux machine.

Start up a simple container, and log the customary hello world.

docker run -it --rm busybox echo -n "Hello, World"
Hello, World%

Not only does docker output this to the terminal (as instructed with -it flags), it is also saved on the filesystem.

CONTAINER_ID=`docker run -d  busybox echo -n "Hello, World"`
jq . /var/lib/docker/containers/$CONTAINER_ID/$CONTAINER_ID-json.log

In fact, docker is quite nice and structures the logs in a JSON format, adding the time and stream.

  "log": "Hello, World",
  "stream": "stdout",
  "time": "2019-11-17T01:59:21.854865946Z"

These files can be kept on disk for viewing while the container is on the machine. Even if the container is stopped, the logs will exist. If the container is removed, they will be cleaned.

Losing logs when a container is removed presents a problem. When using a container orchestration system like Kubernetes, containers are removed frequently.

Enter log collection tooling! Something like fluentd, fluentbit, filebeat, or logstash. Hopefully these are familiar, but if not, they are simply programs that you run to help track these files, and send each log message to a configured upstream destination that eventually persists the logs for long term storage.