DevOps
How to Send Docker Logs to Grafana: A Step-by-Step Guide for New Learners

How to Send Docker Logs to Grafana: A Step-by-Step Guide for New Learners

Urvish Suhagiya's photo
Urvish Suhagiya
·

6 min read

Table of contents

Introduction

In today's DevOps landscape, monitoring and logging are critical for maintaining the health and performance of applications. Grafana, a powerful open-source analytics and monitoring solution, can help visualize data in real-time. This guide will walk you through the process of sending Docker logs to Grafana, making it easy for beginners to understand and implement.

Step 1 : Setting Up a Linux EC2 Instance

First, we need a Linux EC2 instance. Follow these steps to launch one on AWS:

  1. Log in to AWS Management Console and navigate to the EC2 dashboard.

  2. Click "Launch Instance" and choose an Amazon Machine Image (AMI). The "Amazon Linux 2 AMI" is a good choice.

  3. Choose an Instance Type. The t2.micro instance is sufficient for this tutorial and falls under the free tier.

  4. Configure Instance Details. Ensure you enable "Auto-assign Public IP."

  5. Add Storage and Tags (optional).

  6. Configure Security Group. Add rules to allow SSH (port 22), HTTP (port 80), and Grafana (port 3000).

  7. Review and Launch. Create a new key pair or use an existing one to connect to your instance.

Step 2 : Install Docker and Start Docker Service

We will use User Data to automate Docker installation when the instance launches. In the "Configure Instance Details" step, add the following script under "Advanced Details" in the User Data field:

shCopy code#!/bin/bash
yum update -y
yum install -y docker
service docker start
usermod -a -G docker ec2-user
chkconfig docker on

This script updates the package manager, installs Docker, starts the Docker service, adds the ec2-user to the Docker group, and ensures Docker starts on boot.

Step 3 : Connect to Your EC2 Instance

After launching the instance, connect to it using SSH:

ssh -i /path/to/your-key.pem ec2-user@your-ec2-public-ip

Step 4 : Create Docker Containers

Let's create two Docker containers running a simple todo application. First, create a directory for the app:

mkdir my-todo-app
cd my-todo-app

Create a Dockerfile with the following content:

FROM node:14
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]

Create a package.json file:

{
  "name": "todo-app",
  "version": "1.0.0",
  "description": "A simple todo app",
  "main": "app.js",
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "express": "^4.17.1"
  }
}

Create an app.js file:

const express = require('express');
const app = express();
const port = 3000;

app.get('/', (req, res) => {
  res.send('Hello World! This is a simple todo app.');
});

app.listen(port, () => {
  console.log(`App running on http://localhost:${port}`);
});

Build and run the Docker containers:

docker build -t todo-app .
docker run -d -p 3000:3000 --name todo-app-1 todo-app
docker run -d -p 3001:3000 --name todo-app-2 todo-app

Step 5 : Install and Configure Grafana

Install Grafana on the EC2 instance:

sudo tee /etc/yum.repos.d/grafana.repo<<EOF
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
EOF

sudo yum install -y grafana
sudo systemctl start grafana-server
sudo systemctl enable grafana-server.service

Access Grafana by navigating to http://your-ec2-public-ip:3000 in your browser. The default login is admin for both the username and password.

Step 6 : Enable Docker Plugin in Grafana

  1. Log in to Grafana and go to the Configuration (gear icon) > Data Sources.

  2. Add a Data Source and search for the Docker plugin.

  3. Configure the plugin with the appropriate settings (e.g., Docker endpoint URL).

Step 7: Integrate Docker Logs with Grafana

  1. Install Docker Logging Driver for Grafana on your EC2 instance:
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
  1. Configure Docker Daemon to use the Loki logging driver. Edit /etc/docker/daemon.json:
{
  "log-driver": "loki",
  "log-opts": {
    "loki-url": "http://localhost:3100/loki/api/v1/push"
  }
}
  1. Restart Docker:
sudo systemctl restart docker

Step 8 : Verify Logs in Grafana

Navigate to Grafana, add a new dashboard, and configure a panel to display logs from your Docker containers.

Bonus: Integrate Prometheus with Grafana

  1. Install Prometheus on your EC2 instance:
sudo yum install -y prometheus
sudo systemctl start prometheus
sudo systemctl enable prometheus
  1. Configure Prometheus to scrape Docker metrics. Edit /etc/prometheus/prometheus.yml:
scrape_configs:
  - job_name: 'docker'
    static_configs:
      - targets: ['localhost:9323']
  1. Restart Prometheus:
sudo systemctl restart prometheus
  1. Add Prometheus Data Source in Grafana and create dashboards to monitor Docker containers.

Detailed Explanation

Why Docker?

Docker is a popular tool that makes it easy to create, deploy, and run applications by using containers. Containers allow developers to package an application with all of its parts (like libraries and dependencies) and ship it all out as one package.

Why Grafana?

Grafana is an open-source platform for monitoring and observability. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. It is commonly used to monitor applications and infrastructure.

What is Prometheus?

Prometheus is an open-source system monitoring and alerting toolkit. It is particularly useful for gathering and analyzing time-series data, making it a perfect tool to work alongside Grafana.

Step-by-Step Breakdown:

  1. Setting Up the EC2 Instance:

    • Why EC2? Amazon EC2 provides scalable computing capacity in the AWS cloud. Using EC2 eliminates the need to invest in hardware upfront, so you can develop and deploy applications faster.

    • Security Group: It is essential to set up security groups correctly to allow necessary traffic (SSH for access, HTTP for web, and specific ports for Grafana).

  2. Installing Docker:

    • User Data Script: This script is executed automatically when the instance starts. It saves time by automating the installation of Docker.

    • Docker Group: Adding the ec2-user to the Docker group ensures that you can run Docker commands without needing sudo.

  3. Creating Docker Containers:

    • Simple Application: We use a simple Node.js application to demonstrate how to run applications inside Docker containers.

    • Dockerfile: A Dockerfile is a text document that contains all the commands to assemble an image. In our case, it sets up a basic Node.js environment.

  4. Setting Up Grafana:

    • Installation: Installing Grafana on your EC2 instance allows you to visualize and monitor data from various sources.

    • Access: By navigating to the instance's public IP on port 3000, you can access the Grafana UI.

  5. Integrating Docker with Grafana:

    • Loki Logging Driver: This plugin allows Docker to send logs to Loki, which Grafana can then visualize.

    • Daemon Configuration: Editing the Docker daemon configuration ensures that logs are sent to Loki by default.

  6. Bonus - Prometheus Integration:

    • Prometheus: It collects and stores metrics, making it easier to monitor your Docker containers.

    • Grafana Integration: By adding Prometheus as a data source in Grafana, you can create detailed dashboards to monitor your application's performance.

Conclusion

Congratulations! You've successfully integrated Docker logs with Grafana and set up monitoring with Prometheus. This setup not only enhances your monitoring capabilities but also adds valuable experience to your DevOps toolkit. Keep exploring and pushing your boundaries! 🚀

DevopsDockerGrafanamonitoring#90daysofdevopsTrainWithShubham#Junoon