Connecting EC2 Instances with Grafana for Server Monitoring

Introduction

In our previous article, we successfully set up Grafana locally. Now, we will take a step further by connecting both Linux and Windows EC2 instances to Grafana to monitor various server components.

This guide will walk you through the process step by step, making it easy for new learners to understand and implement.

Prerequisites:

AWS Account: You need an AWS account to create EC2 instances.
Grafana: Grafana should be installed and running. If not, refer to the previous guide on setting up Grafana locally.
EC2 Instances: You should have one Linux and one Windows EC2 instance running. If not, create them in the AWS Management Console.

Step 1 : Setting Up the EC2 Instances

1.1 Creating EC2 Instances:

Creating a Linux EC2 Instance:

Go to the AWS Management Console.
Navigate to the EC2 Dashboard and click “Launch Instance”.
Select an Amazon Machine Image (AMI). Choose Amazon Linux 2 AMI (HVM), SSD Volume Type.
Choose an instance type (e.g., t2.micro).
Configure instance details (default settings are fine).
Add storage (default settings are fine).
Add tags (optional).
Configure Security Group. Add rules to allow HTTP (port 80), HTTPS (port 443), and SSH (port 22).
Review and launch the instance. Download the key pair (.pem file) for SSH access.

Creating a Windows EC2 Instance:

Repeat the above steps, but select Windows Server AMI.
Configure Security Group. Add rules to allow RDP (port 3389) and other necessary ports (HTTP, HTTPS).

Step 2 : Installing Monitoring Agents on EC2 Instances

2.1 Installing Node Exporter on Linux EC2 Instance: Node Exporter is a tool used to collect hardware and operating system metrics.

Connect to your Linux EC2 instance using SSH:

  ssh -i "your-key-pair.pem" ec2-user@your-ec2-public-dns

Update the package list and install Node Exporter:

  sudo yum update -y
  wget https://github.com/prometheus/node_exporter/releases/download/v1.1.2/node_exporter-1.1.2.linux-amd64.tar.gz
  tar xvfz node_exporter-*.tar.gz
  cd node_exporter-1.1.2.linux-amd64
  sudo cp node_exporter /usr/local/bin

Create a systemd service file for Node Exporter:

  sudo nano /etc/systemd/system/node_exporter.service

Add the following content:

  [Unit]
  Description=Node Exporter
  After=network.target

  [Service]
  User=node_exporter
  ExecStart=/usr/local/bin/node_exporter

  [Install]
  WantedBy=default.target

Start and enable the Node Exporter service:

  sudo systemctl daemon-reload
  sudo systemctl start node_exporter
  sudo systemctl enable node_exporter

2.2 Installing WMI Exporter on Windows EC2 Instance: WMI Exporter is a tool used to collect Windows metrics.

Connect to your Windows EC2 instance using RDP.
Download WMI Exporter from the official website.
Run the installer and follow the prompts to install WMI Exporter.
By default, WMI Exporter runs as a service on port 9182.

Step 3 : Configuring Prometheus to Scrape Metrics

Prometheus needs to be configured to scrape metrics from Node Exporter and WMI Exporter.

Open the Prometheus configuration file (prometheus.yml) and add the following scrape jobs:

  scrape_configs:
    - job_name: 'node_exporter'
      static_configs:
        - targets: ['<Linux-EC2-Public-DNS>:9100']

    - job_name: 'wmi_exporter'
      static_configs:
        - targets: ['<Windows-EC2-Public-DNS>:9182']

Restart Prometheus to apply the changes:
```
  sudo systemctl restart prometheus
```

Step 4 : Adding Data Sources in Grafana

Open your Grafana web interface.
Go to Configuration > Data Sources > Add data source.
Select Prometheus from the list.
Configure the Prometheus data source by providing the URL (e.g., localhost:9090) and save it.

Step 5 : Creating Dashboards in Grafana

In Grafana, go to the Dashboard section and click on “New Dashboard”.
Add a new panel, choose your Prometheus data source, and use queries to display metrics from Node Exporter and WMI Exporter.
- Example for CPU usage (Linux):
```
  node_cpu_seconds_total{instance="<Linux-EC2-Public-DNS>:9100", mode="idle"}
```
- Example for Memory usage (Windows):
```
  wmi_memory_bytes_total{instance="<Windows-EC2-Public-DNS>:9182"}
```
Customize the panels to display various metrics like CPU usage, memory usage, disk I/O, network traffic, etc.

Conclusion

By following these steps, you have successfully connected Linux and Windows EC2 instances to Grafana and set up monitoring for various server components. This setup provides you with real-time insights into the performance and health of your servers, making it easier to manage and optimize your infrastructure.

Happy monitoring !