Monitoring Linux Nodes Using Prometheus & Node Exporter

In my previous blog , I have explained in detail on how you can install and configure Prometheus on Ubuntu , Check here

In this guide , We will install and configure node_exporter to monitor target Linux hosts.

What Is Node Exporter?

Node-exporter is one of the exporters of Prometheus.It is used to collect metrics of the Linux hosts such as CPU , Memory and Disk usages and also kernel level metrics.

Node_exporter should be installed in the target Linux operating systems which will listen on port 9100.

Installing And Running Node Exporter

The node exporter package can be downloaded using the below link

https://prometheus.io/download/#node_exporter

Once the node_exporter package is downloaded , Extract it using tar command.

tar -xvzf node_exporter-0.18.1.linux-amd64.tar.gz

A folder named node_exporter-0.18.1.linux-amd64 should be created having the binary file of the node exporter.

Copy The Binary File

Now we need to copy the binary file of the node exporter to /usr/local/bin directory

cp /opt/node_exporter-0.18.1.linux-amd64/node_exporter /usr/local/bin/

Creating Node Exporter User

We need to create a user for the node exporter service and the binary file of the node exporter should be owned by it.

useradd --no-create-home --shell /bin/false node_exporter

Now we have to change the ownership of the node_exporter binary file.

chown node_exporter: /usr/local/bin/node_exporter

Creating Systemd For Node Exporter

We need to create a systemd unit file to mange the node exporter service.

We are going to create a file under /etc/systemd/system and the file should have extention as .service

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Save and close the file.

Starting Node Exporter Service

First we need to run the below command to use the newly created service file.

systemctl-daemon reload

To start the node exporter service run the below command.

systemctl start node_exporter

Enable the node_exporter service to start if on system boot

systemctl enable node_exporter

To check the status of the node exporter,

systemctl status node-exporter

Check The Version Of Node Exporter

To check the version of the node exporter installed.

node_exporter --version

To check on which port node_exporter is listening ,

netstat -nltp |grep node_exporter

By default , Node exporter listens on the port 9100.

Check The Metrics Of The Node Exporter

Run the below command , To check the metrics already collected by the node_exporter.

You can see the collection of system metrics.

curl http://localhost:9100/metrics

Configuring Prometheus To Scrape Node Exporter

Installation and configuration of prometheus is explained in detail here.

Now we have Prometheus and the Node exporter is installed and configured.

Now Its time to send the metrics of the node_exporter to the prometheus server.

But wait , before configuring the prometheus We need to setup hostname on the node_exporter server.

To setup a hostname , run the below command.

hostnamectl set-hostname node_exporter

Logout and login to check the changes , Run hostname to check the hostname set for the server.

Also We need to configure hosts file on prometheus and node exporter servers.

Open the /etc/hosts file in the prometheus server and add the node_exporter details as shown below

vi /etc/hosts

172.16.23.55    node_exporter

save and close the file.

Same way we need to configure hosts file in the node exporter server.Here we need to add the details of the prometheus

172.16.58.62    prometheus

Go to /etc/prometheus directory and open the prometheus.yml file.

Under scrape_configs We need to add the below configuration as shown below.

- job_name: 'node_exporter'
 scrape_interval: 5s
  static_configs:
    - targets: ['node_exporter:9100']

Finally the prometheus.yml file should be as shown below.

Save and close the file and restart the prometheus service.

systemctl restart prometheus

If you have installed and configured prometheus and node exporter on the EC2 Instance , The port 9100 should be whitelisted in the security group so that the prometheus server is able to collect the metrics of the node exporter.

Verifying The Installation

To verify the installation and the scraping of metrics , Open the prometheus web interface ,

http://publicipaddress:9090

Under Status , Choose Targets

You should be able to see the node_exporter server is added for monitoring.

Go to http://publicipaddress:9090/graph and run the below query

node_exporter_build_info

You can check the version of the node exporter installed.

Conclusion

We have successfully installed and configured Prometheus and Node exporter.

Hope you find it helpful. Please do check out my other articles.