What is Prometheus?

Prometheus is an opensource monitoring and alerting system.

Prometheus works well for recording any purely numeric time series. It fits both machine-centric monitoring as well as monitoring of highly dynamic service-oriented architectures. Its support for multi-dimensional data collection and querying is a particular strength.

Prometheus can be easily integrated with Grafana and the grafana can use the prometheus as datasource for monitoring and alerting.

What are the components of prometheus?

The prometheus consists of many components as follows.

  • Prometheus server – Prometheus is well-known for collecting and storing the metrics of the client servers and the applications. IT scraps the information and store it as time series data
  • Exporters – exporter such as node_exporter , blackbox_exporter are used to collect the metrics form the client server and sent it to prometheus server
  • Alert Manager – prometheus has an alerting mechanism known as alert manager which helps us to alert us in case of abnormalities.
  • client libraries – which is used for instrumenting application code

Installing Prometheus on Ubuntu

As we are going to install and configure prometheus on Ubuntu EC2 instance , You can refer this detailed article , explaining how to create EC2 instance using AWS console.

Prometheus can be installed using the precompiled binaries or using docker images.

The prometheus packages are provided for Linux , MacOs and Windows operating systems.

We can download the respective packages as per the operating system from the below link

https://prometheus.io/download/

I have used wget to download the prometheus package in the Ubuntu EC2 instance.

wget https://github.com/prometheus/prometheus/releases/download/v2.18.1/prometheus-2.18.1.linux-amd64.tar.gz
prometheus

Once the archive is downloaded , Extract is using the below command.

tar -xvzf prometheus-2.18.1.linux-amd64.tar.gz
prometheus

Once the archive is extracted , You will find a folder prometheus-2.18.1.linux-amd64 with all the prometheus files.

we need a seperate usre and the group for the prometheus.

Create prometheus system group

prometheus runs with its own username and the group.

Lets create a system group named prometheus

groupadd --system prometheus

Create prometheus system user

We have created a system group for prometheus , lets create a prometheus user and add the user to the group.

useradd -s /sbin/nologin -r -g prometheus prometheus

Setup directories for prometheus

We need to create separate folders to store prometheus data and the prometheus configurarion files.

/var/lib/prometheus – Will be used to store prometheus data

/etc/prometheus – will be used to store configuration files

mkdir -p /var/lib/prometheus
mkdir -p /etc/prometheus

Copying prometheus binary files

We need to copy the prometheus binary files to the directory where all the user libraries will be stored.

cp /opt/prometheus-2.18.1.linux-amd64/prometheus /usr/local/bin/
cp /opt/prometheus-2.18.1.linux-amd64/promtool /usr/local/bin/

Copying prometheus console and console libraries to the prometheus configuration directory

Prometheus consoles files and console_libraries should be stored under /etc/prometheus diretory

cp -r /opt/prometheus-2.18.1.linux-amd64/consoles  /etc/prometheus/
cp -r /opt/prometheus-2.18.1.linux-amd64/console_libraries /etc/prometheus/

cp /opt/prometheus-2.18.1.linux-amd64/prometheus.yml  /etc/prometheus/

Changing directory ownership

We have created separate user and the group for the prometheus as the configuration files and the data directory should be owned by prometheus to run the prometheus service.

chown -R prometheus:prometheus /etc/prometheus/  /var/lib/prometheus/
chmod -R 775 /etc/prometheus/ /var/lib/prometheus/

Setup systemd file for prometheus

We need to create and configure systemd unit file for the us to manage the prometheus service at ease.

The name of the systemd file should end with .service and it has to be created under /etc/systemd/system directory.

vi /etc/systemd/system/prometheus.service

and paste the below content into the file.

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/console_libraries
[Install]
WantedBy=multi-user.target

Save and close the file.

Configuring prometheus configuration file

The configuration file prometheus.yml is stored in /etc/prometheus directory.

prometheus

Here the target is prometheus itself.

Start the prometheus service

To start the prometheus service , run the below command.

systemctl start prometheus

Command to start the prometheus service on system boot

systemctl enable prometheus
Created symlink /etc/systemd/system/multi-user.target.wants/prometheus.service → /etc/systemd/system/prometheus.service.

Lets check the status of the prometheus service.

systemctl status prometheus

You should that the prometheus service is UP and running.

prometheus

We can also check , On what port prometheus is listening.To check that , run the below command.

The prometheus service is listening on the port 9090.

netstat -nltp |grep prometheus
tcp6 0 0 :::9090 :::* LISTEN 2296/prometheus

As we are setting up prometheus on the EC2 instance , We need to whitelist the port 9090 on the security group.

To do that , Login to EC2 console , Choose Instances

prometheus

Select the instance where you have configured prometheus.

Under the description , select the security group.

prometheus

and then click Inbound rules

prometheus

select Edit inbound rules, click Add rule

Add the rule as shown below. Temporarily 9090 is allowed for public which is not recommended.

prometheus

and then click Save rules

Now get the public ip address of the EC2 instance , We can access the prometheus web interface as shown below

http://publicipaddress:9090

You will get the following page.

prometheus

Under Status , Click Targets , We can see all the targets monitored by the prometheus.

prometheus

Conclusion

We have installed and configured a monitoring setup , Prometheus using which we can store the metrics of the target servers and applications.

Hope you find it helpful.Please do check out my other publications.