In my previous article , I have explained How to setup ELK stack on Ubuntu.
In this guide , I will show to how to setup Filebeat , Where our applications or web servers are running and then we will configure filebeat to send data to Logstash which will then transfer the logs to Elasticsearch.
What is Filebeat?
Filebeat is a lightweight shipper for forwarding and centralizing log data.
It will be installed as an agent on your servers where our application or web servers are running.
Filebeat monitors the log files or locations that you specify, collects log events, and forwards them either to Elasticsearch or Logstash for indexing.
Install & Configure Filebeat
Before installing filebeat in the server , We need to copy the logstash certificate which we have generated during the Logstash setup.
The Logstash certificate will be stored under /etc/logstash/ssl folder.
using the scp command , We will copy the certificate from the ELK stack server.
scp - r [email protected]:/etc/logstash/ssl/logstash-forwarder.crt .
Next , We will add the elasticsearch repo and install the filebeat package.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
Update the repository and let’s install the filebeat package from the repo.
sudo apt-get update
sudo apt-get install filebeat -y
Copy the logstash certificate to /etc/filebeat folder.
Then go to /etc/filebeat folder and open the filebeat.yml file ,remove the exising configuration and paste the below configuration.
filebeat.inputs: - type: log enabled: true paths: - /var/log/nginx/access.log output.logstash: hosts: ["ELK-SERVER-IP:5443"]
Save and Close the file.
In this example , I am sending Nginx access logs from Filebeat to Logstash.
Once the Logstash collects the data from filebeat , It will be then sent to Elasticsearch.
Lets start the Filebeat service and enable to start on system boot.
sudo systemctl start filebeat
sudo systemctl enable filebeat
To check the status of the filebeat,
sudo systemctl status filebeat
To check the logs of the filebeat ,
sudo tail -f /var/log/filebeat/filebeat
We have successfully installed and configured filebeat and for example , We have configured filebeat to send Nginx access Logs to logstash.
Now that We have a complete Log management tool called ELK STACK which the agent called FILEBEAT to send the Logs to ELK stack.
Hope you find it helpful. Thanks for reading this article.
Please do check out my other articles.