Elasticdump To Backup & Restore Indexes

What Is Elasticdump?

Elasticdump is an import and export tool for elasticsearch to backup and restore Elasticsearch indices into JSON file and store it in the Disk or in S3 Bucket.

Use Cases:

Elasticdump works by sending an input to an ouput.It can either be a elasticsearch URL or a File.

Elasticsearch:

format: {protocol}://{host}:{port}/{index}

Example: http://127.0.0.1:9200/index_name

File:

format: {filepath}

Example: /home/rahulk/backup/esdump.json

Before installing elasticdump, Make sure you have npm and node.js packages installed in your machine.

If you don’t have, follow this article on how to install Node.js and npm packages and then we will install elasticdump using npm.

Installation Of Node.Js & NPM:

NPM – Node package Manager.

To install npm and node.js on Ubuntu, perform the following steps.

01) Enable the Nodesource repository by running the following command.

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -

02)Once the repository is enabled in the system,Install Node.js and npm using the below command.

apt-get install nodejs

It contains both the node and npm packages.

03)To verify the installation and to check the versions, Run the following commands.

root@IA-767-RahulK:~# node -v
v10.18.1
root@IA-767-RahulK:~# npm -v 6.13.4

Once we have node.js and npm in place , Lets go ahead and install elasticdump.

Installing Elasticdump:

Now run the below command to install elasticdump in your machine.

Local installation:

npm install elasticdump

Global installation:

npm install elasticdump -g

Now we have elasticdump installed in our machine.

root@IA-767-RahulK:~# elasticdump --version
6.16.2

Multiple Ways Of Managing Indexes:

You can use elasticdump in multiple ways , Let’s see one by one on how it helps to backup and restore indexes.

Backup Index Data Into A File:

First we need to backup the mapping of the index and then we need to backup the data of the index:

Using the below commands,

elasticdump --input=http://host:9200/index_name --output=/es-backup/index_name_mapping.json --type=mapping
elasticdump --input=http://host:9200/index_name --output=/es-backup/index_name.json --type=data

Copying Index From Production To Staging ES Using Analyzer And Mapping:

Make sure production server is able to communicate with staging server on port 9200.

We can directly copy an index from production to staging elasticsearch using the below command,

elasticdump --input=http://productiones.com:9200/index_name --output=http://staginges.com:9200/index_name --type=analyzer
elasticdump --input=http://productiones.com:9200/index_name --output=http://staginges.com:9200/index_name --type=mapping
elasticdump --input=http://productiones.com:9200/index_name --output=http://staginges.com:9200/index_name --type=data

Backup Index To S3 Bucket:

We need Access key Id and Secret Access key with S3 permissions to upload Elasticsearch index to the S3 Bucket.

Using the below command,

elasticdump --s3AccessKeyId "${access_key_id}" --s3SecretAccessKey "${access_key_secret}" --input=http://production.es.com:9200/my_index --output "s3://${bucket_name}/${file_name}.json"

Import Index From S3 Bucket To Elasticsearch:

If you already have a backup of indexes in the S3 bucket and you want to import it to the existing Elasticsearch, Use the below command.

elasticdump --s3AccessKeyId "${access_key_id}" --s3SecretAccessKey "${access_key_secret}" --input "s3://${bucket_name}/${file_name}.json" --output=http://production.es.com:9200/my_index

Restoring Mapping And Index Data:

If you have a index backup in the Disk and you want to import it to the existing Elasticsearch setup, Use the below commands.

elasticdump --input=/es-backup/index_name_mapping.json --output=http://host:9200/index_name --type=mapping
elasticdump  --input=/es-backup/index_name.json --output=http://host:9200/index_name --type=data

Backup And Index To Gzip Using Stdout:

In order to maintain the Disk space in the server while storing the Elasticsearch index backup , You can compress and store it, Using the below command.

elasticdump --input=http://productiones.com:9200/index_name --output=$
| gzip > /esbackup/index_name.json.gz

To Know More About Elasticdump:

If you would like to know more about elasticdump , You can use the below command.

elasticdump --help

Thanks for reading this articles.

Please do check out my other posts.