Elasticsearch is very good in collecting data. A lot of this data only usable for a short amount of time and Elasticsearch is well known to consume lots of ram when there are many indices(shards) added. To archive or delete these indices we can use the tool Curator. It’s very easy to install and configure and can be up and running within 10 minutes. I added a small example to delete Filebeat indices after 10 days.
Git repo: https://github.com/GeekintheMiddle/SysadminScripts
Configfile Language: YML
Operating system: Debian/Ubuntu
How to install
First we install the Curator through the PIP installer.
pip install elasticsearch-curator
After the installation we add the necessary configuration files. The YML configuration files can be added in the /etc/elasticsearch folder. These files are found within my SysadminScripts git repo.
We start by doing a dry run of the curator command to see if it has the desired outcome.
/usr/local/bin/curator /etc/elasticsearch/delete_filebeat.yml --config /etc/elasticsearch/curator.yml --dry-run
If the output of the command is correct we wil add the following command to your crontab (/etc/crontab). This will run the Curator command every night at half past 12.
30 0 * * * root /usr/local/bin/curator /etc/elasticsearch/delete_filebeat.yml –config /etc/elasticsearch/curator.yml