Description

This watcher download provides the possibility to move existing indices into other ILM phases. This specific case was made to move based on disc usage per zone. This is especially helpful if you are short on storage capacity in one of your data tiering zones.

With the default implementation of ILM a user only can move the indices based on the time after the rollover or creation of the index has happened. While this works fine to configure your retention time it could be also challenging to configure it correctly to not hit high storage water marks e.g. in Elastic Cloud.

This watcher job can prevent hitting the thresholds in your data tiering zone by moving an index before the retention has reached. Be careful: This could also lead into situations where your data is moved very early. In those cases it can also influence the performance of your Kibana dashboards or Logs UI in case your timeframe is including data from an index that has already been moved to warm.

To move an index to the next ILM phase using this watcher jobs this download includes:

A job that is extracting the index information using the Kibana API. It will store all the relevant information into a separate index that can be used for further analysis.
A job that is taking the data of the first job and is checking for specific rules like disc threshold for all indices in one single zone. If the disc threshold is higher than expected the watcher job is triggering the ILM move API of Elasticsearch. This is an expert API and hard to configure. You need to know which is your target state of ILM.