[[configuration]]
= Configuration

[partintro]
--

These are the higher-level configuration settings used by the configuration
files.  <<actions,Actions>> and <<filters,filters>> are documented separately.

* <<envvars,Environment Variables>>
* <<actionfile,Action File>>
* <<configfile,Configuration File>>
--

[[envvars]]
== Environment Variables

WARNING: This functionality is experimental and may be changed or removed +
    completely in a future release.

You can use environment variable references in both the
<<configfile,configuration file>> and the <<actionfile,action file>> to set
values that need to be configurable at runtime. To do this, use:

[source,sh]
-------
${VAR}
-------

Where `VAR` is the name of the environment variable.

Each variable reference is replaced at startup by the value of the environment
variable. The replacement is case-sensitive and occurs while the YAML file is
parsed, but before configuration schema validation. References to undefined
variables are replaced by `None` unless you specify a default value. To specify
a default value, use:

[source,sh]
-------
${VAR:default_value}
-------

Where `default_value` is the value to use if the environment variable is
undefined.

[IMPORTANT]
.Unsupported use cases
=================================================================
When using environment variables, the value must _only_ be the environment
variable.

Using extra text, such as:

[source,sh]
-------
logfile: ${LOGPATH}/extra/path/information/file.log
-------

is not supported at this time.
=================================================================

=== Examples

Here are some examples of configurations that use environment variables
and what each configuration looks like after replacement:

[options="header"]
|==================================
|Config source	       |Environment setting    |Config after replacement
|`unit: ${UNIT}`       |`export UNIT=days`     |`unit: days`
|`unit: ${UNIT}`       |no setting             |`unit:`
|`unit: ${UNIT:days}`  |no setting             |`unit: days`
|`unit: ${UNIT:days}`  |`export UNIT=hours`    |`unit: hours`
|==================================


[[actionfile]]
== Action File

NOTE: You can use <<envvars,environment variables>> in your configuration
  files.

An action file has the following structure:

[source,sh]
-----------
---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True.  If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
  1:
    action: ACTION1
    description: OPTIONAL DESCRIPTION
    options:
      option1: value1
      ...
      optionN: valueN
      continue_if_exception: False
      disable_action: True
    filters:
    - filtertype: *first*
      filter_element1: value1
      ...
      filter_elementN: valueN
    - filtertype: *second*
      filter_element1: value1
      ...
      filter_elementN: valueN
  2:
    action: ACTION2
    description: OPTIONAL DESCRIPTION
    options:
      option1: value1
      ...
      optionN: valueN
      continue_if_exception: False
      disable_action: True
    filters:
    - filtertype: *first*
      filter_element1: value1
      ...
      filter_elementN: valueN
    - filtertype: *second*
      filter_element1: value1
      ...
      filter_elementN: valueN
  3:
    action: ACTION3
    ...
  4:
    action: ACTION4
    ...
-----------

It is a YAML configuration file.  The root key must be `actions`, after which
there can be any number of actions, nested underneath numbers.  Actions will be
taken in the order they are completed.

The high-level elements of each numbered action are:

* <<actions,action>>
* <<description,description>>
* <<options,options>>
* <<filters,filters>>

In the case of the <<alias,alias action>>, there are two additional high-level
elements: `add` and `remove`, which are described in the <<alias,alias action>>
documentation.

[[description]]
=== description

This is an optional description which can help describe what the action and its
filters are supposed to do.

[source,yaml]
-------------
description: >- I can make the description span multiple
    lines by putting ">-" at the beginning of the line,
    as seen above.  Subsequent lines must also be indented.
options:
  option1: ...
-------------

[[configfile]]
== Configuration File

NOTE: The default location of the configuration file is `~/.curator/curator.yml`,
    but another location can be specified using the `--config` flag on the
    <<command-line,command-line>>.


NOTE: You can use <<envvars,environment variables>> in your configuration
  files.

The configuration file contains client connection and settings for logging.  It
looks like this:

[source,sh]
-----------
---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - 127.0.0.1
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  username:
  password:
  timeout: 30
  master_only: False

logging:
  loglevel: INFO
  logfile:
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']
-----------

It is a YAML configuration file.  The two root keys must be `client` and
`logging`.  The subkeys of each of these will be described here.

[[hosts]]
=== hosts

This can be a single value:

[source,sh]
-----------
hosts: 127.0.0.1
-----------

Or multiple values in the 3 acceptable YAML ways to render sequences, or arrays:

WARNING: Curator can only work with one cluster at a time.  Including clients
from multiple clusters in the `hosts` setting will result in errors.

Flow:

[source,sh]
-----------
hosts: [ "10.0.0.1", "10.0.0.2" ]
-----------

Spanning:

[source,sh]
-----------
hosts: [ "10.0.0.1",
    "10.0.0.2" ]
-----------

Block:
[source,sh]
-----------
hosts:
  - 10.0.0.1
  - 10.0.0.2
-----------

You can also provide these hosts with optional ports, and bypass the port
option:

[source,sh]
-----------
hosts:
  - 10.0.0.1:9200
  - 10.0.0.2:9201
-----------

WARNING: When adding a port to the end of a host or IP, the YAML Flow and
    Spanning styles require `host:port` to be single `'` or double `"` quote
    encapsulated or you will receive an error.  The Block style does not have
    this limitation.

[[port]]
=== port

This should be a single value:

[source,sh]
-----------
port: 9200
-----------

The default is `9200`.  This value will only be applied to <<hosts,hosts>>
without a port affixed, e.g. `localhost:9202`.

[[url_prefix]]
=== url_prefix

This should be a single value or left empty.

[source,sh]
-----------
url_prefix:
-----------

In some cases you may be obliged to connect to your Elasticsearch cluster
through a proxy of some kind. There may be a URL prefix before the API URI
items, e.g. http://example.com/elasticsearch/ as opposed to
http://localhost:9200. In such a case, set the `url_prefix` to the
appropriate value, 'elasticsearch' in this example.

The default is an empty string.

[[use_ssl]]
=== use_ssl

This should be `True`, `False` or left empty.

[source,sh]
-----------
use_ssl:
-----------

If access to your Elasticsearch instance is protected by SSL encryption, you
must use set `use_ssl` to `True`.

The default is `False`

[[certificate]]
=== certificate

This should be a file path to your CA certificate, or left empty.

[source,sh]
-----------
certificate:
-----------

This setting allows the use of a specified CA certificate file to validate the
SSL certificate used by Elasticsearch.

There is no default.

include::inc_filepath.asciidoc[]

[[client_cert]]
=== client_cert

This should be a file path to a client certificate (public key), or left empty.


[source,sh]
-----------
client_cert:
-----------

Allows the use of a specified SSL client cert file to authenticate to
Elasticsearch. The file may contain both an SSL client certificate and an SSL
key, in which case <<client_key,client_key>> is not used. If specifying
`client_cert`, and the file specified does not also contain the key, use
<<client_key,client_key>> to specify the file containing the SSL key. The file
must be in PEM format, and the key part, if used, must be an unencrypted key in
PEM format as well.

include::inc_filepath.asciidoc[]

[[client_key]]
=== client_key

This should be a file path to a client key (private key), or left empty.

[source,sh]
-----------
client_key:
-----------

Allows the use of a specified SSL client key file to authenticate to
Elasticsearch. If using <<client_cert,client_cert>> and the file specified does
not also contain the key, use `client_key` to specify the file containing the
SSL key. The key file must be an unencrypted key in PEM format.

include::inc_filepath.asciidoc[]

[[aws_key]]
=== aws_key

WARNING: This feature has not been fully tested and should be considered BETA.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS IAM access key, or left empty.

[source,sh]
-----------
aws_key:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

IMPORTANT: This feature may be deprecated in a future release. You should consider
    using <<aws_sign_request,aws_sign_request>> instead.

[[aws_secret_key]]
=== aws_secret_key

WARNING: This feature has not been fully tested and should be considered BETA.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS IAM secret access key, or left empty.

[source,sh]
-----------
aws_secret_key:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

IMPORTANT: This feature may be deprecated in a future release. You should consider
    using <<aws_sign_request,aws_sign_request>> instead.

[[aws_region]]
=== aws_region

WARNING: This feature has not been fully tested and should be considered BETA.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS region, or left empty.

[source,sh]
-----------
aws_region:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

[[aws_sign_request]]
=== aws_sign_request

WARNING: This feature has not been fully tested and should be considered BETA.

WARNING: If installing via `pip`, this setting will not work unless the `requests-aws4auth`
    and `boto3` Python modules have been manually installed first.

WARNING: Credentials found in your environment will replace the data specified in
    <<aws_key,aws_key>> and <<aws_secret_key,aws_secret_key>>

This should be `True` if you want your requests to be signed with credentials retrieved
    from your environment. The order in which credentials will be searched for is:

    . Environment variables
    . Shared credential file (~/.aws/credentials)
    . AWS config file (~/.aws/config)
    . Boto2 config file (/etc/boto.cfg and ~/.boto)
    . Instance metadata service on an Amazon EC2 instance that has an IAM role configured.

The default value is `False`.

[source,sh]
-----------
aws_sign_request: True
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.



[[ssl_no_validate]]
=== ssl_no_validate

This should be `True`, `False` or left empty.

[source,sh]
-----------
ssl_no_validate:
-----------

If access to your Elasticsearch instance is protected by SSL encryption, you may
set `ssl_no_validate` to `True` to disable SSL certificate verification.

Valid use cases for doing so include the use of self-signed certificates that
cannot be otherwise verified and would generate error messages.

WARNING: Setting `ssl_no_validate` to `True` will likely result in a warning
    message that your SSL certificates are not trusted. This is expected
    behavior.

The default value is `False`.

[[http_auth]]
=== http_auth

**NOTE: This setting is now deprecated. Please use `username` and `password` instead.**

This should be a authentication credentials (e.g. `user:pass`), or left empty.

[source,sh]
-----------
http_auth:
-----------

This setting allows basic HTTP authentication to an Elasticsearch instance.

The default is empty.

[[username]]
=== username

The HTTP Basic Authentication username

[[password]]
=== password

The HTTP Basic Authentication password

[[apikey_auth]]
=== apikey_auth

This should a base64 encoded token (e.g: `id:api_key`).

[source,sh]
---------
apikey_auth:
---------

This setting allows API Key authentication to an Elasticsearch instance.

The default is empty.

[[timeout]]
=== timeout

This should be an integer number of seconds, or left empty.

[source,sh]
-----------
timeout:
-----------

You can change the default client connection timeout value with this setting.

The default value is `30` (seconds) should typically not be changed to be very
large.  If a longer timeout is necessary for a given action, such as
<<snapshot,snapshot>>, <<restore,restore>>, or <<forcemerge,forcemerge>>, the
client timeout can be overridden on per action basis by setting
<<option_timeout_override,timeout_override>> in the action <<options,options>>.
There are default override values for some of those longer running actions.

[[master_only]]
=== master_only

This should be `True`, `False` or left empty.

[source,sh]
-----------
master_only:
-----------

In some situations, primarily with automated deployments, it makes sense to
install Curator on every node. But you wouldn’t want it to run on each node.
By setting `master_only` to `True`, this is possible. It tests for, and will
only continue running on the node that is the elected master.

WARNING: If `master_only` is `True`, and <<hosts,hosts>> has more than one
    value, Curator will raise an Exception.  This setting should _only_ be used
    with a single host in <<hosts,hosts>>, as its utility centers around
    deploying to all nodes in the cluster.

The default value is `False`.

[[loglevel]]
=== loglevel

This should be `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`, or left empty.

[source,sh]
-----------
loglevel:
-----------

Set the minimum acceptable log severity to display.

* `CRITICAL` will only display critical messages.
* `ERROR` will only display error and critical messages.
* `WARNING` will display error, warning, and critical messages.
* `INFO` will display informational, error, warning, and critical messages.
* `DEBUG` will display debug messages, in addition to all of the above.

The default value is `INFO`.

[[logfile]]
=== logfile

This should be a path to a log file, or left empty.

[source,sh]
-----------
logfile:
-----------

include::inc_filepath.asciidoc[]

The default value is empty, which will result in logging to `STDOUT`, or the
console.

[[logformat]]
=== logformat

This should `default`, `json`, `logstash`, `ecs` or left empty.

[source,sh]
-----------
logformat:
-----------

The `default` format looks like:

[source,sh]
-----------
2016-04-22 11:53:09,972 INFO      Action #1: ACTIONNAME
-----------

The `json` or `logstash` formats look like:

[source,sh]
-----------
{"@timestamp": "2016-04-22T11:54:29.033Z", "function": "cli", "linenum": 178,
"loglevel": "INFO", "message": "Action #1: ACTIONNAME", "name": "curator.cli"}
-----------

The `ecs` format looks like:
[source,sh]
-----------
{"@timestamp": "2020-02-22T11:55:00.022Z", "log.origin.function": "cli", "log.origin.file.line": 178,
"log.level": "INFO", "message": "Action #1: ACTIONNAME", "log.logger": "curator.cli"}
-----------

The default value is `default`.

[[blacklist]]
=== blacklist

This should be an empty array `[]`, an array of log handler strings, or left
empty.

[source,sh]
-----------
blacklist: ['elasticsearch', 'urllib3']
-----------

The default value is `['elasticsearch', 'urllib3']`, which will result in
logs for the `elasticsearch` and `urllib3` Python modules _not_ being output.
These can be quite verbose, so unless you need them to debug an issue, you
should accept the default value.

TIP: If you do need to troubleshoot an issue, set `blacklist` to `[]`, which is
an empty array.  Leaving it unset will result in the default behavior, which is
to filter out `elasticsearch` and `urllib3` log traffic.
