[[configuration]]
= Configuration

[partintro]
--

These are the higher-level configuration settings used by the configuration
files.  <<actions,Actions>> and <<filters,filters>> are documented separately.

* <<envvars,Environment Variables>>
* <<actionfile,Action File>>
* <<configfile,Configuration File>>
--

[[envvars]]
== Environment Variables

WARNING: This functionality is experimental and may be changed or removed +
    completely in a future release.

You can use environment variable references in the both the
<<configfile,configuration file>> and the <<actionfile,action file>> to set
values that need to be configurable at runtime. To do this, use:

[source,sh]
-------
${VAR}
-------

Where `VAR` is the name of the environment variable.

Each variable reference is replaced at startup by the value of the environment
variable. The replacement is case-sensitive and occurs while the YAML file is
parsed, but before configuration schema validation. References to undefined
variables are replaced by `None` unless you specify a default value. To specify
a default value, use:

[source,sh]
-------
${VAR:default_value}
-------

Where `default_value` is the value to use if the environment variable is
undefined.

[IMPORTANT]
.Unsupported use cases
=================================================================
When using environment variables, the value must _only_ be the environment
variable.

Using extra text, such as:

[source,sh]
-------
logfile: ${LOGPATH}/extra/path/information/file.log
-------

is not supported at this time.
=================================================================

=== Examples

Here are some examples of configurations that use environment variables
and what each configuration looks like after replacement:

[options="header"]
|==================================
|Config source	       |Environment setting    |Config after replacement
|`unit: ${UNIT}`       |`export UNIT=days`     |`unit: days`
|`unit: ${UNIT}`       |no setting             |`unit:`
|`unit: ${UNIT:days}`  |no setting             |`unit: days`
|`unit: ${UNIT:days}`  |`export UNIT=hours`    |`unit: hours`
|==================================


[[actionfile]]
== Action File

NOTE: Starting in Curator 4.1, you can use <<envvars,environment variables>> in
    your configuration files.

An action file has the following structure:

[source,sh]
-----------
---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True.  If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
  1:
    action: ACTION1
    description: OPTIONAL DESCRIPTION
    options:
      option1: value1
      ...
      optionN: valueN
      continue_if_exception: False
      disable_action: True
    filters:
    - filtertype: *first*
      filter_element1: value1
      ...
      filter_elementN: valueN
    - filtertype: *second*
      filter_element1: value1
      ...
      filter_elementN: valueN
  2:
    action: ACTION2
    description: OPTIONAL DESCRIPTION
    options:
      option1: value1
      ...
      optionN: valueN
      continue_if_exception: False
      disable_action: True
    filters:
    - filtertype: *first*
      filter_element1: value1
      ...
      filter_elementN: valueN
    - filtertype: *second*
      filter_element1: value1
      ...
      filter_elementN: valueN
  3:
    action: ACTION3
    ...
  4:
    action: ACTION4
    ...
-----------

It is a YAML configuration file.  The root key must be `actions`, after which
there can be any number of actions, nested underneath numbers.  Actions will be
taken in the order they are completed.

The high-level elements of each numbered action are:

* <<actions,action>>
* <<description,description>>
* <<options,options>>
* <<filters,filters>>

In the case of the <<alias,alias action>>, there are two additional high-level
elements: `add` and `remove`, which are described in the <<alias,alias action>>
documentation.

[[description]]
=== description

This is an optional description which can help describe what the action and its
filters are supposed to do.

[source,text]
-------------
description: >- I can make the description span multiple
    lines by putting ">-" at the beginning of the line,
    as seen above.  Subsequent lines must also be indented.
options:
  option1: ...
-------------

[[configfile]]
== Configuration File

NOTE: The default location of the configuration file is `~/.curator/curator.yml`,
    but another location can be specified using the `--config` flag on the
    <<command-line,command-line>>.


NOTE: Starting in Curator 4.1, you can use <<envvars,environment variables>> in
    your configuration files.

The configuration file contains client connection and settings for logging.  It
looks like this:

[source,sh]
-----------
---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - 127.0.0.1
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 30
  master_only: False

logging:
  loglevel: INFO
  logfile:
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']
-----------

It is a YAML configuration file.  The two root keys must be `client` and
`logging`.  The subkeys of each of these will be described here.

[[hosts]]
=== hosts

This can be a single value:

[source,sh]
-----------
hosts: 127.0.0.1
-----------

Or multiple values in the 3 acceptable YAML ways to render sequences, or arrays:

Flow:

[source,sh]
-----------
hosts: [ "10.0.0.1", "10.0.0.2" ]
-----------

Spanning:

[source,sh]
-----------
hosts: [ "10.0.0.1",
    "10.0.0.2" ]
-----------

Block:
[source,sh]
-----------
hosts:
  - 10.0.0.1
  - 10.0.0.2
-----------

You can also provide these hosts with optional ports, and bypass the port
option:

[source,sh]
-----------
hosts:
  - 10.0.0.1:9200
  - 10.0.0.2:9201
-----------

WARNING: When adding a port to the end of a host or IP, the YAML Flow and
    Spanning styles require `host:port` to be single `'` or double `"` quote
    encapsulated or you will receive an error.  The Block style does not have
    this limitation.

[[port]]
=== port

This should be a single value:

[source,sh]
-----------
port: 9200
-----------

The default is `9200`.  This value will only be applied to <<hosts,hosts>>
without a port affixed, e.g. `localhost:9202`.

[[url_prefix]]
=== url_prefix

This should be a single value or left empty.

[source,sh]
-----------
url_prefix:
-----------

In some cases you may be obliged to connect to your Elasticsearch cluster
through a proxy of some kind. There may be a URL prefix before the API URI
items, e.g. http://example.com/elasticsearch/ as opposed to
http://localhost:9200. In such a case, the set the `url_prefix` to the
appropriate value, 'elasticsearch' in this example.

The default is an empty string.

[[use_ssl]]
=== use_ssl

This should be `True`, `False` or left empty.

[source,sh]
-----------
use_ssl:
-----------

If access to your Elasticsearch instance is protected by SSL encryption, you
must use set `use_ssl` to `True`.

The default is `False`

[[certificate]]
=== certificate

This should be a file path to your CA certificate, or left empty.

[source,sh]
-----------
certificate:
-----------

This setting allows the use of a specified CA certificate file to validate the
SSL certificate used by Elasticsearch.

There is no default.

[[client_cert]]
=== client_cert

This should be a file path to a client certificate (public key), or left empty.

[source,sh]
-----------
client_cert:
-----------

Allows the use of a specified SSL client cert file to authenticate to
Elasticsearch. The file may contain both an SSL client certificate and an SSL
key, in which case <<client_key,client_key>> is not used. If specifying
`client_cert`, and the file specified does not also contain the key, use
<<client_key,client_key>> to specify the file containing the SSL key. The file
must be in PEM format, and the key part, if used, must be an unencrypted key in
PEM format as well.

[[client_key]]
=== client_key

This should be a file path to a client key (private key), or left empty.

[source,sh]
-----------
client_key:
-----------

Allows the use of a specified SSL client key file to authenticate to
Elasticsearch. If using <<client_cert,client_cert>> and the file specified does
not also contain the key, use `client_key` to specify the file containing the
SSL key. The key file must be an unencrypted key in PEM format.

[[aws_key]]
=== aws_key

WARNING: This feature allows connection to AWS using IAM credentials, but
    <<faq_aws_iam,Curator 4 does not currently work with AWS>>.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS IAM access key, or left empty.

[source,sh]
-----------
aws_key:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

[[aws_secret_key]]
=== aws_secret_key

WARNING: This feature allows connection to AWS using IAM credentials, but
    <<faq_aws_iam,Curator 4 does not currently work with AWS>>.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS IAM secret access key, or left empty.

[source,sh]
-----------
aws_secret_key:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

[[aws_region]]
=== aws_region

WARNING: This feature allows connection to AWS using IAM credentials, but
    <<faq_aws_iam,Curator 4 does not currently work with AWS>>.

WARNING: This setting will not work unless the `requests-aws4auth` Python module
    has been manually installed first.

This should be an AWS region, or left empty.

[source,sh]
-----------
aws_region:
-----------

IMPORTANT: You must set your <<hosts,hosts>> to the proper hostname _with_ port.
    It may not work setting <<port,port>> and <<hosts,hosts>> to only a host
    name due to the different connection module used.

[[ssl_no_validate]]
=== ssl_no_validate

This should be `True`, `False` or left empty.

[source,sh]
-----------
ssl_no_validate:
-----------

If access to your Elasticsearch instance is protected by SSL encryption, you may
set `ssl_no_validate` to `True` to disable SSL certificate verification.

Valid use cases for doing so include the use of self-signed certificates that
cannot be otherwise verified and would generate error messages.

WARNING: Setting `ssl_no_validate` to `True` will likely result in a warning
    message that your SSL certificates are not trusted. This is expected
    behavior.

The default value is `False`.

[[http_auth]]
=== http_auth

This should be a authentication credentials (e.g. `user:pass`), or left empty.

[source,sh]
-----------
http_auth:
-----------

This setting allows basic HTTP authentication to an Elasticsearch instance.

The default is empty.

[[timeout]]
=== timeout

This should be an integer number of seconds, or left empty.

[source,sh]
-----------
timeout:
-----------

You can change the default client connection timeout value with this setting.

The default value is `30` (seconds) should typically not be changed to be very
large.  If a longer timeout is necessary for a given action, such as
<<snapshot,snapshot>>, <<restore,restore>>, or <<forcemerge,forcemerge>>, the
client timeout can be overridden on per action basis by setting
<<option_timeout_override,timeout_override>> in the action <<options,options>>.
There are default override values for some of those longer running actions.

[[master_only]]
=== master_only

This should be `True`, `False` or left empty.

[source,sh]
-----------
master_only:
-----------

In some situations, primarily with automated deployments, it makes sense to
install Curator on every node. But you wouldn’t want it to run on each node.
By setting `master_only` to `True`, this is possible. It tests for, and will
only continue running on the node that is the elected master.

WARNING: If `master_only` is `True`, and <<hosts,hosts>> has more than one
    value, Curator will raise an Exception.  This setting should _only_ be used
    with a single host in <<hosts,hosts>>, as its utility centers around
    deploying to all nodes in the cluster.

The default value is `False`.

[[loglevel]]
=== loglevel

This should be `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`, or left empty.

[source,sh]
-----------
loglevel:
-----------

Set the minimum acceptable log severity to display.

* `CRITICAL` will only display critical messages.
* `ERROR` will only display error and critical messages.
* `WARN` will display error, warning, and critical messages.
* `INFO` will display informational, error, warning, and critical messages.
* `DEBUG` will display debug messages, in addition to all of the above.

The default value is `INFO`.

[[logfile]]
=== logfile

This should be a path to a log file, or left empty.

[source,sh]
-----------
logfile:
-----------

The default value is empty, which will result in logging to `STDOUT`, or the
console.

[[logformat]]
=== logformat

This should `default`, `json`, `logstash`, or left empty.

[source,sh]
-----------
logformat:
-----------

The `default` format looks like:

[source,sh]
-----------
2016-04-22 11:53:09,972 INFO      Action #1: ACTIONNAME
-----------

The `json` or `logstash` formats look like:

[source,sh]
-----------
{"@timestamp": "2016-04-22T11:54:29.033Z", "function": "cli", "linenum": 178,
"loglevel": "INFO", "message": "Action #1: ACTIONNAME", "name": "curator.cli"}
-----------

The default value is `default`.

[[blacklist]]
=== blacklist

This should be an empty array `[]`, an array of log handler strings, or left
empty.

[source,sh]
-----------
blacklist: ['elasticsearch', 'urllib3']
-----------

The default value is `['elasticsearch', 'urllib3']`, which will result in
logs for the `elasticsearch` and `urllib3` Python modules _not_ being output.
These can be quite verbose, so unless you need them to debug an issue, you
should accept the default value.

TIP: If you do need to troubleshoot an issue, set `blacklist` to `[]`, which is
an empty array.  Leaving it unset will result in the default behavior, which is
to filter out `elasticsearch` and `urllib3` log traffic.
