********************************************
omelasticsearch: Elasticsearch Output Module
********************************************

===========================  ===========================================================================
**Module Name:**             **omelasticsearch**
**Author:**                  `Rainer Gerhards <https://rainer.gerhards.net/>`_ <rgerhards@adiscon.com>
===========================  ===========================================================================


Purpose
=======

This module provides native support for logging to
`Elasticsearch <http://www.elasticsearch.org/>`_.


Notable Features
================

- :ref:`omelasticsearch-statistic-counter`


Configuration Parameters
========================

.. note::

   Parameter names are case-insensitive.


Action Parameters
-----------------

Server
^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "array", "none", "no", "none"

An array of Elasticsearch servers in the specified format. If no scheme
is specified, it will be chosen according to usehttps_. If no port is
specified, serverport_ will be used. Defaults to "localhost".

Requests to Elasticsearch will be load-balanced between all servers in
round-robin fashion.

.. code-block:: none

  Examples:
       server="localhost:9200"
       server=["elasticsearch1", "elasticsearch2"]


.. _serverport:

Serverport
^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "integer", "9200", "no", "none"

Default HTTP port to use to connect to Elasticsearch if none is specified
on a server_. Defaults to 9200


.. _healthchecktimeout:

HealthCheckTimeout
^^^^^^^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "integer", "3500", "no", "none"

Specifies the number of milliseconds to wait for a successful health check
on a server_. Before trying to submit events to Elasticsearch, rsyslog will
execute an *HTTP HEAD* to ``/_cat/health`` and expect an *HTTP OK* within
this timeframe. Defaults to 3500.

*Note, the health check is verifying connectivity only, not the state of
the Elasticsearch cluster.*


.. _searchIndex:

searchIndex
^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "word", "none", "no", "none"

`Elasticsearch
index <http://www.elasticsearch.org/guide/appendix/glossary.html#index>`_
to send your logs to. Defaults to "system"


.. _dynSearchIndex:

dynSearchIndex
^^^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

Whether the string provided for searchIndex_ should be taken as a
`rsyslog template <http://www.rsyslog.com/doc/rsyslog_conf_templates.html>`_.
Defaults to "off", which means the index name will be taken
literally. Otherwise, it will look for a template with that name, and
the resulting string will be the index name. For example, let's
assume you define a template named "date-days" containing
"%timereported:1:10:date-rfc3339%". Then, with dynSearchIndex="on",
if you say searchIndex="date-days", each log will be sent to and
index named after the first 10 characters of the timestamp, like
"2013-03-22".


.. _searchType:

searchType
^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "word", "none", "no", "none"

`Elasticsearch
type <http://www.elasticsearch.org/guide/appendix/glossary.html#type>`_
to send your index to. Defaults to "events"


.. _dynSearchType:

dynSearchType
^^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

Like dynSearchIndex_, it allows you to specify a
`rsyslog template <http://www.rsyslog.com/doc/rsyslog_conf_templates.html>`_
for searchType_, instead of a static string.


.. _pipelineName:

pipelineName
^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "word", "none", "no", "none"

The `ingest node <https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html>`_
pipeline name to be included in the request. This allows pre processing
of events before indexing them. By default, events are not send to a pipeline.


.. _dynPipelineName:

dynPipelineName
^^^^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

Like dynSearchIndex_, it allows you to specify a
`rsyslog template <http://www.rsyslog.com/doc/rsyslog_conf_templates.html>`_
for pipelineName_, instead of a static string.


.. _skipPipelineIfEmpty:

skipPipelineIfEmpty
^^^^^^^^^^^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

When POST'ing a document, Elasticsearch does not allow an empty pipeline
parameter value. If boolean option skipPipelineIfEmpty is set to `"on"`, the
pipeline parameter won't be posted. Default is `"off"`.


.. _asyncrepl:

asyncrepl
^^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

No longer supported as ElasticSearch no longer supports it.


.. _usehttps:

usehttps
^^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "binary", "off", "no", "none"

Default scheme to use when sending events to Elasticsearch if none is
specified on a  server_. Good for when you have
Elasticsearch behind Apache or something else that can add HTTPS.
Note that if you have a self-signed certificate, you'd need to install
it first. This is done by copying the certificate to a trusted path
and then running *update-ca-certificates*. That trusted path is
typically */usr/local/share/ca-certificates* but check the man page of
*update-ca-certificates* for the default path of your distro


.. _timeout:

timeout
^^^^^^^

.. csv-table::
   :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
   :widths: auto
   :class: parameter-table

   "word", "1m", "no", "none"

How long Elasticsearch will wait for a primary shard to be available
for indexing your log before sendids.net/> of action.resusted path
es at
the nfiguratio1tured Co. csv-table::
                      root                                                                                                                                                                                                                   Configuration Parameter Types
=======ble func            e-block:andatorutp")

 arametertry was found in the

============Enable tout
^^^^^^^

.. cshether ts.


.. _omclier1" pwd=41, "none"

Specifies the number of milliseconds to wait for a successful health check
on a server_. Before trying to submit events to Elasticsearch, rsyslogmany times fases API serongly die, rathef
tha                               root                                                                                                                             faultNetstreamDriverCertFile** <unc dged pro Note t check the man eger", "3500", "nwtb parameter-tabmv. csv-o Elasticsnory", "|FmtObshis module is
t specifie           rulele i    `  rulele m  . not"

-  **p compatible.

To 217756 026270  0    sage services ("on", the RELP *n queues
***t", "manhed.


user
^^^, mmjsonparse will
nonetheless return that the JSON parsing has failed.

The "CEE cookie" is the character squence "@cee:" which must prepend the
actual JSON. 
Name
^^^^

.. csB          sources/configuce "@cee  "binary", "off", "no", "none"

When POST'ing a documen`"

Stameter-table

   "binary", "on", "no", "``$InputUnixL will
execute       ==========p

   ...

   AMQP 1.0 Options:
      --domain DOMAIN           Domain of this broker
      --queue-patterns PATTERN  Pattern for on-demand queues
      --topic-patterns PATTERN  Pattern for on-demand topics


If no AMQP 1.0 related options appear in the help output then yuse. Logs are bat
===s appear       h/to/non-fixed.log") a                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 