Elasticsearch Setup

Setting up an Elasticsearch Cluster

Elasticsearch Supported Versions

The Nuxeo Platform communicates with Elasticsearch using the transport client JAVA API, as stated in the Elasticsearch documentation: "You are encouraged to use the same version on client and cluster sides. You may hit some incompatibility issues when mixing major versions".

The Nuxeo Platform 7.3 (and above) uses Elasticsearch 1.5.2 library and has been successfully tested against 1.1.2 to 1.7.x cluster.

We recommend to use the same JVM version for all Elasticsearch nodes and Nuxeo.

The default configuration uses an embedded Elasticsearch instance that runs in the same JVM as the Nuxeo Platform's.

This embedded mode is only for testing purpose and should not be used in production.

For production you need to setup an Elasticsearch cluster.

Installing the Elasticsearch Cluster

Refer to the Elasticsearch documentation to install and secure your cluster. Basically:

Don’t run Elasticsearch open to the public.

Don’t run Elasticsearch as root.

Disable dynamic scripting (disabled by default since 1.2.X).

Use an explicit cluster name by setting the cluster.name in the /etc/elasticsearch/elasticsearch.yml file, this will avoid conflicts with other environments.

Recommended Tuning

If you have a large number of documents or if you use Nuxeo in cluster you may reach the default configuration limitation, here are some recommended tuning:

Consider disabling the OS swapping or using other Elasticsearch option to prevent the heap to be swapped.

In /etc/default/elasticsearch file you can increase the JVM heap to half of the available OS memory:

elasticsearch.addressList points to one or many Elasticsearch nodes. Note that Nuxeo connects to the API port 9300 and not the HTTP port 9200.

elasticsearch.clusterName is the cluster name to join, elasticsearch being the default cluster name.

elasticsearch.indexName is the name of the Elasticsearch index for the default document repository.

elasticsearch.indexNumberOfReplicas is the number of replicas. By default you have 5 shards and 1 replicas. If you have a single node in your cluster you should set the indexNumberOfReplicasto 0. Visit the Elasticsearch documentation for more information on shards and replicas.

audit.elasticsearch.indexName is the name of the Elasticsearch index for audit logs.

seqgen.elasticsearch.indexName is the name of the Elasticsearch index for the uid sequencer, extensively used for audit logs.

If you want to disable Elasticsearch and use the SQL database as the default backend for audit logs you can simply update this property in nuxeo.conf:

audit.elasticsearch.enabled=false

Triggering SQL to Elasticsearch Audit Logs Migration

When upgrading a Nuxeo instance from a version lower than 7.3 to 7.3 or higher, if you decide to use Elasticsearch as a backend for audit logs you need to add the following property to nuxeo.conf to trigger the migration of existing audit log entries.

audit.elasticsearch.migration=true

This will launch a background job at server startup to migrate data from the nxp_logs, nxp_logs_extinfo and nxp_logs_mapextinfo tables of the SQL database to the ${<span style="color: rgb(0,0,0);">audit.elasticsearch.indexName</span>} Elasticsearch index.

Migration uses batch processing. The number of log entries processed per batch can be configured by adding the folllowing property to nuxeo.conf:

audit.elasticsearch.migration.batchSize=5000

Default value is 1000. As an example, we successfully tested migration of 22.000.000 log entries with an average speed of 1500 entries / second using audit.elasticsearch.migration.batchSize=10000 on a Linux virtual machine with two cores, 4 GB RAM, a local PostgreSQL instance and an embedded Elasticsearch instance.

Once the migration is done you should remove the audit.elasticsearch.migration property from nuxeo.conf, else you wil see a warning about it in the logs.

Rebuilding the Repository Index

If you need to reindex the whole repository, you can do this from the Admin > Elasticsearch > Admin tab.

You can fine tune the indexing process using the following options:

Sizing the indexing worker thread pool. The default size is 4, using more threads will crawl the repository faster:

elasticsearch.indexing.maxThreads=4

Tuning the number of documents per worker and the number of document submitted using the Elasticsearch bulk API:

# Reindexing option, number of documents to process per worker
elasticsearch.reindex.bucketReadSize=500
# Reindexing option, number of documents to submit to Elasticsearch per bulk command
elasticsearch.reindex.bucketWriteSize=50

Changing the Mappings and Settings of Indexes

Updating the Repository Index Configuration

Nuxeo comes with a default mapping that sets the locale for full-text and declares some fields as being date or numeric.

For fields that are not explicitly defined in the mapping, Elasticsearch will try to guess the type the first time it indexes the field. If the field is empty it will be treated as a String field. This is why most of the time you need to explicitly set the mapping for your custom fields that are of type date, numeric or full-text. Also fields that are used to sort and that could be empty need to be defined to prevent an unmapped field error.

The default mapping is located in the ${NUXEO_HOME}/templates/common-base/nxserver/config/elasticsearch-config.xml.nxftl.

Updating the Audit Logs Index Configuration

Here the index is a primary storage and you can not rebuild it. So we need a tool that will extract the _source of documents from one index and submit it to a new index that have been setup with the new configuration.

Update the mappings or settings configuration by overriding the {NUXEO_HOME}/templates/common-base/nxserver/config/elasticsearch-audit-index-config.xml(follow the same procedure as the section above for the repository index)

Use a new name for the audit.elasticsearch.indexName(like nuxeo-audit2)

Start the Nuxeo Platform.
The new index is created with the new mapping.

Stop the Nuxeo Platform

Copy the audit logs entries in the new index using stream2es. Here we copy nuxeo-audit to nuxeo-audit2.