Elasticsearch: Making EC2-Discovery Work In AWS Mumbai Region

With the recently launched AWS Mumbai region, many of us might need to migrate our infrastructure there. In one such scenario, we have to setup Elasticsearch cluster on AWS EC2 instances and test it in Mumbai region.

Challenge: There is a plugin which enables Elasticsearch cluster to discover new nodes automatically and added them to the cluster called “AWS Cloud Plugin“. The plugin used EC2 instance tags and region to add nodes to the cluster. Once we have installed the plugin, we need to configure the “elasticsearch.yml” file. The plugin segment should look similar to this:

Here, we updated the cloud.aws.region to ‘ap-south-1’ (Mumbai) and restarted our Elasticsearch nodes but the nodes could not get added to the cluster and there were errors which lead us to conclude that the plugin does not support Mumbai region yet.

The Solution: Download/Clone the plugin and get ready to make some changes. Basically, we want to update the source files and add support for Mumbai region. Once we have downloaded the plugin, we will search for the files that contain AWS Regions in them. The desired file could be searched as shown below:

$ grep -rnw ./ -e 'ap-southeast-1'

We will get all the filenames and paths that contain region names and now we know where to make changes. To be precise there are three files:

S3Repository.java

InternalAwsS3Service.java

AwsEc2Service.java

There is a segment of code in all these files where ‘if’ operator checks for the Region and Endpoint and it is where all the Regions and Endpoints are to be updated. Just add one more entry here for Mumbai region (ap-south-1) in all the files: