Elasticsearch with Rails – How to Create a Simple Search Engine in 1 Hour

Have you ever had a problem in your application with a slow search engine? It acts like a simple SQL LIKE query and it’s really slow? Or maybe you want to create an efficient and simple search form on your webpage? If you answered yes to either – this is the tutorial for you!

In this tutorial, I’ll cover how to integrate your Rails application with Elasticsearch and create a simple search engine in 1 hour!

First, let’s cover what Elasticsearch is and how it works. It’s a search engine based on Apache Lucene – an information retrieval software library. It provides distributed full-text search based on an HTTP interface. It’s written in Java and developed by Elastic.co. It’s similar to Apache Solr or Sphinx (maybe you have used one of them before).

It’s also used by a lot of big companies like Facebook, Netflix or Mozilla. You can find out more on their official webpage.

What will we build?

We will build a simple Rails 5 application integrated with Elasticsearch. I’ll cover basic search methods, how to use Elasticsearch and an integration with a database-elastic, how to index records, how to customize the index process and indices structure. We will also build a simple endpoint which will return serialized search results based on a search query. Now let’s get start and write some code!

Basic setup

We’ll start by creating a new Rails 5 app from the scratch. We’ll use PostgreSQL as our database system:

$ rails new elasticsearch_with_rails -d=postgresql

Now, we need to install Elasticsearch. If you have already installed it, just skip this part.

I’ll cover how to install it on macOS using brew manager. If you use Ubuntu, this is a great tutorial which covers an installation process.

To install Elasticsearch, just run the following command:

$ brew install elasticsearch

I’m using Elasticsearch 6.1.1 and this version will be used in this tutorial. If you want to run it simply type:

$ elasticsearch

Just to make sure that everything is ok, go to http://localhost:9200/ and check what you get. Your response should be similar to mine:

There is also an elasticsearch-rails gem but we won’t need it in this tutorial. It contains some useful commands for Rails you can find out more about it here.

Let’s install everything by running bundler:

$ bundle install

By default, Elasticsearch uses http://localhost:9200 as an instance endpoint. If you want to change it, you need to change the configuration in the client setup. Also, you can use AWS Elasticsearch but for this purpose, you need to install a gem:

gem 'faraday_middleware-aws-signers-v4'

Here is an example AWS config. It should be added in an initializer, for example elasticsearch.rb:

Models setup

Elasticsearch is basically used for searching, so we need to create a few models and populate a database with some data. Let’s do it, I want to use simple schema, a book and an author. Author has many books and we will be able to search for an ISBN, a book name, author name etc.

Author model, first name and last name:

$ rails g model author first_name last_name

Book model, ISBN, author_Id, published_at, number of pages and a name:

Ok, let’s add a basic Elasticsearch setup to our book class. For now only include two modules, Elasticsearch::Model and Elasticsearch::Model::Callbacks. The first module adds a lot of methods, for example, a search method (Book.search which is also available via __elasticsearch__ namespace – Book.__elasticsearch__.search) which search through all indices to find a result. You can find out more here.

The second module just adds callbacks to our model, basically, when you add a model to a database, remove it or update – Elasticsearch is updated.

require 'elasticsearch/model'
class Book < ApplicationRecord
belongs_to :author
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
end

It looks more or less like this – see attached logs after updating or deleting a record:

Index definition

Maybe you’re wondering what an index looks like? Yeah… we don’t know how a new record will be added and serialized. But what about a situation where a model has any associations? We need to handle it by ourselves. We need to redefine the as_indexed_json method. We want to add some fields, not all, so only – id, name, isbn, published_at, pages and info about an author. We can do it like this:

Controller and basic API

Our models and Elasticsearch are ready but let’s add an API endpoint which will return a serialized records based on a query.

Let’s start from adding new routes. Add a new resource to the routes.rb:

resources :books, only: [:index]

How can we return records now? Well, we should use the __elasticsearch__.search. There are a lot of ways to perform a query in Elasticsearch. Basically, Elasticsearch offers tons of different options how to search for a requested result.

It’s API is really, really rich and it could take months to really feel that you know something about the API 🙂

But for now, let’s focus on some basic definitions.

We can define a different search method or search for a request by scanning all fields or use selected fields.

Book.__elasticsearch__.search(params[:query]).results

The first definition defines the most basic search. Scan for results based on all fields.

We forgot about one thing – what happens when we update an author name? We should update books’ indices too! How we can do it? Well, the easiest way is to run the index_document method to force reindex a record. We can do it in an after_save callback. How? Like this:

Honestly, this isn’t very powerful. Imagine indexing huge amount of records in the same in the rails app. It’s not a good idea. We can also do it using a worker and Sidekiq. So run a sidekiq job after an author was updated/created.

Summary

Now you know how to implement Elasticsearch with yours Rails application and add some basic stuff, like custom indexing, searching and indexing records from an associated class. I just wanted to cover same basic stuff instead and describe them instead of writing about a lot of things and going through them quickly.

I hope that it would be useful for your apps and you will discover a new passion for Elasticsearch 🙂

Related Posts

As you've probably guessed by the title of my article, I still consider Ruby on Rails as a relevant technology that offers a lot of value, especially when combined with ReactJS as it's frontend counterpart. Here's how I approach the topic.

Gitlab Pipeline for Rails is the main part of a powerful GitLab CI/CD tool and can be a useful alternative for other applications like Jenkins and TeamCity. If you’re looking for some more detailed information on exactly how it works, we’ve compiled an example configuration that can help you.