Getting Started with Elastic Using .Net NEST Library, Part Four

Welcome to part four of my tutorial on getting started with Elastic using .Net NEST library. In part one, I covered the reasons for choosing NoSQL, an explanation of Elastic, and the installation of and communication with Elastic. In part two, I went over the creation of your first Elastic index, CRUD operation, and simple search with paging and sorting. Part three focused on queries, nested types, and retrieving only part of a document. In this, the final installment, I will cover empty properties, dynamically-constructed queries, custom analyzers and NEST Bulk API document migration. Let's dig in.

Find an Empty Property

Null and empty values cannot be indexed, therefore usual queries cannot be used. A filtered query should be used instead. Let’s put a new post with an empty body property into our index as such:

We tell Elastic to match all documents, but additionally filter results with a “missing” filter in order to return only those having a null or empty “body” property. You can read more about missing filters here.

How To Dynamically Construct a Query

Exactly how to dynamically construct a query is initially unclear using NEST. This example should put you on the right track:

As you can see, we are dynamically changing the "must" query part and adding a necessary filter part.

Defining and Using a Custom Analyzer (Autocomplete Example)

The analyzer performs the following three functions: character filtering, tokenization and token filtering. A character filter cleans text blocks by removing or replacing unwanted text sequences, for example, removing HTML tags. Tokenization splits a block of text into individual terms, which are suitable to use in an inverted index. Token filters normalize terms into standard forms. Elastic comes with the following analyzers available out of the box: standard analyzer, simple analyzer, whitespace analyzer and language analyzer. The real power of Elastic is in the ability to create custom analyzers suitable for different purposes by combining different character filters, tokenizers, and token filters provided by Elastic out of the box.

We will construct a custom analyzer by using a “keyword” tokenizer and “lowercase” token filter. Keyword tokenizer emits the entire input as a single input. Lowercase token filter normalizes token text to lowercase. This custom analyzer could be used where the whole input (not splitting it into individual terms) should be matched in a case-insensitive way. To define our custom analyzer, we must modify the index creation code in the following way:

Here we are using a “custom_lowercase_analyzer” in the BlogPost Title property mapping. We tell Elastic to use this analyzer, then we perform a search on the blog post title. For example, this analyzer, in combination with a “wildcard” query, could be used to drive autocompleting functionality. We created a new index, so let’s populate it with the following data:

Using NEST Bulk API to Migrate Documents Into a Newly-created Index

Many times in this article, we have changed mappings, then created a new index and populated it. This is ok for learning purposes, but in a real production project you don’t want to populate an index with new data and lose old documents every time you change mappings. What we can do about this? Here comes document migration and the wonderful NEST Bulk API into our rescue. You can use "Bulk()" to create any bulk request you'd like. Let’s create a minimum example for the migration of old documents into a newly created index. We have created a new index and named it “my_fourth_index” by using the same code as in the previous example.

Now save the build and run the program. You can run a simple “match all” query to see that all old documents from “my_third_index” were migrated into the new “my_fourth_index”. We are using scan queries and scroll because it is the optimum solution to access documents in a sequence.

That is all. Thank you for your time. The following is a “Reading resource” section if you want even more information about Elastic and NEST.