.NET Sample App Backend Tutorial

The .NET SDK tutorial bridges the gap between simple and advanced concepts by walking through a complete web application.

The full source code for the tutorial is available on GitHub couchbaselabs/try-cb-dotnet.
The primary focus of the tutorial is to explain the function and theory behind the Couchbase .NET client and how it works together with Couchbase Server, and especially new features in versions 4.0/4.5 like N1QL, FTS and sub-document.
It makes use of the travel-sample data set.
The code that generates the web application is provided with the source code, but is not discussed in this tutorial.

There’s currently no "fill-in-the-blanks" branch so you’ll have to delete method bodies and try to take it from there.

This tutorial focuses on querying through N1QL and FTS rather than mapreduce views.
If you want information about using views, see Views.

Walking Through the API

The following sections lead you through the primary functions of the sample application.
All of the REST API application code is in several files according to function.
The following shows you how to program with the various features and services of Couchbase including: connecting to a cluster and bucket, key/value iteraction, document query through N1QL and full text searches.

Configure and Bootstrap the SDK

The first step in the Couchbase SDK is to configure your connection to the cluster.
This can be done either programattically or through configuration files.
For simplicity, in this sample app we’re going to do it programmatically.

In the sample app, the CouchbaseConfig.Register() method is responsible for reading the hostnames and/or IP addresses for the cluster and initializing the connection.
First it reads the CouchbaseServer setting from the web.config and then creates a ClientConfiguration, setting the Servers with the hostname from the app setting.
The ClientConfiguration is then passed into the ClusterHelper utility which uses the configuration to setup everything required.

The ClusterHelper is a utility built into the .NET SDK to help with managing the cluster connection and can be used to get references to a bucket (which is your entry point for the whole storage API).
The cluster connection should be reused as much as possible, typically called a singleton, because it is expensive to create.
The ClusterHelper stores the connection to the cluster for easy reuse within your applications.

To get a references to a bucket, with which you can then execute operations such as Get, Insert and Remove, you can use the GetBucket method on ClusterHelper.
This returns an IBucket which exposes the majority of the operations available in Couchbase.

An example of getting a reference to a bucket and retrieving a document is below:

Couchbase Server is a document-oriented database which provides access to your data both through its document ID (for high performance access), as well as through map/reduce views and N1QL (SQL like query language).

This is noticeable in the API, where the methods reflect Key/Value operations (get, create, etc...) and work with a Document<T> interface that has an Id and a Content.

Typically, in this type of scenario you would implement the Repository pattern to separte functional logic from business logic.
However, to make this sample as easy to get up to speed with controllers are used instead.

Creating New Users

For the purposes of this sample application users are required to register before they can book flights.
All the user related operations are going to exist in the UserController.

The SignUp method is where new users are registered.
The method takes a LoginModel (src/Models/LoginModel.cs) as the only method parameter.
ASP.NET has an automatic model binder where it can map a POST request’s properties to a custom class, like the LoginModel.
The property types and names have to match for it to assign the value.

public async Task<IHttpActionResult> SignUp(LoginModel model)

The model is validated to ensure that the username and password fields are non-empty and then checked to see if the username already exists.
For the purposes of this sample app, the user document keys are in the format user::<username>.
The ExistsAsync method is used to do a non-blocking check to see if the document already exists in the bucket.

Next a document with the user details is created and stored in Couchbase Server.
The userKey created earlier is used and alsoa document expiry time is included if one was set on the model.
As you do not want to store plain text passwords, the passwords are MD5-hashed before storing it in the user document.

The non-blocking InsertAsync method is used to ensure that a thread waiting for the response from Couchbase Server is not blocked.
Once the code receives the response, it continues where it left off.

The last thing to do is to create a security token for the browser, so that the front end knows that additional requests are for a valid user.
For this a JWT (JSON Web Token) is used, which includes the username in a list of claims and is then encrypted with a secret key.
The secret is stored in the Web.Config.

The response content has two parts, the first is the JWT and the second part is a narration string which is something the frontend app understands and will display in a console.
The narration enables the users of the application to get an idea of what is going on on the server side while browsing the app.
It is similar to a log, but sent to the frontend.

Loging in Signed up Users

The Login method enables users who have already signed up to sign in and use the application.
The Login method signature looks like this:

public async Task<IHttpActionResult> Login(LoginModel model)

The LoginModel includes Username and Password properties that can be used to find the user document and verify the passwords.

First the user document must be retrieved and the password checked to ensure that it matches with the model.
The user document key needs to be built using the model’s username property, the document is then retrieved from Couchbase Server.

Now the application has a user document, it can check the passwords match.
Remember the password was hashed in the document for added security so the model’s password will also have to be hashed before they are compared.

This is going to be the first time that the application checks for a valid security token; it has only generated these so far.
The security token is fairly simple and only includes the username of the user, but that is enough for this sample application.
To verify the token the authentication header needs to be fetched and then decrypted.
If this fails for any reason, the application returns either a 401 (Unauthorized) or a 403 (Forbidden) response.

N1QL: Flight Paths

In the SDK, there is a query method that accepts all variants of querying with Couchbase (views, spatial/geo views, N1QL and FTS).
For N1QL, the IQueryRequest is expected.
This allows to wrap a N1QL Statement, use positional parameters and provide query tuning (eg Timeout).

N1QL is a super-set of SQL, so if you’re familiar with SQL you’ll feel at ease.

This controller has one method, GetFlights, which provides flight routes between two airports.
It uses a N1QL query to get them.
The method has three parameters; from, to and leave (string for departure date).
The first thing the application does is validate the parameters, returning a 500 (InternalServerError) if it’s not.

After doing some checks to ensure there are results from both airport codes, the application then does some Geo-location calculations to find the distance between the two airports and the estimated travel time.
The distance and travel time are then used when calculating ticket prices.

Yes, you read that right, N1QL can do joins (on a single bucket or on several).
It works as long as the "foreign key" described by ON KEYS clause can be mapped to a document’s key in the joined bucket.

A specificity of N1QL that is seen in the second statement is UNNEST.
It extracts a sub-JSON and puts it at the same root level as the bucket (so its possible to do joins on each element in this sub-JSON as if they were entries in a left-hand side bucket).

The application now has all flights between the from and to airports but there are not any prices any prices.
These are then calculated.

LINQ: Airports

Goals: Use the LINQ provider to build N1QL queries to retrieve Airport details.

LINQ is a standardised way of constructing queries over a data storage engine, such as in-memory collections, SQL and even NoSQL like Couchbase.
It’s a very simple yet powerful tool that enables developers to write complicated queries programatically.

In this Controller the application is trying to find the aiport name, given some additional information about the airport.
It uses the LINQ provider to build the queries.

Indexing the Data: N1QL & GSI

Index management is a bit more advanced (and is already done when loading the sample), so now that you’ve learned about N1QL, you can have a look.
For N1QL to work, you must first ensure that at least a Primary Index has been created.
For that you can use the DSL from the BucketManager class:

Goals: Use the Index DSL to make sure data is indexed ready for N1QL to query it.

bucketManager.CreateN1qlPrimaryIndex(false); // create primary index, and don't defer building it

The fluent API will guide you with the available options, you just have to declare that you want to CreateN1qlPrimaryIndex().

You can also create secondary indexes on specific fields of the JSON, for better performance:

In this service, hotels are searched for using more fuzzy criterias, like the content of the address or the description of an hotel.
This is done using Full Text Search (FTS).
When some results match the specified criteria, only the relevant data for each result to be displayed in the UI is fetched using the subdocument API.

Let’s have a look at the FindHotel method.
It accepts two parameters, location and description, which are the two possible refining criterias for an hotel search.

A ConjunctionQuery allows you to combine multiple FTS queries into one, in a logical AND fashion.
This query includes an exact match criteria that restricts it to the hotel data type (as reflected in the type field of the JSON document).

If the user provided a location keyword, a second component is added to the FTS query that will look for that keyword in several address-related fields of the document.
That is done in an OR fashion, using a Disjunction query:

The matchPhrase FTS query can contain several words and will search for variations of the words (eg.
including plural forms or words with the same root).

The compound FTS query is now ready to be executed.
A SearchQuery object is built out of it, which also determines which FTS index to use ("hotel") and allows you to set various parameters (like a limit of maximum 100 hits to return).
The query is logged (and kept for narration) then executed, returning an ISearchQueryResult object:

The FTS results are then iterated over, and the document corresponding to each result is fetched.
In actuality, only the parts of the document that will be displayed in the UI are required.
This is where the sub-document API comes in.

The sub-document API allows you to fetch or mutate only a set of paths inside a JSON document, without having to send the whole document back and forth.
This can save network bandwidth if the document is large and the parts that we’re interested in are small.
So here the results of the FTS search are iterated over and appropriate subdoc calls are triggered:

Each FTS result is represented as an ISearchQueryRow which exposes the document’s Id.
The sub-document API can then be used to fetch data (bucket.LookupIn<T>(documentId)) and specify what parts are wanted: name, description, address, city, state and country.
The application then Execute() the sub-document query.
In the rest of the code, the address-related fields are aggregated together and the data obtained is returned.

Now the results are obtained, the application can build up the Hotel objects and return them along with the FTS query narration.