Getting Started with Node.js LoopBack Connector for Couchbase

Introduction

LoopBack is the leading open source, enterprise-ready Node.js framework for helping developers to create APIs integrated with legacy and next gen backends that at the same time enables mobile and micro services architectures.

LoopBack models connect to backend systems like databases via data sources that provide create, read, update and delete (CRUD) functions through the LoopBack Juggler: a modern ORM/ODM. These data sources are backed by connectors that implement connection and data access logic using database drivers or other client APIs. In addition, there are connectors for REST and SOAP APIs, NoSQL, messaging middleware, storage services and more. For example, StrongLoop maintains connectors for:

You may have used one or more of these connectors, but did you know that in addition to these connectors maintained by StrongLoop, there are many community contributed connectors as well?

This tutorial is the first in a series of posts that will help you get started with some of the many user contributed NoSQL connectors for LoopBack. In this series we will cover usage of the connectors for:

Couchbase Server

Couchbase describes their Couchbase Server product as “a high-performance NoSQL distributed database with a flexible data model. It scales on commodity hardware to support large data sets with a high number of concurrent reads and writes while maintaining low latency and strong consistency.”

Now that we know a little bit more about Couchbase Server, let’s get it installed and ready to power our LoopBack API.

Installing Couchbase

Head over to the Couchbase website and download the version for your platform. Couchbase offers both community and enterprise editions of Couchbase Server. We’ll be using the community edition version 3.0.1 for this tutorial. We’ll also be developing on Mac OSX, but Couchbase also offers downloads for Windows and various flavors of Linux.

Installation on OSX is extremely simple. After you download the file, expand the archive and drag the Couchbase-Server.app to your Applications folder. Double-click the application to launch Couchbase.

That’s it! Couchbase Server is now running on your Mac. You’ll see the Couchbase Server icon in your tray.

Setting Up Couchbase

We need to take a few steps in order to get Couchbase Server ready for use. Head to http://127.0.0.1:8091/index.html in your browser to begin the Couchbase Server setup. As this tutorial is more focused on connecting LoopBack to Couchbase Server, we’re going to gloss over some of the Couchbase installation steps, but you can review the full Couchbase Server installation documentation for your platform if necessary.

There are 5 steps that you need to run through in the Couchbase admin to get your server up and running. We’re going to accept the defaults for everything, so just click Next on all 5 pages.

When you’re complete with the rest of the setup steps, you’ll be taken to the Couchbase Server administrator panel. Click on the Data Buckets so we can create our bucket for this tutorial.

Click “Create New Data Bucket” and enter the name as “loopback-example”. Leave all the settings at their defaults and click “Create”.

Now that we have our sample bucket, we need to take one more step in preparation for connecting to our Couchbase Server with LoopBack. We need to install the Couchbase N1QL engine.

Setting Up N1QL

N1QL (pronounced “Nickel”) is a query language developed by Couchbase used for finding data in Couchbase Server. N1QL allows for allows for joins, filter expressions, aggregate expressions and many other features.

Head back to the Couchbase website again to download the N1QL file for your platform.

Note: At the time of this writing (as of this commit) loopback-connector-couchbase only works with N1QL Developer Preview 3. Even though Preview 4 is the most current version of N1QL, make sure you download Developer Preview 3.

After you’ve download the Developer Preview 3, expand the archive and cd into the directory.

If you’d like to run through the N1QL tutorial – highly recommended if this is your first time using Couchbase Server and N1QL – run the start_tutorial.sh script in your terminal and navigate to http://localhost:8093/tutorial/. The tutorial covers many of the different concepts of N1QL with a series of examples that you can run right in your browser.

Next we need to connect the N1QL engine to our Couchbase Server. If you followed along above, your Couchbase Server is running on port 8091.

$ ./cbq-engine -couchbase=http://127.0.0.1:8091/

Finally, per the loopback-connector-couchbase README, we are going to start the N1QL command lines tool and create a primary index on our loopback-example bucket that we created during the Couchbase setup phase. The N1QL engine will run on port 8093 by default.

Note that since this is a community contributed connector, StrongLoop does not have a generator for the datasource definition. What you will get after running slcloopback:datasource above is a skeleton definition that you will need to fully complete.

Now, edit the datasources.json file in the root of your application with the correct Couchbase connection information. Remember that the default ports are 8091 for our Couchbase Server and 8093 for the N1QL engine. Our database is named “loopback-example” and we also set some timeouts for connecting to the server and for server operations.

We’ll take a look at using this relationship a little bit later in the tutorial, but first let’s finish telling LoopBack about our Couchbase Server.

We want to tell LoopBack to expose our two new models via the REST API. After adding both the employee and company models to model-config.json in the application root, your file should look like this. Please note this can be auto updated by the loopback generators as well.

Tip: You can also execute slc run with debugging arguments to see exactly what N1QL is sending to Couchbase Server. This is very helpful in development as you’ll receive valuable debug information from the connector. Here’s an example that you could receive when calling GET /companies.

$DEBUG=connector:couchbaseslcrunconnector:couchbasecouchbaseconnectorinitializeDataSource():options:[{"dsconnector":"couchbase","host":"127.0.0.1","port":8091,"password":"","n1qlport":8093,"bucket":"loopback-example","env":"debugging","connectionTimeout":20000,"operationTimeout":15000,"connectUrl":"couchbase://127.0.0.1","n1qlUrl":"127.0.0.1:8093"}]+0msBrowseyourRESTAPIathttp://0.0.0.0:3000/explorerWebserverlisteningat:http://0.0.0.0:3000/connector:couchbaseCouchbaseDB.prototype.all():["company",{}]+16sconnector:couchbaseCouchbaseDB.prototype.all()finalqueryis:SELECT*FROM`loopback-example`WHERE(`docType`='company')+7msconnector:couchbasecbquery():queryis:[{"str":"SELECT * FROM `loopback-example` WHERE (`docType`='company')"}]+21msconnector:couchbasecbquery():success![null,[]]+21ms

You probably noticed the (`docType`='company') where clause in the query. It’s important to note that loopback-connector-couchbase automatically adds a docType field to documents when they are created. This docType field is equal to your LoopBack model name.

Tip: Be careful if you are using an already existent bucket in Couchbase with the connector. Your documents will not have the docType field that the connector is looking for. Queries will initially return empty sets as the connector cannot find any documents that meet criteria for the clause of WHERE (`docType`='modelName'). You’ll need to get the docType property in all of your documents first in order to work with this connector.

Take note of the docType and id fields. We already discussed the docType field, but you’ll see that loopback-connector-couchbase also created an id field. Currently the connector automatically generates a UUID for the id if you do not pass in an id. If you want to set the id yourself when creating a document, you can simply pass in an id field.

Input the other company via Explorer and do the same for the first 4 employees in the sample data set. We’ll create the last one using curl and then discuss querying and updating records.

Let’s enter our last sample employee using curl which is more representative of how you may actually use your API.

Updating data

Updating records in Couchbase Server with the connector is equally as easy. We’ll use the id of the record that we queried in the section above. To update a record, send a PUT request to the employee resource with the id and specify which portion of the document you want to update.

The preserve key

There is one setting you can apply to a property in your model when using the Couchbase connector. If you set the preserve property to true the connector will not update that property when a document is updated – even if you ask it to.

Wait a second. That response makes it look like the record was updated with the new employee number! However, if you check the document you’ll see that the response was not quite true and the document was not actually updated.

Conclusion

The community contributed loopback-connector-couchbase is an excellent addition to the LoopBack family, albeit with a few quirks. If you are careful to take note of things like the automatically generated UUIDs and the docType property, the connector can be very powerful and is certainly currently the best way to connect LoopBack and Couchbase Server.