Now that you’ve installed Couchbase and have created a cluster of servers, you
need a client library to read and write data from the cluster. The Couchbase C
client library, also known as libcouchbase, can be used from your application to
access Couchbase Server.

Building and installing on Microsoft Windows requires nmake and tools in
Microsoft Visual Studio 2010.

Open the Visual Studio Command Prompt and navigate to the directory for the
extracted archive for libcouchbase. The NMakefile defines an INSTALL variable
as C:\local. Edit the NMakefile if you want to change the installation
location. Then build and install libcouchbase:

shell> nmake -f NMakefile install

From libcouchbase version 2.1.0, you can also use CMake system to generate
correct MS Visual Studio project, for example:

The C client library, libcouchbase, is a callback-oriented client which makes
it very easy to write high performance programs. There are a few ways you can
drive IO with the library. The simplest approach is to use the synchronous
interface over the asynch internals of the library. More advanced programs will
either call the libcouchbase_wait() function after generating some operations
or drive the event loop themselves.

To connect, you first configure the connection options and then create an
instance of the connection to the cluster:

Callbacks can be set up for all of your operations called in libcouchbase. In
the API, you’ll note the use of a cookie. This is metadata from your application
which is associated with the request. The libcouchbase library will not inspect
any cookie or send the cookie to the server.

When you put the connect logic and the get callback together and plug them into
a complete program with the include headers, you get:

This tutorial assumes you have installed libcouchbase on your systems per the
installation instructions in the Getting Started section of this guide. Because
the approach for building a program based on libcouchbase may vary between
Linux/Mac OS and Windows, this tutorial will focus on the components of the
program rather than how to build it.

If you need to set up a server node or data bucket, you can do so with Couchbase
Administrative Console, Couchbase Command-Line Interface (CLI), or Couchbase
REST API. For information and instructions, see:

Before getting into a more complex example of the programming model to this
library, we will walk through a straightforward example of a program which
builds with libcouchbase, connects to a server, and sets and gets a value:

In addition to the other sections of this manual, such as the Getting Started
guide and the API reference, the libcouchbase package includes an examples
directory and a tools directory. Each of these show simple Couchbase tools and
an example libcouchbase programs.

This tutorial assumes you have installed libcouchbase on your systems, following
the installation instructions in the Getting Started section of this guide.
Because the approach for building a program based on libcouchbase may vary
between Linux/Mac OS and Windows, this tutorial will focus on the components of
the program rather than how to build it.

The libcouchbase is written in C and can be integrated in various ways with your
application. The simplest integration scenario is when your application is
written in a scripting language (supported by a Couchbase client library) that
already has a libcouchbase wrapper. Couchbase has built and maintains following
libcouchbase-based libraries:

Users of these libraries automatically get an API, which looks natural for their
platform, so that it doesn’t require knowledge of libcouchbase APIs. This
tutorial is mostly about two other use cases:

Add libcouchbase into an existing application to implement persistence layer.
This section will contain two parts: first we will create a simple echo server
written using libev library.
Second, we will show how to persist each message going through the server to
Couchbase.

Build applications around libcouchbase. Here we’ll show how to build a proxy
server for regular memcached clients. Building this proxy server is just like
building a moxi server, but
the proxy server will be more limited more limited in abilities.

Let’s start with a specific existing application for this example. Our
application is the asynchronous single-threaded echo server, which leverages
libev to solve C10K
problem and efficiently serves a huge number of
concurrent connections. It should listen to the given port (in this sample 4567)
and send back everything the client submitted to it.

The libev library implements reactor
pattern which uses the term
event loop to represent an entity which encapsulates the register of handlers
for various IO events and the timers. To start, we have the application code
from the main function:

Here we are create an event loop instance, and create and bind a new socket to
local port PORT_NO (4567). After that we can mark this socket as listening and
ask libev to call the function accept_cb when the socket become ready to read,
that is when the new client will come to us. The final step is to run ev_run()
which does all the server interaction.

The accept_cb is in charge of allocating client structures 60 the buffer for
messages. The sample is using ring buffers from the libcouchbase code for easier
managing memory for IO buffers.

The most interesting part of the function above is in the last two lines. It
registers the new handler, client_cb, upon reading an event for the client
socket. This handler will be called each time when the kernel receives new data
from the socket, so that it could be read with a non-blocking call.

The last snippet of our application depicts the client_cb function, which
handles all the application logic. It reads everything from the socket if it is
ready, registers itself for write events, and then sends all the contents out to
the network. It is also the handling end of a stream situation, which means that
the client closed the connection with the Couchbase server.

Now that we described our application, let’s integrate Couchbase as the database
using libcouchbase. The server.c file will have some small additions. In the
main function, before initializing server socket, we will call storage_init
and assign the connection information to the server structure variable.

Also in the client_cb handler, each time libcouchbase receives data from the
client, it will send the data to Couchbase server with storage_put where
the_key is a string "example".

storage_put(client, the_key, val, nbytes);

Here, the initial application from step one is already using its own custom
event loop to handle other IO in the application. The application could be more
complex, but libcouchbase has a separate interface to inject your IO
implementation into the library. In the simplest case, for example if your
application is using one of the IO libraries supported by libcouchbase, you can
just pass your event loop instance into the initializer, and libcouchbase won’t
create new one. Instead it will register all its events on this external loop.
In case your application is using its own implementation of a reactor pattern,
such as nginx, or you use linux epoll API, you can easily write your own IO
plugin and pass it to the connection initializer.

The echo server is using libev library, and libcouchbase provides a plugin to
libev out-of-the-box. In this case we can imagine that it doesn’t already have
it. In this case, we can copy the plugin from libcouchbase distribution into the
directory step2/lcb-plugin/ and update build files appropriately.

Now we inspect the storage.c, where our storage_init() and storage_put()
defined.

The end of the function may look familiar, but the io_opts structure is more
interesting. It defines how to look up our custom IO plugin. The
io_opts.v.v1.sofile is set to NULL to specify that our plugin compiled into
the current executable image. In this case, symbol is the string name of the
function with the following signature:

This initializes the IO plugin and then the cookie will be passed to this
function as the loop argument. You can refer to the lcb_create_io_ops(3)
manpage for other ways to initialize IO subsystem.

The libcouchbase client is purely asynchronous; therefore, each data operation
isplits into two parts: a function-scheduler, which validates and copies all
arguments to internal buffers for further handing, and a function-callback,
which will be called with the results of the operation. It is possible to
implement a wrapper which will provide a more synchronous API, but it is
difficult to write a library API that is both generic and efficient. Since all
communication is done by using function arguments, it is easier to maintain
backward compatible APIs by versioning both incoming structures and results.
Let’s see how we did it in our sample:

Here, the storage_put() function is a wrapper over lcb_store(3). The
storage_put() function translates arguments to the versioned structure and
runs lcb_store to pass it to the library. The data won’t be sent immediately,
but when the library connects to the data socket, it will be ready to accept
data. After the server processes the request, the application will be notified
asynchronously via storage_callback. This callback will copy CAS value to the
output buffer to be sent to the client.

Time to demonstrate an application. This application is built the same as
before, except you need to navigate to step2/ directory. As previously noted,
the script will be split to demonstrate both server and the client:

Now you can install cbc tools from package libcouchbase2-bin. If you
installed libcouchbase from source, you most likely have it already. Then you
can check that it created a key example and that the CAS values are matching:

For clarity, we skip parsing and initialization of the Couchbase connection from
the main function. The proxy code begins in run_proxy, which is similar to
the main function in the previous section. The important difference here is that
we are using the IO abstraction from libcouchbase, to register events, and drive
an event loop.

As in the previous example, we create a listening socket, and then we bind a
handler to process read events. The handler is proxy_accept_callback, and it
will be triggered for each new client. In turn it will create a new client
structure, defined as follows:

This initializes input and output buffers,and makes the socket descriptors
non-blocking. At the end, the code will register the handler function
proxy_client_callback for read events, and then wait for incoming data.

The proxy implements a very limited number of protocol commands: GET, SET, and
VERSION. But other commands can be easily added. Here’s an example:

The first part occurs when the event loop notifies us that the socket is ready
for non-blocking reading. At this point, the function tries to read all the data
available on the socket. It finds the packets and passes aligned byte arrays to
handle_packet. Once the code gets a special code from the IO subsystem get
from the IO subsystem special code, that it cannot read anything without
blocking, it stops processing and registers itself for write events, because in
nearest future, responses will appear in the output buffer.

The second part of the function reacts to write events and is much simpler. It
just checks to see if there is data in the output buffer. If there is data in
the output buffer, it will try to send that data to the client. If not, the
functionwill unregister itself from write events to save CPU cycles.

The function handle_packet is a good place to start playing with this example
if you’d like to add new features there. It decodes protocol packet and
translates it into libcouchbase calls.

To maintain a logical link between requests and responses for the clients, the
protocol defines a special field opaque, which can be a special tag or
sequence number. To preserve this field in the response, we will put it into a
cookie_t structure. During the proxy initialization, we defined two
callback-functions for libcouchbase:

In this kind of application, these functions just build a protocol response from
libcouchbase return value. They also register the proxy_client_callback on
write events to send out data recently written into the buffer.

These are viritually all the integration points with libcouchbase. As we
demonstrated, libcouchbase more than a protocol parser; it also an abstract and
portable IO framework, allowing you to build asynchronous applications around
Couchbase Server.

To conclude, let’s run your favourite memcached client and verify our proxy.
First you need to clone and build it:

Use cluster type connection for cbc-bucket-flush. Although
flush command is accessible for bucket type connections,
cbc-bucket-flush doesn’t use provided bucket name to connect to,
therefore it will fail if the bucket name isn’t “default”.

Allow to make connect order deterministic. It allows the
user to toggle between deterministic and random connect order for
the supplied nodes list. By default it will randomize the list.

Fixes in 2.1.3

Updated gtest to version 1.7.0. Fixes issue with building
test suite with new XCode 5.0 version being released later this
month.

Skip unfinished SASL commands on rebalance. During
rebalance, it is possible that the newly added server doesn’t have
chance to finish SASL auth before the cluster will push config
update, in this case packet relocator messing cookies. Also the
patch makes sure that SASL command/cookie isn’t mixing with other
commands

Fig segmentation faults during tests load of
node.js. Sets inside_handler on socket_connected. Previously we
were always using SASL auth, and as such, we wouldn’t flush packets
from the cmd_log using server_send_packets (which calls
apply_want). apply_want shouldn’t be called more than once per
event loop entry – so this sets and unsets the inside_handler
flag.

Fallback to ‘select’ IO plugin if default plugin cannot be loaded. On UNIX-like
systems, default IO backend is ‘libevent’, which uses third-party library might
be not available at the run-time. Read in lcb_cntl(3couchbase) man page in
section LCB_CNTL_IOPS_DEFAULT_TYPES about how to determine effective IO
plugin, when your code chose to use LCB_IO_OPS_DEFAULT during connection
instantiation. The fallback mode doesn’t affect application which specify IO
backend explicitly.

Skip misconfigured nodes in the list. New lcb_cntl(3couchbase) added to control
whether the library will skip nodes in initial node list, which listen on
configuration port (8091 usually) but doesn’t meet required parameters (invalid
authentication or missing bucket). By default report this issue and stop trying
nodes from the list, like all previous release. Read more at man page
lcb_cntl(3couchbase) in section
LCB_CNTL_SKIP_CONFIGURATION_ERRORS_ON_CONNECT

Use provided credentials for authenticating to the data nodes. With this fix, it
is no longer possible to use Administrator credentials with a bucket. If your
configuration does so, you must change the credentials you use before applying
this update. No documentation guides use of Administrator credentials, so this
change is not expected to affect few, if any deployments.

Do not disable config.h on UNIX-like platforms. It fixes build issue, when
application is trying to include plugins from the tarball.

New backend select. This backend is based on the select(2) system call and its
Windows version. It could be considered the most portable solution and is
available with the libcouchbase core.

API for durability operations. This new API is based on lcb_observe(3) and
allows you to monitor keys more easily. See the man pages
lcb_durability_poll(3) and lcb_set_durability_callback(3) for more info.

New backend libuv. This backend previously was part of the
couchnode project and is now available
as a plugin. Because libuv doesn’t ship binary packages there is no binary
package libcouchbase2-libuv. You can build plugin from the source
distribution, or through the libcouchbase-dev or libcouchbase-devel package
on UNIX like systems.

New backend iocp. This is a Windows specific backend, which uses “I/O
Completion Ports”. As a part of the change, a new version of plugin API was
introduced which is more optimized to this model of asynchronous IO.

New configuration interface lcb_cntl(3) along with new tunable options of the
library and connection instances. In this release the following settings are
available. See the man page for more information and examples.:

LCB_CNTL_OP_TIMEOUT operation timeout (default 2.5 seconds)

LCB_CNTL_CONFIGURATION_TIMEOUT time to fetch cluster configuration. This is
similar to a connection timeout (default 5 seconds)

From the release the 2.1.0 package libcouchbase2 will not install an IO
backend automatically. If you are upgrading, there are no changes because you
have already libcouchbase2-libev or libcouchbase2-libevent packages
installed. For new installations, a backend must be selected for the client
library to work correctly.

If for example you are using the PHP SDK, the old way, which works for pre-2.1.0
versions is:

Improve lcb\_get\_replica(). Now it is possible to choose between three
strategies:

LCB_REPLICA_FIRST : Previously accessible and now the default, the caller will
get a reply from the first replica to successfully reply within the timeout for
the operation or will receive an error.

LCB_REPLICA_ALL : Ask all replicas to send documents/items back.

LCB_REPLICA_SELECT : Select one replica by the index in the configuration
starting from zero. This approach can more quickly receive all possible replies
for a given topology, but it can also generate false negatives.

Note that applications should not assume the order of the replicas indicates
more recent data is at a lower index number. It is up to the application to
determine which version of a document/item it may wish to use in the case of
retrieving data from a replica.

Fix segfault when rebalancing. When a (!connected) server is reconnected, the
tasks in its “pending” buffer will be moved into “output” buffer. If its
connection is broken again immediately, relocate_packets() will go to wrong
path.

Reset internal state on lcb_connect(). Allow caller to use lcb_connect()
multiple times to implement reconnecting using the same lcb_t instance. Also it
sets up the initial-connection timer for users who don’t use lcb_wait() and
drive IO loop manually.

Ensure HTTP works even when the network may be unreliable. This changeset
encompasses several issues which had been found with HTTP requests during
network errors and configuration changes. Specifically some duplicate code paths
were removed, and the process for delivering an HTTP response back to the user
is more streamlined.

Allow the user to specify a different hash key. All of the data operations
contains a hashkey and nhashkey field. This allows you to “group” items together
in your cluster. A typical use case for this is if you’re storing lets say data
for a single user in multiple objects. If you want to ensure that either all or
none of the objects are available if a server goes down, it could be a good idea
to locate them on the same server. Do bear in mind that if you do try to decide
where objects is located, you may end up with an uneven distribution of the
number of items on each node. This will again result in some nodes being more
busy than others etc. This is why some clients doesn’t allow you to do this, so
bear in mind that by doing so you might not be able to get your objects from
other clients.

Return zero from do_read_data() if operations_per_call reached. The
`operations_per_call' limit was introduced to prevent from freezing event
loop. But in the function variable rv could store two different results and in
case of reaching this limit it is returning number of the processed records,
which is wrong. The function should return either zero (success) or non-zero
(failure).

Refactor the API. This is a full redesign of the current libcouchbase API
that’ll allow us to extend parts of the API without breaking binary
compatibility. Also it renames all functions to have lcb prefix instead of
libcouchbase and LCB/LIBCOUCHBASE in macros.

Implement getter for number of nodes in the cluster: lcb_get_num_nodes()

Add lcb_get_server_list() to get current server list

Deliver HTTP headers via callbacks

Implement RESTful flush in the cbc toolset

Merge lcb_get_locked into lcb_get function

Bundle libvbucket

Fixes in 2.0.0beta

Fix a problem with allocating too few slots in the backup_nodes. Fixes illegal
memory access.

Include sys/uio.h. Needed by OpenBSD

Added --enable-fat-binary. Helps to solve issues when linking with fat
binaries on MacOS.

Differentiate between TMPFAILs. This allows a developer to know if the temporary
condition where the request cannot be handled is due to a constraint on the
client or the server.

Don’t try to put the current node last in the backup list. This may cause
“duplicates” in the list if the REST server returns another name for the server
than you used. Ex: you specify “localhost” and the REST response contains
127.0.0.1

Claim that server has data in buffers if there are HTTP requests pending.
Without this patch the event loop can be stopped prematurely.

Make libcouchbase_wait() re-entrable

Fix to handle the case when View base doesn’t have URI schema.

Bind timeouts to server sockets instead of commands. This means that from this
point timeout interval will be started from the latest IO activity on the
socket. This is a behavior change from the 1.0 series.

Implement function to execution management requests. Using
libcouchbase_make_management_request() function you can configure the
cluster, add/remove buckets, rebalance etc. It behaves like
libcouchbase_make_couch_request() but works with another endpoint.

Add support for notification callbacks for configuration changes. Now it is
possible to install a hook using function
libcouchbase_set_configuration_callback(), and be notified about all
configuration changes.

A fix for a buffer overflow with the supplied password as has been integrated.
While it is a buffer overflow issue, this is not considered to be a possible
security issue because the password to the bucket is not commonly supplied by an
untrusted source

Don’t wait for empty buffers. If called with no operations queued,
libcouchbase_wait() will block forever. This means that a single threaded
application that calls libcouchbase_wait() at different times to make sure
operations are sent to the server runs the risk of stalling indefinitely. This
is a very likely scenario.