REST APIs with Node Tastypie and Mongo - Part 1

EST Apis seem to be taking over the world as of late, and Node has been the platform of choice for building them. However, when you take a look at some of the tooling for building robust REST APIs in the node community, the options are bleak at best. When you take a look at all of the properties that would make an API RESTful, it is easy to see that it is rather complex.

REST

There are a few fundamental properties that make apis RESTful - they should be a Scaleable, Stateless, Cacheable, fit into a Layered System which is apart of a Client / Server model that provides a Uniform Interface. In the confines of a Restful HTTP API, it is common to use the HTTP verbs ( GET, PUT, POST, DELETE, etc ) to describe what action the API should perform.

When it comes right down to it, there isn't really anything the facilitates this in the Node world. Most frameworks are limited and leave a lot of this work up to you which leads to a lot of repetitive boilerplate work. And like many DIYers in the JavaScript community, this frustration has lead me to build one myself. For the better part of a year I have been working on the side on a REST framework heavily inspired by a Django project called Tastypie which focuses on implementing most of the REST principles with an emphasis on HATEOAS - And wrote it in JavaScript!

Tastypie

Tastypie is built on top of Hapi from the fine folks at Walmart. Tastypie lets you Define a Resource which will provide basic CRUD operations which map to the HTTP verbs complete with customization serialization, Validation, Caching, etc. Plus easy-as-pie Mongoose support to boot. To create Your first CRUD API we need 3 things - a Data Model, a tastypie Resource definition, and an API Namespace to hold your resources.

Define A Resource

The next step is to define a Resource for this data model. While tastypie resources are not inherently tied to mongoose, it does ship with a Resource class specifically for mongoose. Basically, give it a query constructor, and a field definition. Done!

This is the bare bones set up of a resource which will give us full CRUD support. You must give it a query constructor using the toConstructor method on a mongoose query. We're using the simple case here. And a fields definition which defines the data fields you want to expose. The resource class will auto add an id field and uri field. But we'll come back to that.

That is it! The Tastpie API class is a Hapi plugin which adds the url prefix you defined - ( api/v1 in our case ).

Interacting With Your API

Under the hood our resource creates 7 routes by default.

GET /api/v1 - endpoint schema

GET /api/v1/user - Get all users

GET /api/v1/user/schema - User schema

GET /api/v1/user/{pk} -get user by id

POST /api/v1/user - create a user

PUT /api/v1/user/{pk} - update a user

DELETE /api/v1/user/{pk} - delete a user

Tastypie uses the standard HTTP Headers Accept and Content-Type to deal with serialization / serialization of data. You can use whatever HTTP client you like. Curl will do us OK here.

API Schema

Each API instance registers an endpoint which provides all of the defined routes under it as a means for endpoint discovery. In our example it is just the list, detail, and schema endpoints, as you add more resources, this becomes a bit more dense.

Internally Tastypie maps each of the routes to an instance method using the HTTP verb. For example creating a user with POST /api/v1/user maps to post_list, where as GET /api/v1/user maps to the get_list method. Most functionality on a resource is granular and function based so you can subclass & change things if you like. If not granular, there is probably a config option for it.

From here we can take a look at the list endpoint which will return a paged set of data. The default page size is set to 25.

You can see here we have a structured response with two primary blocks, meta and data. The meta block gives you some information about the response. Namely how many records are in the actual data set ( of the query you created ), the current page size or limit,and URIs to the next and previous page, if they exist.

The data block is an array of objects mapping be to the model we defined with the auto included id and uri field which is the unique identifier for that specific object. You can also see here that there is a companyName field which surfaced the nested value from our model. This is because our field definition specified a name path in the attribute property.

companyName:{type:'char', attribute:'company.name', readonly:true}

There are a number of ways to reshape the data, the attribute field is the easiest. It works in both directions, so one could actually send data in the companyName field and tastypie would resolve it to company.name. But in general, it is best to use a data structure that mirrors your model, so we marked this as readonly.

Resource Schema

Each defined resource comes standard with a simple schema definition mapped to the /schema uri which defines default format, allowed methods, and field definitions.

This sort of information can come in very hand for auto generating user interfaces for example.

If you don't like this schema format, you can alter this by changing the get_schema method on your resource to return a JSONSchema definition if you prefer. Or to return what ever you like for that matter.

Paging

You can use a number of special query string params to control how data is paged on the list endpoint. Namely -

limit - Page size ( default 25 )

offset - The starting point in the list

limit=25&offset=50 would be the start of page 3

Sorting

sorting is handled query param orderby where you can pass it the name of a field to sort on. Sorting is descending by default. Specifying a negetive field ( -<FOO> ) would flip the order

orderby=-age

Advanced Querying

You might have noticed the filtering field on the schema. One of the things that makes an API "Good" is the ability to use query and filter the data to get very specific subsets of data. Tastypie exposes this through the query string as field and filter combinations. By default, the resource doesn't have anything enabled, you need to specify which filters are allowed on which fields, or specify 1 to allow everything

Filters are added by appending a double underscore __ and the filter type to the end of a field name. Given our example, if we wanted to find people who were older than 25, we would use the following URI syntax

http://localhost:3000/api/v1/user?age__gt=25

Filters are additive for a given field. For example, if we we only wanted people where we between 25 and 45, we would just add another filter

http://localhost:3000/api/v1/user?age__gt=25&age__lt=45

The same double underscore __ syntax is used to access nested fields where the filter is always the last parameter. So we could find people whos age was greater than 25, less than 45 and whose Company Name starts withW

Resources provide a simple and expressive syntax to query for very specific subsets of data without any of the boilerplate work to set it up. And as you would expect, regular params will map back to exact where applicable

http://localhost:3000/api/v1/user?age=44

Serialization

Tastypie supports multiple serialization formats out of the box as well as a way to define your own custom formats. The base serializer class supports xml, json & jsonp by default. You add formats or create your own serialization formats by subclassing the Serializer class and defining the approriate methods. Each format must define a to_<FORMAT> and a from_<FORMAT>. For example, tastypie defines the to_xml and from_xml methods. JSON is defined by to_json, from_json

To get back xml just change the Accept header NOTE:Hapi deals with most application/foo formats, but is blind to text/foo. So the safe bet here is to use text/xml

Validation

At the time of writing, Tastypie ( 0.2.2 ) delegates validation responsibility to Hapi. You can specifiy a validation object in the Resource options which is passed to the Route configuration untouched. So to validate that the Age Param of incoming post / put data is an integer and less than 100, we could update our resource to hold the following

This is a pretty good overview of tastypie, and we are still just scratching the surface. However, with less than 100 lines of code, we have a pretty feature rich and expressive CRUD API. Next time we'll cover adding custom endpoints beyond CRUD, advanced data preparation through Resource fields and access control.

I'm a software developer and system architect working at help.com. Most of my day is spent working with Javascript & Node.js. I've also done a good deal of web and print design work in my day. I created this space to share my experiences with the world and hopefully learn something in the process.

This Space

Here you will find my ramblings and rants about web development. My focus is around JavaScript( MooTools, Sencha, NodeJS ), Python & Django, HTML & CSS. Most things here target a wide range of skill levels - from the very simple to the moderately complicated. You may also find the occasionaly personal ranting and I may stand on a soap box from time to time.