A little cup'a Kromer with your code

Farewell JSON API Gems

In the past, testing JSON APIs tended to be a bit painful for me. Most of this
pain revolved around setting expectations on the response body.

If you treat the response as a raw string, attempting to use regular
expressions ends up being an exercise in how you handle frustration. While a
JSON body is a string, it has structure. Using regular expressions for parsing
them is akin to using a hammer on a screw. It’ll get the job done, but it’s the
wrong tool for the job.

Ruby gives us JSON.parse.
Which will convert a valid JSON string into a more familiar object structure.
Now comes the “fun” part of actually verifying that structure:

Sometimes you only care about part of the response

Sometimes you care about validating the entire response

Sometimes the response is very complicated consisting of many smaller, more
logically meaningful, structures

Sometimes you only care about the general structure (e.g. this value must be
a number, that value must be either an empty array or an array of strings,
etc.)

It is possible to do all of these validations out of the box. In my experience,
writing them tended to be tedious. Often the resulting code left something to
be desired in terms of readability. This was especially true when validating the
general response structure.

I like to follow the “one expectation per spec” guideline. However, this lead
to writing many small specs. Normally, this is perfectly fine and something I
advocate you do. However, in terms of a JSON response, it means I need to have
more discipline to keep everything explicitly organized.

Naturally in the Ruby community, many gems have sprouted up to help with this
problem set. I’ve had a bit of success with some of those gems in the past.
However, with the release of RSpec 3, several new
features
have eliminated my need for these JSON gems.

Often people don’t realize that the matcher messages (i.e. exist, be, eq,
include, etc) are just factories helpers (see
endnotes). They are just helper methods which create
the matcher object for you. That means, we can easily write our own using our
app’s domain language.

Let’s jump right into an example!

These examples are assuming a JSON structure like one of the ones listed on the
jsonapi.org
site. Though I am assuming integer value are represented as numbers and not
strings, since that is valid JSON and more meaningful:

require'rails_helper'# Use common JSON helpers such as: `json_response`, `be_an_empty`, `all_match`require'support/json_api_helpers'RSpec.describe"/api/kits",type::requestdodefbe_kits_root_jsonbe_kits_json.and(include('meta'=>{'first'=>anything,'last'=>anything,'current'=>anything,}))enddefbe_kits_jsoninclude('version'=>'1.0','links'=>{'kits.beacons'=>"#{beacons_url}/{kits.beacons}",'kits.overlays'=>"#{overlays_url}/{kits.overlays}",'beacons.attributes'=>"#{beacon_attributes_url}/{beacons.attributes}",},'kits'=>be_an_empty(Array).or(all_match('id'=>Fixnum,'name'=>be_nil.or(be_aString),'api_token'=>String,'account'=>be_nil.or(match('id'=>Fixnum,'name'=>be_nil.or(be_aString),)),'links'=>{'self'=>/\A#{kits_url}\/\d+\z/,'beacons'=>be_an_empty(Array).or(allbe_aFixnum),'overlays'=>be_an_empty(Array).or(allbe_aFixnum),},),),)enddefinclude_linked_resources(*resources)resource_maps=resources.each_with_object({}){|resource,mappings|mappings.store(resource.to_s,be_an(Array))}include('linked'=>resource_maps)endcontext"a basic user","with a kit having no beacons or maps"do# Setup world statedescribe"requesting the kits root"doit"conforms to the expected JSON structure"dogetkits_path,*optionsexpect(json_response).tobe_kits_root_jsonend# More specific specsenddescribe"requesting a kit"doit"conforms to the expected JSON structure"dogetkit_path(kit),*optionsexpect(json_response).tobe_kits_jsonend# More specific specsendend# More state specscontext"a developer user","sending request with parameter 'include'"do# Setup world statedescribe"requesting the kits root"doit"conforms to the expected JSON structure with included resources"dogetkits_path(include:"beacons,beacon_attributes"),*optionsexpect(json_response).tobe_kits_root_json.and(include_linked_resources(:beacons,:beacon_attributes))endenddescribe"requesting a beacon"doit"conforms to the expected JSON structure with included resources"dogetkit_path(kit,include:"beacons,beacon_attributes"),*optionsexpect(json_response).tobe_kits_json.and(include_linked_resources(:beacons,:beacon_attributes))endendendend

The possibilities are fairly endless. We could improve this further by allowing
the factories to take model instances or attribute hashes. We can use those to
check specific content when available:

defaccount_resource(account=nil,allow_nil:false)returnnilunlessaccount||!allow_nilifaccount{'id'=>account.id,'name'=>account.name}else{'id'=>Fixnum,'name'=>be_nil.or(be_aString),}endenddefkit_resource(kit=nil,allow_nil:false)returnnilunlesskit||!allow_nilifkit{'id'=>kit.id,'name'=>kit.name,'api_token'=>kit.api_token,'account'=>account_resource(kit.account,allow_nil:true),}else{'id'=>Fixnum,'name'=>be_nil.or(be_aString),'api_token'=>String,'account'=>be_nil.or(matchaccount_resource),}endendcontext"a basic user","with a kit having no beacons or maps"do# Setup world statedescribe"requesting the kits root"doit"conforms to the expected JSON structure"dogetkits_path,*optionsexpect(json_response).tobe_kits_root_jsonendit"has only the expected kit"dogetkits_path,*optionsexpect(json_response).toinclude'kits'=>[kit_resource(basic_users_kit)]endenddescribe"requesting a beacon"doit"conforms to the expected JSON structure"dogetkit_path(kit),*optionsexpect(json_response).tobe_kits_json(basic_users_kit)endendend

Happy RSpec’ing!

Updates 2014-09-30

Thanks to everyone who provided feedback on this post. I’ve take it all into
consideration and made the following changes:

The first code sample now shows the full spec file structure. This hopefully
makes the important distinction that the helper methods are not being defined
on main.

The line require 'support/json_api_helpers' isn’t loading another library.
Instead it is loading an extracted set of shared helper methods common to
nearly all JSON API request specs for this project. These have been extracted
to a module to keep them off of main and placed in spec/support.

This follows the new guidance that specs should only load those files which
they need. It also makes it easier for your future self and your co-workers
to come back to the file later and try to find where things are defined.

It was pointed out that all match will handle the empty Array case.
It is possible to amend the above be_an_empty(Array).or(all_match())
to instead read: be_an(Array).and(all_match()).

This is a helper method. My reference to it
as a factory was more explicitly attempting to describe it as a helper method
which instantiates another object. In Rails, people often know of “factories”
from the “factory vs fixture” debate. Often those factories are relatively
simple wrappers around constructors. After researching this a little more it
seems in the larger programming world “factory” is not the proper term.
Perhaps “creator” is. I will move to
calling them helpers in the future.