So last time I documented the Another TODO API using API Blueprint and now I'm going to take advantage of that to have some test against the API to ensure documentation is up to date with the actual API code. For this task, I'm going to be using Dredd.

Some notes about what has been done here. Dredd has created a dredd.yml file at the root of the project with a bunch of properties based on the replies it has received.

dredd.yml

# https://dredd.readthedocs.io/en/latest/usage-cli.html#configuration-filedry-run:nullhookfiles:nulllanguage:nodejssandbox:falseserver:npm start# Command to start the API server server-wait:3init:falsecustom:apiaryApiKey:''names:falseonly:[]reporter:apiaryoutput:[]header:[]sorted:falseuser:nullinline-errors:falsedetails:falsemethod:[]color:truelevel:infotimestamp:falsesilent:falsepath:[]hooks-worker-timeout:5000hooks-worker-connect-timeout:1500hooks-worker-connect-retry:500hooks-worker-after-connect-wait:100hooks-worker-term-timeout:5000hooks-worker-term-retry:500hooks-worker-handler-host:127.0.0.1hooks-worker-handler-port:61321config:./dredd.yml# Source of Dredd config file blueprint:docs/main.apib# The API Blueprint file to get API definitions endpoint:'http://127.0.0.1:3000/v1'# The base URL where the test will run

Also, at the end, if the dredd config file has the reporter as apiary, there will be a link (similar to https://app.apiary.io/public/tests/run/123456) to this page:

Note: The provided link is a temporary page and will be removed after a while.

In this panel is a lot of info about how the tests did go. Another TODO API has some errors in the docs, one of them is the definition of content-type. Let's fix that and run the tests again.

After the changes and running dredd this is the new report:

This time some of the endpoints have been validated but not all. The endpoints that require a task ID to work are returning 404 responses and causing the test to fail. this is because the task IDs specified in the API docs are only exposed as an example and doesn't really exists in the DB. Here is when Dredd hooks come handy.

Dredd Hooks

The hooks allow executing some code between, before or after each test case. This time I'm going to use one hook to get the ID of the task created on the "Create a New Task" definition to use that created task for the tests that need a taskId to work.

docs/hooks.js

// Import the hooks library to work with them (injected by dredd)consthooks=require('hooks')// Create some shorthand functions for the hooksconstafter=hooks.afterconstbefore=hooks.before// Because the action is going to be the same in all the hooks lets create a functionconstreplaceUrlForCreatedTaskId=function(transaction){// Gets the taskId from the response objectlettaskId=JSON.parse(responseStash['Tasks > Tasks Collection > Create a New Task'].body)._id// Gets the predefined request urlleturl=transaction.fullPath// Replaces the wrong taskId with the correct onetransaction.fullPath=url.replace('586e88337106b038d820a54f',taskId)}// Instantiates an object to store the responsesletresponseStash={}// Sets a hook to be executed after creating a task to store the responseafter('Tasks > Tasks Collection > Create a New Task',function(transaction){// Stores the response inside the temporary objectresponseStash[transaction.name]=transaction.real})// Sets hooks before the requests are made to replace the URLsbefore('Tasks > Task > View a Task',replaceUrlForCreatedTaskId)before('Tasks > Task > Edit a whole Task',replaceUrlForCreatedTaskId)before('Tasks > Task > Edit a Task partially',replaceUrlForCreatedTaskId)before('Tasks > Task > Delete a Task',replaceUrlForCreatedTaskId)

After setting the hooks the dredd.yml file needs to be modified.

dredd.yml

# https://dredd.readthedocs.io/en/latest/usage-cli.html#configuration-filedry-run:nullhookfiles:./docs/hooks.js# Here, we are telling dredd where are the hooks files language:nodejssandbox:false...

There is being used a data structure for the response field but it's indented by 8 spaces , and for API Blueprint documents that means a block of code, so by reducing it to 4 and running the tests again:

Wow, very cool. I actually read the test from the API and write tests with standard unit testing libraries if I want to learn about an API. Never heard about DDD, it seems like dredd can be extremely in that regard. Thank you!

I think that DDD could be a good way to design APIs before beign developed and ensure they works as the designs says. That is one of the problems in my actual job. Backend designs the API Docs for new functionalities and all the teams discuss them and request changes and additions but there are some times that the development doesn't fullfil the designs or the designs doesn't gets updated. This would solve those problems.

Dredd is more of a RDD tool (tom.preston-werner.com/2010/08/23/...), it won't completely replace all your unit tests. But otherwise it exists to solve exactly the problem you described. It makes sure the docs users are presented with are always correct, and that the API is always correct according to the design.

I don't think RDD would be a good practice in this case, as the author says:

By restricting your design documentation to a single file that is intended to be read as an introduction to your software

It may be overwhelming to have a README with the whole API specification, what if the user only want to read only an introduction to the API or some small examples, maybe read the contribution section or some other info that readme files should include.

It's true that the more documents you have the worst you will maintain them updated, but in my opinion, is better if you have a README file that refers to your API docs and offers some friendly introduction and examples, maybe a FAQ and some project info.

The API description document (API Blueprint, Swagger, OpenAPI) usually describes the API in form of some expected happy scenarios. Usually, it doesn't contain examples for all the corner cases. In this sense, it is something like README for a software project. It helps user to understand how to use the API, but it doesn't replace the reference book. For that reason, if you test your API against your API description by Dredd, you get very very very useful thing - an assurance that what users read in the docs is exactly how the API works. But you don't get all the negative scenarios and all the corner cases, so you still need to add some unit/integration tests. That's why I think this is more of a RDD than DDD, but anyway, this is pretty much bike shedding :D

I completely agree an actual README should not be equal to an API description.

And by the way, thanks for a great article. We should definitely link it from the Dredd docs! It's better than our own tutorials. It's amazing to see when people find Dredd so useful they even write articles about it.