Featured in
Architecture & Design

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Operations & Infrastructure

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Enterprise Architecture

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Introduction to Interface-Driven Development Using Swagger and Scalatra

Since it began life a little over three years ago, the Scalatra web micro-framework has evolved into a lightweight but full-featured model-view-controller (MVC) framework with a lively community behind it. Scalatra started out as a port of Ruby's popular Sinatra DSL to the Scala language. Since then the two systems have evolved independently, with Scalatra gaining capabilities such as an Atmosphereintegration and Akka support.

It's been used by BBC Future Media for their Linked Data Writer API, managing large datasets in a scalable and manageable way, and has also been used by gov.uk.

One of the things that Scalatra's been most successful at is the construction of APIs. Over the past several years, REST APIs have become the lifeblood of the web. A relatively recent addition to Scalatra's capabilities is an integration with the Swagger toolset, which is produced by the folks at Wordnik.

What is Swagger?

Swagger is a specification which allows you to quickly define the functionality of a REST API using JSON documents. But it's more than just a spec. It provides automatic generation of interactive API docs, client-side code generation in multiple languages, and server-side code generation in Java and Scala.

Although they're the most eye-catching component of the project and will impress users, the docs produced by Swagger are also great for fostering communication between API producers and consumers during the API design phase.

Let's take a look at how it all works. We'll build out a small REST API using Scalatra, then use Swagger to document our routes.

The first thing you'll need to do is install Scalatra. The easiest way to do this is by following the installation instructions at the Scalatra website. See the notes at the bottom for setting up Eclipse or IntelliJ if you use those IDEs, but you should be able to do this tutorial in any text editor.

Once you've got a JVM installed, along with cs, giter8, and sbt, you'll be able to generate a new Scalatra project.

Getting started

Type this at the command line:

g8 scalatra/scalatra-sbt --branch develop

You'll be asked a series of questions about your project. Answer like this:

Hit to accept the defaults for the organization, scala_version, and version questions.

Once you answer the last question, a full Scalatra project will be generated. Let's check that it works. Change directory into the flowershop folder, and run Scala's simple build tool by typing:

cd flowershop
sbt

You'll need the latest sbt 0.12.1 for this. sbt 0.12.0 will work, just downgrade the version number in the file project/build.properties.

sbt will take care of downloading all of Scalatra's dependencies. This can take several minutes when you're doing it for the first time, as you're getting a full Scala development environment, an embedded webserver (jetty), Scalatra itself, and several companion libraries.

When sbt finishes setting everything up, you should be able to start the application by typing the following at the sbt prompt (which looks like a ">").

container:start

This will start jetty on http://localhost:8080. Visit that URL in your browser, and you should see a Hello World application.

You don't want to have to manually recompile your app and restart Jetty whenever you make a code change, so type this at the sbt prompt:

~; copy-resources; aux-compile

This tells sbt to automatically recompile and reload the application whenever you change a file.

So, now we've got a controller for our flower shop. Let's set up a RESTful interface allowing us to browse flowers.

Open up the FlowersController.scala file found in src/main/scala/com/example/swagger/sample. What you start out with is a very simple generated controller. It's got a "hello world" action mounted on the application's root path get("/"), which can be accessed using an HTTP GET to the path "/". It's also got a way to handle 404s, and Scalate templating, which we don't need for our API.

Scalatra allows you to easily add functionality to your controllers by mixing in Scala traits to your class definitions. Let's slim things down a tiny bit by removing the ScalateSupport, since we don't need HTML templating support for our API. Delete the with ScalateSupport part of the class definition, so it looks like this:

classFlowersControllerextendsScalatraServlet {

You can also remove the import scalate.ScalateSupport, it won't be needed.

You can remove the notFound and get("/") actions as well. You'll be left with an empty Scalatra controller:

As you make these changes and save your files, check sbt's output in the terminal. You should see your source code automatically recompiling itself and reloading the application. The terminal output will turn green once you've slimmed down your controller. You've now taken all of the routes out of FlowersController, so you won't be able to see anything in your browser.

Setting up the data model

Let's get some data set up. Since we want to focus on learning Swagger, we won't attempt to actually persist anything in this tutorial. We can use Scala case classes to simulate a data model instead. If you'd like to see how to find out how to set up ScalaQuery, a Scala ORM, check out Jos Dirksen's tutorial at SmartJava.

First, we'll need a flower model. Add a new directory in src/main/scala/com/example/swagger/sample, and call it models. Then put the following code in a new file, Models.scala, in there:

Models.scala

packagecom.example.swagger.sample.models// A Flower object to use as a faked-out data modelcase classFlower(slug: String, name: String)

A Scala case class automatically adds getters and setters to the class definition, so we get a lot of functionality here without a lot of boilerplate.

Let's add some flower data.

Make a data namespace by adding a new data directory inside src/main/scala/com/example/swagger/sample. Then add a new file in that directory, calling it FlowerData.scala.

That gives us enough to work with in terms of data that we can at least demonstrate our API's functionality.

Let's make a new controller action to retrieve flowers. Add some imports so that we get access to our models and data in the FlowersController class, by adding this at the top, after the package definition:

Retrieving flowers

Now let's make our first API method, a Scalatra action which lets clients browse flowers. Drop this into the body of the FlowersController class:

get("/"){
FlowerData.all
}

It doesn't look like much, but this is faking out data retrieval for us, by calling the all method of the FlowerData object we just defined. This is broadly equivalent to calling a static method in Java or C#, or a class method in Ruby.

Scalatra has found all the flowers for us and returned the data. Looking at what we've got so far, though, it's not a very descriptive API. What resource is actually being retrieved? It's not possible to tell by looking at the URL. Let's change that.

Setting the mount path for better API clarity

Every Scalatra application has a file called ScalatraBootstrap.scala, located in the src/main/scala directory. This file allows you to mount your controllers at whatever url paths you want. If you open yours right now, it'll look something like this:

The only change was to replace the "/*" mount point with "/flowers". Easy enough. Let's make sure it works. Hit the url http://localhost:8080/flowers in your browser, and you should once again see the same results as before:

This is a much more descriptive URL path. Clients can now understand that they're operating on a flower resource.

Automatic JSON output for API actions

Take a closer look at the output. What's going on here? Scalatra has converted the FlowerData.all value to a string and rendered its Scala source representation as the response. This is the default behaviour, but in fact we don't want things to work this way - we want to use JSON as our data interchange format.

Let's get that working. Scalatra 2.2 includes some new JSON handling capabilities which makes this a snap.

In order to use Scalatra's JSON features, we'll need to add a couple of library dependencies so that our application can access some new code. In the root of your generated project, you'll find a file called build.sbt. Open that up, and add the following two lines to the libraryDependencies sequence, after the other scalatra-related lines:

build.sbt is somewhat equivalent to a maven pom.xml in Java, or a Gemfile in Ruby, insofar as it keeps track of all your project's dependencies and can take care of downloading them for you.

Restart sbt to download the new jars. You can do so by first hitting the "enter" key (to stop automatic recompilation), and then typing exit at the sbt prompt. Then type sbt again. You should see some messages telling you that sbt is downloading the new dependencies, and then you'll be back at the prompt. Start the container and recompilation again:

container:start
~; copy-resources; aux-compile

Add the following imports to the top of your FlowersController file, in order to make the new JSON libraries available:

Now we can add a bit of magic to the FlowersController. Putting this line of code right underneath the controller class definition will allow your controller to automatically convert Scalatra action results to JSON:

Just like its Sinatra forebear, Scalatra has a rich set of constructs for running things before and after requests to your controllers. A before filter runs before all requests. Add a before filter to set all output for this controller to set the content type for all action results to JSON:

// Before every action runs, set the content type to be in JSON format.
before() {
contentType = formats("json")
}

Now mix JacksonJsonSupport and JValueResult into your servlet so your controller declaration looks like this:

The JValueResult and JsonJacksonSupport traits which we mixed into the controller, combined with the implicit val jsonFormats, are now turning all Scalatra action result values into JSON.

Making the flowers API searchable

Next, let's make our API searchable. We want to be able to search for flowers by name and get a list of results matching the query. The easiest way to do this is with some pattern matching inside the / in our controller.

Currently that route looks like this:

get("/"){
FlowerData.all
}

We can change it to read a query string parameter, and search inside our list of flowers.

The nice thing is that since our before() filter runs on each and every action, the JSON format converter is still operating on our output. We get automatic JSON support with no extra effort. Sweet.

Interface-driven development using Swagger

At this point, we've got the beginnings of a REST API. It defines two actions, and offers a way for API clients to see what flowers are available in our flower shop. With our desired functionality achieved, we could stop here. But we're missing two things: human-readable documentation and client integration code. And without these, the API is a lot less useful than it could be.

An API is a way for machines to exchange data, but the process of designing and building an API also requires a lot of communication between people. The best API designs happen when the API's users have a way to get involved and detail what it is they need, and the API implementers boil those conversations down into an interface that works for the desired user stories or use cases. Historically, the fact that you've needed to use cURL or read WSDL to understand what an API does has severely limited the ability of non-technical people to participate in the API design process. The technical complexity of just making a connection has masked the fact that, at their core, the concepts inherent in a REST API are not particularly hard to understand.

Making the API's methods, parameters, and responses visible, in an engaging, easy to understand way, can transform the process of building REST APIs. The people at Wordnik, the word meanings site, have built a toolset called Swagger, which can help with this.

Swagger is a bunch of different things. It's a specification for documenting the behaviour of a REST API - the API's name, what resources it offers, available methods and their parameters, and return values. The specification can be used in a standalone way to describe your API using simple JSON files.

The Swagger resources file

If you want to, you can write a Swagger JSON description file by hand. A Swagger resource description for our FlowersController might look like this (don't bother doing so, though, because we'll see how to automate this in a moment):

{"basePath":"http://localhost:8080","swaggerVersion":"1.0","apiVersion":"1","apis":[{"path":"/api-docs/flowers.{format}","description":"The flowershop API. It exposes operations for browing and searching lists of flowers"}]}

This file describes what APIs we're offering. Each API has its own JSON descriptor file which details what resources it offers, the paths to those resources, required and optional parameters, and other information.

A sample Swagger resource file

{"resourcePath":"/","listingPath":"/api-docs/flowers","description":"The flowershop API. It exposes operations for browing and searching lists of flowers","apis":[{"path":"//","description":"","secured":true,"operations":[{"httpMethod":"GET","responseClass":"List[Flower]","summary":"Show all flowers","notes":"Shows all the flowers in the flower shop. You can search it too.","deprecated":false,"nickname":"getFlowers","parameters":[{"name":"name","description":"A name to search for","required":false,"paramType":"query","allowMultiple":false,"dataType":"string"}],"errorResponses":[]}]},{"path":"//{slug}","description":"","secured":true,"operations":[{"httpMethod":"GET","responseClass":"Flower","summary":"Find by slug","notes":"Returns the flower for the provided slug, if a matching flower exists.","deprecated":false,"nickname":"findBySlug","parameters":[{"name":"slug","description":"Slug of flower that needs to be fetched","required":true,"paramType":"path","allowMultiple":false,"dataType":"string"}],"errorResponses":[]}]}],"models":{"Flower":{"id":"Flower","description":"Flower","properties":{"name":{"description":null,"enum":[],"required":true,"type":"string"},"slug":{"description":null,"enum":[],"required":true,"type":"string"}}}},"basePath":"http://localhost:8080","swaggerVersion":"1.0","apiVersion":"1"}

These JSON files can then be offered to a standard HTML/CSS/JavaScript client to make it easy for people to browse the docs. It's extremely impressive - take a moment to view the Swagger Pet Store example. Click on the route definitions to see what operations are available for each resource. You can use the web interface to send real test queries to the API, and view the API's response to each query.

Swagger language and framework integrations

Let's get back to the spec files. In addition to enabling automatic documentation as in the Pet Store example, these JSON files allow client and server code to be automatically generated, in multiple languages.

The framework integrations allow you to annotate the code within your RESTful API in order to automatically generate JSON descriptors which are valid Swagger specs. This means that once you annotate your API methods, you get some very useful (and pretty) documentation capabilities for free, using the swagger-ui. You also get the ability to generate client and server code in multiple languages, using the swagger-codegen project. Client code can be generated for Flash, Java, JavaScript, Objective-C, PHP, Python, Python3, Ruby, or Scala.

Add the dependencies

Exit your sbt console and once again type sbt in the top-level directory of your application in order to pull in the dependencies. Then run container:start and ~; copy-resources; aux-compile to get code reloading going again.

You'll now need to import Scalatra's Swagger support into your FlowersController:

// Swagger supportimportorg.scalatra.swagger._

Auto-generating the resources.json spec file

Any Scalatra application which uses Swagger support must implement a Swagger controller. Those JSON specification files, which we'd otherwise need to write by hand, need to be served by something, after all. Let's add a standard Swagger controller to our application. Drop this code into a new file next to your FlowersController.scala. You can call it FlowersSwagger.scala

That code basically gives you a new controller which will automatically produce Swagger-compliant JSON specs for every Swaggerized API method in your application.

The rest of your application doesn't know about it yet, though. In order to get everything set up properly, you'll need to change your ScalatraBootstrap file so that the container knows about this new servlet. Currently it looks like this:

In order to make our application compile again, we'll need to add a name and description to our FlowersController. This allows Swagger to inform clients what our API is called, and what it does. You can do this by adding the following code to the body of the FlowersController class:

The summary and notes should be human-readable messages that you intend to be read by developers of API clients. The summary is a short description, while the notes should offer a longer description and include any noteworthy features which somebody might otherwise miss.

The nickname is intended as a machine-readable key which can be used by client code to identify this API action - it'll be used, for instance, by swagger-ui to generate method names. You can call it whatever you want, but make sure you don't include any spaces in it, or client code generation will probably fail - so "getFlowers" or "get_flowers" is fine, "get flowers" isn't.

The responseClass is essentially a type annotation, so that clients know what data types to expect back. In this case, clients should expect a List of Flower objects.

The parameters details any parameters that may be passed into this route, and whether they're supposed to be part of the path, post params, or query string parameters. In this case, we define an optional query string parameter called name, which matches what our action expects.

Lastly, the endpoint annotation defines any special parameter substitution or additional route information for this method. This particular route is pretty straightforward, so we can leave this blank.

The Swagger annotations here are mostly similar to those for the get("/") route. There are a few things to note.

The endpoint this time is defined as {slug}. The braces tell Swagger that it should substitute the contents of a path param called {slug} into any generated routes (see below for an example). Also note that this time, we've defined a ParamType.Path, so we're passing the slug parameter as part of the path rather than as a query string. Since we haven't set the slug parameter as required = false, as we did for the name parameter in our other route, Swagger will assume that slugs are required.

Now let's see what we've gained.

Adding Swagger support to our application, and the Swagger annotations to our FlowersController, means we've got some new functionality available. Check the following URL in your browser:

You should see an auto-generated Swagger description of available APIs (in this case, there's only one, but there could be multiple APIs defined by our application and they'd all be noted here):

{"basePath":"http://localhost:8080","swaggerVersion":"1.0","apiVersion":"1","apis":[{"path":"/api-docs/flowers.{format}","description":"The flowershop API. It exposes operations for browing and searching lists of flowers"}]}

Now for the wonderful part.

Browsing your API using swagger-ui

If you browse to http://petstore.swagger.wordnik.com/, you'll see the default Swagger demo application - a Pet Store - and you'll be able to browse its documentation. One thing which may not be immediately obvious is that we can use this app to browse our local Flower Shop as well.

Paste your Swagger resource descriptor URL - http://localhost:8080/api-docs/resources.json - into the URL field, delete the "special-key" key, then press the "Explore" button. You'll be rewarded with a fully Swaggerized view of your API documentation. Try clicking on the "GET /flowers" route to expand the operations underneath it, and then entering the word "rose" into the input box for the "name" parameter. You'll be rewarded with JSON output for the search method we defined earlier.

Also note that the swagger-ui responds to input validation: you can't try out the /flowers/{slug} route without entering a slug, because we've marked that as a required parameter in our Swagger annotations. Note that when you enter a slug such as "yellow-tulip", the "{slug}" endpoint annotation on this route causes the swagger-ui to fire the request as /flowers/yellow-tulip.

If you want to host your own customized version of the docs, you can of course just download the swagger-ui code from Github and drop it onto any HTTP server.

The reason it works is that Scalatra has Cross-Origin Resource Sharing (CORS) support built-in, allowing cross-origin JavaScript requests by default for all requesting domains. This makes it easy to serve JS API clients - but if you want, you can lock down requests to specific domains using Scalatra's CorsSupport trait. See the Scalatra Helpers documentation for more.

Conclusion

Without much in the way of boilerplate code, you've now constructed a simple REST API, set up model-class to JSON output functionality, and auto-generated API documentation by annotating your Scalatra routes with Swagger information.

This is one way to use Swagger, but the Wordniks swagger differently: rather than starting with the API and using Swagger to just generate the docs, they start out by writing the JSON descriptor files by hand. They then look at the API using the HTML docs browser, and have all the parties who are interested in the API sit down and discuss what's needed. After changing the JSON files based on the discussion, they use swagger-codegen to generate the client and server code. This is called interface driven development, and it's well worth a look. With its ease of use, multi-framework integration, and innovative way of involving people in the design process, Swagger is at the forefront of REST API construction tools.

The code

You can download and run a working version of this application by installing Scalatra as detailed at the start of this tutorial, doing a git clone https://github.com/futurechimp/flowershop.git, and running sbt in the top-level of the project.

About the author

Dave Hrycyszyn is Technical Director at Head London, a digital innovation agency in the UK. He is passionate about APIs and application architectures, and is a member of the team working on the Scalatra micro-framework, which has been used by LinkedIn, the BBC, the Guardian newspaper, and gov.uk. He has a keen interest in organizations and institutions, and the interplay between social structures and software.

Adding "setters" to that sentence was just a twitch of the keyboard - I guess OO has really left its mark on me! You're right, no setters are generated when defining a Scala case class as in the article, or like this:

case class Todo(id: Integer, name: String, done: Boolean)

However, it's not quite true to say that case classes are immutable by definition - it is in fact possible to make them mutable, by adding "var" to the property definitions, like this:

case class Todo(var id: Integer, var name: String, var done: Boolean)

You can then set field values (setters are in effect generated by the use of "var"):