Tutorial: Creating Web Services using FME Server

This article shows you how to use FME Server to drive the operations of a REST API. Here, we’ll look at common scenarios for integrating FME Server with an API gateway, and go through a more specific example.

Requirements

Introduction to REST Web Services

Web services allow you to interact with a server or another node on a web network using the simplicity of the HTTP standard. The REST concept itself is not a standard, but rather a set of guidelines that promote simple, easy-to-use web APIs.

The majority of REST APIs will accept and respond with specific variants of either JSON or XML formatted data. In the last few years, we have seen a movement away from XML towards JSON as it is more easily supported in web applications. A web service built on FME Server could support XML or JSON, but we’ll use JSON in this tutorial.

SOA, Microservices and APIs

It is becoming increasingly common to design applications and systems as a collection of services, rather than a monolithic whole. Depending on the scale of implementation, you may hear terms like service oriented architecture (SOA) or microservice architecture. While there are many details open to debate, the core idea is that there should be clearly-defined boundaries between components of a system, and they should communicate using well-defined protocols. This results in a collection of services that carry out smaller tasks, rather than a single application that fulfills all requirements. All these services may be implemented using different technologies, and are likely managed by different teams.

However, it is often still desirable to provide a unified interface (API) that synthesizes the data and operations from the different services. This is done using an API gateway - essentially another service that provides the unified interface, and communicates with all the disparate services in the background.

Borrowing from this concept, we will be using FME Server to provide a workspace-based service. We will then build a REST API using AWS API Gateway, and link it to the FME Server web service. In future iterations of this system, additional workspaces, FME Server instances, or even web services provided by entirely different products, could be added for additional functionality.

Alternatives to AWS API Gateway

We have chosen to use AWS API Gateway in this tutorial because we already use AWS for FME Cloud and other solutions. However, there are several other options, both cloud based and self-hosted (on-premises).

If you want to use a different cloud service provider, you could look into solutions such as Apigee Edge or 3scale. If you're looking for an on-premises solution, you can consider products such as Nginx API Gateway or Kong. (This is by no means an exhaustive list).

Scenario

In this tutorial, a city is starting to roll out an API for public and internal use. The Parks Board is the first department to build a web service, but other departments will be doing the same soon. The Parks Board has decided to use FME Server to implement the service, but this will actually be invisible to users of the API. Other departments may choose to use FME Server, other software providing GIS web services, or simply host some static files if their data is fairly simple and not updated frequently. This will all integrate neatly into the same API.

Required functionality

Users need to be able to carry out the following using the parks service:

Retrieve all of the parks

Retrieve a specific park

Search parks by name

Add a new park

Update a park

Delete a park

The operations should properly be restricted to specific users. Querying the service is available to members of the public, but only internal users are permitted to modify the data.

The operations should be accessible through a RESTful web service, which can be expanded in the future to include services from other municipal departments.

Implementation

Designing the API

Each of the required actions on a resource needs to be mapped to an HTTP action on a URL endpoint in the API:

Action

Endpoint

Retrieve all of the parks

GET /parks

Query parks dataset

GET /parks?key=value

Retrieve a specific park

GET /parks/<id>

Add a new park

POST /parks

Update a park

PUT /parks/<id>

Delete a park

DELETE /parks/<id>

Technical decisions

We’ll be using FME Server to build the parks web service, since FME makes it easy to properly format the data from the Parks Board internal database. However, the job-based workflow of FME Server isn’t ideally suited to building a REST API. Instead, we are going to use a specialized API Gateway. Specifically, we’ll be using aptly-named Amazon AWS API Gateway.

Since we are distributing GIS data through the web, a natural choice for request and response types is GeoJSON.

Setting up a simple API call, end-to-end

It’s good practice to first implement a simple example end-to-end to demonstrate that your systems are communicating properly. In this case, we will implement the /parks query, which just involves a translation from MapInfo TAB to GeoJSON. This will require configuring data streaming, security, and the API Gateway.

Authoring the workspace

The workspace is very simple for this first API call. We’ll just be using the Parks.tab file from the FME sample data package. For your convenience, there is a workspace template available.

Note that future methods you create can also use the token you specified in the stage variable, so if the token changes, you will be able to just set at the stage level. You could do something similar for the server URL if you have separate development and production FME Servers.

Next Steps

Using this example, we can implement further API calls using FME Server workspaces. Some ideas to work with:

Published parameters are easily set using query string parameters

POST data can be read using a Text File reader, or a GeoJSON reader if the data is of that format

API gateways can often do sophisticated mapping of data. For example, you could use the id of parks to add a link to the URL for that park's data in the summary /parks call. You would want to avoid adding this logic to a workspace, as it requires the web service to have awareness of the API layer.