Scrubbing Get Params in Phoenix

Elixir • Phoenix Framework

Phoenix applications can be a mixture of pure REST API endpoints, as well as HTML endpoints rendering content dynamically using views, templates, and partials. However, some convenience methods that exist when creating HTML endpoints are missing when working on an API. One such example is input parameter validation and type coercion.

Introduction

This article is written for Elixir 1.2.5, Phoenix 1.1.4, and Ecto 1.1.7. Please note that future versions of Elixir, Phoenix or Ecto may change in behaviour and invalidate what’s written here. The sample code to this article is available on Github.

What is Phoenix?

Phoenix is a modern web framework, written in the Elixir programming language, which in turn executes on the Erlang Virtual Machine (BEAM). Phoenix could probably best described as an evolution of Ruby on Rails, taking many of the lessons learned from Rails and improving on them.

Phoenix for Rest APIs

Unlike Rails 5’s upcoming API module, Phoenix applications can be a mixture of pure REST API endpoints, as well as HTML endpoints rendering content dynamically using views, templates, and partials. However, when working with Phoenix, some convenience methods that exist when creating HTML endpoints are missing when working on an API.

One such example is input parameter validation and type coercion, which we will take a closer look at in this article.

Preparation

For the rest of this article we will be working with a small demo API that we are going to set up now using the mix phoenix.new command. If you haven’t installed the Hex Package Manager yet, run the command mix local.hex.

First, we create a new Phoenix application using the mix command. I am using the sqlite database here, the default is postgresql. Either is fine, for our sample app sqlite is sufficient.

$ mix ecto.create
...
The database for SampleApi.Repo has been created.

Let’s open up sample_api/web/router.ex and add a new endpoint /hello that will call HelloController.index/2.

defmoduleSampleApi.RouterdouseSampleApi.Web,:routerpipeline:browserdoplug:accepts,["html"]plug:fetch_sessionplug:fetch_flashplug:protect_from_forgeryplug:put_secure_browser_headersendpipeline:apidoplug:accepts,["json"]endscope"/",SampleApidopipe_through:browser# Use the default browser stackget"/",PageController,:indexget"/hello",HelloController,:indexendend

Finally we create HelloController under sample_api/web/controllers/hello_controller.ex as follows:

$ http localhost:4001/hello\?name=John\&age=44
HTTP/1.1 200 OK
I see, John is 44 years old!

Using Ecto Models for Parameter Validation and Type Coercion

The naive implementation shown above gives us a very basic mechanism to ensure that required parameters are present in the GET request. From there on out, we would now have to implement our own logic for checking whether a specified parameter adheres to the specification.

For instance, if we specified in our API contract that age is an integer type with allowed valued between 1 and 120, we might end up adding additional validation code like this:

defmoduleSampleApi.HelloControllerdouseSampleApi.Web,:controllerdefindex(conn,%{"name"=>name,"age"=>age}=params)doifvalid_age?(age)doconn|>text("I see, #{name} is #{age} years old!")elseconn|>put_status(400)|>text("Error, age parameter must be and integer between 1 and 120!")endenddefindex(conn,_params)doconn|>put_status(400)|>text("Error, wrong parameters supplied!")enddefvalid_age?(age)doif(is_integer(age))do(age>=1&&age<=120)elsecase(Integer.parse(age))do:error->false{val,_}->valid_age?(val)endendendend

This kind of code will quickly become very convoluted and un-maintainable! Luckily, there is a much better way! We can use Ecto Models and the validation logic they offer. For this purpose we will create a new Ecto.Model called HelloParams. However, we won’t be using Ecto to persist HelloParams to any database! The main piece we want from the model is the concept of achangeset!

Notice, how we are matching the changeset cs against %{:valid => true} which is the main logic that tells us whether all of our required parameters were present, and passed the validation and type checks. I’ve used a more detailed match in the example above, to conveniently extract the name and age values into variables that I can immediately use in the String interpolation, but other methods of working with the params is absolutely possible, they are available as cs.params.

Bonus: Error Details

A good API does not only render the proper HTTP status codes, but also gives appropriate error messages and error details. As an extra benefit in using Ecto.Model for our parameter validation and coercion, we can now add error details to our endpoint quite conveniently by parsing the changeset.errors construct:

This assumes that some post "/some/:user" route maps to the SampleApi.SomeController.new/2 function, posting a user form to the endpoint. The scrub_params/2 function will recursively walk through the user struct, replacing all Key-value pairs that look like {k, ""} with {k, nil} (see GitHub for more details). Unfortunately, we can’t use scrub_params/2 with our API endpoint above, where parameters are given as GET parameters.

Scrubber Plug

Let’s fix this by adding our own implementation of Param Scrubbing for ordinary GET params. First, we will create a new file sample_api/lib/sample_api_helpers.ex as follows:

If you read the scrub_params/2 code on GitHub linked above, you will see that our implementation of scrub_get_params/2 is actually quite similar to the original. Our function scrub_get_params/2 is defined as a function plug, thus accepting a Plug.Conn.t and a Keyword List, and returning a Plug.Conn.t. We walk over all GET params with Enum.reduce/3, using an empty map as the intial accumulator and passing in the private scrub/2 function. That function has two matches, {k, ""} and {k,v}. In the first match, we add {k, nil} to the accumulator, in the second match, we add add {k, v} as it were.

We can now plug scrub_get_params/2 into our SampleApi.HelloController: