When we discuss ‘scale’ at Twilio, usually the conversation is about a system’s ability to handle growing amounts of work and how we might expand infrastructure to satisfy increasing demand. But, there is much more to 'scale' than adding servers and supporting more users. It's not just our technical infrastructure that needs to scale; it's also our organization, our team, our codebase, our environments, and our processes. There are a lot of pieces that need to fit together. Much of how we scale depends on the organization itself, the choices we make and the coordination we have as a team.

We’ve been growing faster than ever over the last year at Twilio. Teams have doubled in size. Codebases have grown in terms of complexity and lines of code. The first step in managing this is to break apart the codebase into smaller, reusable components. The challenge of breaking apart the system into loosely coupled services is that it becomes difficult to keep code across the whole engineering team DRY, consistent, and high quality.

For us on the API team, we stepped back and asked, "What are all the qualities we want of a modern API framework?" At the top of the list were flexibility & maintainability, performance & asynchronous processing, and better tools & libraries. Then we looked at where our existing framework stood with respect to our goals and compared it to other alternatives. Over the next few months of building prototypes and iterating on the design we ended up with Flask-RESTful.

Flask-RESTful is a simple, easy to use Flask extension that helps you construct APIs. It gives you a clean interface for easily parsing arguments to your resources, formatting/serializing your output, and organizing your routing. It abstracts away a lot of the HTTP boilerplate code for you. Flask-RESTful does this while staying out of the way of your business logic. It doesn’t prescribe a particular ORM, and it doesn’t have any dependencies other than Flask itself. The goal of Flask-RESTful is to give you a simple and extensible starting point for building your API just the way you want it.

What a Flask-RESTful API looks like

Here’s an example API for managing a todo list. It has two resources:

Todo - an instance of a todo item. Supports GET by id, and DELETE

TodoList - a list of Todo instances. Supports POST to create new items, and GET to list them

fromflaskimportFlaskfromflask.extimportrestfulfromflask.ext.restfulimport(reqparse,abort,fields,marshal_with,marshal)app=Flask(__name__)api=restful.Api(app)TODOS=[{'task':'build an API'},{'task':'?????','otherField':'secret data!',},{'task':'profit!'},]#only output the ‘task’ fieldfields={'task':fields.String}# Todo# show a single todo item and lets you delete themclassTodo(restful.Resource):@marshal_with(fields)defget(self,todo_id):ifnot(len(TODOS)>todo_id>0)orTODOS[todo_id]isNone:abort(404,message="Todo {} doesn't exist".format(todo_id))returnTODOS[todo_id]defdelete(self,todo_id):ifnot(len(TODOS)>todo_id>0):abort(404,message="Todo {} doesn't exist".format(todo_id))TODOS[todo_id]=Nonereturn"",204# TodoList# shows a list of all todos, and lets you POST to add new tasksparser=reqparse.RequestParser()parser.add_argument('task',type=str)classTodoList(restful.Resource):@marshal_with(fields)defget(self):returnTODOSdefpost(self):args=parser.parse_args()task={'task':args['task']}TODOS.append(task)returnmarshal(task,fields),201## Actually setup the Api resource routing hereapi.add_resource(TodoList,'/todos')api.add_resource(Todo,'/todos/<int:todo_id>')if__name__=='__main__':app.run(debug=True)

That’s it! You’ll notice in this example some of the design choices we’ve made for Flask-RESTful.

We map the HTTP methods GET/POST/PUT/DELETE to method names on the Resouce.

You can return any object/dict/list from your method handler and it will get serialized in the output.

You use fields to filter and format data on those objects before rendering to the user.

This allows you to keep your existing business logic & model code intact and not have to worry about exposing the internals of your system to external API clients. It also allows you to decouple the processing of requests from the rendering of output.