Notes from Erin Swenson-Healey

Preface

To quote Rúnar Bjarnason:

One of the great features of modern programming languages is structural pattern matching on algebraic data types. Once you’ve used this feature, you don’t ever want to program without it. You will find this in languages like Haskell and Scala.

I couldn’t agree more myself. That said, I spend most of my time writing programs with languages that don’t have first-class support for algebraic data types (ADTs). So what’s a programmer to do? Continue reading …

Our example application implements a function createEmployee that is used to create an employee from a personId.

To create an employee, our system loads some data from our database, validates that data, and then performs an insert. Some of our functions are written in continuation-passing style (they accept a callback) and some are written in a direct style (they return values and/or throw exceptions). We’d like to compose these functions in such a way that they succeed or fail as a single unit – that any error in any segment of the sequence will cause subsequent steps to be skipped with a failure – but with some of our validations happening synchronously and some asynchronously, this can be difficult to do.

Partial application refers to the practice of filling in a functions parameters with arguments, deferring others to be provided at a later time. JavaScript libraries like Underscore facilitate partial function application – but the API isn’t for everyone. I, for one, feel icky sprinkling calls to _.partial and _.bind throughout my application.

Curried functions (found in Haskell and supported by Scala, among others) can be partially-applied with little ceremony; no special call format is required. In this blog post, I’ll demonstrate an approach to currying JavaScript functions at the time of their definition in a way that enables partial function application without introducing lots of annoying parens and nested function expressions.

In this post, I’ll demonstrate the usage of higher-order functions outside of a collection-processing context – with the ultimate goal of reducing boilerplate code in your real-world applications (as filter did above). We’ll start with partial function application (facilitated by Underscore’s _.partial) and move on to writing our own higher-order functions.

I describe building an application in a functional style as the act of composing many smaller, context-free functions into pipelines of data transformations which map from system inputs to outputs.

Compojure (a Clojure web application library similar to Bottle and Sinatra) represents this input as a simple, immutable hash map which is transformed through a pipeline of middleware functions to a new hash map representing an HTTP response. The code of the resulting application resembles one in which we’re applying a mapping function to values in a static collection rather than ones in which we’re working with HTTP requests. By favoring expressions over statements, immutable data and functions over mutable objects and methods, our web application will share many of the positive characteristics of its static sequence-mapping cousins – in terms of simplicity (we can think about each transformation step independently from others) and robustness (our functions are small; their behavior easy to reason about and test).

Your Rails application has become a monolith. Your test suite takes 30 minutes to run, your models grow to several hundred (or thousand!) lines, and feature development slows to a crawl. New members of your team must read through massive amounts of code before they can feel confident making changes. You’ve watched Uncle Bob’s Architecture the Lost Years, Matt Wynne’s Hexagonal Rails and read Martin Fowler’s Microservices and are convinced that the time has come to start breaking things up into smaller, simpler, faster components – but you’re unsure of where to begin.

In this article, I will demonstrate an approach to breaking a monolithic-style Rails application apart into microservices using plain Ruby classes and a simple RPC framework – all deployed to Heroku. I’ll show you how to work through this migration incrementally; you’ll be able to decompose things down as far as makes sense for your codebase. As a special bonus, I’ll show you how you can use Barrister RPC to alleviate much of the pain associated with documenting and creating API clients for the components of your now-distributed system. I’m skipping the advocacy; if you’re here it’s because you already love yourself some microservices but don’t know how to implement them yourself.

Connecting the components of a distributed systems is no easy feat. Should I use REST? An RPC system like Apache Thrift? Protocol Buffers? SOAP? How do I document these components’ APIs? What’s the best way to write client bindings? The Internet offers a wide array of possible solutions, all of which I’ve found to be either incompatible with my needs (to, at minimum, work in a web browser) or heavy (XML sucks).

For the last few years, I’ve enjoyed great success using a an RPC framework called Barrister. Barrister RPC provides a variety of language bindings (Python, Ruby, browser-based JavaScript, Node.js, Go, Java, and PHP), which means that I have no problems connecting my Backbone application to a Ruby web service, or Node.js image processor to Spring MVC. Barrister separates API design from transport – so I can experiment with HTTP or SMTP or Redis queues… all without changing my API. Remote calls are encoded as JSON, so I can debug using Firefox and all the other tools I’m already familiar with. Imagine SOAP, redesigned for developers who build modern, dynamic, cloud-deployed web apps, don’t work at banks and have no stomach for enterprise service busses or XML. Continue reading …

As someone who consumes many web service APIs – both internal and external – the recent trend in the web development community to forgo writing API documentation has got me worried. While I do understand that publishing and maintaining documentation can be a hassle, APIs with no documentation can quickly become a bottleneck in a team’s workflow, inhibiting the team’s ability to scale, outsource, or distribute themselves geographically. In this article, I’ll walk you through a partially-fictionalized version of a disastrous experience I had working on distributed system whose APIs were largely undocumented.

While attempting to deploy a system of interconnected microservices to Heroku recently, I discovered that processes running in dynos in the same application cannot talk to each other via HTTP. I had originally planned on each microservice implementing a “REST” API – but this wasn’t going to be an option if I wanted to stick with Heroku. Much head-scratching ensued.

The solution, it turns out, is to communicate between microservices through a centralized message broker – in my case, a Redis database (but I’ll show you how do it with RabbitMQ as well, free of charge). The design of each microservice API has been decoupled from HTTP entirely: Client/server communication is achieved by enqueueing JSON-RPC 2.0-encoded messages in a list, with BRPOP and return-queues used to emulate HTTP request/response semantics. The Redis database serves as a load balancer of sorts, enabling easy horizontal scaling of each microservice (in Heroku dyno) on an as-needed basis. Redis will ensure that a message is dequeued by only a single consumer, so you can spin up a lot of dynos without worrying that they’ll clobber each other’s work. It’s pretty sa-weet.

Carbon Five is a full service software consultancy that helps startups and established organizations design, build, and ship awesome products. If you have a project you’d like us to take a look at, or are interested in joining our team, please let us know.