Constructing a Front-end Ecosystem with Ember CLI

duplicating the interoperability that SOA has on our backend

Roughly three years ago at Iora Health I began pitching the engineering group on splitting our monolithic application infrastructure down into a SOA platform. There's a lot of data and problem domain in your medical record. It's not uncommon for startups to be focusing one of the areas of the domain we function in (biometric markers, aggregated lab results, secure messaging, etc etc) and our CEO has long thrown down the gauntlet that we are expected to meet our patients where they are. That means we need access to our data in a vast array of different use cases.

It was pretty easy for our engineering group to agree to an SOA approach to our platform. Actually moving the needle forward on adoption was an entirely different matter. You see, our tech group has this pesky desire to ship valuable software and values this desire over perfect architecture. As the resident architectural astronaut, I had my work cut out for me.

Pushing the needle forward in SOA

Basically, I needed to reduce the cost of developing greenfield software with SOA to be in the same ballpark to that of building existing features onto the monolith. I've developed a sort of hierarchy of needs that needs to be in place to push implementors into SOA over monolithic app development:

Easy build with our preferred technical stack - If I can't get Capybara, Rspec, sass, on a new project with no fuss -- I'm not going to fuss with building a new app.

Authentication / Authorization - everything in the medical application is an authenticated request. If I have to reimplement auth on every app, I will not build new apps.

Access to data existing services - If I'm building an application that combines data from two services into the UI, I better be able to get them both quickly.

Access to common design language - Staff expects to be able to start typing a patients name and have a typeahead style autocomplete, therefor my PM expects it when I submit a feature for acceptance. I do not expect to copy + paste the UI components around.

In the Rails world, we were able to accomplish most/all of this pretty easily (though some tools have been improved upon recently).

We have a Rails template and a CLI application that can instantly give us a new Rails app with our preferred technical stack. Changes to our preferred technical stack happen as PRs against our template. It allows us to move forward with technology in a reasonably controlled way. We are also free to experiment with different components -- engineers just have to understand that they should need to answer questions when maintenance questions arise.

We nearly immediately built a auth/auth Rails Engine when our identity service was built. We authenticate all applications via OAuth. The engine is even installed via our template application, and the glue code is in place by the time we've created a new application. All a developer has to do is set key/secret environment variables and they are good to go. We have also started using RabbitMQ for service to service communication (much of our service to service is still RESTful APIs though) and we have a library to facility that easily as well.

Access to existing services was slightly tricky until Heroku blogged about their API Toolchain which greatly streamlined how things work. We use prmd to generate a JSON Schema document of a RESTful resource, which in turn is used to generate both human and machine readable documentation of our API, then we use Heroics to automatically generate a Ruby client. The automatically generated API client is probably the turning point here -- building a HTTP client isn't particularly painful, it was just a boring/trivial hoop to jump through when building an API and a client application.

We still haven't really figured out the common design language portion of the Rails stack. Our designer experimented with engines to accomplish the task but wasn't particularly excited. In part, it might have been trying to support the monolith and our new applications from the same engine. I can't particularly speak to the difficulties.

The problem: we rarely build boring CRUD applications

Everything is great for SOA development so long as we stop at Rails as the end of our tech stack in regards to application development. Unfortunately, that is increasingly not the case at our company. What I want is an ecosystem of front-end development so that I can build interesting applications that can mash-up our services quickly and easily, and not be directly dependent on aforementioned technical stack working on your development machine to do so.

The case study in why I have up until now failed the client side

Recently, one of our designers built a beautiful functioning prototype for a potential application in Ember-CLI. His application used nothing but dummy data, which is pretty common for a prototype but I felt I failed him as a platform developer. Here's why:

Each model that he created in Ember had an OAuth backed, CORS accessible service that could serve up real data. Instead of working with the equivalent of lorem ipsum, doctors playing with the prototype could've been playing directly with our staging environment. He didn't even consider prototyping with our services because:

If he even had an access token, he'd have to go find other Ember apps and copy/paste the models/adapters/serializers to get them running. App A probably has 33% of the cool functions that'd make his life easier, while App B has the other 67%. He'd build a conglomerate of model bizness without the tests -- probably a sad day.

Clearly, we need to start making the client similar in easy of use to the server.

The solution: build ember addons to make front-end development awesome

As it turns out, fixing this sort of problem is really fucking easy. Developing addons for Ember-CLI isn't too hard and it took me roughly one day to extract out an authentication workflow into a library. Since talking about the addons we're building Boston Ember, we've even figured out some of our existing stumbling blocks (accessing the included application config from the addon). This library isn't in any way a generic authentication framework -- I'm unsure if I even want to head in that direction. It just solves my immediate problem (and might be a blueprint to help solve yours!).

Up next I'll be building a library that takes my JSON schema and translates it into Ember models, so that instead of writing these files once, machines write them once on my behalf. In the meantime, we're adding them as we need them to a common addon so that new Ember-CLI applications don't have to reinvent the wheel when it comes to model access. I think the coolest part of this is that if/when we open our APIs up to third parties (perhaps even directly to our patients) we've essentially given them immediate access to mashing up our services. I'd love to see what people could do :).

I feel most stoked about building a shared design language with an Ember addon. The Ember component system translates extremely well building repeatable and extensible UI widgets and I think we'll reduce a significant amount of widget copy + paste that we've seen in our monolithic Backbone.js app. It's way to early to even talk about patterns in development of this -- we're just starting to extract our first few simple widgets. I hope that in a year we've got some things figured out though.

Wrapping up

I think the important part of this post is mostly that you should be thinking about lowering the bar to developing new applications at your company. Take what Rails/Ember does (building the easy on the back/front end to trivial) and make this apply to your platform. Make it so easy to contribute to a SOA architecture that the alternative seems crazy and counterproductive.