An example of independent Rails engines

Two engines being independent means that they do not require any of each others code. Imagine the following application structure:

Let the registration component be one where a customer can sign up for the service. It has a Web UI that guides through the sign-up process at the end of which the registration is recorded. Typically this creates a user record. Bear with me and imagine that the automatic creation of user records is not possible due to legal requirements demanding the confirmation of certain aspects of the registration. As part of this manual process customer support personnel create a user record in the service component. Only then can the service be used by the customer.

As you can see, the components are not dependent on each other. The databases could stand alone. The question now is: if we need to manage both registrations and users how do we design the administration pieces? I will discuss Option 2 from my tweet first as, in my experience, that is what feels more natural to many.

Obviously, one could argue that since there are no dependencies one should simply build two applications. While it is an option, I won’t get into that here. That is a discussion of its own.

Option 2: A single, separate admin engine

The reasoning behind creating a single administration component is that there ought to be one admin experience. As we discussed, the same people need to manage data in both sides of this application. If you add an admin engine, your app will look like this:

Unless we are willing to create multiple models for the same database table, this alone is not going to work: The admin engine needs access to the data in both original engines. So, unless we are willing to duplicate the models from those engines (which is really Option 3), we have to have the admin engine depend on the other engines in order for it to have access to their models. Those additional dependencies lead to this structure:

Since the admin engine can now require both the service and the registration component, it has access to the models and an admin interface can be written. However, I don’t think that is an ideal solution as the admin engine now has access not only to the models (which it needs), but also to controllers, helpers, views, and other files from these two engines (which it does not need).

I strongly believe that if something is not needed it should not be accessible. In order to fix this, we can split the service and registration components vertically: into a web and a data layer. That change results in the following structures:

I have seen this structure evolve for applications with as few as three components and with as many as thirty. It typically leads to vertically and horizontally sliced applications. In this example we went from two components to five. If you don’t like to see that many components, you might want to consider what was Option 1 in my tweet.

Option 1: One admin interface per component

The reasoning behind creating an admin section separately for every component is that the overall structure of the application can stay simpler. It makes every component self-contained: customer-facing code and admin code live next to each other and don’t depend on anything else. Because of this, in the first iteration, the structure looks like this:

Yep, it is exactly the same as the original #cbra design. That is because the admin interfaces are simply part of the component in which the data resides. Yes, it is that easy. Or rather, it can be.

Let’s be devil’s advocate myself and ask some pesky questions:

What if the admin UIs are supposed to share Visual design?

Authentication and authorization are the same between the two admin sections: do they have to be duplicated?

It is one admin interface! Aren’t we creating a fragmented experience?

Shared visual design

If visual design is supposed to be shared between the admin interfaces, we can create a component with shared admin assets and depend on it in both application parts. That would make the application structure look like this:

Interestingly, we can make the same argument about these new dependencies as we did earlier about the dependency of the single admin engine onto the original components of the application. Parts of registration and service do not need to depend on the admin assets.

By now you probably know the trick: we separate out each admin section into its own component. From before we remember that in order for this to work we can either duplicate the models in the admin engines or extract the data parts of registration and service. The two versions of this extraction look like this:

Shared auth

How do we prevent code duplication if authentication and authorization are the same for all parts of the administration interface? The answer staying in the spirit of what we have done so far is to extract the common part. That leads to this structure.

On one of my previous projects we found an intersting alternative to this. Instead of a new component we used an initializer to add a middleware in front of Rails. This middleware enforced auth for access to admin sections. An unusual side-effect of this is that the engines themselves don’t contain anymore auth code. You will want your integration tests to be watertight to ensure that you don’t have unprotected admin features out there.

Preventing a fragmented admin experience

In any application where there are multiple admin components one is initially dealing with a fragmented admin experience. A component with common admin assets brings the pieces visually back together. But they are still independent. How does one get from one to the other?

A solution to this is to inject the entry points (URLs) of all admin components from the Rails app into all admin components. This way every admin component can render links to all other admin components. In combination with common admin assets, a user of this application will not be able to tell that they are being sent from one component to another when they are navigating through different parts of the admin interface.

Summary

Generally, there are at least three options to dealing with admin components in Component-based Rails applications:

A single admin component depending on the data layers of all necessary parts

An admin engine per component, either within or outside of the component itself

But did I really answer the question of how a Component-based Rails application should be structured when it seems to contain independent parts? Nope, I didn’t. Which solution you should go for is dependent on where you are with your application and how far you want to take things.

One very common observation when working with #cbras should have become obvious:

If you extract one component out of your application, chances are you are going to extract at least two: the component you wanted to extract and a component for the stuff that is shared between the original app and the new component.

In essence, #cbra design is about the discovery of your domain and the application you are building. Zac could have never asked his question about structure his app in his case and I could have never come up with a handful of ways to do it, had it not been for the tool that is #cbra. We would just write a ball-of-mud Rails application and never get to ask or answer any of these questions. Because we would never be able to see them. I am convinced that if you start thinking about your (Rails) applications in terms of components, you will understand your own domain better and will ultimately start writing better applications.

Footnotes

All the graphs in this post were created using cobradeps, a gem to print the dependencies within component-based Rails applications. The application skeletons that are the basis for the different structures discussed can be found on github.

I am writing a book on #cbra: Component-based Rails Applications. I would love to hear your feedback regarding the subject, what you would like to see in the book, and any other comments you might have.

9 Comments

In case it is of interest (regarding the admin auth portion of this article), I extracted this concept, based on our work together, into a Rails Engine that provides this authentication capability found here: https://github.com/bkuhlmann/auther. It has proven extremely useful in several Rails apps I’ve built since.

One thing I’d like to understand better, in regards to Rails Engine components, is how to collaborate between them. I assume exposing a JSON API between the components is the answer? This way as the components start to grow (or even need to broken into separate apps due to performance/high availability concerns), one can easily extract the Engine in the main Rails container app to a stand-alone engine that has all the plumbing for communicating, externally, to the original Rails container app? Anyway, still thinking about how best to approach this (although, have not hit it yet).

On a high level you have two options: inject the dependency or make it explicit within the component. With the former your are able to create two communication (or is “circular dependency better?”), the latter only allows unidirectional communication as you can not have explicit circular dependencies between components. With an injected dependency a component only has to have expectations about what a collaborating component does, not what it is, or where it is.

This decision informs how to implement one of the many technical possibilities for the communication.

As an example of how the implementations differ between the choices in that decision, take the most natural: a Ruby API. If implemented explicitly component A depends on component B and uses its public interface directly. When injected component A expects a collaborator to be passed in on initialization. It expects that collaborator to conform to some sort of interface. Make this bi-directional by doing the analog for the other direction. Lastly, two components can “communicate” bi-directionally by sharing state through a shared component C, say a data access layer component as described in the blog post.

A JSON API adds a level of abstraction to the above options by enforcing message serialization. That prevents accidental sharing of more information than was intended. E.g., if you serialize a message from an ActiveRecord object in component A you can’t get at its associated models in component B. Thus, a JSON API is closer to what things would look like if the app were split and turned into an SOA.

Ryan suggests an evented system. That’s yet another alternative. However, I suggest you do not start there. As beautiful as it is that an event can be sent out by one component and any component can decide to make use of it, it can introduce unnecessary complexity into the system. Here is why:

If component A and B together provide the functionality of the system and B needs to receive messages from A to make it work, then with all of the approaches above this fact would be visible through an explicit or injected dependency. In an event-based system this fact is obfuscated. A and B depend on the message bus component and expect certain messages to be exchanged. The dependency is implicit and thus hidden. You will need to add documentation to clarify what senders there are and what they send.

Hope this helps!

PS: Let me know if you still have problems with the images. They are loading for me now.

Have you come across a situation where is is necessary to restrict access to en entire component based on say ,a tenant? Think in the context of a multi-tenant application where each tenant is detected by middleware based on a subdomain. Tenant A has the lowest subscription and can only access Component 1. Tenant B has the highest subscription and can access Components 1, 2, and 3. Using Rails routing constraints you should be able to restrict access to controllers but how do you limit access to the rest of the classes in Components 2 and 3 from Tenant A?

Having tenant authorization code in the Components would do the job but it would be nice to isolate the Components completely.

@Brooke Depending on the needs of the engines it might be possible to utilize a simple event pub/sub system. Engine A does some action and Engine B can take additional action based of what Engine A has done. All Engine A does is broadcast the event and Engine B subscribes to said event. This helps keeps the coupling low. Here is a talk from Jason Clark at RailsConf 2014 by about this: http://www.confreaks.com/videos/3327-railsconf-make-an-event-of-it

June 5, 2014 at 11:41 pm

Stephan Hagemann says:

Hi Ryan,

Your idea sounds awesome!

While I have not come across exactly what you describe, I have implemented similar. It is very common to see the following roles: global admin, tenant admin, content admin, and customer. I highly recommend always checking whether it makes sense to implement the UIs for each of the roles in individual components. Contrary to your example each component knows which role should be allowed to access it, though.

Rails route constraints feel like the wrong way to implement what you are seeking. Seems to me like there is too much logic going into the decision which components are accessible to a tenant. Here’s what I think would work:

Create components in this structure. The tenant access control magic happens in access_checker, which is added as a middleware. It authorizes every system access. To this end the Rails app container injects the set of access controlled components, which can be mapped to subscriptions by the subscription component. Using this data the access_checker rejects any disallowed requests before it ever hits the components.

Voilà, no auth logic in the components!

June 8, 2014 at 12:12 am

Stephan Hagemann says:

As Brooke pointed out, some images are not loading. What’s more is that this entire post is in a “works for me” state. Working on the fixes. Stay tuned.

Thanks. I needed to go to the page on my phone before I could confirm. Fixing it now!

July 23, 2014 at 1:23 pm

Karl Dawson says:

What metrics are you using to determine the utility/benefit of refactoring to a component-based rails application? What is the impetus to refactor? Code quality metrics (number of classes, lines of code, object dependency graph)? Development effort (velocity, number of exceptions)? How do you know when a refactoring to components has been worth it? Are you measuring (and how) understandability, testability, adaptability? Is the object dependency graph the most important heuristic?

I have seen component-based software work on platforms where there are explicit programming artifacts to support a component model. I wonder how successful is an effort on a platform without a component model. Where is the evidence?

By the way… How does one broach a component-model in Ruby without being accused of failing to embrace the wonderful dynamic nature of the platform? “Interfaces? It is so yesterday!”

March 20, 2015 at 2:31 am

Stephan Hagemann says:

Karl, those are very good questions!

I have not used metrics to guide componentization and I believe it would be difficult to come up with metrics that guide this decision well. Some of the metrics you mention might be positively or negatively affected, for others the effect will be unclear.

The component dependency graph is the most visible high-level change, but I don’t consider it a metric. However, the graph can be used to assess how well the application’s intent is visible in the application structure. It is not a metric. It takes the whole development team to discuss, refine, and refactor the components over time.

What is the evidence of this working? It has worked for every single project I have applied this technique to. This has sometimes resulted in a few, large components, sometimes in many, small ones. I generally see the benefits as better communication, easier collaboration, more flexible creation, better maintenance, and improved comprehensibility.

How do I breach the subject? The major principle for all this is: it is much simpler to put things together then it is to split them apart. Component-based Rails makes the cost of separation much lower (then extracting and publishing gems), which is why it should be considered much more often. To convince someone that this is a good idea, I extract a small, simple component, fix the tests, make it testable by the build server (hopefully all this takes less than 30 minutes) and ask “was this so bad?” Worked every time so far…