Why the n-Layer approach is bad for us all

The "n-layer" design used by developers usually includes 3 horizontal layers, one that houses delivery mechanisms such as user interfaces like websites or applications and services. This layer depends on the next layer in the stack which is the work flow layer, commonly called the “business layer”. This layer houses domain entities and the work flows the entities need to go through based on a set of given business requirements or “business rules”. The final layer is the persistence layer which is usually referred to as the “data layer”. The data layer houses repositories that provide crud based operations, or proxies that communicate with remote services.

The benefit of this design is that it organizes code into “layers” that focus on specific concerns. It’s much easier to organize and locate items and when used (and maintained) properly instead of having the logic of each concern strewn about in each class.

The problem is that developers are lazy by nature and like electricity, will take the path of least resistance. So things get unruly very quickly.

One of the problems with this n-layer design is that it is basically just a few containers that have a bunch of stuff in them. There is rarely any organization beyond that. Determining what classes compose a system is difficult and since “code reuse” is a best practice that developers strive to follow, existing systems and new systems alike become all intertwined and come to have dependencies on parts that make up other systems which makes it hard to change both systems. The astute developer will recognize these dependencies and abstract them away into separate classes which add to the complexity.

Eventually we have logic in bits and pieces all over the place with no logical organization. Reading the code or using profiling tools is the only way to make sense of anything (if it's even possible).

What's more, there is no encapsulation of logic. For example, any controller, class or work flow can spin up a user repository and start doing things with the data. Is this so bad? Yes, it is! Any business rules towards working with users can completely be ignored and this can and will lead to side effects.

Scenario (yes, real world)

Let’s take a look at an example of the retrieval of a data structure that represents the configuration of a customer. Developer A writes the bits needed to retrieve and hydrate the model and puts it nicely in the data layer. He then consumes this model from his workflow in the business layer. But, the retrieval of the data is expensive. Being a smart developer, he decides to cache it in memory. Now in his workflow, if the data is cached, don’t bother with the data layer. Perfect.

Now comes Developer B who needs to work with the same customer data. "Oh, cool" he says as he discovers that the data layer already has some bits in there to do the work of getting the data. So now he continues on with his workflow, calling out to the data layer to get the data when needed. Not realizing that the retrieval process is expensive, he releases his changes and off it goes into QA. Now the complaints about performance come in and after much discussion, Developer A realizes that Developer B hasn't added any caching. So off he goes to update his workflow with the caching logic. Meanwhile, the data structure is being saved in memory twice and the knowledge of this issue fades away until Developer C needs to work with the data. The story continues to loop.

When developer D comes along and finally says, "Hey, why don’t we move the caching logic into the data layer so that we don’t run into this problem again?". Developer D is met with much resistance from arguments such as, "The retrieval method shouldn't be responsible for caching; it should only do one thing! It’s the single responsibility principal, hello?!".

While it may be arguably true that the particular method that retrieves the data should only do one thing, the customer data retrieval system should handle the caching. This leads to my final point about the n-layer design, it causes developers to think in terms of classes and methods and that leads to complexity which eventually drives up development time and cost, reduces maintainability and eventually leads to a re-write.

Being unable to think about a problem from a higher level will never lead to a well-built solution.

A new approach

While there are many different approaches to resolve each individual down side that comes with the n-layer design, sometimes it's just best to not only think outside of the box, but rethink the box completely!

Having read Simon Brown’s, "Software Architecture for Developers", I was inspired to rethink my approach to solution design. I had previously started practicing problem decomposition which given any problem, break it down into small and manageable pieces. But, I still ran into problems as described with the n-layer design above.

This is where Simon’s work came in. His approach to software architecture is basically decomposition starting with the highest level "thing" possible. In software, this is what you’re trying to build. He calls this process, "C4" (Context, Containers, Components, Classes). I won’t explain it here because I simply won’t do it any justice, but at each level of decomposition, you examine how each piece interacts with the others (or not). When you’re done breaking everything down, you have a clear idea of what needs to be built and why.

I highly recommend you take a look at the book to get a fresh perspective on things.

Design by Systems

I’m certainly not going to take any credit for what I’m about to explain, but I will say that I’ve taken inspiration from Simon’s work and a few other resources and I’ve started practicing what I call design by systems.
Uncle Bob Martin states that when you look at an accounting system, it should scream ACCOUNTING SYSTEM! It should not scream WEB SYSTEM!

With the n-layer design, developers usually start off thinking about the delivery or the data and the interaction between the two. Take input and push it through to the database. Whatever has to come in between just does.

The stuff in between is what’s important, not that MVC or SQL Server is used!

Basics

The goal is to end up with a set of independent and autonomous components. Each component serves a very specific purpose and is self contained. By self contained, I mean everything the component needs to do its work is contained within the component including data access, service proxies and so on. The component exposes only a single public interface in which you interact with it. I call this the "service provider". Components contain the business logic and business rules expected of the specific task the component addresses.

From the components, we compose systems. Systems are consumers of the components and are in charge of the work flows, invoking and interacting with each component as stated in the requirements. Systems contain logic only as applicable to the work flow.

Systems are consumed by whatever needs to consume them (usually the delivery mechanism such as a UI). Each system is a façade that is in charge of solving a particular problem (a feature).

To determine which components need to be built, the solution needs to be decomposed iteratively and is done so by examining the requirements, user stories and use cases.

Benefits

Designing a solution this way breaks down the problem into manageable, cohesive chunks and provides a perspective that allows developers to understand the system for what it is instead of what it isn't (a web site, mobile application, etc). Developers can start thinking about the implementation from a much better starting point.

Implementing a system in this way provides a clear and logical organization, encapsulation of functionality, business logic is centralized and each component can be developed independently of the rest of the system.

Case Study – RSVP Application

Let’s walk through a recent request that I received. The requirements were pretty simple, I needed to build an RSVP application for a local user group.

Requirements
- Needs to collect First name, Last name and Email address
- Email addresses need to be verified to make sure they exists and are not fake
- Do not let users RSVP more than once per event
- The RSVP page needs to show information about the upcoming event
- After submitting, show them a thank you page
- User should be able to download an ICS (iCal) file for the event
- Subscribe the user to the MailChimp mailing list

I won’t go into breaking the requirements down into user stories and use cases in this article, but let’s start to decompose them and do some discovery. First, let’s get the nouns

Registration (part of RSVP)

Event

Page

Calendar

Mailing list

Now let’s extract the features

Event registration

See event information

Download ICS file

And now the business rules

Emails must be verified to see if it exists

RSVP’s must have first name, last name and email address

RSVPs must not be submitted more than once per event

RSVP’s must only be allowed for the next upcoming event

At this point, we start taking the features and evaluating the workflow. These workflows will become our systems.

Event registration

Collect First name, Last name, Email address

Verify email address is valid

Fail if not

Submit RSVP

Check if first, last or email is invalid

Fail if so

Check if RSVP for event already exists.

Fail if so

Persist RSVP

Register email address to mailing list

Show thank you page

This is a pretty good break down of the work flow. But, if you notice, #1 and #5 are related to the delivery mechanism which we can infer from the nouns in the requirements, will be a website. We can now break down this work flow into two concerns, delivery and processing:

Event Registration (UI)

Collect First name, Last name, Email address

Submit RSVP

On success show thank you page

On failure show error message

Submit RSVP (System)

Verify email address is valid

Fail if not

Submit RSVP

Check if first, last or email is invalid

Fail if so

Check if RSVP for event already exists.

Fail if so

Persist RSVP

Register email address to mailing list

Now we have the basis for our RSVP system. Now we can break this work flow to pull out the action items. These action items will become components. Comparing the actions in this list to our list of nouns, we can start to see the type of components we need to build.

Email Verification Component

RSVP Component

Mailing List Integration Component

Each of these components will hold the business logic and enforce the business rules as outlined in the requirements.

From this point we have some really great information that we can use to start building our individual components or at least setup the interfaces so that the rest of the solution can be built and tested independently.

Notice we gave almost no attention to the UI and we did not once mention a database. We can put off those details until we actually come to a point where we need to deal with them. Defer them as long as possible.

Follow the same process for the other features until every feature and requirement can be completed via a work flow and every requirement has a home. When everything is satisfied, the solution is complete enough to start implementation.

Implementation

If we were to start implementing the RSVP system and component, it might look like the following.

RSVPSystem – The work flow façade that invokes the components required to complete the work flow

Components – A logical grouping of components

RSVP – A logical grouping for all component related classes

RSVPService – The public interface for the RSVP component

Models – The data structures needed by the RSVP component

Only the RSVPSystem, RSVPService and the input/output models would be public. The rest would be internal and private as needed. This library project can be referenced by the web project and the system can be consumed.

Enforcement

Depending on what environment and stack you’re dealing with, there are tools that will automatically enforce architectural constrains such as Visual Studio’s Layer Diagram or through Aspects such as the ones provided with PostSharp AOP framework.

But those options are not always viable. Since any developer can break the encapsulation by making classes public and using them directly, using a good naming and grouping convention would make it clearly obvious that something isn’t being used correctly.

Shared functionality

You will eventually come across functionality in two or more components that is exactly the same or very similar. When it comes to very similar functionality, treat it as completely different functionality. If it is in fact related, or the same functionality, then maybe a new component needs to be extracted. Evaluate this on a scenario by scenario basis.
If the functionality is global in nature, then that functionality is a good candidate to be moved into a reusable library instead of a component.

Common Models

Components will have input and output which may come in as individual parameters or as input/output models. These models must be independent and specific to the component. Shared/common models break the autonomous nature of the component.
Additionally, shared and common models usually end up being changed over time to fit many other needs and that convolutes the models and makes it hard to determine what the purpose is.

Components depending on other components

Since the goal here is to build autonomous components, no component should rely on another component. If this becomes an issue, the dependency should be evaluated and moved into a system and the results of the dependency should be required as input on the dependent component.

For example, If Component A depends on Component B because Component B is in charge of getting data that Component A needs, then the system that is invoking the components needs to handle the process of getting data from B and giving it to A. Component A should require as input, the data that Component B provides.

If Component A needs to modify the data that Component B provides or deals with, then the responsibility of Component A should be re-evaluated. If data needs to be changed, then this should be invoked by the System.

Components can be depended upon, but must not depend upon anything else with the exception of any 3rd party libraries that can be deployed along with the component.

System using or depending on other Systems

Yes, this is absolutely fine. If work flows get too complex, or certain bits of a work flow need to be shared, then make it possible by breaking up the work flow.

Conclusion

While this proposed solution may end up with its own down-sides, it does offer an alternative to the n-layer design with far more benefits to developers both current and future, the code base and whoever is paying to maintain and expand the product.

In practice, what I propose in this article is mostly a change in the way developers think about and approach solutions (what is being built) and a suggestion as to how the actual code should be structured to support that thought process and make it clear that you're not just looking at a "web system", but a system that was purpose built.