Software architecture is hard to define as the term “software architecture” is used to describe many facets of software structure, behavior, design, activity and documentation. The concept of software architecture is also relevant at different levels of a system as the components of one system become a reusable platform for the next system. Ultimately the emergent behavior caused by layers of architectural dependencies and reuse can be difficult to predict

We use the definition above to indicate that architecture is a simplification of the system, used to communicate the nature of that system and guide its development. However, architecture is more than just a diagram, it is grounded in technical context describing technology stacks, standards adopted and environmental factors.

Architecture is concerned with explaining the structure, behavior, integration and aesthetics of a system (or system of systems). It explains common ways of doing things (patterns and mechanisms), how Non-Functional and Functional Requirements are met, how systems are put together and provide a common technical direction for teams working with them. Architecture provides a shape and look and feel to the internals of a system that are the foundation for the ultimate external behavior.

The idea of an “Architect” role with such a broad focus is analogous to that of a structural architect concerned with buildings. Indeed, in 25 BCE Vitruvius (a well known Roman writer and architect who inspired Leonardo Da Vinci, hence the Vitruvian Man) described an architect as:

The ideal architect should be a man of letters, a mathematician, familiar with historical studies, a diligent of philosophy, acquainted with music, not ignorant of medicine, learned in the responses of jurisconsults, familiar with astronomy and astronomical calculations.

Vitruvius is famous for stating that architecture must be solid, useful, beautiful (De architectura). The same three qualities relate to software architecture. Despite architecture being a fine balance between a subtle science and an exact art we must realize it is only useful if it is aligned to requirements and becomes executable.

Good architecture speeds you up in building some software. Bad architecture is lots of diagrams and documents.

Architecture, not documentation

In line with one of the founding Principles of Holistic Software Architecture we value working software in the form of quality releases from short development cycles over comprehensive documentation, business analysis and enterprise architecture documentation. Choosing how much architectural work is necessary up-front and especially how much architectural documentation is necessary is difficult.

Traditional document focused development methods promoted large up-front effort to detail, in document or model form, the architecture and system design prior to development. As well as the inherent late risk miitigation issues in waterfall processes this could often cause “analysis paralysis” where architecture work was seen as an endless diagram drawing exercise. Because there was no way to test ideas before designs were complete designs often ended up being layer upon layer of speculation.

Agile methods focus on working software over documents and designs, however this doesn't mean that no documents or designs should be produced. It is almost always useful to have some description of an architecture before development even if it's just a statement that the team's usual architecture will be used as otherwise team members won't even know which language to start writing in.

Good architecture addresses how we'll resolve the major technical risks, communicates between the team the overall structure of the solution and works out how our solution will meet the functional and non-functional requirements. Knowing when we've worked out enough architecture so that we've reduced the complexity of the problem into manageable sizes for the team is a difficult challenge.

Doing too much architectural analysis or elaboration, either through abstract design and modelling or practical spiking (writing small amounts of throwaway code to demonstrate the feasibility of a technical approach or architectural idea) will slow down a development project and increase the risk of wasted work. Value is only achieved once working systems are in the hands of the customers/users.

Alternatively, not doing enough architectural analysis can lead to significant architectural refactoring during a projects lifetime, as key requirements can't be met based on the current architecture, again leading to wasted work and slowing down the project.

Architectural work, and corresponding documentation can be more implicit when complexity is low, rapid feedback is working and there are strong, high trust, relationships. Just as with Requirements, architectural analysis and documentation need to be more explicit when work is more complicated, where risk is higher and where there are more cross-team or distributed communication concerns. High complexity architecture, in contrast, is often subject to a lot of change. Up front work on structure and behavior is prone to extensive rework, and although we do not recommend completely emergent architecture, in these situations managing for emergent structure and behavior by architecting for change becomes the best way to succeed.

When doing up front architecture, especially in the form of documents and models, we need to be careful to consider architectural work in the context of the team's definition of done. Typical levels of done don't normally include "analysed", "architected" or "designed". Although these terms might be meaningful to describe internal team development states they do not constitute working software and are only a step on the way to creating value.

Architectural models and documentation do not constitute progress. Good architecture is working software, and that’s progress.

Good Architecture and Design

Having a common understanding of architecture, regardless of the format of that understanding (documents, models, sketches, whiteboards, implicit team knowledge), is described as "Intentional Architecture".

In Holistic Software Development we consider "good" architecture and design to have the following characteristics:

Well described elements: modular elements have simple expressive meaningful names enabling readers to quickly understand their purpose, similar to code cleanliness we should strive for design cleanliness

Runs and passes all defined tests or acceptance criteria

Lightweight documentation

Architecture in Holistic Software Development

Architecture is represented at three levels in HSD in the Big Picture - however these views are not meant to be separate and translations of each other, the most effective holistic architectural description is one that exists in the same format (model, sketch, documents) as part of a cohesive whole.

Enterprise Architecture applies architecture principles and practices to guide organizations through the business, information, process, and technology changes necessary to execute their strategies

System Architecture applies architecture principles and practices to a particular software system focusing on structure and behavior to address functional and non-functional requirements (usability, reliability, performance and scalability etc.).

Architectural Profile

Architectural Profiling is a method of understanding the relative complexity of different areas of concern for an architecture.

Architecture addresses a number of concerns and so a useful early approach is to consider the profile of the architecture in terms of the relative complexity of each of these areas. This helps to give a feel for the shape of the requirements, architecture and overall solution. We can also identify possible areas in which we can reuse existing components, frameworks or maybe standard packages of requirements, components and tests. Architectural profiles may be useful at any level of architecture (Enterprise, Solution and System) and are useful to understand areas of complexity in the requirements space, especially for understanding the relative complexity of non-functionals.

We consider a number of “dimensions” representing the various concerns of architecture. Starting with the FURPS scale mentioned in Requirements, but also considering other important dimensions such as “security” we extend the model in whatever meaningful way is necessary. The dimensions used aren't set in stone, we use the dimensions that are meaningful for the organization and type of project.

Here's an example profile of an application that does some significant data processing, needs to do it reasonably quickly but not excessively so and has got to do some pretty visualizations of the data. Other than that it's fairly straight forward. Initially we'll discuss the non-functional aspects.

The x-axis here is close to the standard FURPS scale with a couple of extra dimensions - as mentioned above we might customize the dimensions to the context of the project or organization.

The y-axis ranges from no complexity to high complexity but it is deliberately not labeled (not something we'd normally recommend!) so we can focus on the relative complexity of these dimensions of the requirements, quality, architecture and therefore solution. Complexity here is a subjective opinion, adding false accuracy of scale is not worthwhile.

The height of one of these bars helps us shape the architectural approach that the work needs, including how implicit or explicit the elaboration and documentation needs to be.

For example, let’s take the security dimension and consider it from simple, complicated and complex.

If it's empty/simple we know that we don’t need users, audit logs etc. Maybe we just need a username for personalization but we could pick that up from the Operating System or session state.

In most organizations, and for most developers, these are very common requirements which have been implemented many times. There is unlikely to be a need to do detailed requirements documentation, design or even significant testing as the quality risk is likely to be low. Hopefully we will be able to reuse corporate authentication and authorization mechanisms or standard mechanisms such as LDAP integration or Oauth and so not have to implement any of this stuff directly. Of course just because quality risks are likely to be low doesn't mean we can assume there are no bugs, a minimal level of testing might still be required by our Definition of Done.

If the bar for security is at the very top, then we are in a high complexity security context. In this case we may be dealing with an unstable cyber security situation such as needing to operate securely in a hostile environment where adversaries are actively trying to compromise our software or operational effectiveness. These situations are not resolved through up-front design, but through architecting for change, experimentation and learning.

As well as the standard non-functional "URPS+" dimensions we also often look at as standard is "Data Processing" which covers the volume and shape of data a system might need to deal with. Simple entity management is typically fairly low, whereas running significant algorithms across that data will push the bar up. Large datasets start to bring in some elasticity concerns, "Big Data" and massively parallelized processing will push the bar up further.

For lower levels of complexity, dealing with simple entity creation, editing and deletion we do not recommend elaborating textual requirements or CRUD stories/simplistic scenarios, similarly we do not recommend creating a lot of designdiagrams that describe CRUD operations for each entity. However a simple persistency mechanism and data model are frequently worthwhile.

Another dimension we frequently use for user facing systems is "Reporting and Visualization" means graphical rendering of data or processing. At its simplest level this dimension can be simple GUI feedback but it can range to interactive touch displays, augmented reality, VR etc. As the bar increases, the requirement for User Experience (UX) practices increases.

The Functional Dimension

The first dimension we title "Differentiating Functionality" which represents the functional requirements which make the product or product family unique. If this bar is low then the product is likely to be a commodity offering rather than a market differentiating product, in which case there should be a strong strategic reason to buy not build. A significant non-functional difference between the proposed product and existing alternatives may be a perfectly valid reason, but we recommend making that explicitly visible.

If, during product evolution, this bar lowers significantly then the product should be considered for retirement or replacement with a commodity solution. Sometimes market disruption will out-move a business in which case cutting losses, and redirecting onto more differentiating business value, is a sensible business decision.

Other than these cases, the functional dimension is typically the source of business value and so is often prioritized by customers and users above the other dimensions. Using an architectural profile is a useful way of balancing the requirements and development work to consider the whole problem. We tend to use a different color for the functional bar to help draw out this distinction and ensure a realistic balance of concerns.

Technical risks, and likely quality risks, will be hiding in any dimension with a high complexity and so will be fertile ground for finding fringe cases. Complex areas are excellent candidates for early iterative development as their implementation can help to de-risk project, programmes and portfolios.

Architectural Synthesis

Architectural Synthesis is the creative problem solving activity that turns a set of requirements or direction into an initial candidate architecture.

The “magic sauce” in software design, architectural synthesis is the activity that shapes initial candidate architecture. Based on requirements (which will have little detail but hopefully represent scope), and investigation into non-functional needs an Architect or team will come up with a number of options for how to meet the needs, or solve the problem. Architectural synthesis is dependant on the complexity of the problem being addresed:

For simple pieces of work, architectural synthesis is implicit as the answer is already obvious to everyone.

For complicated pieces of work, investigation into where areas of complication are (using an Architectural Profile) backed up with experience and experimentation/spiking tend to lead pretty quickly to a candidate architecture.

For complex work we recommend a series of experiments/spikes to investigate areas of complexity or try ideas that might work towards delivering business value.

Architectural synthesis is a creative process, especially in anything other than simple work. It is often the point where the level of complexity will be recognized. Investigations and spiking will often change understanding of complexity and risks, either uncovering them or addressing them.

We do not recommend following a standard process for synthesis since it’s essentially creative idea generation, and it should not be rushed. In our experience critical thinking and logical analysis can help architectural synthesis.

Architectural Spiking

Architectural Spiking is running a small technical experiment, building working software to prove or disprove feasibility or a specific hypothesis. Spikes are throw away code they are not integrated into released Products.

Spiking is intended to investigate an area, reduce complexity, mitigate a risk or otherwise prove/disprove a theory. By diving deep into the solution space (a vertical “spike”) we can understand whether our early architectural ideas are likely to work. Spikes are particularly useful during Architectural Synthesis.

The purpose of a spike is to test a theory, not to create part of a working product. So although they are built as working software, they are throw-away code, usually ignoring all architectural, style and even good coding guidelines in favor of expediency. Spiking is an excellent form of risk mitigation, and a great way to reduce complexity if possible.

We recommend that Spikes are formed by articulating a theory and a simple test or two. Although we describe Spikes as “throw-away code” we don’t actually throw them away. We don’t integrate them into our real code but they are useful to keep, alongside their specification/tests for people to refer to later or find out how something was done.

Here’s a simplistic example of a real spike for a system that was looking to access a SQL Server database using JavaScript. If this spike failed the team had other ideas, but this was a feasibility test they needed to answer.

This was the entire documentation for a Spike. The spike involved doing some web searching, running some command lines and writing around 5 lines of code for each theory, which proved both cases. The team simply ticked the tests and saved the following produced assets:

Spike documentation

Links to online guidance

Install scripts

Produced code

The team were then able to reduce a risk related to technical integration (although had to create some new risks around the security implications of accessing a database using local Javascript – but that’s a different story).

Architectural Overview

An Architectural Overview is a diagram or sketch that describes key architectural elements including: the shape of the system, architectural pattern usage, primary interfaces between subsystems and external systems combined with target deployment platform information, critical data schema and important service/component/class/module structure.

Just as we describe a mix of concerns and processes in the Big Picture we recommend creating a big picture view of an architecture that communicates its key concepts. We don't recommend slavishly following a visual language such as UML, they are generally too restrictive. However, using common de facto symbology where possible is useful and standards like UML are a good source of common symbology. The purpose of an architectural overview is to explain the important structural, logical and physical characteristics of an architecture in a single diagram. In some user-centric pieces of work a User Experience Design may complement an Architectural Overview.

Key elements to include are:

a high level view of the structural pattern of the architecture (e.g. layers or pipes and filters etc.)

a breakdown of the most important contents of each major element (e.g. layer)

indications of target deployment infrastructure

key user/actor interaction

primary interfaces between subsystems and external systems

critical data schema

This is an anonymized overview diagram of a system:

This diagram is a lot richer than a traditional UML architecture diagram because it dispenses with many of the rules of UML, while keeping some of its useful symbology, and incorporates rich graphics to further explain elements. We haven't even used color in this example, which can be used to significantly add clarity if done well.

Horizontally it describes a layered architecture (including target deployment infrastructure and client platforms) as well as user and system actors via a GUI and exposed APIs. Vertically it describes the dependency pattern between architectural layers, the main communication pathways through the architecture and an eagle eye view of its structural composition. We can see that all communication between layers is via exposed interfaces and that the framework layer is composed of two primary subsystems which also have exposed interfaces. The Data layer is managed by a proxy which delegates to a persistency controller and there's an outline view of the data model.

The purpose of a diagram such as this is to provide a starting point to explain the architecture to someone else. We recommend using an architectural overview as the primary way of describing an architecture in a team and to interested stakeholders. For a very simple system there may not be much more needed in terms of architectural documentation however for most systems a number of mechanisms that describe the systems behavior will also be useful.

Integration Scenarios, provide an excellent way to describe an end-to-end thread through an architecture especially in a system-of-systems context. The graphical and textual techniques used in Integration Scenarios can also be used to describe architectural mechanisms.

Enterprise Architecture

Enterprise Architecture applies architecture principles and practices to guide organizations through the business, information, process, and technology changes necessary to execute their strategies

Enterprise architecture is focused on making technical decisions that shape the work of an organisation via constraining or directing Solution Architectures and System Architectures. Enterprise Architecture does not deliver a product but instead provides a context for architectural work within an organization that ensures product architectures are aligned to Strategic Goals.

Enterprise Architecture is best defined as a set of architectural standards, decisions and common services. Having an understanding of Enterprise Architecture is necessary to help focus the Portfolio and reach the Portfolio Scoped Milestone. We have seen many organizations struggle to size their portfolios in an architectural vacuum. Estimations, guesses at the best of times, are particularly hard outside of meaningful architectural context.

The Portfolio and Enterprise Architecture have a bit of a circular relationship in that Enterprise Architecture is necessary to qualify the scope of the Portfolio, but Portfolio Requests may change the Enterprise Architecture, and the evolution of various systems and solutions over time also change the Enterprise Architecture. This circular relationship must be accepted, any Enterprise Architecture that doesn’t change is most likely incorrect.

Although standards and organization wide decisions can aid communication and alignment within an organization they can also inhibit creativity if there are too many rules to follow, and they are too formally applied. In our experience standards chosen by non-technical Business Leaders often have detrimental effects on development efficiency. We recommend that architectural decisions are evidence based and are made by people who are active practitioners. Architectural decisions must be made by those technically skilled enough to understand the advantages and disadvantages. Business Leaders are not expected to know everything and must not feel threatened by technical specialists. A strong Leader delegates decisions to those best suited to make them. If an organization does not trust its technical experts to make technical decisions it is unlikely to succeed and will undoubtedly diminish motivation amongst the technical community.

Enterprise Architecture often has a poor reputation in many organizations as being something that consultants have sold, resulting in many models in expensive tools and extensive documentation that no one uses. Canonical data models that involve many effort-months of meetings to agree that aren’t useful in software design are a poor substitute for real Enterprise Architecture.

Good Enterprise Architecture makes it easier to make new products by providing common starting points, technical direction, useful infrastructure and tooling as well as common organization-wide mechanisms such as authentication/logging etc.

Importantly Enterprise Architecture must not constrain innovative/inventive work, in terms of the Commoditization Scale although such work should be informed by EA to avoid too much technical divergence.

Many organizations struggle to “pull through” work from their research areas into specialist/mainstream development because of a lack of common technology stacks, languages etc. Where a specific technology is used for good reasons in innovation work, such decisions should not be constrained. However, if there is no technical reason to choose one technology over another then the one that is easiest for the rest of the organizations should be chosen.

Enterprise Architectural Standards

Standards will be those adopted by the organization to guide architectural and development work, standards need to be explained in terms of the types of development they are suitable for. Strongly engineering standard architectural patterns and mechanisms may be desirable for complicated work (including many systems of systems architectures) but is unlikely to be appropriate for innovative research and high complexity work. Innovative work needs to experiment with new ways of doing things and high complexity work needs to architect for change, not stable structures.

Enterprise Architectural Decisions

Architectural decisions influence, or even direct, technical decision making in Solution and System Architectures. Not making these decisions at an Enterprise level can lead to technology diversification making long term support, licensing and maintenance expensive or even impossible. Many organizations will rightly choose not to use homogenous technology stacks throughout their enterprise, however, explicitly choosing which technologies should be used where is still necessary.

Enterprise Architectural Services

Enterprise Architectural Services are common corporate services that Solution and System Architectures can make use of. Establishing enterprise reuse can be a misleading goal leading to large enterprise frameworks which become a heavy technical debt on any new endeavor but highly atomic services, with low dependencies, that are truly generic in the organization may be useful.

Enterprise Architecture Services take standards or decisions and actually build them. Providing working starting points, technology stacks or working functionality to new pieces of work to speed up their development. EA services should be automatically metricated where possible to ensure that they remain useful.

Examples of typical Enterprise Architecture services include:

Common authentication and authorization services

Common data models, formats or containers

Common platform configurations for development, hosting or operational use

Common services like these are best described as Architectural Mechanisms, where they don't exist as part of Enterprise Architecture they may often need to be created as part of a Solution Architecture or even a System Architecture. However, if left to System Architecture there is a high probability that each piece of work will duplicate mechanisms, wasting effort.

A Word of Warning

Enterprise Architecture is aimed at enhancing the efficiency of development in an organization but, all too often, becomes a source of impedance to development teams and organizational business agility. This can be especially true in terms of Enterprise Architecture stifling technical motivation, research and innovative experimentation. Enterprise Architecture needs to be as lightweight as possible, decided on by active technical practitioners and open to continuous challenge and improvement. We recommend quarterly (if not continuously) reviewing Enterprise Architecture for appropriateness, efficiency and alignment to strategy.

We strongly recommend that anyone involved in making architectural decisions is able to spend time working on real projects, building real software. Technical skills decay, and as the industry relentlessly progresses the expert coder of today quickly becomes the dinosaur of tomorrow. Mastery, as discussed in the People view, requires practice.

We recommend that technical leaders sit in on retrospectives and architectural show and tell sessions on other people’s projects to keep well informed of the breadth of technical diversity in an organization. Exposing technical decisions help everyone improve. There is no shame in not knowing the answer, but there is shame in pretending to know and deceiving an organization.

As we mentioned in the Architecture View, architecture must be solid, useful and beautiful. In Enterprise Architecture terms that means it must be:

Solid – Enterprise Architecture needs to be stable and yet current. It must be reliable and logically defensible.

Useful – If it’s not useful, retire it immediately. It must be lightweight and simple to use.

Beautiful – Good architecture is elegant and adds beauty to a solution, without being overly clever. Architecture is there to simplify the complex.

Solution Architecture is normally focused on the system-of-systems nature of a programme of work - the challenge being to ensure that separately developed systems (maybe including out-sourced or COTS systems) join up correctly to deliver business value. If the work in question is not a system of systems and is instead a single system (with or without some external integration) then no matter how large it is there is no Solution Architecture, just System Architecture. A Solution is a combination of Products that, on their own, deliver Business Value, but when used in combination deliver more significant Business Value.

A Solution is a combination of Products that, on their own, deliver Business Value, but when used in combination deliver more significant Business Value. Be careful not to confuse a Product with constituent components (even if they are branded differently) as a Solution. Components don’t deliver Business Value outside of the context of a Product whereas Products outside of a Solution still deliver Business Value. Sometimes Solutions will evolve out of opportunities only made available from constituent Products. In our experience, Solution Architecture is one of the most poorly understood fields.

When delivering a complex COTS Implementation, the primary "product" may be Business Change, in which case a Solution Architecture may be considered in the context of related Adoption & Business Change and other communication activities that must work together to achieve Business Value. Although a Solution Architecture is normally a technical systems of systems concern.

To derive a technical Solution Architecture, we initially shape an architecture using an Architectural Profile derived from the early scoping requirements (consisting of Features and Integration Scenarios perhaps on a Programme Backlog). Architectural Profiling is also used at system (or product) level and so constituent system architecture profiles will align broadly with the profile used at Solution Architecture level. Specific systems might be targeted at de-risking part of a Solution Architecture profile and so it is not unusual for lower level systems to have profiles that do not exactly match.

Architectural Profiling is a method of understanding the relative complexity of different areas of concern for an architecture.

Architecture addresses a number of concerns and so a useful early approach is to consider the profile of the architecture in terms of the relative complexity of each of these areas. This helps to give a feel for the shape of the requirements, architecture and overall solution. We can also identify possible areas in which we can reuse existing components, frameworks or maybe standard packages of requirements, components and tests. Architectural profiles may be useful at any level of architecture (Enterprise, Solution and System) and are useful to understand areas of complexity in the requirements space, especially for understanding the relative complexity of non-functionals.

Solution Architecture exists within the context of an Enterprise Architecture and shapes lower level System Architectures. Various modeling languages can be useful in this space such as BPMN, workflows based on SIPOC and some elements of UML or simple activity workflow diagrams. As always in Holistic Software Development we do not recommend slavishly following standards, instead we recommend using elements from appropriate toolkits to communicate as effectively as possible. In our experience a rich picture, not too focused on symbology, is more beneficial.

Solution Architecture is not simply system architecture scaled up, or Enterprise architecture scaled down. It is often concerned with higher complexity than system architectures due to more integration, more moving parts, more concurrency and more distribution. It is also more requirement driven than Enterprise Architecture, focusing on producing Business Value and so is more specific than common technology choices and standards.

Understanding, and communicating, a valid Solution Architecture is a key part of achieving the Product Family De-Risked Milestone.

Solution Mechanisms

Examples of how a system-of-systems communicates and achieves end-to-end value

Solution Mechanisms focus on how a system-of-systems will work together and so is concerned with communication, integration, cross-product User Experience and distribution. Standard non-functional dimensions, normally addressed at system level, may also need special attention at Solution level. For example, the performance characteristics of a single system may not still meet requirements when it’s operation is federated across geographical areas with output combined by a different system before presentation.

Splitting a Solution Architecture into a number of Solution Mechanisms enables iterative and incremental development of Solution Mechanisms, although they are often cross-cutting in terms of the functional requirements.

One of the biggest “common problems” in a Solution is integration. Common choice of integration technologies is useful in ensuring a common development paradigm within a product family. Products within a Product Family may employ different system architectures, and even be written in multiple languages, but will need to communicate with each other and make use of common services. As a result, they need to be able to talk the same language in terms of integration. We recommend the use of standard technologies with low (or no) dependencies such as JSON over http rather than complicated orchestration engines and proprietary binary formats.

Sometimes these integration standards will come from Enterprise Architecture, but if they do not, and would add value in a Solution they should be created. In fact, proof of usefulness in a Solution may prompt adoption at Enterprise level depending on the Enterprise diversity.

Solution Architectures will often describe architectural mechanisms and features related to concerns such as:

Common data models, formats or containers

Messaging technologies

Composite transactional mechanisms

Large scale architectural patterns (e.g. SOA, hub and spoke, etc.)

Distribution mechanisms and target deployment topologies

Common logging or audit mechanisms

Common authentication

Elastic architecture principles to enable cloud deployment

Similarly, the chosen technology stack, if not constrained by Enterprise Architecture, should be described as part of the Solution Architecture.

Solution Mechanisms, like most architectural concerns are best described in terms of structure and behavior.

Structure

Solution Architecture is best described in terms of both structure and behavior. Structural views will tend to be at a higher level than those in System Architecture as they will be less concerned with system internals and more concerned with system dependencies, interfaces and deployment. In many ways a Solution Architecture diagram is essentially a zoomed out Architectural Overview diagram.

We recommend largely dispensing with UML or other standard symbology and rules when describing solution architecture, instead focusing on communicating as clearly as possible the relationships between users, systems and interfaces. We can often use a single structural view for many mechanisms. Don’t create lots of diagrams if you don’t need to.

In the anonymized example diagram below we're identifying different types of users that interact with different Products and primary interfaces between layers. In this case the Solution was a news aggregation platform, the lower layer was textual and the upper level was creative User Experience based on a number of platforms. There's also some interfacing to external systems indicated at the bottom of the stack. We’ve actually used some UML symbology but we’re using more color and layering than the rules allow.

We recommend annotating structural diagrams like the above with deployment information in terms of technology stacks, elasticity requirements and other platform information. This specific example was simply https everywhere, platform information and elasticity was already well understood based on Enterprise Architecture.

Behavior

We can describe the behavior of functional Solution Mechanisms in terms of the requirements it implements, in HSD, these take the form of Integration Scenarios which often provide a graphical view of behavior. Annotating Integration Scenario diagrams, or simply linking elements on them to more technical diagrams works well.

For non-functional Solution Mechanism we recommend creating a simple workflow based diagram, that represents the flow of events, data or control as appropriate.

The point of behavior diagrams at this level is to explain what each contributing Product or System is meant to do, and how they’re expected to communicate with each other. A rich picture that resonates with business process, like a graphical Integration Scenario diagram, is a good option.

In either case it can sometimes be useful to overlay the most important of these on top of a structural diagram. The most important purpose of a Solution Architecture is to explain end to end flows through the system.

Integration Scenarios describe an end-to-end workflow across a Product or Product Family resulting in Business Value for a Product Customer in flexible format.

Integration scenarios are threads across a number of other requirements such as Features and User Stories that don’t deliver Business Value on their own. They are ideally suited to describing the scope of Releases and providing the input for integration testing, necessary to achieve higher levels of acceptance.

Level of Detail

Many programmes will be tempted to elaborate Solution Architecture to a significant level of detail, and drawing the line where Solution Architecture ends and System Architecture begins can be difficult. Overlap between Solution and System architecture can cause conflict and confusion. It is not the jobof Solution architecture to connect every bit of system level information, or to design every part of every system.

Solution Architecture needs to describe:

Overall structure of the system of systems

The implementation of key integration scenarios

Architectural standards to be followed for the programme (such as technology choices, frameworks, data schemas etc. where not provided by Enterprise Architecture)

Solution Architecture should not describe any elements internal to subordinate systems and so should be as lightweight as possible. Spending more effort on describing testable Integration Scenarios, and automating integration, is worth more than architectural modeling at a Solution level.

Executable Solution Mechanisms

The best Solution Mechanism are built in terms of executable code so that contributing teams can simply pick them up and integrate to them. Solution Mechanisms need to be proven through use, and changed in line with feedback as soon as possible to mitigate risk.

System Architecture

System Architecture applies architecture principles and practices to a particular software system focusing on structure and behavior to address functional and non-functional requirements (usability, reliability, performance and scalability etc.).

System architecture describes the overall structure, behavior and aesthetics of a Product. It provides and communicates technical direction to the team on how to structure code and work with each other. It mitigates the significant technical risks by providing mechanisms that offer:

Common solutions to common problems within the Product

Ways of addressing primary Product technical Risks

Ways of reducing significant areas of Product complexity

Examples of how the system “hangs together and works”

Architecture must be:

Solid – System architecture needs to provide a common “scaffolding” for developers to add their bits of a Product to. It must be reliable and logically defensible.

Useful – If it’s not useful, retire it immediately. System Architecture isn’t just diagrams, it’s executable code. If it doesn’t make a Product easier to develop, it’s not doing its job.

Beautiful – Good architecture is elegant and adds beauty to a solution, without being overly clever. Architecture is there to simplify the complex.

Like Solution and Enterprise architecture, System architecture is unlikely to be “right first time”. It must be proven through delivery of working software, refined by the feedback of tests, the development team and users.

As described in the Architecture View, how much architectural elaboration and description is necessary is not easy to define. Various factors may move a project to need more or less documentation but the emphasis should always be on the minimum necessary documentation. We recommend understanding System Architecture in terms of an architectural profile, overview sketch(es) and lightweight mechanisms. Sometimes other elements might be useful such as User Experience Designs, integration scenario workflows, data models etc. but we focus on a core minimal set of products in HSD.

Architectural Profile

Architectural Profiling is a method of understanding the relative complexity of different areas of concern for an architecture.

Architecture addresses a number of concerns and so a useful early approach is to consider the profile of the architecture in terms of the relative complexity of each of these areas. This helps to give a feel for the shape of the requirements, architecture and overall solution. We can also identify possible areas in which we can reuse existing components, frameworks or maybe standard packages of requirements, components and tests. Architectural profiles may be useful at any level of architecture (Enterprise, Solution and System) and are useful to understand areas of complexity in the requirements space, especially for understanding the relative complexity of non-functionals.

Architectural Overview

An Architectural Overview is a diagram or sketch that describes key architectural elements including: the shape of the system, architectural pattern usage, primary interfaces between subsystems and external systems combined with target deployment platform information, critical data schema and important service/component/class/module structure.

Just as we describe a mix of concerns and processes in the Big Picture we recommend creating a big picture view of an architecture that communicates its key concepts. We don't recommend slavishly following a visual language such as UML, they are generally too restrictive. However, using common de facto symbology where possible is useful and standards like UML are a good source of common symbology. The purpose of an architectural overview is to explain the important structural, logical and physical characteristics of an architecture in a single diagram. In some user-centric pieces of work a User Experience Design may complement an Architectural Overview.

Architectural Mechanisms

Architectural Mechanisms are small parts of an architecture that address a specific important concern, provide a common way of doing something or are a good example of how the architecture behaves and is structured. Architectural Mechanisms are described in terms of structure and behavior.

Mechanisms explain how the architecture does something. Mechanisms exist within the context of an architecture which provides overall structure so we recommend mechanisms are tightly bound with an Architectural Overview and are only used to refine the architectural description, where necessary, as indicated by the Architectural Profile. Architectural Mechanisms take different forms in Enterprise, Solution and System Architecture.

Collectively this information provides a lightweight set of architectural descriptions that can be used to communicate an architecture and speed up Product development.

High Complexity Architecture

High complexity architecture involves applying architecture practices and principles to business and technical problems involving many interdependent integrating parts that interact in a number of ways resulting in unpredictable, emergent outcomes.

We’ve been lucky enough to work on some of the most complex architectures in the world. Most organizations work with complicated problems, not high complexity issues. However, to innovate or invent ahead of competitors or adversaries, organizations often have to work in higher complexity areas. By working on very new ideas, there is a lack of knowledge and tried and tested techniques; this widens the cone of uncertainty significantly. Where there are a large number of interacting parts and a large number of interactions complexity can be very high. For more on complexity in general, see Simple, Complicated and Complex.

Part of the job of architecture is to reduce complexity. That’s ok for simple and complicated systems using the techniques mentioned above but it’s a little trickier for very high complexity systems.

Very high complexity architectures are those in which a number, or all, of the following conditions are true:

There is extensive integration between independent components, technologies and/or products

The work is highly speculative and unpredictable, we expect to fail fast and often

There isn’t a large amount of domain knowledge or experience in the field

One or more of the dimensions of an architectural profile are “Very high”

Estimates are extremely uncertain

Risks are very high

Extremely large scale (of data, change and distribution)

There is a complex logical relationship between inputs and emergent properties or behaviors

When working in high complexity systems, the intent is often more important than the detail. Work is often highly speculative as people are inventing new techniques that may or may not work. Empirical feedback becomes more useful than specification, and due to the experimental “sensing” nature of high complexity work details will change significantly.

We have seen many pieces of work attacking high complexity problems that have failed, not due to the inherent complexity and risk, but because they used practices and ways of working suited to complicated problems to try and attack them.

Failure is always likely in high complexity work but it’s made even more likely by trying to “control” complexity. Practices that have worked for previous work or seem sensible become highly dysfunctional. Examples include:

forward planning - We don’t yet know how we’re going to solve the problem if we even can. Plans will change quicker than they can be written down. “Probing and sensing” is more appropriate.

detailing requirements - The intent is important, not the detail of how. Our technical solutions are likely to change significantly meaning that details will change. A sub-optimal solution might be the only cost-effective option significantly affecting scope.

analysis and design - Decomposing problems into manageable chunks isn’t the right answer when the complexity is in the number of “chunks” and their interactions. Instead we need to manage emergent properties and create architectural experiments (spikes) to prove or disprove our ideas.

user-centric design - Users are unlikely to have resolved the high complexity issues and may not even understand the problem fully. Giving the users what they want, often a good idea, is often the opposite of a high complexity solution. Users seek simplicity, and although this is a good idea in terms of interaction with high complexity architecture, designing interaction doesn’t help solve the problem. Since scope and technology are likely to change, UX detail is best left until the solution is more stable.

estimation - By the nature of the complexity estimations will all be extremely uncertain. Numbers may be significant orders of magnitude out. Instead organizations are better off funding experimentation in a number of timeboxes to see how much complexity can be produced in those timeboxes through spiking and experiments.

When working with high complexity architecture, standard Enterprise Architecture may need to be abandoned to deal with emergent behavior. However, the cost of not using Enterprise Architecture should not be considered as an inhibitor to high complexity work if the Business Value is important enough or if the work is related to a strategic direction to beat competitors.

In Solution Architecture for high complexity architectures, a common dysfunction is to try and connect all of the architectural information and system architectures. We’ve seen organizations create massive over-complicated models that try and resolve integration complexity by documenting it all. Well-meaning, but ineffective and wasteful. This approach is counter-productive because the complexity comes not from the number of items, but the dynamism of their interactions and unpredictable aggregate behavior. These large models are, at best, a snapshot view of complexity at a single point in time but they don’t help anyone solve the problems.

Instead, when working with high complexity Solution and System Architectures we recommend architecting for change, not solution requirements. By that we mean that since requirements, scope and emergent behavior are so likely to change the only thing we really know the architecture needs to support is change. Therefore, focus on the principle, technology and mechanisms that enable change such as:

Since the behavior and properties of high complexity systems are emergent we recommend creating a “pull” towards Business Value through rapid, preferably continuous, integration and deployment. Creating measures that reflect desired outcomes (not intermediary stages or logical decomposition) and test every change in terms of moving towards or away from “good” emergent behavior.

At the extreme end of this high complexity scale, using evolutionary techniques to generate possible solutions and test for emergent properties can be useful to speed up iteration cycles or artificial intelligence networks can be used evolve network weighting to recognize successful progress.

Specialists vs. Cross-Functional Developers

As software development practices have evolved people have specialized into specific roles such as:

Developer

Designer

Architect

Tester

Requirements Engineer

Etc.

This is further compounded by processes introducing roles, some process frameworks have 10s of roles! The problem is that as people specialize and go deeper into an area of specialization they increasingly abstract away from actually building software. Sometimes “professionalization” is the enemy of progress.

Chief Software Architect

Software is increasingly important to everyone, it’s everywhere. It’s in our phones, runs our cars, our cities, our healthcare, entertainment and utilities. There are few businesses without a software element and many that are critically software dependent. Many organizations are now beginning to understand that the business of doing software is difficult, in fact it’s complex. It needs C-suite level direction and coverage.

Being good at software isn’t a matter of hiring the good coders, buying the right workflow management tool, using the right version control system, hiring the right management consultants or using the correct technology. These things all have a part to play but they’re supporting parts to the most important factor: culture.

Software Development is a team activity and the team is bigger than we used to think it was (a few developers, testers, direct customers, managers, analysts etc.). A good set of development practices at team level isn’t enough to be effective because software development is a broad business enabling activity. Software development touches all parts of an organization (development teams, human resources, procurement, security, business change, legal, financial, business services, operations, support etc.).

As organizations evolve to make the maximum use of technology they are making increasingly complex software products, often through diverse technology stacks, using a variety of in-, near-, and out- sourcing partners. Delivering systems-of-systems across teams-of-teams is not simple and so organizations are now looking to embrace the importance of software, technology and ways of working to their businesses at board level.

Whether a “Chief of Software”, a “Chief Software Officer“, a “Chief Digital Officer” or “Chief Architect” the role tends to include:

Leading and encouraging a positive software development culture:

Where delivery of business value is the primary measure of success and decisions are based on business value

Where failing fast is a cause for celebration not a career ending issue

Architecture provides a context for making technical decisions in normal development, makes technical decisions on commoditized platform/software/operations choices but does not constrain innovation and research (architecture has a different impact in different parts of a hybrid dynamic model)

Architecture enables control of whole-lifecycle costs by smoothing the flow of work from innovation and research into mainstream development and support

Ensuring that teams can use the tools and environments they want and need to use, not those that had the best sales pitch to a non-technical board balanced by creating a cohesive community that isn’t unnecessarily fragmented by every team needing to set up different tools environments

Tool chains take the pain out of development, testing, build and deployment – automated wherever possible

Development environments are appropriate for actually developing, building, testing and deploying

Ensuring, and embodying, the principles of agility, devops, and agility-at-scale at the organizational level – promoting the evolution of working practices, operating models and ways of working while avoiding endless management fads

Building and championing the needs of the development community, attending to their needs

Ensuring that portfolio selection takes into account technical issues, shaping it in line with architecture and in turn using business priorities to shape architectural choices