Much has been made recently of the imperative to fully transition our businesses into the modern digital world. It now hardly needs to be said at this point. There is even some encouraging news for traditional enterprises: The latest data from Forrester shows that companies are indeed at long last making digital transformation a top priority, with 74% of executives saying that they currently have a strategy to get there.

Yet “having a digital strategy” can also mean just about anything, depending upon who you ask. At this point however, there are basically two main forks in the road to digital for most organizations:

There is the ‘bolt-on’ strategy, which typically means adding a few new digital channels to existing touchpoints — typically social and mobile — and maybe creating an associated but minor sideline business with some digital revenue.

Then there is the ‘digital transformation’ approach to digital. It’s a full-on, meaningful reconception of the business, often using a startup or incubator model, with the intent to re-imagine a digital native organization with all that it entails, from new business models, culture shifts, remodeling of the structure and processes of the business, and rethinking of the very foundations of the enterprise across the full spectrum of digital possibility.

Unfortunately, the latter approach also has many of the characteristics that corporate leadership tends to avoid: a) The big bang initiative which has a high likelihood of failure, b) cross-silo involvement, meaning it will encounter numerous bureaucratic and political obstacles, and b) the likelihood of of success being dependent on securing rarefied talent with scarce expertise that crosses the domain of the business, the world of strategic emerging technology, and next-generation IT.

The reality is that both forks have real risks: The bolt-on approach is too little and too incremental to have the requisite strategic impact, though it’s certainly a valid interim approach (as long as it’s not the only one.) On the other hand, the full digital transformation model entails a major investment and commitment across the organization with a seemingly all-to-uncertain outcome.

Yet, the latest data tells us unequivocally that the act of doing nothing — or just too little — is also sure to fail. The march of technology is wiping out traditional companies faster than ever before, and the pace is only accelerating.

Another way of putting it is that the CEOs, CIOs, COOs, and CMOs — the four roles most directly responsible for guiding this transformation — will secure rewards for their organizations that are directly commensurate with their commitment to drive broad digital adaptation and change. For the data is unambiguous: Those that don’t fully align with the state of the marketplace will be absorbed by those that do.

Many industries even today are resistant to digital. Source: Forrester

Thinking Like a Digital Business

What can organizations do if they are serious about their responsibilities to lead the business into the future? Several clear options are emerging:

Seek out digital change. Avoid having it imposed. Successful next-generation enterprises — see the start of my 2014 NGE target here — won’t wait until adopting new digital channels, tech, and business models are unavoidable. They will pro-actively seek them out, learn them early on, experiment, and be ready to grow when they mature. Even fast-followers will be at risk if they don’t avidly seek out new opportunities. Dave Gray has previously pointed out research from Shell showing that the longest-lived companies are pro-active seekers and explorers of new markets. What’s more, digital change is now nearly continuous, and the organizations must establish long-term processes that tap into and pull these changes into a new “digital metabolism” that makes incorporating strategic innovation both routine and sustainable. Organizations that only respond to change will always be several steps behind those that are change-seekers. Finally, be bold it seeking out these changes. As the latest McKinsey report on digital transition notes, the winners will “be unreasonably aspirational.”

Cultivate capabilities to support multiple operating models. As John Kotter pointed out this week, there is now “an inseparable partnership between hierarchy and network.” We will have two and probably more major operating models in our organizations going forward, at least the legacy and the digital. We must operate and exploit each of these systems to their fullest — and together — to produce competitive and effective results today. To get there, successful leaders will strategically enable the shift of hierarchy into much more network-centric models, while cultivating the strengths of both simultaneously. Since most organizations currently have significantly underdeveloped networked operating models, this will require special investment and integration into the digital transformation process.

This is not to say that businesses have not already extensively digitized. They have, but as Sameer Patel recently pointed out, they generally have not transformed. The single biggest obstacle to successful digital transformation is a broad shift to a ‘native digital’ mindset that will consistently inform broad action. I’ve come to believe that traditional companies can make this transition, but only if they decentralize tech innovation that is coupled with a supportive new network operating model, while carefully controlling downside (typically security, data control, etc.).

So, while bolting-on a digital mindset may lead to some short-term successes, it will certainly stunt the future of your organization. Instead, employ internal and external networks to create a naturally-supportive environment where digital change is far more scalable, emergent, adaptive, and continuous.

Many of you know that over the last several years I’ve tried to make the case that most organizations are currently falling behind the advancing pace of technological change. That business is so centered around technology today is the reason why addressing this has become a top competitive issue. Becoming better adapted to tech change is even tied to the medium-term survival of many organization as I recently explored in my look at digital transformation.

But to say that technology alone is what is disrupting traditional businesses would be inaccurate. We ourselves have changed — have co-evolved — along with technology. Our mindsets have become expanded by the new possibilities of super-connectedness, new models of working, and pervasive data-based insight that today’s networked revolution has wrought.

That’s not to say there aren’t important pros and cons to these advances as well. Along this journey of global, open, and social digital networks, we’ve also encountered enormous challenges in grappling with issues such as individual privacy and equal access, as well as the inherently large inequalities that emerge from the gaps between the digital haves and have-nots. This is precisely because technology is a profound force multiplier of just about everything it touches. There are other potential worries as well.

As The Economist fretted over recently, most technology revolutions have created more employment, not less. We hope that this is true for the next generation, but we’ll see, given how current models show that producer power is generally moving outside of traditional organizations to external networks that have less well-defined employment models:

Everyone should be able to benefit from productivity gains—in that, Keynes was united with his successors. His worry about technological unemployment was mainly a worry about a “temporary phase of maladjustment” as society and the economy adjusted to ever greater levels of productivity. So it could well prove.

Yet to most of us, it’s quite clear that digital channels combined with engagement at scale within them amongst all our stakeholders is at the core of the future of business. But what does this actually mean? What does it look like to most organizations? How can we articulate the changes to structure, process, and management of our organizations in a deeply digital age? It’s my belief we need a comprehensive yet eminently understandable model of how all this reshapes our organizations.

I’ve come to realize we’re trying to hit a fast-moving target with poorly aging models for service delivery and IT governance when it comes to digital transformation. The reality is that it usually takes several years for a large organization to achieve large scale change. By this I mean three to five years, and often more, and that’s just for an individual enterprise-wide initiative.

In today’s operating environment of yearly — sometimes quarterly — waves of highly disruptive enterprise technologies and products, that’s just too long. We need a clearer and more updated sense of where we need to take our organizations, and it must also show us how to increase our technology metabolism as well. This model should include the broad strategic outlines as well as specific adaptations to the latest powerful new digital capabilities such as big data analytics, omnichannel customer engagement, the Internet of Things, social business, and so on. These subjects are all highly strategic to the future of our organizations at the moment, yet they are also interrelated and must fit together relatively well in this model somehow.

Motivation for an Infinitely Renewing Model of Tech & Business

Over the last few years, I’ve adopted a term known as the next generation enterprise or NGE for short. It’s the idea that we can maintain an up-to-date strategic model when it comes to digital transformation. The vision for the next generation enterprise is different from one year to the next and has specific technological phases as well as overall strategic themes at any given time. This vision has its own management theories as well, such as shifting from organizational hierarchy to networked community or reorganizing how we operate to the three new top-level modes, to name just two examples.

In other words, the idea of the next generation enterprise is a relatively complete view — including both business and technology — of the target that typical organizations should be aiming for in their objectives for digital adaptation and growth. For the moment, let’s put aside whether there even is a typical organization, since many of the most important technology innovations are usually agnostic to your particular industry or unique company attributes. In other words, most major technology advances will derail your boat if you ignore them long enough, no matter what business you’re in.

To give us a shared roadmap and a point of reference, I’d like to start putting a clearer definition behind what we think is meant by a next-generation enterprise. Early this year, I mapped out the most important strategic new enterprise technologies, but it was a purely technology view and included a good many tactical elements that aren’t that important when it comes the big picture.

Instead, I’d like to have a more enterprise-centric view that includes the most important advances in business that technology has directly enabled. Some would say that the advent of being digital connected to every human being on the planet at all times (at least in the developed world) is one of those advances, and I agree. This realization is that communities are moving increasingly to the center of our businesses. But it’s more than that. The enclosing strategic conception is really one of ecosystem, whether that’s inside a segment of the enterprise with a single networked team, an external customer community, or a full-on developer network of thousands of application development partners who have welded your digital supply chain to their apps. All of these are ecosystems that must be created (or identified), grown, cultivated, managed, secured, and governed.

In fact, one of the largest issues we have in digital transformation today is that we look at business in a far too simplistic traditional model. In this legacy view, there are functional silos with workers combined with management hierarchies that together actually make decisions and operate our organizations. Then there are suppliers, business partners, and customers, and that’s about it for the big moving parts.

Today’s next generation enterprise plays on a much larger and more complex chessboard. There are thousands of relevant ecosystems that now exist for most businesses, most informal, and across thousands more channels, all with a long tail structure.

This means that while the head of the distribution consists of big channels you’ve heard of — from major social networks and call centers to traditional media and Amazon’s cloud — there are thousands you haven’t heard of and will never be able to deliberately consider and plan for. Business architecture has thus moved from simple planned models to complex and highly dynamic emergent networks across every business function we have. We’ve gone from a few dozen groups of stakeholders to ultimately tens of thousands that we must still manage to somehow. Ultimately, our org structures must adapt to reflect this.

I’ve previously proposed a set of enterprise strategies which have a good chance at addressing many of this issues, which were originally brought forth by the channel fragmentation, scale, and decentralization that we saw greatly exacerbated by IT consumerization a couple of years ago. But I now see that bring-your-own-device was just the forefront of a wave of grassroots led network-enabled change, including bring-your-own-application, bring-your-own-community, and soon, even bring-your-own-workforce.

The Element of The Next Generation Enterprise for 2014

So I’d like to put a stake in the ground and define what I think the next generation enterprise for 2014 should look like. There are several views here, but I’ll start with the more business-centric view of ecosystem and expand to other views as I’m able. In this ecosystem view we have the following components:

A more network-centric enterprise. Less hierarchical and consisting much more of online communities for achieving cost-effective outcomes at scale. This will happen within and amongst the workforce (network/social collaboration), business partners, customers, and the marketplace. Management and leadership through networks will become an essential skills and will require knowledge of the concepts and operation of digital and social businesses.

Business partner communities. One of the least developed models of networked communities, there are however good examples that can be pointed to. Strategic partners, affiliates, and suppliers can be engaged together in operations, in particular — as John Hagel famously pointed out — with exception handling scenarios.

Marketplace. The single most scalable asset that businesses have is networked access to their customers and the broader marketplace. While this constituency also includes regulators and influencers, two groups that can be hard to manage, it also includes online advocates, crowdsourcing participants, software developers, and other interested parties. If you’re surprised to see developers in this list, then don’t be: Developers have become one of the single most important new constituencies as their innovations can drive primary growth and network effects. This is a very different view of business than before, where companies directly engaged their stakeholders.

New channels. The next generation enterprise will still have some legacy aspects including physical offices/stores — just smaller and more virtual — it will be the Web and especially on mobile devices that value is primarily created and captured, both. Social business environments (communities of all functional types and audiences) and the application as the new CRM will be key channels here as well. Ultimately, however, APIs — which I define as open digital supply chains — will be the most strategic channel for many industries because it scales faster and creates far more robust outcomes for very little investment.

Using this model, we can also baseline the various states of maturity of each part of the modern enterprise ecosystem for comparison, as in how far along are we? The essential point here with this view of the next-generation enterprise is that it’s the current target model, not what you should look like today. It’s what you should be aiming for, although you should certainly have some elements of it in place today (see figure 2.)

What do you see as other essential views of the next generation enterprise? What else needs to be added?

I’m looking forward to traveling to Paris, France the week after next to provide the opening keynote to the Intersection Conference. Intersection is an intriguing new multi-disciplinary event organized by Milan Guenther that’s intended to explore how we should design our organizations for the future. Milan also wrote a terrific book by the same name, which I urge you to read as well. The write-up for the conference itself says it best:

The role of design in economy and society is shifting. We see disciplines such as Service and Interaction Design moving beyond individual services and their digital components, to tackle experiences between enterprises and their audiences.

Enterprises and entrepreneurship are everywhere, playing a vital role in our lives. They are ubiquitous in the mass of organisations of all sizes we are in touch with as consumers, employees, investors, or in other roles.

This event is about designing the new enterprise, making it less awkward and more humane. We will explore how to design enterprise-wide brand experiences, social organisations, and digital businesses. To do this, design practitioners, consultants and architects combine methods and models from Service and Experience Design, Information and Enterprise Architecture, Systems and Design Thinking, to drive innovation and transform complex enterprise ecosystems.

Those who’ve followed my most recent musings on how our organizations are trying to change to adapt to a rapidly transforming world, and being changed even more by external forces outside their control know this has been a prime focus of mine. Technology in particular is the author of so much of what is reshaping markets, communities, corporations, and even our cultures. The growing question is whether our organizations will break under the loads of so much change, or are there paths we can navigate and steps we can take to transition more gracefully?

In addition, to make matters worse, due to misalignment between our constituents’ respective goals, we also see that most workers in the typical enterprise are generally are poorly engaged. Only 40% of workers are ‘well-engaged’ according to recent analyses. This is a unacceptable state of affairs, but we’ve really only ourselves to blame. I believe we can do better.

How do we adapt sustainably to constant change?

The big question is there an intersection between change, the role and function of businesses, and the future of work that will allow us to adapt more readily? Can we still do this while creating an environment that enables far better and more satisfying work and outcomes for everyone, including employees, customers, and yes, even shareholders? What skills must be brought to bear to realize a fundamental restructuring — in the face of the many major new modes of work — for how we absorb, adopt, and manage the external march of change facing us — and therefore frequently imposing serious business challenges — that we are encountering at an ever accelerating pace?

Personally, I believe getting to equilibrium between our organizations and external change will require a specific set of modifications to the core of most of our organizations. First, achieving this will require a much-more pragmatic and decentralized view of business and enterprise architecture. It will also require that we honestly look at technology and how we metabolize and assimilate it into the way we work and then seek ways to reconcile with it. We’ll also need to cultivate workers that have effective skills in design thinking, which they can use within and across the organization to locally redesign it for current realities. And finally, we need to update how we collaborate in a fundamental and more focused way, and by that, I mean all modes of collaboration between all stakeholders.

These then are the issues I’d like to explore in Paris in just over a week. I hope you will join me. Milan has assembled an all-star cast for this discussion, including corporate strategist, author, and futurist Chris Potts, branding, innovation, and design expert Erik Roscam Abbing, and the lead designer for Dassault, Anne Asensio, to name just a few of the leading thinkers who will be speaking and collaborating there. Either way, I encourage you to join this vital conversation.

Why is it that so many traditional companies with an enormous wealth of assets largely fail to transform them for the digital era? By assets here, I mean established customer base, closely held relationships with trading partners, mountains of data and IP, as well as their bread and butter, the actual products and services they offer. For large organizations, these assets typically represent many billions in long-term investment and accumulated value that is being stranded beneath a digital ceiling they cannot seemingly break through. The lesson has been a hard one: It’s been surprisingly difficult for many companies to make a genuine transformation to digital.

For those just joining this conversation, this transformation is about opening up and digitally enabling the strategic assets of our organizations for better consumption and participation, with as low a barrier as possible. It’s also means doing so in a way that continually maximizes their value over time in today’s deeply networked marketplace. Achieving this triggers the primary engine of growth for digital ecosystems, namely network effects. This is how Apple, Facebook, Google, Amazon (new guard), and Microsoft, IBM, SAP, Oracle, and many others (old guard) eventually built hundreds of billions in combined value. They tapped into the relevant power laws of networks by carefully and deliberating cultivating and then closely managing them by harnessing peer production over the network.

How exactly was this accomplished? They did it by digitally platforming their businesses in specific ways: Enabling self-service on-boarding, viral adoption, open participation, best-of-breed data capture, ownership and control, and took advantage of the fact that relationships — and therefore, ultimately transactions — must take place on the network with as little friction and cost as possible. They realized that we are now all connected together continuously in a single global network and then designed their organizations around this central fact of the digital age. They are now reaping the results of this mindset:

Networked ecosystems must be a core focus and competency of modern business.

This begs the increasingly urgent question: Why then are a large number of older organizations neglecting their digital ecosystems, often failing to meaningfully cultivate them at all for many of their most valuable assets?

But for every Apple which has gone one way, I see so many others piss away this huge asset that is their ecosystem. I hear about musicians and filmmakers auditing, even suing studios for accounting disagreements. I hear SAP mentors complain about legal issues getting licenses and other access to new technology. It’s easy to dismiss Joe Konrath’s litany of complaints against book publishers as one from an unhappy author, but the 230+ comments it has drawn shows a deeper angst about how poorly publishers are managing their author ecosystems.

Ecosystems, communities – call them what you want. They are a vibrant organism which deserve far more ink from all of us. And they need professional managers at the companies at the middle who nourish them and not just treat them as railcars to be hustled away whenever inconvenient.

As companies remain inadequately connected to their customers, partners, and workers via digital ecosystems, many of which they do not control, they are missing a rapidly narrowing opportunity. That’s because it’s very, very hard to disrupt a well-established network effect, which is much more powerful than the equivalent notion in the pre-digital era: traditional market share. Network effects are primary focused on pull distribution, while marketshare is heavily based on push, which is much harder and much more expensive to sustain. As digital natives sew up more and more industries and lay down network effects years ahead of their traditional brethren, any chance to reclaim the throne will be very unlikely.

Why is this? There are a number of reasons but a few are particularly significant. What I wrote in the EI thread in response to Vinnie’s original question was this:

I find that in general, the farther you are from the tech business, the less native skill or familiarity there is with system thinking, which is perhaps the critical capacity to have in order to regard your business in ecosystem terms. This is something that in tech is standard fare with constant discussion and focus on platforms, network effects, SDKs, open APIs, app stores, etc.

Traditional publishers are typical of the technically challenged industries that are being blind-sided by newer, much savvier, techno-centric, network-oriented new digital businesses.

Business leaders that can’t deeply see the way forward for their organizations as flexible, highly dynamic, and organic digital networks of customers, partners, workers, and other (likely and unlikely) participants will ultimately fail. But this isn’t the set of skills or mindset that made them successful in the first place, so they don’t value it and don’t think in these terms. They literally throw off digital rethinking like a sort of corporate immune system. Surprisingly, from my talks in the C-suite the last few years, everyone individually seems realizes they have to change, but collectively they are resistant, it’s fascinating to watch.

A big part of the problem boils down to this: Companies are inherently designed to perpetuate the problem they were invented to solve. It’s a particularly thorny instance of the Innovator’s Dilemma, which ensures that a company is unlikely to aggressively re-invent itself until it’s in the process of being disrupted. Unfortunately, this often means it’s already too late.

In fact, it may be too late for a growing number of industries to fully make the transition to being ecosystem-centric. This includes media, publishing, telecom, retail, and many software companies. Under looming threat is real estate, higher education, financial services, professional services, accounting, and even venture capital. In each of these categories, ecosystem-centric firms are building network effects with open network-based products increasingly built by worker/open communities and delivered to customer communities. Those products are in turn built upon by hundreds or thousands of loyal 3rd party partners to bring their own customers and ecosystems to the table. This is an embarrassment of riches that only a few companies, again mostly digital natives, seem interested or able to tap into.

As Fred Wilson once said, the Web (and therefore digital business) is all about “building networks on top of networks“, which leads to even more powerful outcomes, like 2nd order network effects.

Fortunately, the force multiplier of the ecosystem model can be stated in a simple, fundamental way: It allows one to tap into the vast size and strength of the external network to drive growth, innovation, and revenue for your own ecosystem. As Peter Kim and I wrote in our new book, the fundamental principle of business in the ecosystem era must be to let anyone participate in every aspect of the business, primarily by inverting the facilitation process of driving shared value (i.e. network effects by default.) Being able to elicit the network (Internet, community, shared data, whatever) to maximum effect to fuel and growth your ecosystem is thus the core competency of the digital era. Unfortunately, this lesson is being lost to most organizations that were built well before this next-generation business model was understood. It will be a great loss that doesn’t necessarily have to happen in my opinion, but will ultimately result in the needless disruption of a large number of companies that just aren’t able to become digital natives.

The title of this post is almost right. The workplace of tomorrow will look like a lot of things actually, including the Internet; just not a whole lot like the way our organizations look today. For one, the workplace itself has steadily begun to disappear as teleworking becomes more and more prevalent, though the latest data shows this will take longer than other more imminent changes. These other disruptive forces, such as next-gen mobility, social networking, cloud computing, and big data, are so close at hand that most organizations are already extensively affected by them. It’s not a stretch to say they are eclipsing how IT is applied to business in many ways, even as IT shops are significantly underestimating their current impact, according to brand new research from Unisys.

Over the last couple of years, I’ve been following this set of closely interrelated trends, each one that began “out there” on the Internet or in the consumer world, and have little or no roots in the enterprise world. It’s this singular fact that induces in so many IT executives and business leaders a profound feeling of disquiet. Yet the ones I’ve spoken to this year realize that they have to respond to these changes. Why? Because technology innovation today is driven mostly by the Internet or the consumer world, yet technology is one of the leading ways we use to automate and drive productivity improvements in business. High technology — and particularly the fundamental architecture of the Internet — also has an innate tendency to dislocate the old ways of working. It tends to tear down the traditional — yet less effective — means of operation, along with their associated cultures, norms, and expectations. However, it’s fair to say that no one being held to a quarterly earnings cycle or holding a market leading position vulnerable to technology change (media, software, travel, education, etc.) likes to experience dislocation. So it’s up to organizations to get (much) better at realizing an effective digital strategy, just as innovation and change is happening much faster than any other time in human history.

Recently, the phenomenon of “CoIT” has been growing. It’s a new concept that says that the adoption of IT is now proceeding rapidly outside of the CIO budget, often in entirely unsanctioned initiatives by lines of business. In its more mature form, CoIT also stands for a much closer yet decentralized notion of IT where innovation and technology leadership is driven on the ground by the business, yet supported by IT. The business — as well as IT — brings in the latest new cloud services, mobile apps, APIs, data sources, and mobile devices. IT then makes it safe, secure, and manageable, or provides guidelines for doing so. It’s a smart, efficient, scalable new partnership. The former is the “Consumerization of IT” while the latter model is the “Cooperation of IT”. Both are represented by the moniker, CoIT, which was originally coined by Computerworld Editor-in-Chief Scot Finnie a little while back.

Clearly there’s widespread interest in the topic, as one of my most popular writings this year was the exploration of the “Big Five” IT trends of the next half decade, one of which is consumerization, for which it could be argued it’s actually an encompassing supertrend. All of this ultimately culminated in a gracious invitation by Eric Norlin to come and present my research at Defrag 2011, which I did last week.

Below is the deck itself, which I gave as a keynote last Thursday morning:

If you don’t have time to review the deck, the key points to take away are the following:

5 Strategic Points about CoIT

Evidence is growing that current productivity gains aren’t coming from traditional IT investments. They are coming from somewhere else, or the cost of IT is collapsing radically. Almost certainly both are true by comparing slides 3 and 4.

There is far too much new tech for any centralized process (like IT) to absorb. New types of processes must be created that can unleash and scale the application of powerful new technologies (next-gen mobile, social business, cloud computing, big data, etc.) to the business..

If the only real constant is change, change must be in our DNA. But these ‘genes’ are usually not present in large enough quantities in the enterprise. This is the concept of moving from fixed processes to dynamic relationships embodied by the Big Shift in order to transform the enterprise as we know it.

Ten to hundreds of times more apps and data are coming soon, get ready for it. Cultivate the skills, create enterprise app stores, build social layers into the organization, define decentralized enterprise architectures (really, business architectures), and create a new CoIT playbook. Or this will all route around you. 30% of IT is already outside the purview of the CIO and growing fast.

I’ll be exploring this more soon with new data and examples. In the meantime, I’d love your thoughts on where you are seeing IT going in a rampantly mobile, social, big data world. In addition, here are 10 strategies for coping in the CoIT era.

Over the weekend my friend and industry colleague JP Rangaswami wrote an insightful post that pondered how we have gone about delivering on customer experiences as connected to our back-end capabilities. Specifically, he explored an issue that is increasingly challenging many of the large-company CIOs I speak with these days: That the present rates of change demanded of the accumulation of 20-30 years of legacy business systems is greatly exceeding the ability of our enterprises and associated software “stacks” to deliver on them, particularly as cloud, social, and mobile dramatically transform computing today.

The problem lies in our classical views of enterprise architecture and business architecture both. But JP puts it more poetically:

Development backlogs are endemic, as the sheer complexity of the grown-like-Topsy stack slows the process of change and makes it considerably more expensive to change. The stack has begun to fossilise, just at the time when businesses are hungrier for growth, when the need to deliver customer-facing, often customer-touching, applications is an imperative.

Which makes me wonder. What Tainter wrote about societies, what Shirky wrote about companies, are we about to witness something analogous in the systems world? A collapse of a monolith, consumed by its own growth and complexity? As against the simpler, fractal approach of ecosystems?

This fractal aspect of user systems, Web 2.0, and SOA is one that I deeply explored in the 2005-2007 timeframe, and my ideas on this even made the cover story of the SOA/Web Services Journal at one point. This is when we began to see successful, composite systems sprouting up “in the wild” that were eminently natural and highly successful, such as Amazon’s, Flickr’s, and many others. Innovative new open API-based businesses had definitively emerged and shown us — in fact, virtually proved to anyone willing to observe — that business/technology ecosystems could be routinely produced that were far more successful than the ones virtually any traditional business had managed to produce for itself internally with methods like SOA. It was clear something important and new was happening and we tried to learn from it.

The signature lesson in this time period was that being “in the business” of ecosystems, at multiple levels, was the key to resilience, growth, and sustainability. It’s still a far cry from how most organizations are structured today, but I do see the first hints that this is starting to change. Companies like Best Buy and Sears have been getting the message among the old guard firms and looking at integration, partnership, and engagement as an open network activity.

The Web has also shown us that complexity, as important as it is to address and resolve the many inherent vagaries of dealing with the real world, is largely the enemy, whenever it exists needlessly. Our assumptions of where the complexity should be, in the transports for example, was wrong. It was in the ecosystem itself and we paid dearly for half a decade (and running) based on that misapprehension. But we’ve learned. SOAP usage has almost completely shifted to REST, complex WS-* stacks have largely moved on to simple standards and methods like Web-Oriented Architecture.

But technical discussions obscure the very important truths about what we’ve learned in the interim between the “aha” moment when we realized that what was happening so successfully “out there” was something that would make us just as successful inside our organizations, for internal and customer-facing needs both. After all, this holistic ‘integratedness’ — usually triggered via our connections to the Internet — was a part of our businesses increasingly and we simply had to realize this and engage as Internet natives. But back then, Andrew McAfee worried that focusing on the plumbing was the wrong emphasis to have on the exciting changes taking place, and he was largely right from a communication perspective.

However, building fractal architectures that addresses the old and the new isn’t as easy as just deciding to cast off the old, less effecive ways, as JP notes:

What I’d established in my own mind was a growing belief that the issue was to do with rates of change and costs of change. Vertical integration paid off when the rate of change was low. Networked small-pieces approaches paid off when the rate of change was high.

We are clearly living in times of much greater rates of change. Unfortunately, the old systems, the old architectures, the old business “stack” is often running the core of the business. It’s usually deeply vertically integrated, and not made of small networked pieces, loosely joined that are truly agile, deeply harness innovation on the edge, and turn IT into a profit center instead of a cost center.

4 Ways to Create Sustainable Business Ecosystems

If resilient, networked, recomposable, open business capabilities are the future of business and IT architecture, getting there then requires a substantial change in stance:

Open Your Business Stack. To everyone, internally and externally, generally in the order of the systems and data that are most often requested for integration. Make it easy for anyone to onboard themselves and use the simplest and most egalitarian technology possible to deeply connect the world to your business. Self-onboarding is crucial because it enables the killer asymmetric business advantage of systems that respond to external engagement best: Export most of the change effort to outside your organization and impose it on those that wish to participate and integrate with you. Enterprises that close the “clue gap” with Big Data will have an enormous advantage.

Go Into the Business of Ecosystems. Stand behind your open supply chain: Invest in it for the long term, market it, evangelize it, support it, and stand behind it like your customers — again, internally or externally — will be building world changing products and services from them (if you do this, the track record is that they will.) Be fair and generous with your IP and enterprise data; you are under-utilizing it by several orders of magnitude until you do this, leaving much or even most of the potential of your business on the floor. Worse, until you treat it like a business, you won’t get external contributions to enrich your ecosystem that will drive down the cost of change while greatly increasing the velocity of new solutions and local adaptations.

Begin Fractal Reconciliation of Your Classical Systems. While hanging a trendy Web API off an ancient COBOL accounting system might be a step in the right direction, long-term success will mean going well behind the API. The existing business/IT landscape must be converged and rationalized so they themselves are made of “networked small pieces” that are faster to change as well as more resilient and responsive to being so much more connected (and therefore useful and valuable) to the world.

Spin Up New 21st Century Business Models. All new possibilities, many of which will be foreign to non-digital native firms, are reachable once you have created a fractal enterprise stack. Dominating classes of data, metered business knowledge, all new monetization methods, and more become possible once the three transformations above have started.

Unfortunately, after having advised and directly assisted many large companies in doing much of what I’m describing here over the years, it’s clear that most organizations are at a major disadvantage in making this transformation. The organizational will required is actually quite low, since most of this is fairly easy to do now with cloud technologies, increasingly capable technology partners, and the latest development stacks. Instead, it’s the understanding of how important it is to kill excessive complexity and focus on ways of thinking about business and technical architecture that will propel companies into bright futures.

We must leave behind vertical integration and radically collapse the size of our seams (APIs) in our businesses, while radically increasing the number of useful endpoints such that we look at them as the very fundament of our enterprises. The benchmark for success is how many others are depending on your ecosystem for their own success. Starting now, the activity in your organization must be focused on optimizing for growth of this number. That is the only way we’ll ensure we’re building businesses that are the most relevant, meaningful, and valuable to us going forward. The enterprises that don’t simply won’t survive in their present form. I believe that much is at stake here. Unfortunately, the “cultural gap” John Hagel wrote of back when this conversation started between the enterprise crowd and the Internet crowd is still much too large and is still hindering the move forward for many organizations.

I look forward to continuing this discussion and exploring just how much we’ve recently learned about the future of business and enterprise architecture.

Most of my readers know that I’m a software architect by trade. I’ve been creating software large and small for over twenty years. And I’ve experienced movement after movement in software design from object-orientation in the 1980s and early 90s to component-based design, distributed objects, Web-based software, service-oriented architecture and too many others to even mention. I’m pretty jaded at this point because I’ve learned, in general, the problems that you couldn’t solve in the previous generation of technique are often only marginally more solvable in the next generation (which is invariably invented to "fix" the previous problems.)

Alas, a genuinely better mousetrap is really hard to find.

So in the end, if you couldn’t do whatever it is you wanted to do with the previous generation of technique, it’s actually not that likely you’ll succeed in the next. Certain software problems remain hard, and in general, it mysteriously happens to involve the juncture between technology and people in some way. To paraphrase this, I could say that the software and techniques get better fairly constantly, but people remain the same.

And please, bear with me because I’m going to try out a real zinger on you in a minute.

Because every once in a long while, something new and big actually does come along. Or at least something that looks new and big. One of the new and big things that came along about ten years ago was the concept of design patterns. It was pretty neat stuff. It said that despite the current technology we have, the processes that continue to evolve, there are certain timeless solutions to certain software design problems. It was a revelation at the time. And the writers of the book that explained this got both famous and very successful. Why? Because these design patterns really worked is why. And anyone who has read the books and has ever really built software recognizes these patterns. And what was strange was that no one really expected it. One day, we just had them. And the kicker was, they were always there, but now they were in our conscious thought and we had real names for them. My point: They were in our face all the time but most of us couldn’t see them.

We are in a similar place with the Web right now. We’ve done this Web stuff enough now that we are just beginning to see the design patterns. What works, and why, in a specific situations, bounded by forces. Some folks have had the hubris to give this next generation a name and to tease out these patterns. Some people are just now going aha, and some people haven’t got it yet, and most of the rest of us either aren’t ready for it or just haven’t heard of it. But, I will tell you this. It’s quite real. The best practices and design patterns of Web software are just starting to become understood. The strange part is, we’re discovering the same things over again. What’s old is new again.

Now, before you get all worked up or worse, I bore you and you stop reading, I will give you a nice list of the the forces behind these patterns. If you recall, design patterns are a solution to a problem in context. We are starting to get the context and even the outlines of the patterns of this "new" generation of software. But we have a long way to go still. The Web is a monstrously big space with big problems, and it’s not getting better. There are one billion of us out here now. Clearly understanding what it takes to create great software on the Web that is successful, useful, and vibrant will be an ongoing challenge for a long time. But it will get easier because we are codifying our knowledge of this exciting and huge place where we now find ourselves.

Figure 1: The driving forces in modern software.With a rough comparison between SOAand The Timeless Way (Web 2.0 by any other name).

Now is where I’m going to hit you with a flight of fancy. I’m going to use Christopher Alexander’s opening chapter of a Timeless Way of Building and tailor it to describe this old-but-new way of building the Web and software for it. We are lacking for a little inspiration and this book in particular continues to sell upwards of 10,000 copies a year, 25 years after it was frst published. And Christopher Alexander, for those of you who may not know, was the person that originally discovered the design pattern. But it wasn’t for software. It was for creating great, timeless buildings. He was one of the first that realized that his field of endeavor has certain elemental, timeless cores, no matter the technique, building material, or the people. It was an amazing discovery that poured over into the software world with considerable success.

My assertion is that nothing has really changed in software, we might understand the forces better but they are almost always the same. People want software that does what they want, is available when they need it. They want software that grows with them, helps them, teaches them, and lets them do the same with others. They want software that gets out of their way, disappears, and is more convenient by far than inconvenient. And they want to pay as little as possible for it, but enough so that it’s worth it. They are willing to have software get right into the middle of their lives. If it’s the right software. And as long as we’ve had software, they’ve always wanted this. But now they might actually start getting it.

In any case, I don’t literally believe every phrase in this take-off, but I do believe the overall concept deeply and profoundly as a software professional. And I will continue to update the diagram above (clearly marked beta 1) until we have more of the forces in it. And some are definitely missing. Please, as always, leave any comments and suggestions for improvement below.

And now, without further ado, here is the The Timeless Way of Building Software, with sincere apologies to Christopher Alexander:

The Timeless Way of Building Software

Inspiration For The Next Generation of Web Software

There is one timeless way of building software. It is decades old and is the same today as it’s always been. And because it is timeless, it will always remain this way.

The great software of our time has always been created by people who were close to this way. It isn’t possible to create great software – software that is satisfying, and useful, and makes itself a natural extension of life – except by following this way. And as you will see, this way will lead anyone who looks for it to elegant, vibrant software which is itself timeless in its form.

It is the process by which the function of a piece of software grows directly from the inner nature of people and naturally out of the raw bits, the otherwise meaningless digital medium, of which it is made.

It is a process which allows the life inside a person, or a group of people, or a community to flourish, openly, in freedom, so vividly that it gives rise, of its own accord, to the natural order which is needed to be contained within it.It is so powerful and fundamental that with its help you can create software that is as beautiful and enriching as anything else you have ever seen.

Once you understand this way, you yourself will be able to create software that is alive, that is intertwined comfortably with your life and the lives of others. You will design worlds where you and others will want to work, play, and co-exist together; beautiful places where you can sit and dream comfortably.

This way is so powerful, that with its help hundreds or thousands, or even hundreds of thousands of people, can come together together to create software and community which is as alive and vibrant, peaceful and relaxed, as any living experience has ever been.

Without the central control of authorities and experts, if you are working in this timeless way, a genuine place will grow right from underneath your fingertips, as steady as the grass in the fields or the trees in your backyard.

And there is no other way in which a software which is fundamentally good can possibly be made.

That doesn’t mean that all ways of making software are identical. Quite the contrary. It means that at the core of all successful software and at the core of all successful processes of creation and evolution, there is one fundamental invariant feature which is responsible for their success. Although this way has taken on a thousand different forms at different times, in different places, still, there is an unavoidable, invariant core to all of them.

Take a look at the some of the great Web software like Google’s search page, Flickr or del.icio.us. They all have that unique, yet unhurried, grace which comes from perfect ease and natural balance. But what is it they have in common exactly? They are beautiful, ordered, harmonious – yes, all of these things. But especially, and what strikes to the heart, they live.

Each one of us yearns to be able to bring something to life like this. Or just be a part of it somehow.

It is a fundamental human instinct, as much a part of our desire as the desire to be part of something greater than ourselves. It is, quite simply, the desire to make a part of nature, to complete a world which is already made of mountains, streams, stones, buildings, ourselves, our living systems, and our increasing connectedness together.

Each one of us has, somewhere in our heart, the dream to make a living world, a universe, and place of our own for us to share with others.

Those of us who have trained as software designers have this desire perhaps at the very center of our lives; that one day, somewhere, somehow, we shall build a software experience which is wonderful, beautiful, and breathtaking; a place where people can go and live their dreams.

In some form, every person has some version of this dream; whoever you are, you may have the dream of one day creating a most beautiful place, virtual or otherwise, where you can come together with others and freely share your knowledge, learn, participate in your community or government, and otherwise conduct your daily interaction with the rest of the world.

In some less clear fashion, anyone who is concerned with communities and other large group efforts has this same dream, perhaps for the entire world.

And there is a way that software can actually be brought to life like this.

There is a definable sequence of activities which are the heart of all acts of software design, and it is possible to specify, precisely, under way conditions these activities will generate software which is alive. All this can be made so explicit that anyone can do it.

And just so, the process by which a group of independent people can make software become alive and create a place as real as any other can equally be made precise. Again, there is a definable sequence of activities, more complex in this case, which are the heart of all collective processes of software creation. And it is also possible to specify exactly when these processes will bring things to life. And once again, these processes can be made so explicit, and so clear, that any group of people can make use of them.

This process is behind the design of community built software like Linux, Apache, Wikipedia, and many others. It was behind the design of the great virtual places for people to live and work: the Internet, Usenet, and the World Wide Web. It was behind the creation of simple, satisfying software of the kind that powers the iPod, the Blackberry, and Firefox; of SourceForge, Wikipedia, and BitTorrent. In an unconscious form, this way has been behind almost all ways of creating software since the beginning.

But it has become possible to identify it, only now, by going to a level of analysis which is deep enough to show what is invariant in all of the different versions of this way.

This hinges on a form of representation which reveals all possible design processes, as versions of one most fundamental set of patterns.

First, we have a way of looking at the ultimate constituents of the environment: the ultimate "things" which a piece of software is made of. As we shall see, every piece of software is made of certain fundamental entities known as design patterns; and once we understand software in terms of its patterns, we have a way of looking at them, which makes all software, all of their parts and function, all members of the same class of thing.

Second, we have a way of understanding the generative processes which give rise to these patterns: in short, the source from which the ultimate constituents of software come. These patterns tend to come from certain combinatory processes, which are different in the specific patterns that they generate, but always similar in their overall structure, and in the way they work. They are essentially like languages. And again, in terms of these pattern languages, all the different way of building software, although different in detail, become similar in general outline.

At this level of analysis, we can compare many different software creation processes.

Then, once we see their differences clearly, it becomes possible to define the difference between those processes which make software vibrant, alive, and useful, and those which make them the opposite.

And it turns out that, invariant, behind all processes which allow us to make great software, there is a single common process.

This single idea is operational and precise. It is not merely a vague idea, or a class of processes which we can understand: it is concrete enough and specific enough, so that it functions practically. It gives us the power to make software and virtual communities live, as concrete as a match gives us the power to make flame. It is a method of a discipline, which teaches us precisely what we have to do make our software what we want it to be.

But though this method is precise, it cannot be used mechanically.

The fact is, that even when we have seen deep into the processes by which it is possible to make software alive, in the end, it turns out this knowledge only brings us back to that part of ourselves which is forgotten. Although the process is precise, and can be defined in exact scientific terms, finally it becomes valuable, not so much because it shows us things which we don’t know (though it may do that as well), but instead, because it shows us what we know already.

Of course, this way of building software has never be named. It’s not service-oriented architecture, or the personal software process, or agile methodology, or the unified process, or CMM, or any of the others. It’s the actual things that are conceived and done and worried about when software is created and used. For now, because all software is quickly becoming connected to all other software, and because the Web is becoming the place where more and more of the relevant software is, and finally because it is a more complete reconception of what we thought we knew, we’ll give it a name temporarily. An unsatisfying name, but one that we can remember for now.

We will call it Web 2.0.

What do you think? Are we at a place where we can really identify the design patterns in Web 2.0?