Wicked (1)

I've been reading a lot of blog posts, and comments to same, that highlight the seemingly intractable quality of current world problems. This recent post by Steelweaver is a great example. So are a lot of the comments to my previous "Beyond Prediction" post. Steelweaver in particular hits the nail on the head with the idea that "people no longer inhabit a single reality. ... Collectively, there is no longer a single cultural arena of dialogue." This is definitely the case when you examine the cultural and political dialogues arising around the Greek and U.S. debt crises, or global warming.

--And this is a brilliantly insightful idea, but it's a little bit sad, too, because it seems as though a lot of people are just discovering this problem, and yet it's been well known for decades.

I spend a lot of time with people who have a very particular way of looking at problems: define it, decompose and scope it, solve it, implement it. A lot of engineers, scientists, and programmers of my acquaintance take this approach. And they sometimes get very, very angry when faced with real-world problems that can't be approached this way; in fact, I keep ending up in circular arguments with technologists who insist on using this approach on, eg., climate change. Or the debt crises. (Encountered angry trolls in any comment threads lately? Do they tend to make sweeping generalizations about the nature of problems, their causes, and their solutions? Hmmm...)

But often, in the human sphere, there are what're called "wicked" problems. In 1973, Horst Rittel and Melvin Webber defined a wicked problem this way:

There is no definitive formulation of a wicked problem (defining wicked problems is itself a wicked problem).

Wicked problems have no stopping rule.

Solutions to wicked problems are not true-or-false, but better or worse.

There is no immediate and no ultimate test of a solution to a wicked problem.

Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly.

Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.

Every wicked problem is essentially unique.

Every wicked problem can be considered to be a symptom of another problem.

The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution.

The social planner who tackles a wicked problem has no right to be wrong (planners are liable for the consequences of the actions they generate).

Climate change is a great example of a wicked problem: Quick, somebody tell me what the acceptable maximum amount of CO2 in the atmosphere should be, in parts-per-milion! Provide me with the answer to that question, and you win a pony! (And now, dear trolls, fair warning: if you argue over climate change at all in the comment thread, I will mod you off the island. 'Cause it's not what this post is about, it's just an example.)

It is not the case that wicked problems are simply problems that have been incompletely analyzed; there really is no 'right' formulation and no 'right' answer. These are problems that cannot be engineered. The anger of many of my acquaintances seems to stem from the erroneous perception that they could be solved this way, if only those damned republicans/democrats/liberals/conservatives/tree-huggers/industrialists/true believers/denialists didn't keep muddying the waters. Because many people aren't aware that there are wicked problems, they experience the failure to solve major complex world issues as the failure of some particular group to understand 'the real situation.' But they're not going to do that, and granted that they won't, the solutions you work on have to incorporate their points-of-view as well as your own, or they're non-starters. This, of course, is mind-bogglingly difficult.

Our most important problems are wicked problems. Luckily, social scientists have been studying this sort of mess since, well, since 1970. Techniques exist that will allow moderately-sized groups with widely divergent agendas and points of view to work together to solve highly complex problems. (The U.S. Congress apparently doesn't use them.) Structured Dialogic Design is one such methodology. Scaling SDD sessions to groups larger than 50 to 70 people at a time has proven difficult--but the fact that it and similar methods exist at all should give us hope.

Here's my take on things: our biggest challenges are no longer technological. They are issues of communication, coordination, and cooperation. These are, for the most part, well-studied problems that are not wicked. The methodologies that solve them need to be scaled up from the small-group settings where they currently work well, and injected into the DNA of our society--or, at least, built into our default modes of using the internet. They then can be used to tackle the wicked problems.

What we need, in other words, is a Facebook for collaborative decision-making: an app built to compensate for the most egregious cognitive biases and behaviours that derail us when we get together to think in groups. Decision-support, stakeholder analysis, bias filtering, collaborative scratch-pads and, most importantly, mechanisms to extract commitments to action from those that use these tools. I have zero interest in yet another open-source copy of a commercial application, and zero interest in yet another Tetris game for Android. But a Wikipedia's worth of work on this stuff could transform the world.

If Google+ can attract millions of people in just a few days to an app that does little more than let them drag pictures of people into circles, surely we can build a simple app that everybody can use that does even one useful thing, like, say, mitigate the Erroneous Priorities Effect when you're attending a meeting.

106 Comments

Personally, I think that writing about complex issues is a wicked problem. Why? Writing is a linear (one-dimensional) medium. In writing, one strings words together in a line. A complex issue like terraforming, climate change, or whatever, is a massively multidimensional problem. Mapping a multidimensional problem in a one-dimension medium involves quite a lot of distortion. Since mapping is about lying anyway, in the sense that things are distorted and details are left out (there's even a book entitled How to Lie With Maps), I'd say that writing is a wicked problem, by this definition.

I was recently watching a TV show about visual illusions. One point from the show is that, when people see things, about 10 percent of the activity is in the visual cortex (directly stimulated by sight) and about 90% of the activity was from other areas of the brain feeding into the visual cortex, to provide meaning for the raw stimuli.

That was given as the basic for illusions and magic: one's brain is telling the visual cortex what it is seeing. Thus, the idea that people live in different realities appears to have a biological basis.

These numbers reportedly came from functional MRI studies, so we should always remember the dead salmon scan and that this was being reported on a TV show. I haven't chased the study down in the literature, so someone who knows more should certainly correct this statement.

Still, rather than talking about how people use only 10% of their brain power, perhaps we should say that people only see 10% of what they think they see, and the other 90% is their brain telling them what it means.

These are problems we are starting to tackle in Zero State as the number of active members grows beyond the point where informal chat is sufficient. However, as you say, it's difficult problem. However, the payoff for solving it within any given org will be huge. It will provide a significant advantage over the "competition", as well as a speed, flexibility and response time matched only by the best dictators or Mr Jobs.

Surely SOME aspects of climate change/Global Warming are NOT a wicked problem?

We presumably don't want the CO2 level to go up much more, if at all. But we need, and will continue to need lots of energy.

So: PARTIAL solutions must include:
Energy saving, greater energy-efficiencies, less "Carbon" in our power-generation (which in my opinion means we MUST "go nuclear") and a greater educational realisation of the "Interconnectedness of things".
Now this WON'T "solve" the problem, but it sure would help a lot.
Now, we don't even seem to be able to get that far.
And Jared Diamond's book "Collapse" postulates quite scary possible explanations for why this might be so.
Which is another "wicked" problem, .... oh ... bugger, it's gone recursive on me!

Is this part of the status of "Wicked" problems?
That they can and do go recursive?
And are "Partial" solutions acceptable?
If not, why not?
Answers, double-spaced, write on one side of the paper, only, please ... oops.

If anything people are being finally forced to inhabit the same reality by globalization and all the difficulties we're facing are a product of all these jagged edges coming together, I mean I can't think of any previous period of history where people really were on the same wavelength, globally or even socially. Different strata just didn't come into conflict as often due to geographic/class/cultural separation

The trouble is that when it's a politician trying to solve a wicked problem, they wilfully ignore definition 10, because they claim that because they were elected to address the wicked problem, then they have a mandate to do whatever they see fit to address it, and by definition that mandate means that they cannot be wrong, only right. And equally by definition, those who oppose them are therefore wrong. (Or, in the case of the UK, they wrangled their way into power to address the wicked problem and are addressing it without a mandate - but they're still right. But that's a different argument.)

On that basis, your solution is a technocratic rather than a democratic one. Just as long as you realise that...

You're idea of an app for collaborative decision making is very cool. I think the key is to distill the space of arguments down into some some of structure that removes needless rehashing of the same arguments over and over again. There are a lot of valid arguments, but there are also a lot of invalid ones, and our current political discourse tends to bring up the invalid ones over and over again.

The idea is to combine voting and logical argument with the goal of removing the limitations of both. Pure voting is bad because people can vote up statements which are clearly false, and pure logic is bad because we have to make decisions in an environment of uncertainly.

part of the issue is that people don't agree on co2 - some evidence suggests that co2 lags behind temperature rises, so therefore it's not worth worrying about according to those who ascribe to that viewpoint. leaving aside whether that's correct or not as irrelevant to the problem domain at hand (i.e. wicked problems themselves), co2 is basically an example of point 9 above: the choice of explanation determines the nature of the problem's resolution.

the science still isn't proven to some people's satisfaction, and that's why it's a wicked problem. you'll never get everyone to agree, so the only option is to basically ignore that problem and attack it from a different angle. for example, if clean energy was available and cheaper, you'd want that instead, wouldn't you? so make e.g. nuclear power a cheaper source of energy, along with electric vehicles and so on. make it about saving the individual money, or giving them a tangible benefit now, and they'll play along.

Techniques exist that will allow moderately-sized groups with widely divergent agendas and points of view to work together to solve highly complex problems. ... Structured Dialogic Design is one such methodology.

I'd be really interested to know what the conditions are required for this to work. Do participants have to ultimately "share a reality" to make it work, or can they keep their separate realities while forging another? Does it require "reframing" a new shared reality that everyone can participate in, much like a game or simulation?

[The US Congress looks like it wants remarkably similar things whichever side of the aisle the representatives are. Much of the sturm und drang is just posturing. For example, "Obamacare" ~= "Romneycare", but you wouldn't know it by the rhetoric.]

Karl, if you want a programmer, I'm on board. Give me interesting enough specs in Wicked (2) and I'll start without you.

Last time I wrassled with this notion, though (two years ago) the group was too large and diverse to start. Which sounds like the writing of a wicked-problem tool might itself be a wicked problem, or we were just not focused enough on a wicked-problem tool.

Actually, come to think of it, we weren't. I certainly saw the effort in those terms, but I was probably a lot more alone than I'd thought.

Anyway, please write Wicked (2) to coincide with a day when I have time to think. Thank you. (Ha.)

Isn't the fact that a technological solution exists, but is not being implemented, confirm Karl's view about the issue being communication and coordination? Or perhaps the larger view (from the Steelweaver blog) about different realities?

so make e.g. nuclear power a cheaper source of energy, along with electric vehicles and so on. make it about saving the individual money, or giving them a tangible benefit now, and they'll play along.

Except that it isn't. So no useful solution.

Even worse, there is a faction that actually thinks the earth would be better in a warmer state. [Naturally they don't live in the tropics]. For them, the solution is make the world wetter and with higher CO2.

Then there is a faction that says that geo-engineering is the way to go. This increases GDP as new businesses are created to do this, which would be a good thing.

These are hard realities to share with the "use clean energy, and more efficiently" crowd.

This seems to miss what about this problem is wicked -- we're talking about planetary scale problems, where a small part of the population can only keep their privilege by NOT solving these problems.

Inherently, if you actually build a collaborative functional system, the distribution of benefits would have to map to a function of your inputs to the system plus a weight of your capacity to disrupt the system -- a minimum "dignity" effect.

That would make everyone on the planet much more equal than they are today -- if an American collaborates with an African and he wants the relationship to function, he can't expect the African to take 1/100 of the benefits.

To some extent you can focus on non-zero sum solutions -- but at the end of the day, we can't be all Bill Gates. A lot of people will have to give up a lot of bennies, if they're serious about such collaboration on global problems.

Somehow, I think a lot of people are willing to risk apocalyptic catastrophe, rather than sacrificing a known standard of living.

This problem scales at all levels -- between the US and Africa, between Seattle mansions and Compton ghettos, ... There are very vested interests in not taking the only path to solve these problems.

As you point out -- methods are known, yet somehow Congress refuses to use them. I wonder why?

I've come across similar intractability - even in seemingly simple problem areas (technology consulting in HR, where decision support is a major challenge).

One challenge I see frequently with any 'problem definition' is the cultural assumptions (always unstated) that underlie that definition. Take such definitions, pass them through the filter of multiple organizations (outsourced to consulting company A with same-country personnel, but with limited client-company understanding; pass the transformed requirements to technology group B to build, with a completely different culture and understanding) and the results would be laughable if they weren't so costly.

We can mitigate somewhat for regional culture, for language-choice and -familiarity, and for differing corporate cultures - when we know about them early enough. What we can't seem to do is mitigate for all of these automatically and simultaneously - which becomes more important and more challenging as the problems become more complex. As we approach the scale of wicked problems - where there are many more, seemingly intractable viewpoints and directions - the ability to mitigate among the differing constraints becomes almost impossible (wickedly so...)

Alex and Dethe: Most of an SDD session consists in coming to agreement about the meaning of words. This makes it tedious and, in some people's eyes, frustrating because "things are clear." Whether things are clear to YOU is never the point, however, and no large-scale collaborative system is going to work, however, unless everybody using it shares a common language. This requirement, I suspect, makes it an unsexy project for your average programmer, because you basically have to build a nitpicking tool. ("What's my definition?")

SDD then provides a method to avoid erroneous priorities: the choosing by the group of an 'averaged' or lowest-common-denominator main problem to tackle. This method is best implemented as a computer program to reduce time and cognitive load; it does in fact exist in the form of the Cogniscope software created by Christakis et al.

I can't find a non-technical explanation of how the DDD stuff works, but the gist of it is the government maintains a social networking infrastructure where you can either directly vote on issues or gift people your vote on certain issue categories.

The theory is that you'll end up with a kind of representative democracy with a lot of supernodes that is far more fluid then current systems, as people can gift and ungift their vote whenever they feel like it

“The best argument against democracy is a five minute conversation with the average voter.”

interesting thing about a social networking approach to direct democracy is you could weight the voting power based on some of this stuff charlie is referring too, "stakeholder analysis, bias filtering" etc.

Want your full vote on capitol punishment? You have to visit a prisoner on death row and talk to him for thirty minutes. Or read a book on it.

I think the DDD stuff is interesting THEORY mind you, not in any way saying it doesn't have massive issues in implementation

Most of an SDD session consists in coming to agreement about the meaning of words

IOW, sharing a reality...and a desire to abide by that sharing to reach a goal.

And if people do not want that? Must a a reality be forced on those with another?

I'm reminded of a friend of mine who wanted a world where people could live in different economic regimes, so that the Darwinian process of capitalism could be help back for those not wishing to be subjected to it. A different reality if you like. How can you make such differing realities stable unless they are physically separated from each other?

Some projects I've done failed precisely because of a desire to tackle wicked problems. When I heard that I wanted to "boil oceans", it was an apt criticism. However, I learned a lot.

The article need not be taken as a criticism of engineering or scientific reductionism. These approaches to problem solving are correct for certain phases of tackling a problem (like implementation).

The problem we collectively have with wicked problems is that they are vastly interconnected, and so many small moving parts rapidly changing that we collectively can not keep up. Even if climate is slow moving, all the parts that affect it are not, and we are not fast enough or smart enough to keep up.

It is our self-righteous stance against nature that helps us survive, but admitting that a problem is bigger than us isn't ... natural.

My belief/hope is that computers will increasingly tackle wicked problems.

And that's not a problem of technology. We can adequately feed the entire world with the food we produce now and the food we could produce without implementing any new technology. The problem with global hunger is food distribution.

I sat through 24 years of work groups like that. Each was set up to "fix" a problem only to end up with a so called solution that involved us spending millions on some "Consultant Solution(tm)" that was abandoned soon after implementation.

Hmmm... Isn't this Structured Dialogic Design essentially Jürgen Habermas' "communicative rationality" theory in a different dressing? From this quick explanation on "wicked problems" it seems to me that the biggest difference is that your definition of a "wicked problem", according to Habermas, would apply to communication itself, not just one of its subsections.

People will tend to believe things that make them comfortable. Many people are made less comfortable by worrying about CO2 emissions, so there is a lot of pressure to not think about it.

One result of this is that they deny the problem's existence.

Other people are made comfortable by telling other people what to do...without accepting responsibility. So this group will tell the first group that they ought to worry about X, and the first group refuses. This annoys both groups, so they get mad at each other, and neither group works on solving the problem.

Now, clearly, I've left out a lot of intermediate positions, who are all caught in the crossfire. Including people who, for one reason or another, lie about either their goals or about the problem. This, at minimum, confuses things. (Reasons for lying include, but aren't limited to, "it will last my time" and "I'm making a lot of money this way".)

Just, e.g., consider the people who drive to a "Green conference". And then plant a tree to remediate, but don't bother to care for it, and don't own the land it was planted on. And then feel that they're doing their part to solve the problem.

The essential feature of the climate problem is probably that it's too complex for even specialists to hold the whole thing in their minds. This isn't, in and of itself, enough to make the problem wicked, though it's a good start. The wicked part comes because different people have motivations which cause them to see the problem in intractably different ways. (Ever see an oil company ad about how environmentally sensitive they are? Consider that they may not be intentionally lying.)
Note that because the problem is too complex for anyone to understand, there are LOTS of defensible positions to take. And lots of reasons to attack any one of those.

P.S.: A part of the complexity (not all) is that the problem is not modular. You really can't defensibly take a single part of the problem and just say "If we can solve this, we'll simplify the problem." Not even (to go back to climate) CO2. (If you want I could give you a long, and not totally convincing, explanation as to why I say that. But the point is that even though it isn't totally convincing, it isn't unreasonable, either.)

FWIW, *I* think that resolving the CO2 problem would ameliorate things significantly *IF* it could be done without side effects that were worse. But so could an orbital sunshade. (You'd BETTER be able to fold that thing up when you need too, though. It could easily be a worse problem than the one it was solving.)

But do note that "resolving the CO2 problem" contains no hint of how this would be done. Throwing iron dust at the ocean didn't work out. Asking people to restrain their consumption is laughable when the population is increasing, and much of the world is below poverty level. The only people it would be even half way reasonable to ask to restrain their consumption are the wealthy. And most of the wealthy refuse to consider that they *are* wealthy. And another large section don't understand what you're asking or why. And another large section don't care. The long and the short is that "restraining consumption" isn't going to work. People will be dying by thousands and it still won't convince people. (Besides which, the oceans are already warmer. When CO2 is reigned in, expect a glaciation. If the oceans get warm enough, we may be headed for another "snowball earth" episode. [Warm oceans take a long time to cool off. Warm ocean surface evaporates more water. When the air cools, the ocean stays warm because of thermal ballast. Precipitation in cold weather becomes snow. Snow reflects lots of light. Things get colder. It rains more. It snows more. Glaciers start moving towards the equator. And, eventually, spreading out onto the ocean surface. The ocean cools, and stops evaporating lots of water, but now the earth is hugely reflective.])
N.B.: I'm *NOT* a climatologist. I read some popularizations decades ago. So don't believe this, but consider it.

Hmm. Wicked problems have no uncriticisable description using the language, imagary, graphs and other descriptive tools at our disposal. Furthermore, they have no single description that can be used to solve the problem. Any single idea that looks like it solves the problem is in fact being taken outside of it's correct scope, although we could do with a variable notion of scope.

Since human being learn new ways of looking at things all the time, a problem can be brought out of the field of wicked problems, although some sort of patchwork definition also looks possible. So, I would think that scientific breakthroughs are often examples of such a moving of a problem. Moving the collection of taxonomies into the tree of life, via evolution, is probably a major breakthrough. In fact, anything which organises a field of knowledge could be viewed as such a thing. However, building the original patchwork will be just as necessary.

I think you missed the main reason why climate change is such wicked problem. And that is systematic false reporting of likely effects, following the Al-Gore-Doctrine that people won't act unless you overstate the case you're arguing for and that you must not admit to any doubt over any of it.

In fact, that only leads to massive distrust in one part of the population, indifference in another and a completely distorted view of the problem in those who believe the propaganda (to call it by its proper name).

These days, just about every natural disaster is attributed to climate change - yet absolutely nobody is talking about what the climate *is* like. Somalia or Australia are drought prone places - and recent droughts are absolutely not unprecedented. Nor are the rainfalls in Korea or Japan. The flooding of Pakistan last year was the worst in 80 years. It's not a question of climate change, it's a question of being unprepared for local weather extremes.

This is in fact rampant in the USA. New Orleans gets hit by hurricanes a couple of times each century, people had been warned for decades until Katrina struck. Tornadoes are far from the overwhelming force that people keep pretending they are. Properly build facilities can withstand any tornado - St. John hospital easily withstood an F5 tornado (and is not the first building to do so), the only casualties being patients on ventilators who suffocated when electricity and emergency generators failed.

The droughts of the 1930ies, the decade when 25 out of 50 US state maximum temperature records were set, were well before any climate change whatever. Yet, nobody prepared for their reoccurence. The Missouri floods that threatened a nuclear power plant earlier this year were nothing out of the ordinary - the dams were build to hold such floods back, however, an unresponsible breed of environmentalists made the operators of the dams hold back water in spring in order to simulate the annual flooding of the river. While this may help to conserve the ecosystem as it was (instead of changing it to one adapted to new circumstances), it also obliterated the dams ability to mitigate the floods they were designed to hold back.

Yet, instead of recognizing that those are normal events and that people have a responsibility to prepare and deal with the local weather - these days people claim it's all just climate change and carry on, while writing doomsday articles about the weather getting out of control.

Guess what: the weather was never in our control to begin with and there is absolutely nothing that suggests it's getting worse. We have some 200 countries in the world - there are plenty of 50-year floods to around each year. Plenty of 50-year droughts, snowfalls, heatwaves, storms, tornado outbreaks - take your pick.

You want an answer on how much of a climate change is acceptable? Well, you won't find one so long as people think that "normal" climate is everything that historically happened in between the disasters. First, you must know and understand what business as usual means in terms of weather. Second, you have to prepare for that. And only then will you have a reasonable basis for discussing such matters.

Wicked problems are never about the problems. They are about the people debating the problem being irrational. The constant drone to instill fear into people's minds is what is making them irrational and the reason for the problem turning into an unsolvable mess.

At one point I had a close friend who occasionally went bananas. He was supposed to take some anti-psychotic drugs, but every year or so he decided that the side effects were so bad, that he'd rather go crazy. He lived in the apartment next door, and one year when he knew he was starting to lose it, he appealed to me for help. He *really* didn't want to go back to the psych hospital. So I and three-four other friends tried to help. He stopped sleeping. He got progressively crazy. At the start he wrote a lot in a novel he was working on, but at some part he started to believe that it was reality. Then he got paranoid. Not about his friends, anyone he'd ever met, but about secret a mystical organization whose character kept changing. Eventually, we couldn't take it any longer, and we had to take him to the hospital. Nobody liked that answer. Not us, not him, not the hospital. (He couldn't pay.)

It was so painful that I cut my connection with him. I still regret it. But I couldn't do anything else. Occasionally I see him and we say hi. And I feel unhappy all over again, even though he's never blamed me.

That's not a solution. Not for any of us. But we can't figure out how to do any better.

Outside of the platonic universe of dreams and delusions is there any clarity? Has there ever been?

Is there any cut-off point for complexity, where it suddenly ends?

I totally agree with #26 that this is not new and I totally agree with #1 that language is widely overrated and part of the problem.

I'd like to think, that in a self-organizing interconnected society cultures with a critical worldview (i.e. "nothing is clear") have a survival advantage.
Not necessarily enough to actually survive...

In my eyes, communication is an impossible dream and you should forget to even think about it.
In my eyes, cooperation is a valid aim and achievable at least within a culture or within cultures sharing a meta-culture.

And on an emotional, subliminal level (not physically) that means war. Incompatible, competing patterns of cooperation, the oldest game in the wolrd...

No, climate change per se is not a wicked problem, and your framing of it is completely bogus. But perhaps that was your point. If not, it's a point that needs to be made.

"The climate change problem" is, however, a symptom of a wicked problem: the problem of propaganda, the methods by which one group can persuade everyone else to "freely" act in various ways, including and especially acting against their own interests. See also religion, tobacco, and, yes, the completely synthetic US debt crisis.

This post looks to me like an "if all you have is a hammer..." problem.

The Fox friends will say anything true or not, to win. That's Wicked. Lets not do anything about Global Warming because we say Al Gore lied about it. That's Wicked. Remember when those in power said that tax cuts would more than pay for themselves. Any body who could count knew it was wrong. That was Wicked. People still cutting taxes and saying its to help the economy, are they dumb or Wicked? About 40% of the temp rise is from other things in the air besides co2. So what! The co2 can be cut back with tax money. And its is still making the seas acid. When its too acid, in not that long a time, most or all of the sea live will die. Then all most all of us will. The word must have gone out over the net to mob this blog with the Wicked?

While not a "complete solution" to the full breadth of the social problem you describe here, http://hyperarchy.com/ is a start (I'm not affiliated with them, just admire the idea). It's a way to use the delegated proxy voting, a kind of "fluid democracy" in conjunction with a voting system that is better (more accurately reflects the will of a group of people) than the "pick just one" method we have now.

To Dirk (#33): the other key element of SDD is stakeholder involvement. Participant selection is built around the Law of Requisite Variety in order to ensure that all interested parties are part of the decision making process.

#41 and #48, no more talk about climate change, please. You're inching into exactly the argument I asked you to avoid. I WILL mod you off.

By the way, it's discouraging how many commentors in this thread seem to think I'm Charlie. :-(

I'm in the business of doing this, or at least providing the evidence that feeds into policy discussions for these kinds of questions. (http://www.royalsociety.org.nz/publications/policy/2011/ecosystem-services/).

To make progress on wicked problems requires two things: the first is a desire for cooperation, the second is a collaborative process.

On the desire, one of the reasons that the Scandinavian countries do so well at handling wicked problems is that there is a cultural recognition that some problems can only be handled by cooperation. There's various drivers for this, crap weather, shared history, common values, small size, but whatever the drivers are, they are far more willing to get opposed groups to sit down together around a table, agree that everyone is not going to get everything they want, but everyone is going to get something. Without that desire for collaboration, wicked problems don't get solved. The political culture of the US prevents that collaboration, so the problem there is the need to change to political culture.

The second part, the process, requires everybody involved to understand how a consensus is to be formed. Like all social processes, experience matters, and participants need to be experienced in the process, no matter what methodology you choose. Structured Dialogic Design is just one option. Choosing methodologies and then running them without allowing them to be co-opted remains a hard problem.

For instance, (one of) our wicked problem in NZ is that we've seen a massive growth in irrigation for dairy farms, pushing us up against the limit of freshwater use (that's by NZ standards for freshwater, by European standards we're in clover). You can conceptualise this as farmers making lots of money by polluting the environment, or as environmentalists trying to drag down good honest hard working farmers who are just trying to make a living. Neither approach really gets you anywhere, so the government set up the Land & Water Forum, modeled very deliberately on Scandinavian consensus building processes, to get everybody around the same table, to get them to agree to the science, to get them to state their values and goals for the environment, to build a shared language for those values, and to agree to an approach to making freshwater management better, where what better meant had been agreed upon.

This social approach is slow and messy, and the best approach we have to dealing with environmental problems.

Blades@12: Let's be precise here. The issue is not that "people don't agree on CO2". Scientists agree on CO2. Politicians find reasons to ignore scientists. Deniers muddy the waters.

The science isn't proven to some people's satisfaction, but that has nothing to do with the science and everything to do with the deliberately, ludicrously high standards that some people will set when they insist on sticking their fingers in their ears and shouting "I can't hear you". That doesn't help anyone come to a solution.

But, really, a non-wicked problem is just a special case of the wicked problem.

You have a sheet of paper in front of you, printed "2+2=?". How wicked could that be? But maybe someone will be doing a DNA scan on the pen you use to write down 4. There can always be something over your horizon that complicates things beyond your capacity to understand them.

A non-wicked problem is a wicked problem with a clearly defined boundary. But any sufficiently conspiratorial theory could say that the boundary is just there to better fool you.

In a sense, life is itself a black-box. We have never, by definition, a complete definition. We have incomplete definitions that, sometimes, include contained context inside of which we can operate in a "engineering" frame of mind. But each and every one of those contexts can be a part of something bigger. We can never really know.

(I wish i had the time to write a better reply to this, i'll be checking the next episodes...)

Indeed, it seems that a good first step (or maybe it comes later?) in solving a wicked problem would be to identify and decompose the parts that *are* "tame" or easily soluble. I think a failure to deliberately segregate the two sides of a problem like that could definitely be impeding resolution.

"it's discouraging how many commentors in this thread seem to think I'm Charlie." So be happy about it! If you wrap all your problems into one big ball like pols do it can't be fixed. If you do what engineers are suppose too, you look at the ball, grab a piece of it and fix it. Then grab the next one till there is no ball. look up R.B. Fullers WORLD GAME. It was fixing things that had not happened yet. but money was needed to pay for Black Studies and they killed it. Now we need it.

The reason why I wrote at length and didn't just write a sound-bite on the topic was to illustrate the central point. Namely, that you can set up any number of wicked problems if you fail to take the perception of the problem out of the loop.

If perception of the problem is not objective but subject to part of the problem itself (in this case: the population), you will very easily end up with a wicked problem.

A major part of the wicked problem of US economic policy is that government spending is perceived to be much less effective in supporting the economy and austerity is perceived to be much less harmful to the economy than is actually the case. But as public perception is a major part of the politics themselves, this is creating a feedback loop and is basically how the US ended up with the current economic mess.

A major part of the wicked problem of climate change is that consequences are perceived to be much worse and the solutions (e.g. renewable energy, nuclear energy) are perceived to be much easier to implement by the agents who are part of the problem than you can account for from an objective point of view. Which is radicalizing both sides to the point of creating a lot of problems.

E.g. Germany turning away from nuclear power and towards burning a lot more coal, gas and bio-fuels/bio-gas. Bio-fuels are being justified by the argument of carbon-neutrality while ignoring their already huge impact on food supply on international markets (which happen to be what a lot of African countries must depend on after the IMF told them to substitute their food crops for cash crops). You don't burn some 8-10% of the world grain crops as bio-fuel without a major impact on food prices - which went up by some 200-300% within 3-4 years in countries were food makes up the largest part of expenditure. (Imagine having to deal with a 200-300% rise in rent for your apartment within 3-4 years!)

This is a big issue, and you wouldn't be doing yourself a favor if you were to ignore it.

All these vague talks about properties of wicked problem remind me of quantum mechanics, unless there's a equivalent of Bell experiment to prove these wickedness, I just don't buy it. As far as I can see, the so called wicked problem is just normal problems in a highly interconnected and interdependent system (like the problems in human body: cancer, aging), with miscommunication and personal greed throw in.

I don't see a political culture in the US. Depending on the viewpoint, I see either two of them (or even three) or none (no overarching one).

But the lack of an overarching culture in the US might be actually a good thing, because the reason for it is a global dynamic, an emergent global structure. Not Western, not anglo-saxon and not necessarily codeable in time-worn English soundbytes.

If you understand the Tea-Party as an acute immune-reaction, an ageing organism in hyperdrive to stave off "infection", a lot of real world details can be integrated in your picture.

The answer to the underlying problem is "Reality meet Tea-Party, Tea-Party meet Reality": a return to purity is a desperate defense, that is ultimately bound to fail, because it involves a loss of freedom of action.

Osama bin Ladin will be remembered as the last nail in the coffin of radical Salafism, Mussolini was the last nail in the coffin of an outdated code of the "chivalrous warrior". The extreme and the pure at a certain point become transparent and utterly laughable. The rest is patience.

SDD is "..coming to agreement about the meaning of words. ...... unsexy project for your average programmer, because you basically have to build a nitpicking tool."

A nitpicking tool is something IT industry needs too. (And uses, even if I still dont know something satisfying and open source. Key words, e.g.: requirement analysis, requirements engineering) I dont know what you need, If you find something interesting, please keep talking about it.

Then, can you please give some start links to read about SDD, I'd be happy.

So the problem-solving depends a lot on everyone getting together, and understanding each other?

Long-term, that suggests some things are going to have to change. Look at how government power in the UK seems to be concentrated in a quite closely-defined group, with a narrow range of education and experience. Each of us might ask, "How can they understand my situation?"

Is it such a surprise that the last great change was done by a government which didn't all go to the right schools, and had a wide range of personal experience? Yes, I'm talking about the 1945 Labour government (which also had been part of the wartime Coalition). Just pointing at the obvious Left-Right difference misses rather a lot.

And it doesn't necessarily mean that there is anything wrong with an Eton education. The Public Schools of England, at their best, do find and bring out the strengths of their pupils. But when that sort of background becomes commonplace, do we have a small tribe running the country?

Some of the things I see happening on the TV, the way the politicians argue in Parliament, and answer questions, puts me in mind of summoning spirits from the vasty deep, and Hotspur's reply: "Why, so can I, or so can any man; But will they come when you do call for them?"

If we can't solve that problem, "wicked" or not, we're going to struggle.

For once, I agree with Greg. Wicked problems are not "normal problems in a highly interconnected and interdependent system". They have emergent properties that make them non-reduceable to optimisation.

When (some) engineers say "this problem is perfectly solvable by breaking it into little pieces whilst ignoring the human dimension", they're making an argument that's exactly analogous to libertarians saying "we can solve all of the political problems in the world by just assuming that human beings will no longer behave like human beings".

Why can we not accept that for "unbounded"/"wicked" problems, we need to accept partial solutions?

From the practical, problem-solving point of view, I think this is spot on.

Competing, incompatible and at least temporarily unprovable explanations and world-views will always be bothersome and untractable, but since we have to live with it, partial solutions should be the way to go.

Agreed. Much of the issue with this type of problem, is that once you break it down into its constituents and solve for each of those, or more often attempt to model for each constituent to find a solution, what you end up with has very little bearing on what you were originally trying to solve.

It all reminds me of the old saw .. "First, we assume a spherical cow"

The biggest issue seems to be very few people agree on a common definition of exactly what the problem is, and even fewer can agree on what result they want.

If you ever have to write a proper requirements specification you see this issue in micro scale - even after extensive consultation, you never get everyone agreeing so you eventually force a compromise on the least worst solution. And then afterwards, all the participants convince themselves that when *they* were putting forward their requirements they made everything very clear and simple and *someone else* clearly obstructed things. Give it six months, and they will be so certain of events that even looking at the minutes of a meeting where you show them what they said, they will insist it was recorded incorrectly. I swear, getting rid of the Human factor is the hardest problem of all.

From your description of wicked problems, it appears as though proving whether or not a given thing is a wicked problem is itself a wicked problem (i.e., it's the P vs NP situation all over again: some problems are known not to be wicked, but the only way to know whether or not a given problem is wicked is to prove that it is not wicked by solving it)

I think you missed the main reason why climate change is such wicked problem. And that is systematic false reporting of likely effects, following the Al-Gore-Doctrine that people won't act unless you overstate the case you're arguing for and that you must not admit to any doubt over any of it.

What is and is not persuasive is not necessarily set in stone.
In the 16th century an oratory voice with a big gilded book as a prop was a major success. Try that today.

While I don't doubt that certainty, authority and generalized language still work in most age brackets, in a lot of urban youths and ad professionals of all ages they only trigger instant all out defense.

If used too often it becomes recognizable and then it becomes transparent.

How do you reconcile the two? Well, a first step is: you don't. Instead, you compromise and don't radicalize. You compromise on trying to persuade at all cost and instead take time to explain. And you compromise on trying to persuade by selective argumentation and blatant lies and instead present the full argument including those parts that don't look so good - provided that you first make sure to have enough time to do so and otherwise be steadfast in declining to even make one single statement.

If there is not enough time to explain reality, a narrative trying to give a full description of reality will be just as incoherent (and incomprehensible) as a narrative that is cherry-picking among whatever caught the narrators fancy or is a blatant lie to convince people of something or other. The single most important aspect of judging a narrative is coherence, because reality is coherent.

If your narrative of reality is truncated, its coherence will be compromised. And especially when it has to be severely truncated, it is compromised to the point that it can be much easier to come up with narratives that cherry-pick or are lies *and* are more coherent and believable than a narrative trying to describe reality.

Radicalization is inherent in the current media environment, by the simple fact of the short form reporting. The sound bite culture is more than just a complaint by some head-in-clouds romantics. There simply is no room for either subtlety or even a minimum of factual correctness, if you have to make your point in less than a minute. Half an hour can sometimes be insufficient.

There is an incompressible core of an argument and with some complicated problems that simply takes time to convert into human language in a way that it can still be unpacked by the audience.

It is simply not true, although common wisdom says otherwise, that the world today is facing more complicated problems than ever. No. It's simply that the communication of problems has degraded to the point where problems simply cannot be adequately explained. And a problem that can't be explained, can't be solved.

Each time I watch a TV program from the 60ies or 70ies, I'm flabbergasted by both the patience of the hosts and the depth of discussion. The reporting on the moon landings was out of this world - it had much more scientific content and deeper explanations than anything you could find in the 21st century so far (of which I have a lot more hands-on experience than of a time when I wasn't born).

Can I just throw in my own 2cents regarding different ways of discussing and formulating problems and solutions.

Specifically an important factor to remember is that a large number of the entities involved in the problem spaces we're discussing are corporations. Unfortunately these don't and can't have ethics and yet are counted as legitimate parties to the process.

Until we figure out a solution for how to deal with Corporations they will continue to find solutions on how to deal with us.

Greg @4,
The problem is that you and many others here are thinking rationally The issue is not what's reasonable.

I think Karl is discussing how we can breakthrough individuals' emotional "opinions". Right now, when it comes to climate change, we're definitely - at least in the U.S. - on the "worse" end of the spectrum. Facts seem to be irrelevant to a significant part of the human population.

Right wing political movements have understood this for years. They constantly appeal to emotions. The obvious result is political victory and the passage of legislation which generally makes everything worse.

d.brown@49. You're confusing the wickedness of evil with the wickedness of complexity.

I'm a trained mediator and, you should all understand that most disputes involving humans (are there any other kind?) are wicked.

I often cite H.L. Mencken's analysis of problem solving, "For every complex human problem there is a solution which is simple, straightforward and wrong."

Karl, I'm a major fan. I also think you seem to be following Charlie's general thrust in this blog: release the hounds!!

Rick York @ 77
Sorry I do appreciate that difficulty, hance my call (partially as a result of my engineering training) for a PARTIAL solution, as better than none.
The real difficulties arise when certain participants refuse to accept there is a problem AT ALL.
GW being the prime current example.

I just now wrote a comment where I said that on reading the post a second time I realized that most of my questions had already been answered; the information just hadn't stuck in my brain on first reading. The comment is being held for moderation because it included a URL to some notes I took; if in the meantime anyone would like a copy of my notes please contact me directly.

ok, here's my current status: a Google search for "Structured Dialogic Design" has yielded a lot of information *about* SDD.

What I haven't found yet (aside from the occasional workshop, apparently I just missed one in Cyprus) is information on how to *learn* how to actually do SDD.

...either because I haven't found it yet, or because the information is being passed around from practitioner to practitioner and shared in workshops, but hasn't been published in a complete form so far.

HOW PEOPLE ARE HARDWIRED TO ACT! ---'The size of the lie is a definite factor in causing it to be believed, because the vast masses of a nation are in the depths of their hearts more easily deceived than they are consciously and intentionally bad.
The primitive simplicity of their minds renders them more easy victims of a big lie than a small one, because they themselves often tell little lies but would be ashamed to tell big ones.
Such a form of lying would never enter their heads. They would never credit others with the possibility of such great impudence as the complete reversal of facts. Even explanations would long leave them in doubt and hesitation, and any trifling reason would dispose them to accept a thing as true.
Something therefore always remains and sticks from the most imprudent of lies, a fact which all bodies and individuals concerned in the art of lying in this world know only too well, and therefore they stop at nothing to achieve this end.
~ Adolph Hitler, Mein Kampf

I stopped by the library today and skimmed The Talking Point: Creating an Environment for Exploring Complex Meaning. I was curious if it just talked about the SDD process or went into detail (I hadn't been able to tell from the blurbs). It does look like it gives a detailed description of the process, so I've ordered a copy from Amazon.

No, really. They're held. When you submit a comment with a URL, you get a page that says the author (now other mods, too) will check the comment. Then it asks if you want to go back to the original entry, which has a link back.

Do you keep track of which post you comment on? And look up where the comment would be if it didn't have a link? If it's not there, you should probably ask Charlie. His edress is in the second division of the gray part on the top right.

On the desire, one of the reasons that the Scandinavian countries do so well at handling wicked problems is that there is a cultural recognition that some problems can only be handled by cooperation. There's various drivers for this, crap weather, shared history, common values, small size, but whatever the drivers are, they are far more willing to get opposed groups to sit down together around a table, agree that everyone is not going to get everything they want, but everyone is going to get something. Without that desire for collaboration, wicked problems don't get solved. The political culture of the US prevents that collaboration, so the problem there is the need to change to political culture.

No the problem is the US is really about 300 Scandinavian countries. The Polish section of Chicago is basically a social group. The far western Kentucky area I grew up in was an alien culture compared to that one. And far eastern "hillbilly" areas of Kentucky were also foreign to either of these.

Comparing the US to Scandinavia is not valid. It needs to be compared to the EU. And we all know how well they get along and not have issues between members in the EU.

well, I do not agree 100% with Josh G.... I am sure that with sufficient communication, coordination and cooperation we can minimize this number... i think it's more a diffusion / adoption problem than a technology problem...

I don't know. Not so very long ago, Scandanavians weren't known for their peaceful and cooperative nature. Remember the vikings and the berserkers?

Peoples change.

As for the US, I wouldn't look for deep historical roots for our propensity to violence. Rather, I'd suggest that the US has a propensity to a culture of fear, particularly in the 20th Century. It's not about gun ownership or even weapons ownership. Rather, it's about people telling you that you will need to use those weapons.

For example: I've already been held up at gunpoint. Statistically, that's almost certainly never going to happen to me again. Yet I have various, unspecified arms in my house. I'm paranoid too.

I've seen this attitude over and over again. Years ago, on a knife website, someone asked what kind of giant knife they should carry on a desert hiking trip. I suggested that they should carry a light knife and use the extra weight allowance for more water. For that statement, I was excoriated by a number of incredibly afraid blade freaks who were afraid that they'd need to kill a mountain lion/bear/outlaw with their giant knives, and would rather deal with that imaginary threat than prepare for the certain problem of dehydration.

Again, peoples change. Now, "maintaining a frontier spirit" is used as an excuse for raging paranoia, as is the thought that Christianity (the majority culture in the US) is under radical threat from...whoever, just as it was in the 1st Century AD. Sad, but there you are.

Personally, I think "pay attention" is a better motto than "be afraid," but even for me, it's difficult to buck that current.

I don't know. Not so very long ago, Scandanavians weren't known for their peaceful and cooperative nature. Remember the vikings and the berserkers?

Think about the high level of co-operation viking required, and the fact Scandinavian living arrangements at the time were longhouses shared by multiple families in a climate that meant you didn't leave the house if at all possible during winter.
They may have had an ingroup/outgroup bias in their violence. ",)

I've read about half of The Talking Point and skimmed the rest. It looks like a key part of the SDD process is putting together a root cause influence relationship diagram (this is what helps avoid the Erroneous Priorities Effect).

Apparently SDD has an algorithm to avoid having to compare the influence of all pairs of ideas in order to build the diagram. (As the book says, to compare all pairs of 14 ideas would be 172 comparisons, which at a rate of two minutes per assessment, would be an exhausting six hours of work).

The algorithm is not explained in The Talking Point, but it does say that "this process [...] is fully explain[ed]" in How People Harness Their Collective Wisdom and Power to Construct the Future. So I've ordered that book as well.

Would love to hear more about Erroneous Priorities - is it the same as a board meeting deciding to spend millions on a research project in minutes but discuss what espressomachine they should have in the lunch room for hours? (Since everyone knows something about coffee but not very many bother to delve inte advanced science and choose to trust the scientists and their own colleagues instead..)

What you talk about as wicked problems seem to be the same as http://en.wikipedia.org/wiki/Complexity - unless I've missed something.. Isnt it making things unnecessarily complex to use several terminologies for the same problem?

"is it the same as a board meeting deciding to spend millions on a research project in minutes but discuss what espressomachine"

not quite... what you describe it a very real phenomenon, but Erroneous Priorities would come into play if the group did decide to discuss the research project in detail and in good faith, but ended up making poor decisions because if they only look at what they think is important, they'll probably miss the root cause or underlying issues that will make those important things successful.

Progress of sorts... I've found out that the algorithm I'm looking for is called (or is implemented by) something called Interpretive Structural Modeling (ISM). This was mentioned in The Talking Point, but in a different section than the one that describes the process... so I had missed it in my first reading.

My copy of How People Harness Their Collective Wisdom and Power arrived today. It describes ISM in a bit more detail but doesn't explain the actual algorithm either. It does say that it was invented by John Warfield and gives references. I'm making heavy use of interlibrary loan to see if I can track this down. (It feels very weirdly old fashioned -- I'm not used to following links taking days or weeks instead of minutes or seconds :-)

At this point I don't know yet if the information required to implement SDD (in particular, the ISM component) is in the public domain, whether it's published in academic papers or books, or not. If it isn't we may need to plug in some other algorithm to generate the root cause map part of the process.

The search for unknown information is always interesting: if I look, and don't find anything at first... does that mean it's probably not available, or that I should look some more? In a finite search space (looking for my lost keys in a house) it's easy to have a sense of whether I've looked "everywhere", but in an infinite search space...?

Reminds of the opening to The Prisoner that starts off with "We want Information. Information... information... information!" Yes! I want *INFORMATION*!

Anyway ^_^, an actual description of the ISM process (Interpretive Structural Modeling, a component of SDD), complete with algorithms and boolean and matrix algebra :-) can be found in the nondescriptly titled book "Societal Systems" by John Warfield. Happy programmer is happy.

If an implementation of SDD to mitigate the Erroneous Priorities Effect in meetings is a useful place to start in creating a Facebook for collaborative decision-making (i.e., by analogy Facebook itself started just at Harvard at the beginning), then I'm thinking a modern, collaborate web-based implementation of ISM would be a good place to get started with implementing SDD (i.e., even the initial implementation of Facebook just at Harvard started somewhere).

I have a clue to a 'Wicked Solution'. In 1991, at a state wide meeting of educators in California, an ISD superintendent told me of a solution that raised the education level of students that participated, using fewer instructors, at no increase in education cost. After 2 semesters of testing at a named ISD, it was to be implemented. But a teacher realized, it would reduce the number of teachers required, so appealed to their union. Organized, they threatened to quit. The teacher that had devised the plan was fired; the idea abandoned. A forward thinking HR manager at a Texas ISD in 1996, embraced the idea and instructed me to start a search for the author. Before I could implement an organized search, the HR manager was fired for 'incompatible management practices'. A few jobs were saved in California over what might have been a 'wicked solution'.