How would U.S. and Israeli strategists deal with a much riskier Middle East?

You don't have to fire a nuclear weapon to gain a strategic advantage from it. This is perhaps the most important lesson from the decades of the Cold War. Yet many commentators on the possibility of a nuclear Iran overlook this truth and argue that we could handle this radically new situation.

A nuclear Iran is usually discussed at a cosmic level of abstraction, in terms of deterrence and containment. But it needs to be examined from the bottom up, in concrete detail. War games, which have a long history in U.S. defense planning, are one way to do this. These simulations bring out insights that never show up in academic theories. Over the past five years, games featuring an Iran that possesses a small, crude nuclear arsenal have been played repeatedly by government officials, the military and outside strategic experts in the U.S. and Israel—games in which participants are assigned roles as decision makers in different countries—and I have been involved in several of them.

The insights that have emerged from these exercises are not necessarily true, of course. They have to stand on their own merits. But I have not been encouraged by what I have seen.

The game might begin with a seemingly familiar train of events, not unlike what has unfolded this week in the Middle East: The Shiite militant group Hezbollah kidnaps Israeli soldiers. Israel hits back with airstrikes on villages in Lebanon believed to be Hezbollah ammunition dumps. The West Bank and Gaza flare up, and Hezbollah begins firing long-range missiles into Haifa and Tel Aviv. The weapons come from Iran, and there are even Iranian "advisers" with them.

But then the tempo of the game slows down. Everyone notices caution, even hesitation, in the Israel team. The Israelis refrain from airstrikes on Syria (Hezbollah's other key patron), and the Israeli navy backs off from the Lebanon-Syria coast for fear of losing a ship. If a ship were lost, Israel would have to escalate, and that is the heart of the matter: Escalation in a nuclear context isn't like escalation in earlier conflicts without the bomb.

Israel knows how to escalate in a conventional war or against an intifada or insurgency. But this is different. The conflict is no longer about how much pain to inflict before the other side gives up. It is about risk. An unwanted spiral of escalation might drive the game in a very bad direction.

The Israel team considers firing a demonstration nuclear shot, a missile warhead that would explode 100,000 feet over Tehran. Israeli plans since the 1970s have called for doing this as a last-ditch alternative to firing all-out atomic attacks. The blast would shatter windows in downtown Tehran, but it wouldn't kill anyone, or hardly anyone. Surely it would shock Iran into a cease-fire.

But before that can happen, Iran ups the ante by declaring a full nuclear alert. Rockets on truck launchers are flushed from their peacetime storage bases, along with hundreds of conventionally armed rockets and shorter-range missiles that can hit U.S. bases throughout the Middle East.

The Iran side in this game has given a great deal of thought to the political uses of its primitive nuclear arsenal. A few of its nuclear missiles are in hardened, underground silos. These are for quick-reaction firing, ready to launch on short notice. Mobile missiles can take hours to move and set up. Iran also understands the psychology of its enemies. The West does not want to kill millions of innocent people, so the Iran team places some mobile missiles in city parks in Tehran, Esfahan and Mashhad. Camouflage nets are placed over many parts of these cities to conceal the missiles and to mislead American satellites.

To bring attention to their dire situation, the Israel team orders two Jericho missiles to go on alert. They are timed to move to their launch positions just as the U.S. satellites are passing overhead. The intent, obviously, is to shock the White House. "We hope it leaks to the media, too, maybe we should make sure it does," one member of the Israel team says.

Israel's move forces a U.S. decision. Washington wants to restrain Israel, defend Israel and scare the living daylights out of Iran. So the U.S. publicly gives Israel a guarantee: If one atomic missile hits Israel, the U.S. announces, that would be it for Iran. The guarantee is cleverly worded. Maybe too cleverly. It doesn't specify which weapons America would use. The term nuclear refers only to Iran's attack on Israel.

The U.S. hasn't fired an atomic bomb in anger since 1945. It hasn't conducted a realistic nuclear exercise since the end of the Cold War, and in this war game, the person playing the U.S. president says that he doesn't want to go down in history as the first leader to kill five million people in an afternoon.

Some on the U.S. team call instead for a massive conventional strike, one that would destroy Iran's military power for decades. The person playing the president asks if Iran might simply sit back and watch such an attack unfold over several weeks. And he is angry: Why haven't better options and intelligence been developed over the years? Iran's bomb program isn't exactly a surprise, after all. It is the most closely watched in history. Why hadn't anybody thought about this before?

The Iran team's next move jacks up the tensions to a fever pitch. Without saying anything, Iran evacuates its big cities. The urban population packs into buses, cars and trucks and rides out to distant suburbs and beyond. In a day, Iran's big cities are at 25% of their normal population. Iran is now poised to absorb a nuclear attack. Its missiles are on a hair trigger, and most of the country's population will survive an Israeli counterstrike.

Israel, by contrast, is in chaos. There is nowhere for the Israelis to go. Ben Gurion Airport has been closed by rocket fire. Hundreds of thousands of Israelis are mobbing the coastal marinas, desperately trying to escape to Cyprus in small boats. TV shows the panic. Deterrence, and the myth of Israeli invincibility, the bedrock of Israeli security, is disappearing.

Suddenly, Iran declares that, in the interest of world peace, it will step back from the brink, having exposed the true nature of "the Zionist nuclear entity." So the game ends. Nuclear war has been avoided. Deterrence worked. But who in Israel, the U.S. or Iran will claim that this was the real lesson? Iran had used a small nuclear force to undermine long-standing perceptions of Israeli military strength and to rupture the Middle East order.

Among the U.S. and its allies, there is widespread agreement that Iran should not be allowed to develop nuclear weapons. Some favor sanctions, some favor pre-emptive strikes, some favor espionage and sabotage. All agree, however, that a nuclear Iran would be a danger to the world.

But despite everyone's best efforts, Iran may still get the bomb. Then what? American strategic planning has avoided this uncomfortable question, but we can no longer afford to keep our heads in the sand.

—Mr. Bracken, a professor of management and political science at Yale, is the author of "The Second Nuclear Age: Strategy, Danger and the New Power Politics" (Times Books), from which this article is adapted.

This copy is for your personal, non-commercial use only. Distribution and use of this material are governed by our Subscriber Agreement and by copyright law. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.