I arrived in Berlin last week hoping to see something rare: a country that's prosperous, well-governed and even happy, if only because it was just crowned champion of soccer's World Cup.

Germany is all those things, and Germans know they are doing well. They need only look around to notice that their economy is stronger and their government more effective than those of their neighbors. Their chancellor, Angela Merkel, an indispensable ally of President Obama in crises from Ukraine to Iran, is "the uncrowned empress of Europe," in the words of Josef Joffe, publisher of the weekly newspaper Die Zeit.

But I also found a Germany that has been plunged back into angst over its place in the world, with the latest blow coming from the arrest of a mid-level official in the German intelligence service (identified only as "Markus R.") who was pilfering secret documents for the CIA.

The message many Germans have taken from the latest spy scandal — coming less than a year after the revelation that the National Security Agency had been intercepting Merkel's cellphone calls — is that no matter how successful and reliable as allies they may be, they still don't get much respect from the United States.

Documents leaked by former NSA contract worker Edward Snowden also revealed that although Germany was one of the agency's top targets, the NSA gives Germany less access to its findings than it gives to France, the Netherlands or Norway.

As Merkel put it with cruel understatement, "This isn't a trusting relationship."

"Americans need to understand how deep the feeling of being humiliated has been," Malte Lehming, an editor at the Berlin newspaper Die Tagesspiegel, told me. "It's the feeling of being humiliated without the ability to retaliate…. And it reminds us, with a kind of a shock, that Americans and Germans don't understand each other as well as we thought we did."

When candidate Barack Obama spoke in July 2008 in Berlin near the Brandenburg Gate, he told a rapturous German audience that peace and progress "require allies who will listen to each other, learn from each other and, most of all, trust each other." It was supposed to be the opposite of George...

When candidate Barack Obama spoke in July 2008 in Berlin near the Brandenburg Gate, he told a rapturous German audience that peace and progress "require allies who will listen to each other, learn from each other and, most of all, trust each other." It was supposed to be the opposite of George...

(Jacob Heilbrunn)

"The substance of this little spy story isn't serious," agreed Karsten Voigt, a longtime foreign policy leader in Germany's Social Democratic Party. "But underneath, it's quite serious. What you're seeing is two political cultures colliding."

U.S. spying is particularly galling to Germany because it knows the United States doesn't spy on its closest English-speaking allies, Britain, Canada, Australia and New Zealand. That's because those countries have joined in an intelligence compact known as "Five Eyes" in which they share information and watch each others' backs.

Germany has been offered membership in the "Five Eyes" club, but it has always found the entrance fee too high: a much bigger intelligence service that could operate on the same level as the CIA and Britain's MI6.

So Germany is caught in an intelligence policy Catch-22: It wants to be an ally of the United States, but it doesn't want to play by the same rules.

That's partly because almost 70 years after the end of World War II, Germans are still gun-shy.

"Germans don't want a bigger role in the world," said Voigt. "We still get nervous when other countries are unhappy with us."

None of this threatens the core of the U.S.-German alliance — not yet, anyway. Merkel has been a solid partner to Obama in supporting sanctions against Russia's Vladimir Putin, despite Germany's dependence on Russian oil and gas. "Putin is helping to keep us together," a U.S. diplomat in Berlin told me.

A poll released last week by the independent Pew Research Center found that 51% of Germans still say they have a high opinion of the United States; that's a big drop from 64% in 2009 (when the Obama presidency was new and fresh) but still well ahead of the 30% recorded under George W. Bush in 2007.

But another poll by the German magazine Der Spiegel found that 57% of Germans want their government to pursue a foreign policy more independent of the United States.

The saddest thing is that this crisis didn't need to happen. "The Americans have bungled this completely," charged John C. Kornblum, who was U.S. ambassador to Germany during the Clinton administration.

The problem, he said, is that after the terrorist attacks on Sept. 11, 2001, the U.S. intelligence community lifted most of the restraints it once observed on collecting intelligence on Germany — for example, on recruiting mid-level German officials like Markus R. as spies.

"We never would have done this before 9/11," Kornblum said. But the U.S. intelligence community never subjected its post-9/11 projects to serious cost-benefit analysis, he said; the terrorist threat seemed to justify maximum measures, no matter what the cost.

So where do we go from here? Officials on both sides say they want to repair the relationship. Obama has telephoned Merkel; CIA Director John Brennan has offered to visit Berlin. But the United States hasn't offered what Germany really wants: a "no spy" agreement that would exempt Germany from being a target.

The sensible thing for Washington to do would be to impose voluntary limits on its intelligence activities against its most valuable ally in Europe, especially when they have proved so counterproductive once exposed.

But the U.S. also needs to learn a larger lesson: Alliances, even long-standing ones, need careful tending. They can't be taken for granted.