Peter Orszag quit his government job. For three and a half years, he ran two federal budget agencies, and in 2010, he defected to Wall Street. Even though he thinks the government desperately needs more smart people, he has deep empathy for those who flee to the private sector.

“In a hyperpolarized environment in which we effectively have a bipolar Congress with no middle, there are just much smaller returns to being in government,” he said during an interview with The Atlantic’s Steve Clemons on Wednesday. Because of this, smart people working in business see less appeal in taking a pay cut and moving to Washington. “What would excite many of the people I know about being in government would be the opportunity to actually do things, rather than just lob grenades at each other,” he said.

Those who have stuck with their government jobs aren’t very happy about it. Orszag cited a new study published by the Partnership for Public Service (of which he is a board member), which reports dismal public employee job satisfaction levels: On average, government employees reported a 57.8 percent satisfaction rate, compared to a 70.7 percent satisfaction rate in the private sector.

Employee satisfaction rates over the last decade. Data on public employee satisfaction is drawn from the “Best Places to Work” survey conducted by the Partnership for Public Service, which covers 97 percent of government agencies and received a 48.2 percent response rate in 2013. Data on private sector employee satisfaction is drawn from research by the Hay Group, which surveyed 6.7 million employees at 400 companies. (Partnership for Public Service)

Top agency leaders just aren’t that good, Orszag argued, and government jobs aren’t appealing to the top workers in other sectors. “We are at significant risk over the next 20 years of a dearth of leadership in the senior agency level, which is necessary regardless of whether you’re a Republican or Democrat,” he said.

Poor performance by certain agency leaders is aggravated by what Orszag described as a general aversion to risk. Any truth in stereotypes about government work—slow moving, bureaucratic, unoriginal—is the result of a few factors, Orszag said.

First of all, it’s a turf problem. “It’s always important for people in the policy environment to have an outside option – something else they could be perfectly happy doing,” he said. “The people who are the most risk-averse – if there’s nothing else you could imagine doing with your life – it causes you to be really afraid of doing anything that could deprive you of whatever exalted position that you have.”

Intense political divisions among elected officials don’t help. “If you don’t think anything is going to happen anyway, why be bold? If you’re stuck in a world of polarization and bimodal Congress and nothing’s possible, your incentives are to just kind of sit tight and hope nothing bad happens,” he said.

But perhaps most notably, Orszag blames bloggers for government inefficiency. “Leadership requires just a little bit of opacity, a little bit of give to hope that things turn out. Lots of the great leaders of the past would not be able to survive in a Twitter world, a constant blogging, Politico world. I think that leads to risk-aversion: If you’re worried about the next Politico story or the next tweet or whatever, you’re kind of afraid of your own shadow.”

This an interesting position for a Bloomberg columnist to take. But in Orszag’s case, government work might be a lost cause. “What could the government have done to keep you?” Clemons asked Orszag during the interview. He answered his own question. “Maybe the answer is nothing.”