Lehrer, 78, has moderated 12 presidential debates since 1988 and said in a book written last year that he would not do it again. The Commission on Presidential Debates, however, convinced him to return to referee this year’s first matchup.

Romney, who by many accounts was more aggressive than Obama throughout the debate, routinely ignored Lehrer’s pronouncements about time limits.

When the moderator tried ending the first 15-minute segment in an effort to move to another topic, Romney cut him off.

“I get the last word of this segment,” Romney said.

Lehrer later told Obama, “Two minutes is up, sir,” but the president snapped back, “No, I think I had five seconds before you interrupted me,” and went on to finish a lengthy answer.

If Lehrer did not take enough criticism from the debate’s audience, he also got it from Romney.

In an answer on entitlement spending, Romney said he would cut funding to PBS, despite assuring Lehrer: “I love PBS, I love Big Bird. I like you, too, Jim.”

“I thought the format accomplished its purpose, which was to facilitate direct, extended exchanges between the candidates about issues of substance. Part of my moderator mission was to stay out of the way of the flow and I had no problems with doing so,” Lehrer told ABC News in an email.

“My only real personal frustration was discovering that ninety minutes was not enough time in that more open format to cover every issue that deserved attention,” he said.

The next two presidential debates will be moderated by CNN’s Candy Crowley, and CBS’s Bob Schieffer. A vice presidential debate will be moderated by ABC’s Martha Raddatz.