NEW YORK (CNN) -- Prodding Israel and the Palestinian Authority to restart talks aimed at a permanent resolution of their decades-old conflict, President Obama dropped a demand for an Israeli settlement freeze, U.S., Israeli and Palestinian officials said.

Benjamin Netanyahu, left, President Obama and Mahmoud Abbas.

"Simply put, it is past time to talk about starting negotiations. It is time to move forward," Obama told reporters before a meeting with Palestinian President Mahmoud Abbas and Israeli Prime Minister Benjamin Netanyahu.

"It is time to show the flexibility and common sense and sense of compromise that is necessary to achieve our goals," Obama said.

Obama first met separately with Netanyahu and Abbas on the sidelines of the U.N. General Assembly in what he called "frank and productive" talks. The session was the first among the three leaders since Obama took office in January.

George Mitchell, Obama's envoy for the Middle East, attended Tuesday's talks. He said the talks were "at all times cordial," but "direct" and sometimes "blunt."

Obama told Abbas and Netanyahu that, "The only reason to hold public office is to get things done," and that everyone "must take risks for peace," Mitchell said.

Mitchell told reporters the U.S. is "not identifying any issue as being a precondition or an impediment to negotiation."

But the United States and Israel have publicly disagreed on Israeli plans to build more housing on land the Palestinians regard as theirs. Previous Obama administration demands for a freeze have been ignored by the Netanyahu government. Watch Netanyahu address peace, settlements »

Abbas has so far rejected resuming talks with Israel until the Jewish state halts all settlement building in the occupied West Bank and in predominantly Palestinian East Jerusalem. Arab states also have balked at the U.S. request to take confidence-building measures toward Israel until Israel freezes settlement construction.

Sitting with Netanyahu and Abbas, Obama softened his regular language on a settlement "freeze," saying that Israel has had meaningful discussions about "restraining" settlement activity.

Don't Miss

"But they need to translate these discussions into real action on this and other issues," he said.

"Obama told Abbas that he couldn't get the settlement freeze and promised to keep trying, but that it shouldn't be a condition for talks and it was time to move on," one Palestinian aide to Abbas said.

Several U.S. officials said that Obama told Abbas that although the U.S. believe a settlement freeze would create a better atmosphere for talks to begin, the lack of one should not be used an as excuse not to talk.

"It's been apparent for some time that the Israelis were going to fall short of what is necessary on the settlement freeze," one senior U.S. official said. "But our view is let's get to negotiations and settlements will be addressed within those final status issues"

A senior Israeli diplomat said Israel agreed to not building any new settlements, no outward expansion of existing growth and to only build for "natural" growth within existing settlements. He said Israel also agreed not to encourage Israelis to move to settlements, which would increase the population.

"A complete settlement freeze wasn't physically or politically possible, especially in the absence of any Arab or Palestinian concessions," the Israeli diplomat said. "There was a time the gaps had significantly narrowed, but now they were starting to widen. The administration recognized, rather than have them widen further, we should start negotiating."

The Palestinian aide to Abbas said Obama's failure to secure a settlement freeze has weakened him in the eyes of the Palestinian delegation, because it casts doubt on his ability to move Israel during final status negotiations.

The Palestinians did win an important point, however, with Obama making clear that the peace talks would not start from scratch, which Netanyahu had favored. Rather the talks would take into account progress made in previous negotiations between Israelis and Palestinians, the sources said.

The Israeli diplomat said that Israel would "take into account" the previous negotiations, but stressed his country's longstanding position that "nothing is agreed to until everything is agreed to."

Obama also addressed Abbas' desire to have the terms for negotiations more clearly defined, asking Israeli and Palestinian negotiators to come to Washington next week for further talks.

Next week's talks, to be led by George Mitchell, will center on what issues will be addressed at negotiations, which will present the U.S. with its next big challenge. Palestinians want negotiations to resolve the "final status" issues of the conflict, such as borders, Jerusalem and the right of return for Palestinian refugees. Israel wants to begin with talks on a "provisional" Palestinian state.