(CNN) -- President Obama's friendly interactions with Venezuelan President Hugo Chavez has set off a wave of controversy, but analysts said the leaders' next steps will show if relations have truly improved or if Obama was overstepping boundaries.

Venezuelan President Hugo Chavez presents a book to President Obama at the Summit of the Americas.

Chavez and Obama were both at the Summit of the Americas, which ended Sunday in Trinidad and Tobago.

On the first day, Obama reached out his hand to Chavez and was seen smiling and patting the Venezuelan leader on the shoulder as the two shook hands.

"With this same hand I greeted Bush eight years ago, I want to be your friend," Chavez said, his office reported.

On the second day, as cameras jockeyed for position, Chavez got up and gave Obama a book, "The Open Veins of Latin America," which documents centuries of American abuse in the region. After the exchange, the book skyrocketed to become one of Amazon's best-sellers.

Obama joked about the move at a news conference Sunday, saying while he had meetings with all the leaders involved, "I think it's just that President Chavez is better at positioning the cameras."

"And in all these conversations, here's what I emphasized, that we're not going to agree on every issue, but that, as long as we are respectful of democratic processes, as long as we're respectful of principles of sovereignty for all nations, that we can find areas where we can work in common," he said.

Don't Miss

Relations between the United States and Venezuela grew progressively worse under the presidencies of Chavez and George W. Bush.

Chavez -- whose anti-U.S. rhetoric has included calling Bush the "devil" -- announced Saturday he is considering naming an ambassador to the United States, signaling a potential shift in the tense relations between the two nations.

Chavez expelled the American ambassador in September, prompting the United States to expel Venezuela's ambassador.

Sen. John Ensign, R-Nevada, told CNN on Sunday it was "irresponsible" for Obama to have been seen "laughing and joking" with Chavez.

"This is a person who is one of the most anti-American leaders in the entire world," Ensign said on "State of the Union." "He is a brutal dictator, and human rights violations are very, very prevalent in Venezuela. And you have to be careful." Watch criticism of the meeting »

Republican Newt Gingrich joined the criticism Monday, saying enemies of the United States will use Obama's friendly encounter with Chavez as propaganda.

"Everywhere in Latin America, enemies of America are going to use the picture of Chavez smiling and being with the president as proof that Chavez is now legitimate, that he is acceptable," the former House leader said on NBC's "Today" show.

But Democrats such as Sen. Amy Klobuchar of Minnesota downplayed the moment, saying, "All the president did was shake his hand like George Bush [did]."

Obama's senior adviser said Sunday the administration isn't worried about how the gesture is perceived.

"I'm not concerned about the message that it sends. I'm concerned about what flows from it. Words and handshakes are nice, but they're not enough," David Axelrod said on CBS' "Face the Nation."

Stephen Hayes, a CNN contributor and writer for the conservative magazine The Weekly Standard, said it's not the handshake that irks Republicans but that Obama "seemed to actually be enjoying it."

"Conservatives at least were looking for something from the president who would have said, 'Hey, enough is enough, we want to go forward. We want to put the arguments of the past in the past, but you've got to own up to the arguments you've made,' " Hayes said Monday on CNN.

Jennifer McCoy, director of the Carter Center's Americas Program, said Obama was trying to send a message that he wants to start a new beginning with Latin America.

"So President Chavez is also responding to this new atmosphere, this new style and attitude that Obama is presenting and needed to respond in kind," she said.

McCoy, who briefly spoke with Chavez at the summit, said he told her he is ready to give Obama a chance and see if they can have a more cooperative relationship.

But she said, "I don't expect it's going to be smooth sailing. President Chavez's whole foreign policy is predicated on challenging U.S. leadership."

Obama went into office arguing that relations with Latin America had turned sour and that the U.S. needed to form new partnerships.

David Gergen, CNN's senior political analyst, said Obama is trying to make good on that pledge.

"I think most political advisers would tell the president, you know, it's fine to shake hands, hold the smiles," he said, adding that the gesture shows some inexperience on Obama's part.

"What the real test here is going to be in policies and in the actions. And I think Barack Obama has to somehow make a balance between being open, reaching out and also not surrendering or retreating on basic American principles and on showing some toughness," Gergen said.

William Bennett, a CNN contributor and Republican strategist, agreed it's too early to tell if anything has changed.

"Image is one thing. Reality is another. Substance is another. Things that need to get accomplished is another. That will be the test," Bennett said.

Obama said Sunday he didn't know how the images from this weekend would play politically, and he indicated he really didn't care.

"One of the benefits of my campaign and how I've been trying to operate as president, I don't worry about the politics. I try to figure out what's right in terms of American interest, and on this one I think I'm right," he said.