HAVANA, Cuba (CNN) -- Fidel Castro, the longtime Communist leader of Cuba, met with visiting members of the U.S. Congressional Black Caucus on Tuesday, a day after his brother, Raul, who succeeded him as president, did the same, according to a U.S. official in Havana.

U.S. Rep. Barbara Lee, at left with Reps. Marcia Fudge and Mel Watt, was among those who met with Castro.

The meeting with Fidel Castro, 82, comes amid speculation that the United States is considering a shift in relations with the Communist nation that sits just 90 miles from the Florida Keys.

Upon returning to the United States, members of the caucus said it's time to consider an end to the trade embargo and other diplomatic restrictions placed on Cuba for the past five decades.

"Yes, we have history. We have good history and not-so-good history," said Rep. Laura Richardson, D-California. "But the point is, it's history, and we need to move forward."

Three members of the caucus were visiting the Latin American School of Medicine, where students from nations including the United States study, when they were invited to meet with the senior leader.

"Former President Fidel Castro is very engaging, very energetic," said caucus Chairwoman Barbara Lee, also a Democrat from California. "Our conclusion is, given the new direction in our foreign policy, that it's time to look at a new direction in our policy toward Cuba.

"The 50-year embargo just hasn't worked," she said.

There was no immediate response from the Obama administration on the meeting. But asked earlier about Monday's meeting with Raúl Castro, State Department spokesman Robert Wood declined to say what, if any, role it could play in a possible warming of a decades-long diplomatic and economic freeze.

"Members of Congress have the right to travel where they want and to discuss issues with whom they want," Wood said. "And I am sure members of that delegation will be raising some of the concerns that the U.S. government has with Cuba, in terms of allowing Cubans to have some of the same rights and freedoms as other countries in the hemisphere."

Obama has said he is in favor of changing the relationship with Cuba but has not offered specifics. Government officials have hinted that he may soon lift travel restrictions between the two countries.

In a letter published Tuesday in the online version of Granma, a state-run Cuban newspaper, Fidel Castro wrote that an unnamed caucus member told him "he was sure that Obama would change Cuba policy but that Cuba should also help him."

He noted that the group was in Cuba during the anniversary of the Rev. Martin Luther King Jr.'s April 4 assassination and visited a center dedicated to King.

"I value the gesture of this legislative group," Fidel Castro wrote. "The aura of Luther King is accompanying them. Our press has given broad coverage of their visit. They are exceptional witnesses to the respect that U.S. citizens visiting our homeland always receive."

The United States broke diplomatic ties with Cuba in 1961, two years after Fidel Castro took office. The following year, the U.S. government instituted a trade embargo. Both policies still remain in effect.

Early last year, and with his health failing, Fidel Castro announced that he was resigning from the presidency. The Cuban National Assembly appointed Raúl Castro to the post days later.

Fidel Castro led the revolution that, in 1959, overthrew Cuba's Batista dictatorship. He was credited with bringing social reforms to Cuba but criticized internationally for oppressing human rights and free speech.

Despite widely documented health problems the past few years, including intestinal surgery in 2006 that required him to turn power over to Raúl while he recovered, Fidel Castro seemed to be largely in good health, the delegation said.

"Of course, he has been ill, but I think we will agree he was very healthy, very energetic and very clear thinking," Lee said. "He was very engaging."

Two caucus members, Reps. Emanuel Cleaver of Missouri and Bobby Rush of Illinois, preached sermons at two Cuban churches to commemorate the anniversary of King's assassination.

While in Cuba, the delegation also met with family members of the so-called Cuban Five, a group imprisoned in the United States after being convicted of spying. In 2005, a three-judge appeals court panel overturned the convictions, but the full court later reversed that decision.