WASHINGTON (CNN) -- Hispanic groups want history to be made with a Latino or Latina justice when President Obama makes his selection to fill the upcoming Supreme Court vacancy.

President Obama is expected to name his nominee to replace retiring Justice David Souter soon.

Women's rights groups hope Justice Ruth Bader Ginsburg does not remain the only female on the high court bench.

Some progressives, including some civil rights groups, privately would like to see the first African-American president choose a nominee of color.

Other constituencies -- Asian-Americans, gay and lesbian groups, religious leaders and the legal academic community -- also have their "wish lists."

The call from many on the left for diversity on the high court has energized much of the early speculation over whom the president will pick. Legal and political experts, including liberal groups themselves, say a female nominee is a pretty good bet.

"There is only one thing that is essential for this pick, probably: that she be a woman," said Thomas Goldstein, a leading appellate attorney and founder of scotusblog.com. "Beyond that, having the candidate be Hispanic would certainly be a plus, because it would add still more to the diversity of the Supreme Court. It would be historic in that sense, but it's not going to be determinative."

Diversity is not a word that easily equates with the Supreme Court. Of its 110 justices over the past 220 years, all but four have been white males. The exceptions are two African-Americans (Thurgood Marshall and Clarence Thomas) and two women (Sandra Day O'Connor and Ginsburg).

Don't Miss

"This alarmingly low representation of women in the Supreme Court is dwarfed by the representation of women in the lower courts, private practice, business, academia and politics," said Jennifer Maree of the Women's Bar Association of the District of Columbia.

A total of 212 full-time federal judges are female, about a quarter of the entire judiciary. But one need only go back a few years to get perspective on the advance that number reflects.

Before 1977, only eight women had occupied the federal courts. President Carter changed that to a large extent, adding 40 female judges, including Ginsburg to the District of Columbia's prestigious federal appeals court.

When President Reagan named Sandra Day O'Connor as the first woman on the high court, he had relatively few women with conservative judicial credentials from which to choose. O'Connor had been appeals court judge in Arizona when she was picked.

Her retirement in 2005 left Ginsburg the lone woman, a fact she has often and publicly lamented. Gender equality groups agree.

"When there is only one woman on the Supreme Court, it's not just that it doesn't look right, it's that the court can't decide right," said Marcia Greenberger, co-president of the National Women's Law Center.

Some Hispanic groups have been especially vocal in recent weeks. They have publicly and privately urged the White House to broaden its list of possible nominees to include more Latinos and Latinas, saying many qualified candidates serve in the judicial, political and legal arenas. Learn more about possible nominees »

"They're ready, and they're due, not because of what happened in the last election but because what's been going in the past 50 years and what will happen in the next 50," said Ruben Navarrette, a syndicated columnist and CNN.com contributor. He cited strong Hispanic support for Obama in the November election as a sign that members of that group are ready to have their voices heard with a high court appointment.

There is "always going to be another argument" against choosing a Hispanic, he said, "just as there were arguments against Sandra Day O'Connor. Barack Obama simply has to bite the bullet, pick the best Hispanic person and weather the storm."

Liberals see Carter as a benchmark for adding gender and ethic diversity to the federal bench, and they have been pushing the current White House to consider an aggressive diversity strategy for all federal vacancies. Because judges serve for life, they have the potential of enhancing a president's legacy for perhaps decades to come, especially on the Supreme Court.

Carter named 37 African-American judges and 16 Hispanics, in each case nearly double what previous presidents since 1961 had accomplished. And all in only four years in office. Will Obama follow the Georgia Democrat's lead?

"There is a lot at stake with this pick, both in terms of the Supreme Court's jurisprudence -- because you are putting someone on there that will be there for decades -- and also for its symbolism," Goldstein said. "It is going to be President Obama's biggest statement so far about his view about the law and about diversity and the future of the high court."

The two previous presidents, Bill Clinton and George W. Bush, had pretty good records on the judicial diversity front. Almost 30 percent of Clinton's choices were women, 16 percent were African-American, and 7 percent were Hispanic. Bush's numbers were 20, 8 and 10 percent, respectively.

But both of the justices Bush added to the Supreme Court were white men, including Samuel Alito, who replaced O'Connor, a move that disappointed many women, including conservatives. His attempt at gender diversity on the high court failed when Harriet Miers, named first to replace O'Connor, pulled out after strong conservative backlash over her judicial philosophy and qualifications.

Obama also could face resistance from his own party and with liberal activists.

"They are hanging their hopes on a really strong progressive choice for the Supreme Court," said A.B. Stoddard, a columnist with The Hill newspaper. "Because in the last couple of months, they have become dispirited and disappointed over a host of decisions the administration has made" on such issues as national security, assault weapon bans, gays in the military and abortion rights.

"This pick becomes more important, because they feel they have been let down. So he not only faces a fight with Republicans but tremendous pressure from all the interest groups on the left who think now is the time for him to make up for these disappointments."

A majority of Americans are not concerned about the personal characteristics of the next high court justice. In a CNN/Opinion Research Corp. poll last week, 39 percent of those surveyed thought it important to nominate a female to the bench. Only 26 percent said so for a Hispanic and 22 percent for a black justice. When just women were questioned, 43 percent said it is important to name a woman.

A broad range of progressive groups met privately with White House officials shortly after Justice David Souter announced that he will be stepping down from the court. The message from the White House was to ease anxiety over the process and to assure liberal leaders that their voices were being heard and that they would be pleased with the nominee.

But the White House has publicly cautioned about liberal groups going too far, even before the choice is announced.

"This is a decision that [Obama] alone will make," Press Secretary Robert Gibbs said May 12. "I don't think that the lobbying of interest groups will help. I think in many ways lobbying can and will be counterproductive."