RICHMOND — Virginia Gov. Ralph Northam (D) met with his Cabinet early Monday morning and planned to speak with other administration officials as he considers resigning following two days of controversy over a racist photo in his medical school yearbook.

All 140 members of the General Assembly returned to Richmond for the ongoing legislative session, trying to focus on deadlines for approving a state budget in an atmosphere of uncertainty and drama.

House Speaker Kirk Cox (R-Colonial Heights) renewed his call for Northam to step down but in a morning news conference expressed “hesitation” about the possibility that the legislature would try to force him out.

“I think there’s a rightful hesitation about removal from office,” Cox said. “Obviously you have to consider that to some degree you’re overturning an election. I think the constitutional provisions are very specific . . . it really does call for mental or physical incapacitation.”

Cox, who worked closely with Northam in last year’s session to pass Medicaid expansion, called this a “painful and heartbreaking moment for the commonwealth.” He said Republican leaders had resisted calling for resignation until later Saturday.

“It had become clear to us by then, regardless of the veracity of the photograph, the governor’s lost the confidence of the people and cannot effectively govern,” Cox said. The legislature’s Democratic caucuses were both quick to call for Northam to step down.

The drumbeat spread to the state’s public universities. The College of William and Mary on Monday announced that Northam would not attend Friday’s inauguration of new president Katherine Rowe, saying in a statement that “the Governor’s presence would fundamentally disrupt the sense of campus unity we aspire to and hope for with this event.”

University of Virginia president James Ryan issued a statement Sunday suggesting that Northam should resign, saying that if a leader’s “trust is lost, for whatever reason, it is exceedingly difficult to continue to lead. It seems we have reached that point.”

On Sunday night, the governor met with senior staffers of color to discuss his future following two days of defiance against the national clamor that he should resign. People familiar with that meeting said Northam had not reached a decision.

It was unclear who was present, but the group did not include Lt. Gov. Justin Fairfax (D), who would become governor if Northam resigned, the people said. One Democratic official said the meeting was emotional in tone.

Calling the Sunday night meeting was a clear signal of Northam’s effort to weigh support within the administration as he evaluates his options. Although he pledged Saturday to stand his ground, he also said he would reconsider if he thought he could no longer be effective.

Northam spent much of Sunday in seclusion at his family home on the Eastern Shore as confidants delivered conflicting advice about whether he should resign or continue fighting to clear his name, according to people close to the governor.

Some of his strongest allies in Virginia’s Democratic Party took their calls for resignation to the national airwaves Sunday, arguing that the discovery of a racist photo on his 1984 medical school yearbook page had made him unable to continue to lead the state.

Even within the circle of his staunchest defenders, there was debate about whether he should step down, according to one person familiar with the situation who, like others interviewed, spoke on the condition of anonymity. At least one senior staffer and one Cabinet member have reached out to private firms about job possibilities, according to a person who was approached by the two.

One of those urging Northam to stay and defend his honor is first lady Pam Northam, according to two people familiar with the situation.

Some supporters worked Sunday to track down evidence about the yearbook photo that they thought would prove Northam was not in the racist image, which showed two figures, one in blackface and the other in Ku Klux Klan garb, each holding a can of beer.

The uncertainty over what Northam would do next has left Richmond in limbo. No Virginia governor has failed to complete his term since the Civil War, and the current crisis comes as the state officials would rather focus on two big anniversaries — the arrival of the first Africans and the first meeting of representative government, both in 1619.

Lawmakers are working on the state budget in a somber climate of rumors and dismay. Northam is liked by Republicans and Democrats and had compiled a glittering list of accomplishments in his first year in office, but sentiment around the legislature was almost universal that he should step down.

Northam initially apologized Friday night after the racist image surfaced on the conservative website Big League Politics but reversed himself Saturday at a nationally televised news conference and insisted he was not in the photo.

After the news conference, in which he disclosed another incident in which he wore blackface to imitate Michael Jackson at a dance contest in 1984, Democrats in the General Assembly and in Virginia’s congressional delegation intensified their calls for him to resign.

Northam stood his ground in calls to longtime friends and colleagues Saturday night, saying he intended to stay and fight, according to three people who received those calls. Among them was state Sen. Richard Stuart (R-Stafford), a close friend and fishing buddy.

Stuart did not want to disclose what the governor told him. “But I can tell you I firmly believe that he is going to do everything he can to clear his name and do what he thinks is right in the face of this. I don’t think he’s going anywhere,” Stuart said.

Northam’s only public appearance Sunday was at his home church in Capeville on the Eastern Shore. The pastor at First Baptist Church, Kelvin Jones, is African American and had been in Richmond the day before to support Northam at the news conference.

Northam returned to Richmond on Sunday afternoon. A small circle of staffers and trusted advisers has been helping him handle the crisis, including chief of staff Clark Mercer and Brian Coy, a former spokesman who was a holdover from the administration of former governor Terry McAuliffe (D).

Fairfax, who had begun making preparations to take over as governor on Saturday when it seemed as though Northam might resign, spent Sunday with his family in Northern Virginia and returned to Richmond in the evening.

Party leaders, concerned that Northam will become a liability for 2020 Democratic hopefuls and threaten Democratic control of an important swing state, have been unrelenting in their demands that he step aside.

McAuliffe, who helped Northam win the state’s top office and under whom Northam served as lieutenant governor, said on Sunday that he was “heartbroken” over the discovery of the photo. But he said Northam’s insistence that he was not in the picture was irrelevant.

“It doesn’t matter whether he was in the photo or not in the photo at this point,” McAuliffe said on CNN’s “State of the Union.” “We have to close that chapter.”

McAuliffe, a possible 2020 presidential contender, also defended Northam, saying Northam will be remembered for doing some great things, including helping McAuliffe restore the voting rights of Virginians who completed their sentences for felonies. Many of the people whose voting rights were restored are African American.

But McAuliffe said part of Northam’s legacy should be choosing “the right moral course for Virginia” and resigning.

Host Jake Tapper noted that the nickname “Coonman” appeared in Northam’s Virginia Military Institute yearbook and cited other examples before asking McAuliffe if Northam is a racist.

On NBC’s “Meet the Press,” Rep. A. Donald McEachin (D-Va.), speaking as a member of Congressional Black Caucus leadership, said there is nothing Northam can do to persuade McEachin to give the governor more time in office.

“He’s lost the authority to lead,” said McEachin, who served in the Virginia Senate with Northam. “He’s lost the authority to govern. He has to resign. It’s in the best interest of the commonwealth. It’s in the best interest of the party.”

One Democratic official said the state party has had no contact with Northam since calling for him to step down on Saturday. Others said that Northam has lost the ability to lead even if he is not in the original photograph because of the way he flipped his story and because of the disclosure involving the dance contest.

The question of how the photo could have turned up on Northam’s page in the 1984 yearbook from Eastern Virginia Medical School without Northam’s knowledge remained a central mystery.

William Elwood, who helped lay out the school’s 1984 yearbook, said each student was responsible for submitting photos for his or her page in a sealed envelope.

“Every student would provide them — some big envelopes, some small — with their names written on the outside, and we would keep them sealed until it was time to design that student’s page,” recalled Elwood, 68, a retired family practice and emergency room doctor in Mississippi. “I don’t remember exactly after, but I think they were sealed up and given back” to the students, he said.

Elwood said he had served a tour in the Navy before college and so was 29 and more apt to hang out with a group of older students at Eastern Virginia. He said he didn’t know Northam or remember whether he had designed Northam’s yearbook page.

But the yearbook, he recalled, was a laborious project for seniors. Elwood had the time to help, he said, because he had taken a four-year path through medical school, giving him a lighter load his senior year.

Elwood said that in looking at the photo in recent days, he thinks it was probably done in jest. “People back then put on all kinds of costumes at parties,” he said. “I think it probably is a thing of humor more than politics or racist.”

But there was no excusing the photograph for Northam’s fellow Democrats, especially members of the Virginia Legislative Black Caucus. Lamont Bagby, chair of the caucus, said despite his repeated calls for Northam to resign, impeachment isn’t publicly on the table yet.

“I encourage the governor to step aside so that we can start the healing process,” Bagby said on ABC’s “This Week.” “I’m not at a point where I want to publicly have a conversation about impeachment.”

Former Virginia congressman Jim Moran, a Democrat, emerged Sunday as virtually the only person defending Northam and encouraging the governor to ride it out.

“I hate to be on the other side of virtually all of my friends on this,” he said on “This Week.”

“But I do disagree with their judgment because I think it is a rush to judgment before we know all of the facts and before we’ve considered all of the consequences,” Moran said.

Moran said Northam should be given a chance to redeem himself and invoked President Lyndon B. Johnson’s ability to work with conservative Republicans because of his background in Texas.

“No untarnished liberal from the North could ever have gotten the Great Society programs passed,” Moran said, “but he was able to work with his Southern colleagues because he knew where they were coming from. We still have a conservative Republican legislature and, frankly, I think that Ralph will have the highest motivation possible to bring us further away from this horrible past of racism.”

Then Moran brought up Robert C. Byrd, the late senator from West Virginia, who was a member of the Ku Klux Klan in the early 1940s but, after many decades in the Senate, helped set aside land on the Mall for a memorial to the Rev. Martin Luther King Jr.