If Jeb Bush decides to run for president, he’ll have to defend his family’s controversial political name.

But more immediate family concerns might keep him from running at all.

Republican donors and operatives are chattering about Bush’s publicity-shy wife, so worried she isn’t on board with a 2016 White House run that they’re urging people in the family’s orbit to make the case.

Columba Bush has long been deeply averse to the spotlight, especially after an embarrassing encounter with U.S. Customs while her husband was still in office.

Donors also wonder whether Bush is willing to subject his family and their personal lives to the inevitable scrutiny that comes with a national campaign. Two of his children have been in the news in past years for arrests linked to drug problems and public intoxication.

So while the buzz picks up around Bush, the donor class’s new questions about old issues are a sign of not only the dynamics of a potential Bush campaign, but of how seriously a key establishment group is taking the idea of a Bush bid.

“The issues have been brought up by the press and others in the past, and the family issues may well be brought out again,” said Al Cardenas, the chairman of the American Conservative Union, who headed the Florida GOP while Bush was governor. “If these matters are brought up, it would hurt him, not politically but personally. He has to make up his mind whether that’s a burden he wants to bear.”

A top Republican bundler with ties to the Bushes described the rising concern among those lobbying Jeb Bush to enter the race that his wife’s opposition will keep him from running.

“One of the things on the list of why he won’t run has always been the belief that she is opposed to it,” the bundler said. “As you’re moving closer to when people have to make decisions and Jeb is doing some of the things he has been doing, there has been more chatter around it, if for no other reason than there is a huge number of Bush Rangers, Pioneers — whatever the hell we were called along the way — who remain effectively frozen waiting on Jeb to make up his mind.”

People in the Bushes’ inner circle are being lobbied to make the case directly to Jeb Bush’s immediate family, the bundler added.

The former governor’s concerns about his family’s privacy are thought to have played a role in his decision to skip the 2012 presidential race. One Republican donor recalled that the chatter back then among GOP rainmakers was that Bush would ultimately skip a run because his wife didn’t relish the idea.

“They thought he wouldn’t do it because of Columba…[that] has not changed,” the source said.

Bush, the son of one president and the brother of another, would be a favorite with the GOP establishment, though he faces steeper odds among grassroots activists who detest his moderate views on immigration. He has said that he will likely make a decision on whether to run for president by the end of this year, and that he will pursue it only if his family signs off and if he can run “joyfully.”

Comparisons are often drawn between Bush and former Indiana Gov. Mitch Daniels. The latter was popular with Republican elites in 2012, but ultimately chose not to run for president, citing the “interests and wishes of my family” amid a complicated situation with his wife and children.

Bush’s spokeswoman did not offer comment for this story. But just last week, Bush spoke movingly of his 40-year-marriage and his relationship with his wife, whom he first met while on an exchange program in Mexico, her native country.

“I saw her face, and I fell in love. I can’t explain it, but that’s the way it is,” the 61-year-old said at a Connecticut GOP dinner last week. “You’re going to have to trust me on this one. And for 40 years we’ve been married and it’s (been an) incredible joy. She doesn’t get much attention in the political world because she’s pretty normal, actually…”

He added,“Forty years of marriage for me is a big darn deal, and I love her very much.”

Ann Herberger, who has served as a Bush family fundraiser and remains close with the former governor and his family, said Jeb Bush would no doubt make sure his immediate family is on-board with a run, while also talking to his parents and other close relatives.

“Gov. Bush, he has a tough spine,” Herberger said. “Really the first hurdle he’s got to get over is, he’s got a very good life now back in Florida, Mrs. Bush is loving being back in South Florida. Can he do this joyfully?”

He got a taste of the political piling-on that often confronts presidential contenders last week, after saying that illegal immigration could be viewed as an “act of love” — an attempt to keep families together — rather than a “felony.”

The comment sparked fury among the conservatives who would need to tolerate him in order for him to win his party’s nomination. That immigration remark, along with his “joyful” condition for running for president, raised questions in some corners about whether he’s ready for — and enthusiastic about — prime time.

On the stump, Jeb Bush will likely have to counter questions about his family’s role in U.S. political history. His brother George W. Bush’s poll numbers were in the basement by the end of his second term amid a floundering economy and two wars, although he has seen a recent revival in his approval ratings.

And his mother, Barbara Bush, has even said she isn’t keen on seeing Jeb in the White House, though she later walked that back.

The troubles encountered by Jeb Bush’s immediate family also have received significant press coverage over the years.

In 1999, Columba Bush —then the first lady of Florida — catapulted into the spotlight when it became clear that after a trip to Paris, she declared $500 worth of goods at customs even though she actually made close to $20,000 in purchases. She apologized and paid significant fines, but the issue stayed in the news for days.

“I can assure you it was a difficult weekend at our house,” Bush said of his wife at the time. “She knew that what she did was wrong and [she] made a mistake.”

He noted that Columba “is uncomfortable with the limelight, which is why I love her.” She further shied away from the press after that incident, according to contemporaneous accounts. And donors who’ve seen her within the last year describe her as still extremely withdrawn in political settings.

In 2002, Bush’s daughter Noelle —then in her mid-20s— cropped up in media reports for months after being arrested on drug-related charges. Columba Bush said in a Washington Post story the following year that “absolutely” her daughter’s situation was worsened by coming from a famous family. Columba Bush went on to speak out publicly against drug abuse.

And in 2005, Jeb Bush Jr., then 21, was arrested in Texas on Austin’s rowdy Sixth Street for public intoxication and resisting arrest. (He is now a father and a champion of immigration reform).

“His family concerns are well-documented, relatively public, the struggles his daughter has had, and he dealt with them as governor and since then,” said Tony Fratto, who served as a deputy press secretary to George W. Bush, among other roles in the administration. “How a race impacts your family, whatever their conditions are, a lot of the same issues he may be considering are the same as lots of other [potential] candidates.”

Jeb Bush has support from at least one prominent member of his immediate family: donors say that his son George P. Bush is among his dad’s strongest 2016 advocates. When the younger Bush — who is running for land commissioner in Texas — encounters people expressing enthusiasm about a Jeb presidential bid, he urges them to encourage his father directly to get into the race.

If Bush were to run, he could face a range of possible 2016 candidates —from former Secretary of State Hillary Clinton to Republican Sen. Rand Paul of Kentucky, son of libertarian icon Ron Paul — who also have relatives in the public eye.

The difference between Bush and other potential GOP candidates, Fratto said, is that the Florida governor knows what he would be getting himself into if he chooses to run, after seeing the process up close with his father and brother.

Bush and Clinton, a former first lady and the leading Democratic presidential possibility, “have been heavily scrutinized for decades,” he noted. “You have two families here where it’s hard to come up with any more secrets about them. In a way it’s a luxury that they can go into a race without worrying about what kinds of things people might try to dig into their past and unearth…If Jeb runs, and if Hillary runs, neither is going to be shocked at that level of critical commentary, scrutiny and investigation into their pasts.”

Politically speaking, said Cardenas of the American Conservative Union, “if there’s anyone who’s been battle-tested in terms of scrutiny, it’s Jeb Bush.”

“He has been fully vetted nationally. That augurs well for those who would consider supporting him if he does run for president,” said Cardenas, who has known the Bushes for 35 years. “In other words, there’s no potential for crisis.”