NASHVILLE, Tenn. (AP) — Tennessee’s Haslam family is furiously trying to control the damage following a federal investigation into the family business that could threaten to unravel decades of growing wealth and influence that spans business, sports and politics in the state and beyond.

Jimmy Haslam, the CEO of Pilot Flying J, owns the NFL’s Cleveland Browns and recently sold his small holding in the Pittsburgh Steelers. His brother Bill is governor of Tennessee, and their father, Jim, has been a prominent GOP fundraiser for presidents and senators. The family has given heavily to philanthropic causes in the region and elsewhere.

“It would be hard to think in Knoxville of a family that has been more active and more consistently generous and supportive of their fellow citizens in all walks of life,” said Victor Ashe, a former Knoxville mayor and former ambassador to Poland.

“That’s what makes this all the more unfortunate,” he said.

Jimmy Haslam’s wealth is estimated at $1.8 billion by Forbes magazine, which also ranks Pilot Flying J as the sixth-largest privately held company in the country. The company’s estimated revenues of $31 billion come within $2 billion of Tennessee’s annual budget plan.

An affidavit released after an April 15 raid of the company’s headquarters by FBI and Internal Revenue Service agents alleges that sales employees withheld rebates owed to customers so they could boost profits and pad their commissions. The affidavit indicates the practice was widespread and, according to Pilot employees, was known to the top executives.

At first, Jimmy Haslam tried to downplay the criminal investigation. In his most recent public statement this week, he struck a far more serious tone.

“That was the most painful — and still is — 48 hours that I’ve ever experienced in business,” he said. The CEO has said he won’t step aside from the Browns or the company, but he announced an audit of customer contracts, the suspension of several salespeople and personally appealed to customers who Pilot workers belittled in often-crude language.

“We make mistakes like any company does, but there is absolutely no excuse for that kind of behavior,” Haslam said. “I don’t think I’ve ever been as embarrassed as I have been since I read the affidavit.”

No charges have been filed in the case as the investigation continues.

NFL Commissioner Roger Goodell said he met with Jimmy Haslam on Tuesday to discuss the federal investigation into the truck stop chain.

“Jimmy’s doing everything he’s asked. He’s cooperating. He wants to make sure that he’s doing all the right things in that regard, and he’s assured us he’s going to,” Goodell said at a pre-draft event in New York.

The Haslam family’s reputation stretches back to the football dynasty of Gen. Robert R. Neyland at the University of Tennessee, and their political connections are deep.

Company founder Jim Haslam played tackle for the 1951 national championship team under Neyland, who built the Volunteers into a football powerhouse. The family has been a major financial backer of the state’s flagship public university and its athletics programs ever since.

The elder Haslam founded the Pilot Corp. in 1958 with a single gas station in Gate City, Va. He credits his sons with expanding the chain from mostly gas stations and convenience stores to a “travel center” concept featuring branded fast-food service, plentiful truck parking, clean showers and other services.

Under Jimmy Haslam’s leadership, the company acquired its closest rival, Flying J, and has become the country’s largest retailer of over-the-road diesel fuel.

The Haslam family had long been active behind the scenes in Republican politics, mostly as campaign donors. Jim and Jimmy Haslam were among the top fundraisers for George W. Bush’s presidential bids and for other GOP candidates.

Jimmy Haslam has been close friends with U.S. Sen. Bob Corker, R-Tenn., since they were roommates at the University of Tennessee, and their families have vacationed together at Haslam’s summer home on Nantucket, Mass.

The governor and Jimmy Haslam also lead the ownership group of the Tennessee Smokies, a double-A affiliate of the Chicago Cubs. Gary Wade, chief justice of the Tennessee Supreme Court, is part of the group.

Bill Haslam was the first in the family to run for office, narrowly winning the 2003 Knoxville mayor’s race. Seven years later he was elected governor. That race featured heavy criticism from both Democratic and Republican candidates about his refusal to divulge financial details about his stake in Pilot.

A spokesman said the FBI allegations haven’t changed the governor’s views on keeping his Pilot earnings secret or keeping his holdings in the company outside of a blind trust for his other investments.

Democrat Mike McWherter, who lost the 2010 governor’s race to Haslam, said the revelations emerging out of the Pilot investigation renew questions about the governor’s stake in the company.

“It would have been better for Bill Haslam in retrospect, with all of this information coming out, to have gone ahead with a full disclosure,” McWherter said. “Because people would know exactly what his relationship is to Pilot.”

Bond ratings agency Moody’s Investors Service has placed Pilot on review for a possible downgrade following the raid, which would make it more expensive for the company to borrow money. Meanwhile, Standard and Poor’s has said the raid would not change the company’s rating.

The rise and fall of another powerful pair of Knoxville brothers, Jake and C.H. Butcher, provides a cautionary tale for the Haslams. The Butchers’ grew a single bank started by their father in 1950 into an empire of 27 financial institutions in Tennessee and Kentucky, bringing them considerable power and influence.

Jake Butcher ran twice for governor as a Democrat, and the family members were major donors to others in the party. He reached the zenith of his power in his successful bid to bring the 1982 World’s Fair to Knoxville, where the 266-foot-high Sunsphere still stands as a faded icon to the event that was supposed to propel Knoxville into the top tier of Southern cities.

The Butcher businesses began to unravel on Valentine’s Day 1983 when the United American Bank in Knoxville was closed by regulators probing illegal banking practices.

One by one, the others closed under the weight of unsecured loans, paper corporations loaded with debt and a massive shell game in which loans were shuffled from one bank to another ahead of the bank examiners. It was the fourth-largest U.S. banking failure at the time.