To that end, Buffett made Brooks its own separate unit in 2013. It had been part of sports equipment company Russell. Weber joined Berkshire(BRKB) when it bought Russell in 2006.

Randy Watson, CEO of boot maker Justin Brands, preaches the gospel of the brand as well. And he has the most experience of the three working with Buffett. He was with Justin when Berkshire bought it in 2000.

"You can lose money. But you cannot lose the integrity of the brand," Watson said.

Dairy Queen's John Gainor, Randy Watson of Justin Brands and Brooks Running's Jim Weber told us what it's like to work for Warren Buffett.

Still, Berkshire has put its brand name on more subsidiaries lately. That's why one shareholder asked at the meeting if Fruit of the Loom would be renamed Berkshire Undergarments. But it was a joke. It's not going to happen.

Here's the pool. Swim! All three CEOs said they have access to Buffett when they need it.

But they appreciate that Buffett wants them to truly be the boss of their operations. He's not a micromanager like the evil Lord Business in "The LEGO Movie."

"He allows you to run your own business," Watson said. "That puts more pressure on you to succeed."

Buffett celebrates 50 years of Berkshire

Weber added that "deserved trust is part of the culture." And Gainor said Buffett "empowers" him to run Dairy Queen as he sees fit.

Some years, Gainor may only need to talk with Buffett once or twice.

Don't worry about tomorrow. Berkshire is publicly traded. But it doesn't play Wall Street's game of trying to beat earnings expectations every quarter. Heck, it releases results in a dense SEC filing that always comes on a Friday after the market closes.

Gainor said he's learned to take "the long-term view" from Buffett. Weber added that the goal is to build something that will last for decades.

Weber said one of the most important things Buffett taught him was to focus on things you can actually do something about. He asked Buffett how to deal with volatile currency fluctuations a few years ago.

"Warren told me, 'Jim, I have no idea. I don't pay any attention to stuff that I can't control,'" Weber said.

Watson shared a similar story.

During the midst of the 2008 financial crisis, Buffett told him worrying about what would happen to consumer spending in the short term was a moot point. But what Watson could do was keep a close eye on inventories and not extend too much credit to customers.

Don't make me angry. You wouldn't like me when I'm angry. Buffett sounds like a pretty laid-back boss. But he must get mad sometimes? Throw a tantrum? Drop an F-bomb?

Gainor said he's never seen Buffett angry. So did Weber. But he added he's pretty sure what would tick Buffett off -- hurting the brand by selling cheaper sneakers.

Brooks tried that in 2001 and was almost bankrupted. Weber took over soon after to clean up the mess.

Watson also said he hasn't gotten on Buffett's bad side. And he wants to keep it that way.

"You know what makes your mother mad," he quipped. In other words, don't do whatever it is that could make them mad.