Kentucky Derby: Kid Rock & K-Fed's Rap Battle

At the Kentucky Derby on Saturday, Kid Rock did more than just bet on the horses – he ponied up fatherly advice to Kevin Federline.

Federline, 29, should keep his sons with Britney Spears (Sean Preston, 1, and Jayden James, nearly 8 months), and his two kids with Shar Jackson (Kori, 4, and Kaleb, 2), out of the limelight, Rock told him.

"You never see me photographed with the kids," Federline replied, Rock tells PEOPLE.

Earlier that morning, however, it seemed like neither minded the attention. At a party at Louisville's Felt nightclub, sponsored by the New York hotspot Stereo and Pama liqueur, Rock took control of the DJ booth around 4:30 a.m. before engaging in a rap battle with Federline.

"Everybody wanna be a friend of mine – you, you, you, you, you, you and even Federline!" Rock rapped.

"I like calling that kid's name really loud," he joked later.

And while Rock, 36, called Derby Day "a father/son thing" (the musician has a teen son, Robert James Ritchie Jr., whose mother is Kelly South Russell, a Detroit auto worker), he also brought along his girlfriend, model May Andersen.

"She's my Danish sweetheart," he said. "One of the nicest girls I've ever met, and you know what her last name is: 'Andersen.' " (Last year, Rock famously married, then unmarried, Pamela Anderson.)

Of Friday, at the Barnstable Brown gala, Andersen gushed about her beau too, adding that their Derby date was his idea.

"He's great. He's such a nice person," she said. "We have fun and we support each other."

Another Bluegrass Country visitor, Vanessa Minnillo, also admitted that her trip to Derby wasn't necessarily her idea. "I'm here because of [Nick Lachey]," she told PEOPLE.

"I'm a sports fan, and it's a unique sporting event," Minnillo, 26, conceded. "And you get to wear a hat and hold a mint julep!"

So, how did she pick her Derby day outfit? "It has to match my date!" she exclaimed. "It's gotta coordinate."

At the Barnstable Brown bash, Jenny McCarthy, 34, said she selected her Dolce & Gabbana ensemble to match her hat, which was the winner of a contest held by Chrysler – and made entirely of Sebring materials.

One accessory she didn't' have: boyfriend Jim Carrey.

"It's the best relationship I've ever been in," she said. "It's not for the obvious reasons that people think of – 'Oh, you guys are just funny!' It's recognizing the much deeper level. We're both very spiritual people."

And no, the ring she was sporting on her left hand is not an engagement ring. "I bought it myself," she said. "This hand is fatter, so I had to slide 'em on this hand!"