He argues for raising the Earned Income Tax Credit, which gives tax money back to those earning below a certain income level.

"I think you can accomplish way more through the earned income tax credit without negative effects," Buffett said.

He also said he doesn't really believe any studies on either side of the debate which try to estimate job losses from employers having to pay a higher minimum.

"It's very hard to quantify those trade offs," he said. "People come out with these exact studies. They don't know. Usually you just get proponents of the two sides pulling out figures that substantiate their positions."

Buffett is a strong proponent of the government doing more to address income inequality. "That's something a very rich country should address," he said.

He has long proposed a minimum tax for millionaires, as well as higher tax rates for top earners such as himself, a proposal known as the "Buffett rule."

Growing up a Buffett

Buffett said he once earned the minimum wage himself back in the 1950's, when it was at 75 cents an hour, while he was working at J.C. Penney(JCP). He said he doesn't have a figure for how many of the 330,000 employees at Berkshire Hathaway(BRKA) earn the minimum wage but that it's a very small number.

As for the broader U.S. economic outlook, he does believe that job growth will continue to pick up and says he's not worried about the U.S. falling into a new recession, despite some recent signs of a slowing U.S. economy.

"Exactly what's been going on since the fall of 2009 continues," Buffett said. "We've had this moderate but consistent growth. Every now and then we get excited about it speeding up and every now and then we start to worry about a double dip," Buffett said. "But I would say it's been almost a straight line, not at the kind of slope that people would like, but not flat either."