You know times are tough when a banker pulls a knife on a cabby over the fare.

A city investment banker has been busted on charges of stabbing a taxi driver with a penknife and making racial slurs in a fight over the fare from Manhattan to his multimillion-dollar Connecticut home, cops said yesterday.

But Morgan Stanley financier W. Bryan Jennings claims the hack tried to “kidnap” him back to the Big Apple after arriving at his Darien house.

“His arrest for these charges is absolutely wrong,” said Jennings’ lawyer, Eugene Riccio, after his client surrendered on Wednesday on charges including assault and intimidating the Middle Eastern-born driver with racial slurs during the Dec. 22 fare fight.

The lawyer said that the driver had “abducted’’ his client and that it is “mind-boggling’’ that the cabby hasn’t been charged. Riccio also denied that Jennings, 45, made racial slurs.

But Darien Detective Cmdr. Ronald Bussell said Jennings became enraged and shouted the slurs when the cabby insisted he pay the $203 fare they had agreed on. When Jennings refused, the hack drove to nearby Post Road to find a cop, Bussell said.

“As he’s driving down there, the victim says, the passenger tries to stab him with a knife. He puts up his hands and gets stabbed in the hand,” said Bussell, adding the driver got six stitches.

Jennings then fled from the cab.

“Poor guy,” Bussell said of the driver. “He had to pay the cab fare” to the car’s owner because Jennings bolted.

But Riccio said the driver would not let Jennings out of the cab and “told him he was going to take him back to New York City.

“The driver had locked the doors, and by the time my client was able to get the doors unlocked, the cab was already pulling away from the home at a high rate of speed,” Riccio said.

Jennings took out a knife “in an attempt to get the driver to stop and let him out . . . and the cabby attempts to grab the knife from my client. At that point, my client is able to escape and literally runs home.”