President Donald Trump's tariffs will damage the economy to some extent, and if NAFTA talks break down it could be even worse, CNBC senior contributor Larry Kudlow said Thursday.

Earlier in the day, the president announced the U.S. will impose a 25 percent tariff for steel and a 10 percent tariff for aluminum as early as next week.

While it won't unwind all the good work Trump's done on his agenda, like tax reform, "it's a bad omen," Kudlow told "Closing Bell."

"He's so good on tax cuts. He's so good on deregulation, infrastructure. I even like him on immigration. He's never been good on trade," he added.

One big issue is the North American Free Trade Agreement with Canada and Mexico, which Trump has repeatedly threatened to leave. Both Canada and Mexico make steel, Kudlow pointed out.

While the tariffs may annoy them, it could be a different story if NATFA negotiations fall apart, he said.

"We're already hanging by a toenail on NAFTA. If we have to walk out of NAFTA or those negotiations totally break down, then this steel thing turns from a minor irritant to a major calamity for our economy and our stock market. Make no doubt about that," he warned.

Kudlow said he has expressed his anti-tariff views with Trump but this time the protectionists in the administration won.

Proponents of the tariffs argue they will help create jobs and boost the economy. Century Aluminum CEO Michael Bless told CNBC earlier Thursday that his company plans to hire 300 people and invest $100 million in its plant in Kentucky, thanks to the new tariffs.

However, while there are about 200,000 workers in the U.S. steel, aluminum and iron industries, there are 5.5 million people employed by businesses that use steel, Kudlow said.

There are also 154 million Americans who work, and they are going to be affected by price increases on products, thanks to steel price hikes, he said.

"In theory, if you raise the tariff by 25 percent, the price can go up by 25 percent," he said.