The index remained in strongly expansionary territory at 59.3 despite a drop from last month's 60.8 reading. However, it missed an expectation of 60.1. (When the index is above 50, it indicates US manufacturers are expanding.)

"Many comments reflected the negative impact of the Asian holiday period, concerns about tariffs and difficulties in moving containers from ports to using locations," Timothy Fiore, Chair of the ISM Manufacturing Business Survey Committee, said in the press release for the survey.

Respondents in industries ranging from chemicals to machinery expressed worries over the tariffs, but many said changes have not hit their businesses yet. Here's a few responses from the survey regarding the tariffs:

An executive in chemical products: "We are concerned about the impact of tariff and trade wars on demand, but at this time, [there are] no signals that global demand is slowing,"

An executive in primary metals: "Significant price increases in the steel commodity due to 232 [the tariffs]. The price increases will begin to impact our company's performance."

A executive in machinery: "Much concern in the industry regarding the steel and aluminum tariffs recently [imposed]. This is causing panic buying, driving the near-term prices higher and [leading to] inventory shortages for non-contract customers."

Of particular interest is how the tariffs will push up prices, since the additional tax on imports of raw materials will likely increase input costs for many manufacturers that rely on metals or Chinese goods. According to the ISM index, prices jumped in the month of March and are already on the rise.

"The Prices Index registered 78.1 percent in March, a 3.9 percentage point increase from the February reading of 74.2 percent, indicating higher raw materials prices for the 25th consecutive month," the survey said.

Based on the responses, executives for firms directly hit by the metals tariffs are already seeing serious consequences. For others, concerns about the second-order effects, including possible retaliation from other nations, is the dominant concern.