Federal Reserve Governor Daniel Tarullo on Wednesday said policy makers should proceed cautiously in judging when inflationary pressures are building in the economy, given uncertainty that surrounds just how much slack remains in the labor market.

Mr. Tarullo placed himself in the camp of Fed Chairwoman Janet Yellen, saying he believes the labor market is still operating well short of its potential and associating himself with her March 31 speech explaining the reasons why.

Given there is some debate over how to measure labor market slack, “we are well advised to proceed pragmatically,” he said in a dinnertime speech prepared for delivery at a conference organized by the Levy Institute of Bard College.

He stressed Fed officials should await actual evidence that labor markets had tightened enough to trigger inflationary pressures that could push inflation above the Fed’s 2% inflation target. The Commerce Department’s personal consumption expenditures price index, the Fed’s favored measure of inflation, was up 0.9% in February from a year earlier. The Labor Department’s consumer price index, an alternative measure, was up 1.1%.

“But we should not rush to act preemptively in anticipation of such pressures based on arguments about the potential increase in structural unemployment in recent years,” he said.

There is a vigorous debate at the central bank and among economists generally over the extent of remaining slack in the labor market. Minutes from the Fed’s March 18-19 policy meeting released Wednesday showed that while officials generally agreed slack persisted, they disagreed about how much and how well the unemployment rate reflects the degree of slack.

In her March 31 speech, Ms. Yellen pointed to several factors beyond the jobless rate that suggest the labor market is still quite weak, including the large number of long-term jobless and the seven million Americans who are working part-time but would prefer full-time jobs.

Mr. Tarullo suggested he’s not worried economic growth will suddenly take off and leave the Fed flat-footed and fighting rising inflation. “The issue of how much structural damage has been suffered by the labor market is of less immediate concern today in shaping monetary policy than it might have been had we experienced a period of rapid growth during the recovery,” he said.

In light of the economy’s modest performance since the end of the recession, “it seems less likely that we will experience a growth spurt in the next couple of years that would engender concerns about rapid wage pressures and changes in inflation expectations,” Mr. Tarullo said.

Mr. Tarullo’s comments came within the context of a speech raising concerns about “troubling” long-term trends in the U.S. economy, including falling productivity growth and rising inequality.

The Fed’s efforts to battle recession help lay the groundwork for a stronger, more dynamic economy, Mr. Tarullo said. “But there are limits to what monetary policy can do in counteracting” the longer-term trends he is worried about.

Mr. Tarullo said the federal government could address some of the challenges through investment, especially in ways that help “those who have seen their share of the economic pie shrink.” Early childhood education, job training programs, infrastructure and research are areas that could boost the long-term prospects for the U.S. economy, said Mr. Tarullo.

About Real Time Economics

Real Time Economics offers exclusive news, analysis and commentary on the U.S. and global economy, central bank policy and economics. Send news items, comments and questions to the editors and reporters below or email realtimeeconomics@wsj.com.