Law enforcement officers' ability to track those suspected of being involved with terror groups is hindered by concerns about possible racial profiling charges, R. James Woolsey, former director of Central Intelligence, told CNBC on Monday.

"We don't really have a very good system for staying on top of people who get reported unless they are shown to be violent," Woolsey said in a "Squawk Box" interview.

While the constitutional right to freedom of religion needs to be protected, those who show interest in using their faith to wage a holy war should not be, he added.

"[Officers] are scared of being charged with Islamophobia or something if they stay on top of somebody who happens to be a Muslim after there's some reporting until there's violence."

Justin Solomon | CNBC

Flowers are placed near the bombing site in Boston.

The Boston Marathon bombing has shed light on this difficult balance.

Investigators said that dead suspect, Tamerlan Tsarnaev, 26, and his wounded brother in custody Dzhokhar, 19, used social media networks to express radical views and visited a terrorist bomb-making website.

The older brother was interviewed by the Federal Bureau of Investigation in 2011 at the request of Russian officials worried he might have links to extremists. The FBI did not find any at the time, but there are questions now about whether U.S. officials should have done more.

"What happened is that there was no follow-up," said Woolsey, who led the CIA from 1993 to 1995.

He said that the same thing had happened in the case of Umar Farouk Abdulmutallab, the so-called underwear bomber, who was convicted of trying to bring down Northwest Airlines Flight 253 on Christmas Day in 2009.

"[Abdulmutallab's] father reported him to the U.S. Embassy in Nigeria, saying, 'My son has become a real radical—you ought watch him,' " Woolsey said. "It fell on deaf ears. Nobody watched him. And it took the crew and the passengers on the plane to keep the plane from being blown up."