Washington (CNN) -- The danger of Islamic radicalization inside U.S. prisons "remains real and present," said Rep. Peter King of New York, the Republican chairman of the House Homeland Security Committee. But during a Wednesday hearing on the subject, others said there are only a few cases in which prison radicalization has been linked to terrorism.

"Prisons have not served as a major source of jihad radicalization," according to Bert Useem, a Purdue University sociology professor. Useem said that since the September 11 attacks, 178 Muslim Americans have been prosecuted for terrorism or related charges, but there is evidence in only 12 cases that prison radicalization was a factor.

"If prisons were a major cause of jihadist radicalization, we would expect to see a lot of it, but we don't," Useem added.

The ranking Democrat on the committee, Rep. Bennie Thompson of Mississippi, says that judging from the information he's seen, "the risk of terrorism originating from Muslim converts in U.S. prisons is small."

Thompson and several other Democrats on the panel also complained that the King hearing focused only on Muslims in prisons and did not address other groups such as gangs and white supremacists.

"I actually believe that the focus on one particular group on the basis of race or religion can be deemed as racist and discriminatory," California Democrat Laura Richardson said, adding that members of other groups could pose threats as well.

King blasted back that when Democrats were last in control of the House and his committee, they never brought up threats from some of the groups they were mentioning now. King also distinguished radical Muslims from other groups, saying Muslim radicals could be connected to terror groups overseas.

Michael Downing, who heads the counterterrorism unit for the Los Angeles Police Department, told the committee he takes the threat of radicalization in prison very seriously and thinks "we are on the front end" of a growing problem. Downing and other witnesses called for an in-depth study of the problem and for standards on vetting prison chaplains and on reading materials and videos that prisoners are allowed.

"There's radical material inside the prison system still. Anwar al-Awlaki's material is inside the prison systems," Downing said in reference to the Yemeni-American cleric who U.S. officials contend is a key member of al Qaeda in the Arabian Peninsula. He is accused of involvement in the plot to blow up an airliner headed into Detroit in December 2009 and the 2010 plot involving explosives placed in copiers on planes headed for the United States.

King cited a number of terrorism cases involving prison radicalization, including that of Michael Finton, who pleaded guilty in May to wanting to blow up a Springfield, Illinois, courthouse; James Cromitie, who was convicted in a plot to use surface-to-air missiles against a troop transport aircraft in New York and detonate explosive devices at a Jewish community center and a synagogue; and Jose Padilla, who was convicted in Miami in 2007 of providing material support to terrorism.

Another case cited by King and the committee's witnesses involved Kevin James, who was serving time in a California prison where he formed a radical group called JIS and recruited a fellow prisoner named Levar Washington. When Washington was paroled, he recruited a couple of other men, and they plotted to attack a military recruiting station and a Jewish target. James was involved with the plot while he remained behind bars and was later convicted for his actions.

Kevin Smith, a prosecutor on the James case, said James taught himself about Islam and because of his charisma was able to entice others to join his plot. Smith said the prison system was not able to interfere with James preaching about Islam to fellow prisoners.

Perhaps the most emotional moment in the hearing came when Rep. Hansen Clarke, D-Michigan, talked about childhood friends who went to prison but were never able to get their lives back on track. Clarke said too many people go to prison when they should be treated for mental illness or drug use.

"We are spending too much money incarcerating young men, young black men whose lives can be saved," Clarke said. "It's not about Islam. It's about the sentencing policy. It's about this prison system."