Patrick Kennedy, the former congressman from Rhode Island, said the shooting in Tucson, Ariz. has pushed the threat level to members of Congress to an "all-time high" and members are in fear about running for office.

"It steals from democracy," he said, "when you have my colleagues decide not to run for re-election because they're worried about their personal safety.

"This is a major issue because it's got nothing to do just with the personal tragedy of Gabby, but the tragedy to our democracy when people are fearful of the political process."

In an exclusive interview with ABC News' Christiane Amanpour, Kennedy said some members of Congress have been receiving public threats for years, and that "politician" is "an evil word now."

"Some of my colleagues, people don't know about it, were getting 'round the clock protection for some time," he said. "What it represented to me was a change in just the environment."

He added that he and other members have been aware that the atmosphere of today's politics could eventually ignite a "violent act," such as what happened in Tucson.

"We've lived in an environment that sanctioned pretty heated rhetoric," he explained. "That's really bordering on irresponsible in how much it condones violence and the dehumanization of our political leaders."

Rep. Patrick Kennedy with ABC News' Christiane Amanpour

Pima County Sheriff's Office in Tucson said they determined Rep. Gabrielle Giffords, D-Ariz., was the intended target of the alleged shooter, Jared Loughner, when he opened fire at a community event Giffords was hosting last Saturday.

Kennedy said terrorists feed off widespread fear.

"We're in a fear-based position when people are terrorizing our political rhetoric to the point where it's OK for people to attack in the most personal and vicious way," he said.

Kennedy praised President Obama for his leadership in the wake of the shooting, saying the president put politics aside and "spoke about what unites us as Americans."

Patrick Kennedy Says Killings Affected a Nation

"No one asked if that little girl was a Democrat or Republican," he said. "This tragedy is human, and the political tragedy is if we don't learn that we have very much in common."

Kennedy said he had "enormous worry" for Giffords after she was shot, and added that she was an "amazing human being."

As the son of the late Sen. Ted Kennedy, D-Mass., Patrick Kennedy is no stranger to politically-motivated tragedy and spoke of how acts of violence are never just about the victims.

"It's not just that one murder or killing," he said. "It's the whole family. In this case it's the whole community and our country that's affected...the worry that everybody has that this could be a community that they live in where this could happen to them."

After holding office for more than a decade, Kennedy decided not to seek re-election this past November. His decision stemmed from losing his father in 2009.

"It was difficult to lose my dad," he said."I knew things had changed, for good, when he was gone...I knew if there was a time to move on, this was it."

Kennedy said that deciding to enter public life was a "vehicle" to build a relationship with his father, and he felt like a "servant to the family legacy."

"My dad said, 'we had this great political capital, your
uncles, you know, fought for it...and it should be used for
something positive,'" Kennedy said.

While serving in the House, he focused much of his energy on mental health, an issue that hit very close to home for him as former alcohol and drug addict who has been diagnosed with bipolar disorder.

"Whether it's cocaine or it's alcohol, or it's opiates," he said, "those are irrelevant to the fact that you've got a neurological brain disorder that compels you, both environmental and physical, to self-medicate to deal with your pain."

After crashing his Ford Mustang convertible on Capitol Hill in 2006, Kennedy checked into rehab to be treated for an addiction to pain medication. Those years, he said, were "humiliating."

Kennedy: 'We've Got to Come Together'

Kennedy wouldn't directly comment on if he was out of politics for good, but did say he can do more work with his continued involvement with getting funding for brain research.

In the wake of Tucson, Kennedy said Jared Loughner has been characterized as a "deranged" lone gunman with "serious mental health issues." He said this incident should split open the "third-rail issues" of gun control and personal freedoms of those who have mental health problems.

"We've got to come together," he said. "It's going to involve a solution that deals with both the individuals and their suffering and mental health and the notion of freedom and whether we're going to intervene when we know people need help."