FREDERICK, Maryland (CNN) -- Friends say a scientist who killed himself amid an anthrax investigation fit many stereotypes, but biological terrorist was not one of them.

People who knew Bruce Ivins recall a friendly, helpful man whose neighbors had no reason to suspect him of wrongdoing; an eternal graduate student with ill-fitting clothes and an awkward social manner; an apolitical egghead too busy with his work to carry out the crimes the FBI suspected him of.

But, in addition to authorities investigating the 2001 deadly anthrax attacks, at least one person had a more sinister perception of Ivins: He was scheduled to appear in court Thursday after a woman accused him of stalking her.

Sources have told CNN that Ivins knew he was about to be charged in connection with mailing spores of the deadly bacteria anthrax to a number of congressional offices and media outlets in autumn 2001.

Five people died, including two postal workers, and more than a dozen people became ill. No one has been arrested in the case.

Sources told CNN the government planned to seek the death penalty against Ivins, but he had not been charged with a crime at the time he died.

Ivins, 62, worked for more than 30 years as an anthrax researcher at Fort Detrick, Maryland, home of the U.S. Army's Medical Research Institute of Infectious Diseases. He co-wrote a paper only weeks ago outlining the effects of a drug on anthrax in mice, one of dozens of papers in his career.

Ivins was also a member of the American Red Cross and a musician at his church, St. John the Evangelist Roman Catholic Church. He is survived by his wife, a son and daughter, and two brothers.

Don't Miss

People who knew him were shocked to learn of his death and the possible criminal charges.

"Ivins was as mild as they come," said Luann Battersby, a former microbiologist who worked with Ivins at Fort Detrick.

She remembers a man who wore slightly ill-fitting trousers that revealed his white socks -- a man who never grew out of habits from his years as a graduate student.

"He continued to live the lifestyle of a grad student, no frills," Battersby said.

Norm Covert, who worked with Ivins until retiring in 1999, remembers the scientist as "a brilliant man, very intense with his work."

"We're looking at a man with a distinguished 30-something-year career, unparalleled and known around the world," Covert said. "His career and his reputation are trashed and the FBI still hasn't said what they have on him."

But one woman says she saw a different side of the mild-mannered scientist.

When he died, Ivins was under a restraining order on allegations of stalking, threatening and harassing a woman. He was due in court two days after he died. CNN is not naming the woman who filed the complaint.

The District Court of Maryland at Frederick ordered Ivins on July 24 not to threaten harm to the woman, contact her, or enter her home or workplace.

The court dismissed the order on Thursday, noting that Ivins was deceased.

"We are saddened by his death, and disappointed that we will not have the opportunity to defend his good name and reputation in a court of law. We assert his innocence in these killings, and would have established that at trial," said Paul F. Kemp of Venable LLP.

Kemp said Ivins had "fully cooperated" with authorities and blamed the investigation for the researcher's death.

"The relentless pressure of accusation and innuendo takes its toll in different ways on different people, as has already been seen in this investigation. In Dr. Ivins' case, it led to his untimely death."

Neighbor Bonnie Duggan said it did not take "a rocket scientist" to figure out the FBI had been surveilling Ivins on and off for about a year.

"We knew they were looking at someone, we knew that Bruce worked at Fort Detrick, we knew he worked with pathogens," she said.

But she could not believe he would have carried out the attacks.

"It doesn't jibe with anything that I knew," she said. "It's not like you might say the person gave off vibes that they might be a dangerous person."