The women were freed this week with the help of neighbors. The man who lived in the home where the alleged captives were found, Ariel Castro, was charged Wednesday with four counts of kidnapping and three counts of rape, Cleveland chief assistant prosecutor Victor Perez said.

Prosecutors are not pressing charges against Pedro and Onil Castro, who were also arrested along with their brother Ariel, with Perez saying there's "no evidence" that they were complicit in the case.

Neighbors say they had called police about suspicious activity at the home in the past.

Cleveland police deny that, issuing a statement Wednesday saying that a "thorough review of police communications records" show no such calls ever took place.

JUST WATCHED

Smart's advice for 3 women found

MUST WATCH

JUST WATCHED

Abduction story stuns Ohio community

MUST WATCH

Abduction story stuns Ohio community03:52

Police say they went to the address once in 2000, before the alleged kidnappings, when Castro reported a fight outside his home, and in 2004, after two of the three women had disappeared. The latter visit was unrelated to the kidnappings, police said. Child services requested that visit to investigate a complaint that Castro had left a child alone on a bus during the time he worked as a bus driver.

No one answered the door at the home, and investigators later interviewed him elsewhere, police say.

Martin Flask, director of public safety for the city, later told reporters that "there is no evidence to indicate that any of them (the women) were ever outside in the yard in chains, without clothing or any other manner."

Neighbor Nina Samoylicz says that in July 2010, she and her friends saw a naked woman in the backyard of Castro's home. They said something to her, and Castro "told her to get down. So we said something to him. He told her to get in the house," and then he went inside as well, Samoylicz said.

"We called the cops," she says. "They thought we were playing and joking; they didn't believe us."

But Faliceonna Lopez, Samoylicz's sister, told CNN's "Piers Morgan Live" Tuesday night that they called their mother, not the police. Their mother, Annita Lugo, did not contact authorities because, she said, she didn't know what to do.

Within a week or two after they saw the naked woman, tarps went up, shielding the backyard from onlookers, Nina Samoylicz said.

Samoylicz also said she saw a girl who would sit looking out an open window until Castro would come along and board it back up. Sometimes the girl played at a nearby park by herself. "My cousin one day said something to her. She said she's waiting for her dad and then she pointed to (Castro). And she ran up to him, and gave him a hug, and they left."

News reports say another neighbor called police after seeing a woman through an attic window of the home, which Castro never allowed neighbors to enter.

And another neighbor, Israel Lugo -- Annita Lugo's brother -- says he called police in 2011 when he heard yelling in the Castro home. Officers came a half-hour later, knocked on the door about 20 times over a 10-minute span, then left after no one answered the door, Lugo says.

Yet Cleveland police took to Twitter on Wednesday afternoon to dispute such claims. After an "extensive" look at its records, Cleveland police said there were no calls to the Castro house "for women held or women banging on windows."

While denying any tips in the past, authorities are reaching out now to try to get more information about Ariel Castro and his home, in the hopes it can help their investigation.

"We continue to ask the community to send tips and information -- to say something if you see something. We cannot do this alone," police said in a statement after the women were freed.

If someone really heard screaming and police simply knocked on the door and left without further investigation, then "law enforcement clearly dropped the ball," attorney Lisa Bloom said on CNN's "Piers Morgan Live."

"It was almost like a Keystone Cops situation," prominent attorney Mark Geragos added.

"There's got to be much more to this story. Given my experience in handling criminal cases, I can't believe that we're not going to find in the days ahead or the weeks ahead much more that unravels about this that blows your mind."

Deputy Police Chief Ed Tomba, at a press conference Wednesday, said investigators over the past 10 years have often asked themselves whether they missed a significant angle. The answer, he said, is no.

"I'm just very confident in the ability of those investigators and those law enforcement officers that they checked every single lead and if there was one bit of evidence, one shred of a tip, no matter how minute it was, they followed it up very, very aggressively."