ROME -- Pope Francis spoke publicly on Wednesday about his weekend trip to Ireland and the "pain" of the Catholic Church sex abuse scandal -- but again he declined to specifically address an explosive letter by an Italian archbishop who claims the pontiff was involved in covering up the abuses.

CBS News correspondent Seth Doane says Francis addressed crowds of the faithful in Saint Peter's Square -- as he does most Wednesdays. Same time, same place, but this Wednesday morning, CBS News found some Catholics who were struggling with their faith.

Nick, who came to hear the pope, told Doane the scandal has left him and others less willing to "be open about your faith."

"Not ashamed," added a woman in the crowd, "but when other people of other faiths, when they're looking at you, when they're talking to you, that's all they know is what they're seeing on the news."

In his 11-page letter, Monsignor Carlo Viganò called for Pope Francis to resign. The former top Vatican diplomat, who served as its envoy to the U.S., included sweeping conservative ideological commentary in his letter -- including railing against homosexuals in the church.

Pope Francis accused of ignoring sex abuse allegations

Viganò alleged that Francis knowingly allowed a sex-abuser, Cardinal Theodore McCarrick, to continue operating at the highest ranks of the church, even allegedly relaxing church restrictions put on him. Viganò did not provide any documents to support his accusations that at least 35 church leaders were involved in covering up the abuse.

The majority of bishops have condemned Viganò, with some accusing him of being part of a conservative attack on the more liberal pope. Others have called for an investigation.

Monsignor Karel Kasteel has worked for seven popes and has known Viganò since he was a young seminarian. He insists the Italian archbishop "certainly is an honest man, and he's done his job well and I think he felt he had to talk."

Viganò's letter came in the wake of a grand jury report that exposed 300 pedophile priests and more than 1,000 victims in Pennsylvania.

Pa. reopens investigation into decades-old clergy abuse claim

The state's attorney general, appearing Tuesday on "CBS This Morning," said there were "facts, there is evidence that takes this cover-up and what occurred in Pennsylvania directly to the Vatican."

The director of the Holy See press office told CBS News that, "if the prosecutor is referring to something outside the report -- we'll have to wait for that before commenting."

Italian media was reporting on Wednesday that close associates of the pope say he's saddened, but not considering resignation.