(CNN) -- Father Alberto Cutie, an internationally known Catholic priest who admitted having a romantic affair and breaking his vow of celibacy, is joining the Episcopal Church to be with the woman he loves, he said Thursday.

Father Alberto Cutie will deliver a sermon Sunday at a Florida Episcopal Church.

"I will always love the Catholic Church and all its members," he said at a news conference. "But I want to start today by going into a new family.

"Here before this community where I have chosen to serve and where I live, I am going to continue to proclaim the word of God and my love for God," Cutie said.

Cutie (pronounced koo-tee-AY) was received into the Episcopal Church earlier Thursday at Trinity Episcopal Cathedral in Miami, Florida. He will pursue the priesthood in the Episcopalian faith, the Episcopal Diocese of Southeast Florida said in a written statement.

It was not immediately clear how long the process would take.

John C. Favalora, archbishop of the Catholic Church's Miami archdiocese, where Cutie served, said he was "deeply disappointed" by the news.

"Father Cutie's actions have caused grave scandal within the Catholic Church, harmed the Archdiocese of Miami -- especially our priests -- and led to division within the ecumenical community and the community at large," Favalora said in a written statement. "Today's announcement only deepens those wounds."

Cutie -- sometimes called "Father Oprah" because of the advice he's given in Spanish-language media -- shocked the Catholic community when photographs of him embracing a bathing-suit-clad woman emerged this month in the pages of TV Notas magazine.

The Cuban-American priest admitted having a two-year relationship with the woman, a long-time friend. She has not been publicly identified.

"This is something I've struggled with," he said this month. "I don't support the breaking of the celibacy promise."

Don't Miss

Of his relationship with the woman, he said, "Through the photos, it looked like a frivolous thing on the beach, you know, and that's not what it is. It's something deeper than that."

After the photographs surfaced, Cutie was removed from his duties at St. Francis De Sales Catholic Church in Miami Beach and on the Radio Paz and Radio Peace Networks.

In Thursday's statement, Favalora said that, in the eyes of the Catholic church, Cutie now has the worst of both worlds: He may no longer perform a priest's sacraments and any mass he might perform would be "invalid," but he must abide by his vow of celibacy.

"Father Cutie is still bound by his promise to live a celibate life, which he freely embraced at ordination," Favalora said. "Only the Holy Father can release him from that obligation."

He had been president and general director of Pax Catholic Communications, home of Radio Paz and Radio Peace.

In addition to his television and radio appearances, Cutie has written newspaper advice columns and a self-help book, "Real Life, Real Love."

The Episcopal Diocese of Southeast Florida said Cutie will deliver a sermon Sunday at a church that diocese leaders are looking to restore: the Church of the Resurrection in Biscayne Park.

Cutie will be a part of this effort to revitalize this church and others, the diocese said.

Cutie was the first Catholic priest to host a daily talk show on a major secular television network, according to his information on the LinkedIn online professional network.

Favalora also had harsh words for the Episcopal church's decision to accept Cutie.

"This truly is a serious setback for ecumenical relations and cooperation between us," he said.