On August 13, 1958, Carroll was named the first Bishop of the newly erected Diocese of Miami in Florida.[5] His installation took place on the following October 7.[5] At the time of his arrival, the diocese comprised sixteen counties in southern Florida with a Catholic population of 185,000.[11] By the time of his death, the archdiocese was composed of eight counties, and included 700,000 Catholics, 127 parishes, 500 priests, and 750 nuns.[11] A little over a year following his installation, Carroll founded St. John Vianney College Seminary in Miami.[12] He later opened St. Vincent de Paul Seminary at Boynton Beach in 1963.[12] He also established a weekly diocesan newspaper called The Voice.[12]

In response to the Cuban exile, Carroll welcomed over half a million Cuban refugees into the diocese.[11] In 1960, he used the four-story school building of Gesu Church to establish the Centro Hispano Catolico, a welfare agency that provided medical care, child care, legal aid, employment service, food, clothing and cash for Cuban refugees in the diocese. He also helped to coordinate Operation Peter Pan, and even scolded Monsignor Bryan O. Walsh, who headed the diocesan Catholic Charities program, for not agreeing to resettle more unaccompanied children.[13] However, Carroll was accused by some Hispanic Catholics, including a number of priests, of showing little interest in their community.[11] They also claimed he was trying to Anglicize the diocese by limiting Spanish-language education in parochial schools and Spanish-language Masses.[11] Carroll did, however, maintain amicable relationships with local African American and Jewish leaders.[11] He was a frequent visitor of Camillus House, established homes for the elderly and unwed mothers, and opened rehabilitation centers for drug addicts and alcoholics.[13]

Carroll was known for his firm control over his priests and parishioners, as well as for his outspoken conservative political views and progressive social outlook.[11] He was a vocal opponent of a local ordinance in Dade County that prohibited discrimination on the basis of sexual orientation, supporting the Save Our Children campaign led by Anita Bryant.[15] Despite his opposition to gay rights, he was an advocate for racial justice and strongly supported the civil rights movement.[16] On theological matters, he was described as a "hardline Roman Catholic traditionalist" known for his "vociferous opposition to liberalization of the church."[16] He also opposed repealing the practice of abstaining from meat on Fridays.[16]

At age 72, Carroll died from complications stemming from a vascular disease at his residence in Miami Beach.[3] He was buried three days later in the priests' section of Our Lady of Mercy Cemetery in Miami.[2]