It’s hard to imagine Scandal‘s fiery and formidable Olivia Pope played by anyone other than Kerry Washington, but if ABC had their way during casting, Connie Britton would have landed the lead role.

The character was inspired by real-life Washington D.C. crisis fixer Judy Smith, and Scandal showrunner Shonda Rhimes set out to honor her identity and heritage. “Nothing felt more important than the sense of outsiderness,” Rhimes said of the role while speaking with The Hollywood Reporter for an oral history celebrating the series’ 100th episode. “I didn’t know that there hadn’t been a drama series with a leading black woman for 37 years. When the show got picked up [to pilot], I got a phone call from somebody who said, ‘This would be the perfect show for Connie Britton.’ I said, ‘It would be, except Olivia Pope is black.’”

Added casting director Linda Lowy: “The network was reading us their top choices, and it was Connie and all white women. I panicked. Somebody finally piped up, ‘We’re going to have to redo this list.’”

After discussions about how having a black lead might impact the show internationally, Washington, Jill Scott, and Anika Noni Rose were brought in to test for the role. “It was Kerry from the moment I took her to meet Shonda,” said Lowy.

“She could talk Washington more than I could talk Washington,” said Rhimes. “She was different than what I originally envisioned. We were all like, ‘Oh my God,’ because she’s tiny, cute, pretty, and younger — and because she was all those things, she was aware that people would underestimate her.”

Still, Washington can see why ABC thought the Nashville actress could have been a good fit: “This would have been a great role for Connie Britton!” she says.