"I genuinely never thought [fatherhood] was going to happen," Cowell, 53, told Ryan Seacrest on his radio show. "Partly because I was scared of who I was going to have kids with ... . [But] Lauren and I get along really well and once I got used to the idea I thought, 'I think this is going to be very good.'"

So, will he and Silverman, 36, ever tie the knot?

"Let's put it this way, I wouldn't expect her to be a single mother," Cowell told Seacrest.

News of Silverman's pregnancy was reported this past summer before she was officially divorced from Cowell's friend Andrew Silverman, with whom she has a young son.