Talks between top lawmakers on the Senate Banking Committee and a group of Democrats seen as key swing votes to advance a bipartisan overhaul of Fannie Mae and Freddie Mac broke down Thursday, according to people familiar with the matter, raising the prospect that the bill won't advance beyond the committee this year.

Lawmakers last week postponed a committee vote on the overhaul bill, unveiled in March by Sens. Tim Johnson (D., S.D.)...