"I don't think that any of these debates are changing women's views about the politicians or their policies," said Ipsos pollster Julia Clark in a release accompanying the poll.

However, even though Romney isn't gaining ground among women, he is gaining ground overall. He's up 47 percent to 43 percent on Romney overall, which is down from an 11-point lead he had in the March poll. That's because of voters who are flocking to Romney, trusting him more on jobs and dealing with the economy.

"Obama has had to preside over a really tough economy. ... People are sort of dinging him for it and that's really what's going to make this a competitive race," said Chris Jackson, research director at Ipsos public affairs, in a release.

Women, though, don't have much to do with that. Some key numbers where women rated Obama higher: