I had reason to converse with an estate agent this week, and his current difficulties were not at all those that I had imagined. His main problem is that he has a number of clients wishing to spend roughly £1m on large houses, and rather fewer such properties to sell.

Further, the people seeking these palatial south London homes during this economic downturn are often not even families in search of plenty of bedrooms. They are couples, quite often gay couples, he says, who don't want children, but do want acres of communal space – huge kitchens, living rooms suitable for parties – and also acres of private space – giant bedrooms with enormous bathrooms, roomy walk-in dressing rooms, and a place to house their gym equipment.

The agent showed me the particulars for some of these homes, their expansive interiors filled with beautiful pieces of antique and modern furniture, looking like something out of a film. Lots of pale floors and white upholstery. Not in the least suitable for children.

The houses had all been sold at a good bit more than £1m, in streets not far from the shop where five-year-old Thusha Kamaleswaran was shot by youths on bikes last month. They are not situated in a uniformly rich neighbourhood, not by any means.

Maybe David Willetts, the universities minister, knows many more people than I do who live like the clients of this estate agent, because what I was seeing and hearing reminded me immediately of the charmless phrase he used last week in the course of his notorious comments on Britain's lack of social mobility. The latter, he observed, had largely been the result of feminism and the "assortative mating", people of similar status pairing off, that it had fostered. His musings provoked outrage, with shadow home secretary Yvette Cooper going as far as to demand a retraction.

But perhaps Willetts's contention is worth a little examination, before it is summarily dismissed. He says that the rise of educated working women has been "the biggest single factor" contributing to the lack of jobs available to aspiring working men. That's a wild overstatement. But the rise of "educated working women" has, of course, had some powerful socio-economic consequences, and it is useful to acknowledge and understand them, all the better to rebut the single-issue junk sociology of arguments such as Willetts's.

"Assortative mating" was already being remarked on as early as the 1980s, though the terms were slightly different then. Young, upwardly mobile professionals, riding the wave of Thatcherite deregulation, were called yuppies, and not as a compliment. When they came in pairs, they were called dinkies – dual income, no kids. The buying power of these joint incomes fed the property boom of the 80s, and did likewise during the Blair bubble.

Supply and demand did its work, and in the parts of the country where larger salaries were to be had, a hefty dual income became a necessity in securing a mortgage, rather than an advantage. Further, property prices rose so fast in some areas during the boom, that even in households with two adults earning good money, the house just sat there in the garden all day, doing nothing, but earning even more. This is far from the only reason why the distance between the richest and the poorest has become so much more grotesque since the 1970s, but it has to be counted as one of them.

Willetts implies that women have stolen the educations and careers of aspiring men. But that really is nonsense.

Dinkies, with their large disposable incomes, help to drive the consumer boom, notably the dizzying rise in demand for luxury and "designer" goods. And when high-earning couples do have children, they carry on working, their hectic schedules dictating that cash should be thrown at childcare, cleaning, gardening, interior decoration, eating out, elaborate professional grooming, fancy processed food and heavens knows what else.

The entry of women into the professional economy changed the shape of the jobs market, creating new, different jobs for both sexes, but often for women. In this crucial respect, the needs of capitalism emancipated women far more than feminism did. Exactly the same goes for the "pink pound", by the way, as the estate agent knows. Free-market economics did more to emancipate homosexual people than little things like having a just cause, hard fought for.

It is difficult for the left to accept that its great recent success, the advancement of gender and sexual equality, had quite that intimate a relationship with hated neo-liberalism. But acceptance is important, because it also helps to explain why feminism has had such limited success.

Why, as the tide went out on Blair's boom, did it become so apparent that women with children are much more greatly dependent on the state than men? It is partly because just as economic advantage has been hugely increased by "assortative mating", so too has economic disadvantage. For two people on low incomes, or even middling incomes, the option of lavishing cash on keeping the home fires burning, and the career advancing, while the children are small, does not exist. Women struggle on, working part-time, putting together a patchwork of childcare, and accepting low wages and poor conditions because school and nursery pick-up, school holidays and the responsibilities of running a family home conspire to make this necessary. Recourse to the state is very often a consequence of such impossible economic stresses, on fathers as well as mothers.

Feminism may have helped to make family breakdown more acceptable (along with the move away from religious belief). But family breakdown has long been most prevalent in the areas that have experienced de-industrialisation, not in the areas that boasted spectacular sales of books by Germaine Greer. The last Conservative government saw social breakdown as an unfortunate but necessary consequence of fast, brutal, economic restructuring. Then, as now, it was handy to lay the blame on feminism instead.

The harsh truth, however, is that feminism suited the demands of neo-liberalism very well indeed. Feminism itself has been merrily exploited, and people such as Willetts are still playing that convenient, nasty, female-bashing game. In an odd sort of way, he flatters to deceive, talking up feminism's power in order to convince others – and maybe himself – that the values he believes in are not as socially divisive and toxic as they really are.