The Conservatives' first full reshuffle has left the cabinet slightly more male, more Oxbridge, with fewer people from ethnic minorities and with a more pronounced southern bias.

Though the demographics are only slightly changed by the reshuffle, the new cabinet makes David Cameron's goal to have women in one third of government posts by the end of this parliament remote, and his wider modernisation agenda – to make the party more appealing to non-white voters, people outside the traditional, southern Tory heartlands, lesbian, gay and other minority groups – more difficult.

Analysis by the Guardian reveals the changes between the new and old cabinets. The changes are dominated by Cameron's reshuffle since the Liberal Democrat leader, Nick Clegg, left his five cabinet members unchanged.

The cabinet is two posts bigger and more Tory after the former justice secretary Ken Clarke and ex-party chairman Lady Warsi both remained in the cabinet.

But the detail is possibly more telling about the struggle Cameron is having modernising his party at the top.

Before Tuesday's changes, there were five women among the 29 full cabinet members, all of them Conservatives. Caroline Spelman and Cheryl Gillan have lost their seats in cabinet; Maria Miller and Theresa Villiers have been brought in and Warsi was demoted to a minister who attends cabinet but does not vote. On balance the number of women remained the same, but there are two more seats for men at the table.

Ceri Goddard, chief executive of the Fawcett Society, also attacked Cameron's lack of progress. "Despite near universal acceptance that we need more – not less – involvement of women in building our economy, the prime minister has chosen to further marginalise women's influence on politics," said Goddard. "Sweden, Switzerland and France all have equal numbers of men and women in their cabinets. Why can't we?"

Some of the party's female MPs were more pragmatic, however, stressing the importance of promoting women from the big influx of 2010 into junior ministerial ranks to prepare them for future cabinets. "The PM has limited room for manoeuvre at the top end and has done the best he can by promoting both Maria Miller and Theresa Villiers, [and] at the junior ministerial level I am hopeful we will see more women appointed some of whom have great potential," said Margot James, a party deputy chairman, who also welcomed the promotion of a Lib Dem female MP, Jo Swinson.

Penny Mordaunt, a new MP and member of the defence select committee, also warned against token women's appointments: "As with chaps it must be on merit, expertise, talent and their ability to deliver."

Warsi's move demotes the only non-white member of the cabinet, and the promotion of Chris Grayling appears to put two opponents of expanding gay rights in the cabinet since he sits alongside Owen Paterson, who has spoken publicly against legalising gay marriage.

After two years of jibes about the party being led by southern posh boys, epitomised by the infamous photos of George Osborne and David Cameron in the tails of Oxford's Bullingdon drinking club, the Tory side of the cabinet has lost its majority of public schoolboys, with two more members from grammars and two more from state schools, though the number of Oxbridge graduates is slightly higher. Meanwhile there are the same number of northern MPs, but more secretaries of state born in or representing southern constituencies – despite the border being so far south that Birmingham qualifies as "north".