White, male, middle class and Oxbridge educated. It's the traditional makeup of this country's political arena and one that, depressingly, became more entrenched after last week's cabinet reshuffle.

David Cameron had five female cabinet ministers before the reshuffle, four afterwards, and two more (white) men. It was described as a "bad day for women" , but at least this was commented upon. The complete absence of disabled people didn't get a mention. A group truly knows they are marginalised when their absence from power isn't even noticed.

I am aware of only four MPs with a physical disability: Anne Begg, David Blunkett, Gordon Brown and Paul Maynard. No one knows the exact number because the data has never been collected. Forget a place around the cabinet table; disabled people are barely getting in the building.

An MP sharing a characteristic with a group is no guarantee he or she will fight for them. The new female minister for women, Maria Miller, has the necessary reproductive organs, but apparently no interest in protecting a woman's right to make decisions about hers. But this does nothing to reduce the need for women's increased presence in parliament – and it is no different for disabled people.

The makeup of parliament is important. When people from a marginalised group get a foot in the political process, that group – both their faces and their interests – begin to be represented.

For disabled people, this means the interests that affect the majority as well as those that scar them as a minority. Disabled people are more likely to live in poverty, have no formal qualifications or be unemployed than their non-disabled counterparts. Often, the critical issues that affect everyone hit the disabled harder. It's all too easy, though, for a House of Commons with barely a disabled person in it to overlook the disability dimension to mainstream policy, let alone issues that only affect disabled people. As disability benefits are slashed, you can't help but wonder how things would be if Iain Duncan Smith or George Osborne had to ask Atos for Disability Living Allowance.

It is particularly stinging for a group that is routinely patronised to be again placed in the role of passive recipient of "care". Disabled campaigners sit ignored on the sidelines, as, over in parliament, non-disabled MPs make the decisions.

There are 11 million people with a disability in Britain. In light of their absence from power, either we're to believe being disabled makes a person less capable of holding office or there are multiple obstacles standing in their way.

In truth, there are barriers everywhere; even the most basic such as not being able to use public transport with a wheelchair or communicate without a sign language interpreter. In June, the government launched a £2.6m fund to help with this. But little will improve until we address the barriers that go beyond financial; be it the refusal to let MPs job share or perceptions of disability that, in the secrecy of a ballet box, may affect a disabled person's electability. These obstacles are cultural and all the harder to shift for it.

The message that must be understood to get disabled people into parliament is the same one that will get them anywhere: needing support for a physical disability does not mean a person is any less able.

The fact is, as disabled athletes represented their country, there were still no disabled cabinet members permitted to do the same. Who represents us matters. A society that is not represented by disabled politicians excludes all disabled people.

• Frances Ryan is a writer and a political researcher at the University of Nottingham. She tweets @frances__ryan