The Channel 4 series Benefits Street has riled both the left and the right, and angered a popular audience of what David Cameron and Labour's John Denham alike have described as "hardworking families". Understandably, it has also alienated the subjects of the series. The residents claim that they have been misrepresented, while some on the left see the show as "poverty porn" – and for good reason. Benefits Street is clearly premised on a fetishisation of class differences for popular consumption. The right, meanwhile, sees the show as indicting a failing welfare system and a "something for nothing" benefits culture.

Thanks to social media, the now not-so-silent majority of "hardworking families" dutifully pipe along with the right, cocooned in the knowledge that there are people – and British people, not just Romanian immigrants – who are not the same as them. This sentiment is powerfully divisive, to the apparent glee of some in the coalition government. Benefits claimants are seen as a different breed, even though "hardworking families" may also subsist on minimum wages topped up by state benefits.

All these viewpoints have some validity, though for different reasons. The producers of Benefits Street have adeptly edited their film in order to provoke a response from its intended audience. But is the show simply, as some on the left would have it, the cynical representation of an underclass?

The work of Stuart Hall – the pioneering cultural theorist who died last week – is worth revisiting to address this question. In the early 1970s, Hall elaborated the process of encoding/decoding media discourses.

While the producers of Benefits Street and similar exercises encode particular meanings, values and ideologies, Hall argued that it can't be taken for granted that audiences passively accept these encodings. Instead, audiences decode them according to their social experiences and political contexts. Hall had in mind the racially tinged discourses of the Thatcher era and how these were decoded, and potentially resisted, by audiences. The rise of anti-racist and anti-fascist groups, and the persistence (and flourishing) of feminist and socialist currents of thought were clearly antagonistic to the dominant media narratives.

Perhaps, though, there is something else about Benefits Street that Hall's work doesn't intuitively anticipate. It's easy for the left to see how the show's representation of poverty conforms, as anathema, to the values of "hardworking families", and it is not without overt references to race and gender. Yet the narratives in Benefits Street have a human and poignant quality, often presenting decent and compassionate people disenfranchised by an unfair society. The sense of community, at least from the perspective of White Dee, the programme's putative protagonist, is palpable. Despite the characters' obvious individual issues – mental health difficulties, marital strife and drug and alcohol dependency – there isn't always a sense of victimhood or hopeless entrapment. Instead, there is an endearing honesty in their depiction and, at times, a gentle humour.

The issue then is that this encoding is seemingly lost in the decoding by the subjective and ideological biases of various audiences. Leftist commentators read the show as demonising the poor, ignoring these often sensitive depictions of racial, gendered and class-based poverty. Resistance is indeed occurring, but in the wrong direction for the left's traditional goals. Similarly, the right and the audience of "hardworking families" miss these subtle representations, focusing instead on the alleged fecklessness of "scroungers" (though both fascinated and clutching their pearls at White Dee's success in getting a showbiz agent).

Both left and right perspectives are united in what Hall, in his final essay in 2013, deemed a "common-sense neoliberalism" that frames contemporary politics.

The left, Hall argued, so far has not adopted a position that can effectively challenge the right by aiming to "reduce the need for benefits rather than reducing actual payments". Instead, on both sides, there is a reversion to common sense – the "everyday thinking" which does not require reflection and is "a form of popular, easily available knowledge which contains no complicated ideas, requires no sophisticated argument and does not depend on deep thought or wide reading". Uncritically decoding Benefits Street epitomises these dubious qualities, and perhaps this warning could stand as Hall's epitaph.

more on this story

Decca Aitkenhead: When Deirdre Kelly agreed to appear in a documentary about community spirit, she had no idea what she was in for. Benefits Street would make 'White Dee' a household name – pariah to some, hero to others. So what now?