‘Kathleen’, who was a victim of domestic abuse, with one of her children, and the crockery passed down from her mother. Photograph: Martin Godwin for the Guardian

My Big Fat Gypsy Wedding, the television series that ended last week and attracted nine million viewers, was designed to "throw an overdue light on a secretive, marginalised and little-understood segment of our society", according to the blurb from Channel 4. But the show largely avoided the myriad of problems, such as discrimination, poor health and poverty faced by Travellers, except for what felt like a tokenistic final episode, and instead focused on over-the-top wedding dresses and other excesses.

MBFGW was about Gypsies and Travellers, but there was criticism from both communities that C4 failed to properly distinguish between the two. There are around 300,000 Gypsy Roma and Irish Travellers in the UK – Roma Gypsies are originally from northern India, whereas Travellers are of Irish origin – and both groups are nomadic. Since 2002, Travellers have been recognised as an ethnic group and are protected under the Race Relations Act. Last week C4 was accused by the Irish Traveller Movement in Britain of fuelling "hatred and suspicion" of their way of life and have demanded a right of reply.

Kathleen, who lives with her six children in a three-bedroom trailer, is fairly typical of an Irish Traveller woman, except that she is separated from her husband. Along with many other Gypsy and Traveller women in the UK, Kathleen was a victim of domestic violence. Although there is no conclusive evidence about the prevalence of this abuse, a study in Wrexham, cited in a paper by the Equality and Human Rights Commission, 2007, found that 61% of married English Gypsy women and 81% of Irish Travellers had experienced domestic abuse. And a significant number of those women who had reported the abuse appeared to have suffered more severe and sustained violence than those within mainstream communities.

"I left him and went back to my mammy but he kept finding me, taking me home and getting me pregnant," Kathleen says. She now feels safe because she has male family members living on the same site. "With my brother close by, he wouldn't dare come here."

It is rare for women to call the police for help. "You would be seen as a grass and disowned by the whole community," says Bernie O'Roarke, outreach and resettlement worker for domestic violence charity Solas Anois (Gaelic for Comfort Now), which is based in London. The situation probably isn't helped by the fact that there is only one, 10-room refuge dedicated to Traveller women in the UK, also in London. But domestic violence is just one of the issues tackled by O'Roarke during her visits. The welfare needs, particularly those of the women and girls, of this community are vast. The women are three times more likely to miscarry or have a still-born child compared to the rest of the population, mainly, it is thought, as a result of reluctance to undergo routine gynaecological care, and infections linked to poor sanitation and lack of clean water. The rate of suicides among Traveller women is significantly higher than in the general population, and life expectancy is low for women and men, with one third of Travellers dying before the age of 59. And as many Traveller girls are taken out of education prior to secondary school to prevent them mixing with boys from other cultures, illiteracy rates are high.

O'Roarke is a familiar face on the sites around London, offering women and their children help with health care, education and finance. The men leave the women alone to deal with these issues, so if the women do want to talk about violence and abuse, they can do so without fear of the men overhearing.

I visit some trailers with O'Roarke at a site in London and am struck by how the women seem to manage, usually with large families, to keep everything so clean and tidy. There are colourful displays of Royal Crown Derby crockery, handed down from mother to daughter on her wedding day. There is certainly no sign of wealth or excessive spending. Many tell me they struggle to feed their children, and have no savings or bank account.

Things seem set to get worse for Traveller women. Only 19 days after the general election last year, £50m that had been allocated to building new sites across London was scrapped from the budget. O'Roarke is expecting to be the only Traveller liaison worker in the capital before long – her funding comes from the Irish government.

"Most of the women can't read or write. Who is supposed to help them if they get rid of the bit of support they have now?" asks O'Roarke. "We will be seeing Traveller women and their children on the streets because of these cuts. If they get a letter saying they are in danger of eviction but they can't read it, what are they supposed to do?"

Conditions on the site are as grim as the homes are spotless. The trailers are not connected to water pipes, and the toilets, bathrooms and cooking facilities are in a small, unheated shed across the yard. "In the snow and rain, the little ones are always getting colds," says Brigid, who lives in the next trailer to Kathleen. "And there are so many pot holes that the council haven't filled, you can go flying in the dark."

But living on a site is about being part of the community. When Traveller girls are growing up, they are only allowed to go out with other family members, and once married, her husband rules the roost. "The men would never allow a woman out with her friends," says Kathleen. "That's why we want to live on a site, for company." Kathleen, after spending time in a refuge after finally managing to escape her husband, was initially allocated a house, as opposed to a plot on a site. Almost immediately her children became depressed. "It's like putting a horse in a box. He would buck to get out," says Kathleen. "We can't live in houses; we need freedom and fresh air. I was on anti-depressives. The children couldn't go out because the neighbours would complain about the noise."

Since moving to their site two years ago, Kathleen and her children have been far happier. Until MBFGW was screened, that is. "Now every week I go to the school and the parents are talking about that programme. They won't let our kids mix with theirs because they say we stink and don't talk properly. Settled kids won't even play sports with ours in case they touch them."

Mary, Kathleen's 15-year-old daughter, is upset by the series too, and says that she has faced further prejudice since it hit the screens. "That programme didn't show the real way we go on. All my friends are asking if it's true what they show on telly, and I think they've gone different [towards me] since it was shown."

In one episode the viewer was informed that young Traveller men at weddings and other social occasions use something known as "grabbing" to force a reluctant girl to kiss them. One newspaper report called it a "secret courting ritual".

"Grabbing has never happened to me or any of my friends and the first time I ever saw it was on the telly," says Mary. "I wouldn't put up with it, and I don't know why they made out we all do it. It's just one nasty boy they showed."

Brigid adds: "Grabbing has never happened to my kids. I have honestly never heard of it. It's all make-believe."

Helen, a Traveller in her 20s on the same site, is also furious about the portrayal of women in MBFGW. "The way us women come across in the programme is a disgrace," she said. "It shows us as nothing but slaves to the men, only good for cooking and cleaning, and always being available to open our legs to them. We don't want that for our daughters."

Helen is also worried that Traveller women are being portrayed as rich and spoilt when, in fact, life is a struggle for the majority. "I don't know anyone so rich that they can afford to splash out on wedding dresses like that. Mine was secondhand. They'll now be saying we are all criminals, or sponging off the state." I ask a number of Traveller women how representative of the Traveller and Gypsy communities those featured in MBFGW are, and they all come back with a similar answer: the programme focused on a small number of individuals from five sites (out of an estimated 300-plus across the UK), and in any community, there are a minority who have access to large amounts of cash.

I ask O'Roarke what she thinks the future holds for Travellers. She is worried. "That TV programme has put our work back 100 years. And if these women lose the little support they have, they literally will be left to rot."

She is concerned that problems affecting Traveller women and girls, such as lack of education, forced and early marriage, and abuse within the home, are not being taken seriously.

"These issues do not just affect certain Asian communities," says O'Roarke. "We have had Traveller women in the refuge who have been forced to marry someone who they have never met, and marrying cousins is not uncommon."

But some say that things are slowly improving. "I think it's changing an awful lot for the young ones," says Kathleen. "We don't want them to have no education and get married at 16, and have loads of kids and the same life as we did."

Would Kathleen ever marry again? It is out of the question, she tells me. These things are just not done. "You marry for life," she says. "If I was to have another man, my daughters would never be married because I would have brought shame on them."

O'Roarke would like to see changes that include: "Better support for the women to keep their daughters in education, and a serious commitment from the government to challenge the prejudice thrown at these people."

While people are being entertained by watching Katie Price-replica weddings on TV, and girls dressed in Beyoncé-style outfits dirty dancing, women such as Kathleen, Brigid and Helen are living in substandard conditions and facing daily prejudice while trying to give their children the best start in life. The reality is a far cry from the C4 depiction and is rarely aired. O'Roarke tells me that Traveller women are usually reluctant to allow outsiders into their homes, despite the impression given by MBFGW. "But we just want our side of the story put across," says Brigid, "so settled people know we are not like that."