Two victims of the Windrush scandal have told MPs they believed they would not have faced deportation were it not for their race.

Two of the first people whose cases were documented by the Guardian, Paulette Wilson and Anthony Bryan, were questioned about their experience of being classed by the Home Office as illegal immigrants during the joint human rights committee, along with Bryan’s wife, Janet McKay Williams, and Wilson’s daughter, Natalie Barnes.

Both Wilson and Bryan had similar experiences of being wrongly detained and threatened with removal to Jamaica, a country neither had visited since they were children.

They were asked by Doreen Lawrence, whose son Stephen was murdered in a racially motivated attack in 1993 and who now sits in the House of Lords, whether they thought they would have been treated the same if they had they been Canadian, Australians or New Zealanders.

In response, Bryan said: “I hate to say it, but I don’t think I’d have this problem [if] I had come from Canada instead of coming from Jamaica.”

His wife added: “[It was] because of the colour of your skin.”

Lady Lawrence, who emigrated to the UK from Jamaica at the age of nine, then asked whether they saw race as being a big part of the problem, to which all of them agreed, and Bryan said: “In the Home Office, yes.”

Earlier the chair, Harriet Harman, introduced the meeting by explaining that the committee was concerned that their human rights had been breached.

She said: “One of the most important human rights is the right not to be locked up unless you’ve done something wrong, and the right to know what is actually happening to you.

“We have all seen and heard about you not doing anything wrong, but having being detained, so we are very concerned about that on the basis of it being a breach of your human rights.”

The Labour MP added that the committee wanted to be sure of what went wrong so they can be sure it did not happen again.

Following an hour of questions, Harman, who praised the group for coming forward and telling their stories, asked if they would like the Home Office to give them their immigration files, which they said they would. She said the files could help the committee to “try and understand what was going through the Home Office’s head”.

The hearing was at times highly emotional as the group recalled their experiences. When asked how he had felt while in detention, Bryan said: “They had tickets for me [to Jamaica]. To be honest, I was broken. I couldn’t fight no more. I had given up … Six weeks before, I had buried my son, and all of a sudden I was in this lock-up even though I hadn’t done anything. I was upset. It was very hard.”

When asked what they would have done if their families hadn’t helped them fight their cases, Wilson said: “I would be in Jamaica all alone. I didn’t know anyone over there. I thought they were sending me to die.

“I give thanks for having a daughter like the one I’ve got. If it wasn’t for her, I wouldn’t be here.”