Sam Alladyce questioned why Rafael's handball, left, was not punished with a penalty in the same way as Jordan Spence's was. Photograph: Screengrab

Sam Allardyce could face a Football Association charge after appearing to accuse Phil Dowd of bias against West Ham United for not awarding a penalty when Manchester United's Rafael Da Silva handled early in the second half.

West Ham lost the FA Cup third-round tie to a ninth-minute goal by Wayne Rooney, who missed a late penalty when Jordan Spence was adjudged by Dowd to have handled.

Allardyce said: "There was no doubt whatsoever about the difference between Rafael's hand ball and Jordan Spence's. Jordan Spence plays for West Ham away against Manchester United and Rafael plays at home at Old Trafford. It's got to be that simple when you see it. Phil Dowd was in the perfect position. Arm to ball, he [Rafael] pushes the ball away with his left arm. No penalty. Go the other end and the ball hits Jordan Spence's hand. If you give one, you have to give them both. Simple as that. You've seen it for yourself and you can look at the replay as many times as you want. We have a laptop and code the game. I've looked at his position for the Rafael one and it is perfect – straight in line with it, right in front of it. He had a worse position than Jordan Spence's.

"We are very disappointed we are not given a penalty which is blatant and clear for the referee to see. He couldn't find the courage to give it."