Sepp Blatter's leadership of world football has been called into question after he denied racism was an issue and said any race-related incidents during games should be settled by a handshake.

Fifa's president made the claim in two separate television interviews to provoke a furious response from around the world. Blatter later released a statement on the governing body's website saying he had been misunderstood, accompanied by a picture of him hugging South Africa's housing minister, Tokyo Sexwale.

His remarks were described as "astonishing" by Manchester United's Rio Ferdinand and the 75-year-old was branded "worryingly out of touch" by the anti-racism group Kick It Out. The former Premier League players Stan Collymore and Shaka Hislop were among those to call for Blatter to step down.

Asked if racism was a problem on the pitch, Blatter told CNN World Sport: "I would deny it. There is no racism, there is maybe one of the players towards another, he has a word or a gesture which is not the correct one, but also the one who is affected by that.

"He should say that this is a game. We are in a game, and at the end of the game, we shake hands, and this can happen, because we have worked so hard against racism and discrimination."

He also said, on Al Jazeera: "During a match you may say something to someone who's not looking exactly like you, but at end of match it's forgotten."

His claims came on the day the Football Association charged Luis Suárez with racially abusing Manchester United's Patrice Evra, which the Liverpool striker denies. Ferdinand reacted furiously on Twitter to Blatter's remarks: "Tell me I have just read Sepp Blatter's comments on racism in football wrong … if not then I am astonished.

"I feel stupid for thinking that football was taking a leading role against racism … it seems it was just on mute for a while. Just for clarity if a player abuses a referee, does a shake of the hand after the game wipe the slate clean??"

Ferdinand also directly contacted the Fifa president's Twitter page, writing: "Sepp Blatter your comments on racism are so condescending its almost laughable. If fans shout racist chants but shake our hands is that OK?"

Kick It Out strongly condemned Blatter's remarks. "These comments are worryingly out of touch," a statement read. "Shaking hands to compensate for a racial slur is not what the game has signed up to, and trivialises the work of campaigns like Kick It Out, which has been in the vanguard of rooting out discrimination and unacceptable behaviour in our game for the best part of two decades.

"But leadership is needed to make headway. And comments like this don't help in the ultimate goal of kicking racism out football and making it a discrimination-free zone."

Collymore used Twitter to call for Blatter, Fifa's president since 1998, to step down. "Mr Blatter, your comments about racism are ill thought, and condescending in the extreme. You should resign." Hislop, a former West Ham and Trinidad & Tobago goalkeeper, wrote: "How can he possibly remain?"

Gordon Taylor, the chief executive of the Professional Footballers' Association, joined the condemnation of Blatter. "It's very insensitive and inappropriate at this time, when as the president of Fifa he's got to be shouting from the top of the hills that it's unacceptable on the pitch," he told the BBC.

Blatter attempted to douse the controversy in a statement on Fifa's website: "I would like to make it very clear, I am committed to the fight against racism and any type of discrimination in football and in society," it said.

"I have been personally leading this battle against racism in football, which Fifa has been fighting against throughout the past years through campaigns in all of our competitions such as the 'Say no to racism' campaign.

"I also know that racism unfortunately continues to exist in football, and I have never denied this. I know that it is a big problem in society, and that it also affects sport.

"I strongly believe that we should continue to fight all together against racism on and off the field of play, in order to eradicate this plague. My comments have been misunderstood. What I wanted to express is that, as football players, during a match, you have 'battles' with your opponents, and sometimes things are done which are wrong.

"But, normally, at the end of the match, you apologise to your opponent if you had a confrontation during the match, you shake hands, and when the game is over it is over.

"Anyone who has played a football match, or a match in any sport, knows that this is the case.

"Having said that, I want to stress again that I do not want to diminish the dimension of the problem of racism in society and in sport.

"I am committed to fighting this plague and kicking it out of football."