Neil Gaiman has said that he is "seriously disappointed" at the way science fiction fans reacted to the news that Jonathan Ross was – for a brief moment – going to host the Hugos, an invitation which the Sandman author had passed on to the chat show host.

Broadcaster and author Ross withdrew from presenting the prestigious science fiction awards on Saturday, shortly after the World Science Fiction Convention announced that he would be hosting them. His decision followed a series of attacks on Twitter from fans concerned that – in one reader's words – he would mock "women and other minorities". "If people genuinely believe I would upset them or those they care about then I'd rather not spoil their night," tweeted Ross. "I agreed because I love sf. And because Neil Gaiman asked me."

His wife, the Hugo award-winning screenwriter Jane Goldman, has since cancelled her Twitter account, while his daughter has engaged with the author Seanan McGuire on Twitter over McGuire's comment that "You know, I've really enjoyed knowing that, were I to be nominated for a Hugo, the host wouldn't see me and make fat jokes".

"I'm Jonathan's overweight daughter and I assure you that there are few men more kind & sensitive towards women's body issues," she tweeted to McGuire, who subsequently apologised. "My concerns came from what I know of his humor, and not what I know of him as a person."

Gaiman, creator of the Sandman comics and author of the bestselling novels American Gods and Anansi Boys, has now spoken out about the issue. Gaiman said that he was "seriously disappointed in the people, some of whom I know and respect, who stirred other people up to send invective, obscenities and hatred Jonathan's way over Twitter (and the moment you put someone's @name into a tweet, you are sending it to that person), much of it the kind of stuff that they seemed to be worried that he might possibly say at the Hugos, unaware of the ironies involved".

While he understands fans' concerns about Ross – "I'd had it myself, 25 years ago, when Jonathan and I had first met, and he asked me and Dave McKean to be on his chat show to talk about Violent Cases. I said 'No, you make fun of people. This is comics. It matters to me. I don't want you making fun of it.'" Gaiman wrote on his blog that Ross is actually "a huge SF and Comics fan: in many ways, one of the most fannish people I know".

The convention, which will be held in London this summer, should have warned Ross that some people were likely to have a problem with him as presenter, said Gaiman, but "as it was, he and his family didn't know what hit them".

"Twitterstorms are no fun when people are making up things about you or insulting you for things you didn't do or think or say. When scores of people from a group that you consider yourself a part of are shouting at you, it's incredibly upsetting, no matter who you are," blogged Gaiman. "I sympathise with anyone who felt that Jonathan wasn't going to make an appropriate Hugos host, and with anyone who spoke about it to the convention committee, but do not believe a campaign aimed at vilifying Jonathan personally was wise or kind."

The author, who has won Hugo awards himself, went on to explain how the row has changed his relationship with the science fiction community.

"I have won Hugo awards, and I am incredibly proud of all of them; I've hosted the Hugo awards ceremony, and I was honoured to have been permitted to be part of that tradition; I know that SF is a family, and like all families, has disagreements, fallings out. I've been going to Worldcons since 1987. And I know that these things heal in time," Gaiman blogged. "But I've taken off the Hugo nominee pin that I've worn proudly on my lapel since my Doctor Who episode, The Doctor's Wife, won the Hugo in September 2012, and, for now, I've put it away."

The convention, meanwhile, issued a series of apologies to Ross, his family, and SF fans in general. "We invited [Ross] in good faith, as a creator and fan, and a high-profile member of our community, to be Master of Ceremonies at the Hugo awards ceremony, and he graciously accepted, offering us his services as a volunteer," organisers said. "Having invited him to take the role, we failed to brief him about the recent debates in fandom, and failed to help him deal with the controversy which ensued after we announced his participation. He and his family have had a horrible few days, and it was our fault for putting him in that situation."

The organisers said they also regretted "any and all offence caused to those who disagreed with our choice of Jonathan Ross, those affected by the exchanges that followed on social media, and those who are disappointed that he has now withdrawn".