After more than a decade of revelling in the reputation of being aviation's least likeable man, Ryanair boss Michael O'Leary has finally admitted that his image, and that of his airline, may be a problem.

The abrasive Irishman has in the last year responded to negative publicity by branding customers stupid for making "fuck-ups", and has long scorned fat people, environmentalists and Guardian readers. He has often argued cheap fares and punctual flights are the only real requirements for customers. But, he told shareholders at the annual meeting on Friday, the no-frills airline should henceforth try not to "unnecessarily piss people off".

The consumer magazine Which? this week voted the firm worst for customer service of Britain's 100 biggest brands.

Shareholders have complained that the airline's "macho" image is harming its business. O'Leary, who has hosted press conferences in bed with models and encouraged crew to strip for the Ryanair calendar, replied that he was "very happy to take the blame or responsibility if we have a macho or abrupt culture".

He added: "Some of that may well be my own personal character deformities."

Shareholders told of family members refusing to fly Ryanair and verbal attacks they suffered at dinner parties for their involvement with the airline. One told the meeting: "I have seen people crying at boarding gates. There is simply something wrong there that needs to be addressed."

Earlier in the day, front-page stories in Ireland told of a surgeon who had been charged £160 for an early flight home after hearing his entire family had been killed in a house fire.

O'Leary apologised and said the customer would be refunded and similar cases would be dealt with more quickly in future. His about-turn on customer care comes three weeks after Ryanair was forced to admit its profits will be lower this year after a downturn in bookings.

The airline has also said it will be slower to fine customers for carrying slightly oversize bags and will reconsider the way it communicates with them. O'Leary has admitted his wife would like to take more luggage than he feels necessary when they fly Ryanair and that he found it irritating when staff fine customers for large hand luggage. "If it's a millimetre over size, get on with it. We are not trying to penalise people for the sake of a millimetre," he said.

It will also make its website easier for passengers to use, removing the security feature it implemented to stop other flight websites checking its fares. The mobile phone app will be free to download.

More than a quarter of Ryanair's turnover is from ancillary revenues – charges for services such as using credit cards, priority boarding and checking in baggage.

The airline has been notorious for its lack of response to complaints, but will now set up a team to respond to emails and has joined Twitter, a medium frequently derided by O'Leary. However, responding to the Which? survey, the official Ryanair account tweeted: "Just two #Ryanair passengers out of 3m surveyed have ever heard of Which magazine … but neither of them have read it."