A winner from Dirk Kuyt two minutes from time put Liverpool into the FA Cup fifth round at the expense of their fiercest rivals. Manchester United controlled midfield for most of the game and seemed to have done enough to take the game to a replay at Old Trafford that no one really wanted, but when Patrice Evra was caught out of position the Liverpool substitute spared everyone another week of rehearsed hostility.

Liverpool have now knocked both Manchester clubs out of cup competitions in the space of four days, which counts as success in these parts whatever the league situation. Last week Kenny Dalglish was scathing of his players' efforts in losing to lowly Bolton Wanderers, this time the smile was back, at least until he was asked a question about the Liverpool fans' constant booing of Evra.

"It was a fantastic effort from our players, especially after what they went through on Wednesday night against Manchester City," Dalglish said. "It was hard to be asked to play an early kick-off on a Saturday after that but the commitment and the effort were fantastic."

Sir Alex Ferguson felt defeat was hard to take, especially as there was a suggestion (not proven) of offside about the winning goal. "I haven't seen a replay yet but you don't always get decisions here anyway," the United manager said. "We never got anything from the referee. I felt we dominated the game and at least deserved a replay, though I was still trying to win it at the end – that's why I put Javier Hernández on. We played well, hit a post – it's hard to believe we lost when we were so comfortable."

Before considering the football, what there was of it, you will be wanting to know about the chanting. Liverpool obviously won the volume contest. There were far more people singing: "There's only one lying bastard," than were rejoining with: "There's only one racist bastard."

In terms of originality, the home fans gave full voice to their new slogan of "We're not racists, we only hate Mancs", but were stunned into something approaching silence midway through the first half when the United support at the Anfield Road end came back with: "You're always the victim, it's never your fault." A score draw, perhaps. Dalglish said it was nothing but friendly banter, which made you wonder what would strike the Liverpool manager as unpleasant.

With all the pre-match debate focusing on Evra's participation, United's goalkeeping situation had been overlooked, Ferguson sprang a surprise by choosing this high-profile encounter to reintroduce David de Gea to first-team duties. It was not a decision that paid off.

Liverpool began to crowd out the young goalkeeper at corners and took the lead after 21 minutes when Steven Gerrard swung one over from the left and De Gea came for the ball but found himself blocked by Andy Carroll, leaving Daniel Agger an easy header.

United had begun the match looking the more threatening, with Antonio Valencia striking a post after a good run, though until the first goal there was little apart from the rivalry between fans to quicken the pulse and almost no chance of the occasion being remembered for the football, as had been the wish of both managers beforehand.

Playing with only Carroll up front Liverpool rarely bothered De Gea apart from set pieces, and though United also lacked their usual attacking edge in the absence of Wayne Rooney they managed to get back into the game with a well-worked goal before the interval. Antonio Valencia sent Rafael da Silva racing down the right with a backheel flick and, though José Enrique was still favourite to reach the ball first, the United full-back took on his opposite number and delivered a low cross, met perfectly at the near post by Park Ji-sung with a firm finish that surprised Pepe Reina.

Liverpool had an appeal for a penalty turned down at the start of the second half when Chris Smalling stumbled in the area and appeared to put his hand on the ball in falling, then a minute later a rare chance fell Danny Welbeck's way and he failed to take advantage, pushing the ball past Reina towards an empty goal but not firmly enough to prevent Martin Skrtel arriving to tidy up behind his goalkeeper.

Charlie Adam came on for the last half-hour and immediately offered a more direct threat than Jamie Carragher had managed from defensive midfield, launching a high cross intended to test De Gea and presumably gaining encouragement when the goalkeeper fumbled it to concede another corner. When Gerrard tried the same tactic from a similar distance it was clear Liverpool had been instructed to play on De Gea, though the Liverpool captain left the field shortly afterwards, making way for Craig Bellamy.

Just when it looked as though the afternoon would end in a tame draw, Liverpool struck. Carroll beat Jonny Evans to get his head to a high ball, the knockdown fell ideally for Kuyt, and the Dutch substitute lost no time in burying a shot beyond the exposed De Gea. At least no blame attached to the goalkeeper for the winning goal, though he might have been embarrassed had Carroll added another before the end, instead of hitting the bar.

"That's Dirk's 50th goal for Liverpool, and he's scored it against Manchester United, so it's a great day for him," Dalglish said. "It's been a great week for us in the end, even though it started so badly at Bolton. We are in the Carling Cup final and the next round of the FA Cup, and though I was critical of the players a week ago, full credit to them for getting back on track with two really big wins."