The referee Stuart Attwell is surrounded by Watford players after awarding the so-called ghost goal to Reading in 2008. Photograph: Ian Walton/Getty Images

Stuart Attwell has lost his place on the list of Select Group of referees and will return to Football League duty on the national list.

The move, which is said to come by mutual consent, comes after the 29-year-old made a string of high-profile errors after being promoted to the elite group of referees at the start of the 2008-09 season aged 25.

Only weeks into that season Attwell attracted heavy criticism for awarding Reading a so-called ghost goal against Watford when John Eustace had kicked the ball over the byline four yards wide of the goal before it was put back into play by Noel Hunt. Attwell's assistant Nigel Bannister signalled for a goal.

More recently, in December, Attwell sent off Gary Cahill, then of Bolton Wanderers, for fouling Scott Parker only yards over the halfway line, saying he had denied Tottenham Hotspur a clear goalscoring opportunity. The red card was overturned on appeal.

In a statement on Wednesday night Mike Riley, the former referee who is now general manager of Professional Game Match Officials Limited, backed Attwell to improve as a referee and return to the top level. "Throughout his career in the select group Stuart has demonstrated great courage and mental strength in responding to the challenges that he has faced," Riley said.

"He has shown many of the attributes of a top-flight official but the Premier League is an exacting standard with huge demands. It is imperative that PGMOL deliver the best officiating possible to the English game at all levels.

"This is not the end for Stuart. There are several match officials who have returned to the National List and benefited from it. Stuart has a high level of maturity and responsibility and I'm convinced that he has a long-term future as a referee at the very highest level. PGMOL sincerely hopes to be welcoming Stuart back to the select group in the future."

Attwell said: "I have learned a great deal from my involvement in the select group over the last four years and I am now looking forward to building on that valuable experience.

"I strongly believe I have a lot to offer refereeing and I now look forward with commitment and enthusiasm to the new challenge that I am undertaking. This is supported by a determination to continue my career development and return to the select group as soon as possible."

Attwell spent only one year on the national list before his promotion to the select group four years ago. He made his Premier League debut as a referee in the 1-1 draw between Blackburn Rovers and Hull City in August 2008.

After the Watford-Reading game further controversies followed when he was accused of "losing control" of a Derby County v Nottingham Forest derby, having disallowed two Derby goals in the final stages, booking eight players and sending off Forest's Lewis McGugan.

He has also attracted the ire of Wigan Athletic's manager, Roberto Martínez, who accused the referee of lying about his reasons for sending off Gary Caldwell in April 2010, while Stoke City's Danny Higginbotham claimed Attwell could be too easily influenced by players.

Later that year he awarded Liverpool a highly controversial goal against Sunderland. He had awarded the Black Cats a free-kick inside their own half and Michael Turner touched the ball back to Simon Mignolet, apparently for the goalkeeper to take the set piece.

But Attwell ruled the ball active, allowing Fernando Torres to steal in and set up Dirk Kuyt to finish into an unguarded net.