Alex Lawless scored the goal for Luton Town that knocked out Wolves in the FA Cup third round. Photograph: Tom Jenkins for the Guardian

There are football supporters who talk about having seen it all and then there are Luton Town fans, who pretty much have. There was the chairman who proposed building an F1 track around a 50,000-seat stadium off the M1 as he drove the club towards enforced bankruptcy, and another who banned away fans and wanted cat o' nine tails used on hooligans. There were the joys of spring – a Littlewoods Cup victory and a Johnstone's Paint trophy – and a deep midwinter following three administrations, 40 points worth of deductions and four relegations in less than a decade.

At one point, Luton also faced 55 charges from the FA for systematic abuse of transfer regulations and owed the taxman £3.5m. But after resembling a cartoon character who falls so fast off a large cliff that they drill through the earth's surface and are nearly fried by its core, the club are beginning to gaze upwards again. Luton 2020, a group of investors led by the TV presenter Nick Owen, have stabilised the finances. Now their players' exploits in the FA Cup are starting to loosen ties and tongues.

In the third round, non-league Luton beat Wolverhampton Wanderers 1-0 through a second-half goal from Alex Lawless. It was no fluke. Andre Gray, a 21-year-old striker signed from Hinckley for £30,000, worried Wolves' defence every time he got the ball, while Lawless, a neat footballer at the heart of Luton's carousel, ensured they matched up in midfield. Everyone else grinded, mucked in and allowed the home side to compete on level terms.

But beating a Wolves team short of confidence in front of a braying Kenilworth Road crowd is one thing; facing Norwich City, a Premier League club sat 84 places above them in English football's league ladder, quite another. No wonder the Luton manager, Paul Buckle, is refusing to get carried away. "Wolves was a big marker for us," he says. "It showed we could cause problems against a higher level side. But we're up against a Premier League team and the gulf in class is there to see."

Norwich will provide a stiff test and a welcome reunion. Buckle – the journeyman's journeyman as a player, with 10 lower league and non-league clubs on his CV – started his career at Brentford, where the Norwich manager, Chris Hughton, was the senior pro. "I used to travel in with him," he says. "He is a really good guy, a proper man. I'm not surprised he's become an excellent manager. Knowing Chris like I do, they will be well prepared. They will not underestimate us."

That means Luton, sixth in the Blue Square Premier, go into the game with little except history on their side. Since 1985, 12 teams have played in three or more FA Cup semi-finals – a list that includes Manchester United, Chelsea, Arsenal, Spurs, Liverpool, Everton – and a huge outlier, Luton, who reached the last four in 1985, 1988 and 1994. They also have form for giving big boys the heebie jeebies: in 2006 they were 3-1 ahead against Liverpool, then European champions, before losing 5-3, while they also took Liverpool to a replay in 2009, the season in which, having been docked 30 points, they slipped out of the Football League.

Could they surprise everyone again? The bookies make them 10-1 outsiders, which suggests not. But, as Bradford have shown in the Capital One Cup, sometimes an underdog does have its day. "They proved that lower-league clubs can take on the Premier League sides and win," says Lawless. "And remember, they are only 20 places or so above us. If they can do it, who knows?"

Gray, meanwhile, was bullish enough to claim that Luton were a "sleeping giant" this week – although, given that he was stood in the Eric Morecambe lounge, he may have been trying to tickle the funny bone of the club's most famous ever fan. Then again, perhaps not. "I showed what I could do against Wolves but Premier League defenders are another level again," says Gray, who interests clubs further up the food chain. "But this club is massive. It has a massive history."

Context is key here: Luton are never going to be threatening clubs such as Manchester United, say, but their home gates are usually north of 6,000, more than most League Two clubs, and they took 40,000 fans to the Johnstone's Paint Trophy final in 2009. As their goalkeeper Mark Tyler points out: "Everyone knows we should be playing in a higher league but the Conference is tough to get out of. But what Crawley and Fleetwood have achieved in League Two is what we are aiming for. It's not unrealistic given we have a great board and, touch wood, that we're financially stable."

That is usually a phrase to warm only an accountant's heart. But, over the past decade or so, Luton fans have become well acquainted with balance sheets and internecine battles. In December 2012 there were reminders of the grim past when Derek Peter, the club's finance director between July 2004 and February 2007, was kicked out of the Institute of Chartered Accountants for "grave" misconduct, which led to the taxman being owed £3.5m. Peter was one of four former Luton directors, including the ex-chairman Bill Tomlins, Richard Bagehot and John Mitchell, also to be banned from managing companies.

Those with deeper memories will recall David Evans, the right-wing Tory MP who introduced membership cards and banned away fans while chairman during the high Thatcher era. And John Gurney who, having bought the club for £4 in the summer of 2003, tried to rename it London Luton to attract more fans, sacked Joe Kinnear and introduced a "manager idol" vote for a new boss – a vote that was still going on when he was speaking to Mike Newell about the job. To the surprise of no one, Gurney later went bankrupt.

As Kevin Rye of Supporters Direct says: "The key difference now is that the club is run by people who care, and that's a very good starting point."

Most fans, however, remain angry at how the club was treated in those dark days. As Martin Wells, who has watched Luton for 30 years, puts it: "There is still a sense of injustice, a feeling that the club was hit harder than others with the points' deductions – because of the actions of people who should never have been allowed near a football club in the first place."

Nor are supporters completely won over by Buckle, who they believe has a tendency to blame poor pitches whenever his side lose and plays people out of position. "We don't understand why he often puts Jon Shaw, who scored 45 goals in two seasons at Gateshead as a right midfielder," says Wells. "When he stuck him up front against Wolves the difference was obvious."

That third-round win has helped shift opinion. Any sort of result against Norwich would have them hugging and whooping like in the good old days, when Luton's passing often ripped through the finest sides in the Football League and the cries of "Come on you Hatters" could be heard in the town centre.

"We don't want to be sat there at 4.45pm wondering what might have been," insists Buckle, who is then asked whether he would take a replay. "'Take a replay?'" he replies, allowing a warm smile to form across his lips. It was a smile that Eric Morecambe, watching from the photographs on the walls, would have recognised and loved.

Luton fans, meanwhile, travel to Norfolk hoping their team will bring them sunshine to keep them happy all the long. And they – all 4,000 of them – will keep on singing their happy songs.

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.