You tramp through a muddy copse, remove the old manhole cover and clamber down a narrow steel ladder. At first it feels dank and the rungs of the ladder slippery but soon the clay soil turns to a gleaming white and at the bottom there is a wide cavern with crumbly walls that bounces the torchlight back and forth.

These are the chalk mines of Reading - exciting adventures for caver-types who like exploring underground nooks, but increasingly worrying to the residents of the town who live above the network.

A labyrinth of mines was created in and around Reading from the 18th century on to extract chalk for bricks. But over the next 200 years people forgot where most of the mines were and houses were built over many of them. Now mysterious holes have opened up across the town. Walls give off odd creaking sounds as foundations shift and, in extreme circumstances, bits of houses have vanished.

Millions of pounds have been spent shoring up the properties and streets that have been obviously affected but this week engineers warned that more needed to be done to map the forgotten mines.

The borough council has a firm idea of the positions of several old workings but it is feared there could be as many as 40 mines beneath the town. The authorities are in a quandary. If they go looking for more mines they may cause panic, prompt house prices to fall and find themselves having to find millions of pounds to make areas safe. If they do not go looking, disaster could follow.

In the suburb of Emmer Green, above the Hannover Mine cave complex visited by the Guardian, residents are tetchy rather than panicky. "I'm surprised you are interested," said Diana Hartrup, who lives close to the mine entrance.

Ms Hartrup, however, was herself interested enough to allow civil engineers on to her land a few weeks ago to try to work out what is going on underneath. She insists she is not worried. "We've been here since 1967 and never had a problem. And the mine has been here for two or three hundred years. It's not a worry."

Another local, Steve Jones, said: "Oh, you've come about the mine. We don't really like to talk about it. It's a bit like our guilty secret."

On the opposite bank of the Thames in the Field Road area, 30 homes were evacuated in the winter of 2000 after the fronts of two houses caved into an old chalk mine. Some families had to stay away for two years while engineers worked out what had happened and made them safe. The area is still officially earmarked a "derelict land clearance area" as well as a conservation area.

David Warner, who had to move out of his house with his wife and three children, recalls going into his garden to find a hole 3m square and 9m deep. "I was very surprised, to say the least. Up until that time we had no idea the house was built on mines."

The level of anxiety being experienced by other residents depends on what colour their homes are on a survey map of the neighbourhood. If their home is in red it is built on mine workings. If a vivid orange, then their property is "probably affected" by mine workings. If turquoise they "may possibly" be affected.

Joe Doak's house on Field Road is turquoise. "The council has done such a lot of work here that we don't feel under threat. One thing I have been surprised at is that they didn't seem to tap in to local knowledge. There are older people who remember where mines were - some played down them as children," he said.

Clive Edmonds, of the civil engineers Peter Brett Associates who have worked with Reading council over the chalk mines, said they tended to be quite stable when they were built, but 150 or 200 years on could break down.

Peter Brett Associates is continuing to search for new mines and believes there could be as many as 40 - a figure which surprises the council. Mr Edmonds warned they could not be ignored as they posed a risk to property and even to life.

Happily for residents in Field Road, their area is probably one of the safest in the town. Mr Warner said: "They've done so much work here now that we feel safe. Perhaps the rest of the town will fall away and we'll be left here like a little island."

A Reading council spokesman said: "The council is aware that academic research, independent of the council, is ongoing into the possibility of further mines . The council will obviously keep a close eye on the results of this research."