There's a lot to find in the underground passages and it doesn't sound like you are doing anything wrong. The passageways have four entrances. If you look at MaG's walkthrough, you will see that the top one leads to the dungeon, the bottom/right one leads to the forest, the leftmost passage leads to Nancy's room, and the other passage leads to the well. I would just continue to explore.