How to ascertain our legal right to block the pathway and retrieve the land? What are the legal options available to us?

Situation :
Before asking the question I would like to give a brief case history of our property dispute with our neighbour.
We have a residential house of our own in which we stay and it has open space around it. The total area is around 12000 square feet. We had put up a wired fence around it earlier. There was a private pathway which went through the centre of our property to the neighbours house. It is not a public pathway but we had allowed them to use it out of good will. Now they claim it is their right to use that path citing they have been using it for the past many years. We wanted to block the path way and use the entire land which belongs to us by constructing a compound wall. Our bitter neighbour objected and forced us to make two compound walls on the eastern side where their house is. The two walls are separated by a width of 3 feet and runs around 12 feet long. This was done to create another pathway for them within our property. They have another pathway which they use now, which goes through another person's private property. We want to remove the inner compound wall of that 3 feet path and use the entire space for ourselves.
But our neighbour is objecting to it.