Potential Energy and stability

Okay
We all know that the most stable state of a system (say an object undergoing SHM)is when it has minimum Potential Energy.
Can somebody tell me why a local minimum in the potential energy correponds to a higher stability than some other arbitary state?
(Not too much of quantum theory please!!)

It's simple really. The force exerted by a given potential is equal to minus the gradient. If you're sitting at the bottom of a potential well the potential is rising when you move away from the minimum, thus the force will push you back to the minimum. If you're at the top of a potential hill then when you move away from the maximum the force will continue to push you away.

So minimums are stable because small motions away from the minimum will tend to push you back towards the minimum whereas maximums are unstable because small motions away from the maximum well tend to push you away from the maximum.