Clicking on MapBox when in Pan mode

Hopefully an easy question. When in Pan mode and the user single left clicks on the map it moves the map off to one side or another based on where they click. Is there a way to filter this behavior out while in Pan mode?