I didn't go there. - I did not go to the place I'm referring to, but I may have gone in the past, or I may go in the future.I never go there. - Not at any time ever did I go to the place I'm referring to, nor do I plan on going there in the future.