You can remove rust with lemon juice or vinegar - acid is the trick here. Allow a strong acid to sit on the rust spot for a few moments. You may have to apply acid to the same spot a few times, but it will eventually disappear. Photo credit: ihaven'tseenbarbados on Flickr.