The "Writing Posts" page on the codex says Wordpress will deal with quotes and other characters automatically, and proceeds to give an example title that will actually result in the problem - "My Site - Here's Lookin' at You, Kid".