At first I couldn't tell what program was inserting these, so I downloaded my newly updated site to a local drive, and used Notepad++ to do a file search.

It turns out this is due to the new admin bar in WordPress 3.1, and you can only see it if you are logged in as an admin, your users can't see it. Some people think this is a bug in WordPress 3.1, but it's not.

If you want to get rid of this bar, put this line in your functions.php:

add_filter( 'show_admin_bar', '__return_false' );

You can hide parts of the admin bar with CSS. To remove the comments and the appearance options add this to your stylesheet: