In Linux Mint 13, the update manager (the shield icon) always appears in the notification area at the right corner of the bottom panel. Even if you right click on it to quit, it will reappear the next time you boot. To permanently remove this icon, you will need to go to the Menu (in both MATE and Cinnamon), search for Startup Applications. On this window, scroll to the Mint Update option then uncheck this box:

After that, the update manager wont appear again in the notification area anymore. If you need to check for update, just open the Menu, go to the Administration category and click on the Update Manager option.