Controlling inactive icons in the "notification area"

When you right click on the taskbar and choose properties, you get the "Task Bar and Start Menu Properties" box. At the bottom of the box (while under the "taskbar" tab) There is a "Customize" button which allows you to specify which icons you want hidden when inactive, or always hidden or never hidden etc... This we all know.

My question is this: There are quite a few "Past Items" that I no longer have the program installed for, how do I remove these?