In this paper, a new method for a robust and efficient analysis of video sequences is presented; it allows the extraction of foreground objects and the classification of static foreground regions as abandoned or removed objects (ghosts). As a first step, the moving regions in the scene are detected by subtracting to the current frame a background model continuously adapted. Then, a shadow removing algorithm is used to extract the real shape of detected objects. Finally, moving objects are classified as abandoned or removed by matching the boundaries of static foreground regions. The method was successfully tested on both real image sequences acquired in our laboratory and some sequences from the PETS 2006 Datasets.