Recognizing Objects and their Properties

We want visual systems that can provide detailed descriptions and robust predictions of objects. In some cases, it may be sufficient to name the object. In others, we would like the computer to localize parts and infer pose. In many others, the object may be unfamiliar, and a rough description should suffice. For example, if an assisted driving program encounters a cow on the road, it should recognize it as a four-legged animal and predict its movement, even if it has not seen a cow before.

This project includes the following goals:

Build recognition systems that can recognize everything. Rather than applying detectors for specific categories of interest, the system should be able to localize objects, describe them, and infer coarse categories. With more experience, the system should get more detailed.

Precisely localize objects and infer their shape properties and functional affordance. Segmentation is important for inferring shape from contour, and shape is important to know what an object could be used for.

Create systems that are easily extendible and that learn more quickly with experience. We want systems that incrementally learn new categories, parts, and materials, building on what it already knows. This requires creating flexible learning techniques and good ways to relate different recognition tasks.