This paper presents a preliminary approach for environment exploration by a robot in small but cluttered spaces like a bookshelf or a refrigerator. Since the environment is only partially observable, the robot must adopt an observe-plan-act strategy and interleave planning with execution. The goal is to know the state of every voxel - occupied or free - in the region of interest. In this paper, we present a 2D planning solution in a 3D world which implies that the objects are 3-dimensional but are not stacked on top of each other. We explore different versions of a multi-step look ahead planning approach and compare them using various metrics. We also present an implementation on the PR2 robot by Willow Garage and observe that the planning is fast and reliable.