Solar cells incorporating lead halide-based perovskite absorbers can exhibit impressive power conversion efficiencies (PCEs), recently surpassing 15%. Despite rapid developments, achieving precise control over the morphologies of the perovskite films (minimizing pore formation) and enhanced stability and reproducibility of the devices remain challenging, both of which are necessary for further advancements. Here we demonstrate vacuum-assisted thermal annealing as an effective means for controlling the composition and morphology of the CH3NH3PbI3 films formed from the precursors of PbCl2 and CH3NH3I. We identify the critical role played by the byproduct of CH3NH3Cl on the formation and the photovoltaic performance of the perovskite film. By completely removing the byproduct through our vacuum-assisted thermal annealing approach, we are able to produce pure, pore-free planar CH3NH3PbI3 films with high PCE reaching 14.5% in solar cell device. Importantly, the removal of CH3NH3Cl significantly improves the device stability and reproducibility with a standard deviation of only 0.92% in PCE as well as strongly reducing the photocurrent hysteresis.