For symmetrically dividing cells, large variations in the cell cycle time are typical, even among clonal cells. The consequence of this variation is important in stem cell differentiation, tissue and organ size control, and cancer development, where cell division rates ultimately determine the cell population. We explore the connection between cell cycle time variation and population-level fluctuations using simple stochastic models. We find that standard population models with constant division and death rates fail to predict the level of population fluctuation. Instead, variations in the cell division time contribute to population fluctuations. An age-dependent birth and death model allows us to compute the mean squared fluctuation or the population dispersion as a function of time. This dispersion grows exponentially with time, but scales with the population. We also find a relationship between the dispersion and the cell cycle time distribution for synchronized cell populations. The model can easily be generalized to study populations involving cell differentiation and competitive growth situations.