Experimentally and mysteriously, the concentration of quasiparticles in a gapped superconductor at low temperatures always by far exceeds its equilibrium value. We study the dynamics of localized quasiparticles in superconductors with a spatially fluctuating gap edge. The competition between phonon-induced quasiparticle recombination and generation by a weak non-equilibrium agent results in an upper bound for the concentration that explains the mystery.