It seems weird that there should be two vibrational degrees, but there is vibrational potential energy and vibrational kinetic energy.

I believe that the two "vibrational" modes are vibration and libration. The true vibrational mode is akin to a spring compressing and expanding, while the librational mode is akin to that spring flexing back and forth.

I believe that the two "vibrational" modes are vibration and libration. The true vibrational mode is akin to a spring compressing and expanding, while the librational mode is akin to that spring flexing back and forth.

No, libration is restricted rotation, when an external torque opposes full rotation. No such exists in case of free diatomic molecules.
For any vibrational mode, Equipartition Principle assigns 2(1/2 kT) average energy, which corresponds to two degrees of freedom. The energy of a vibration mode is the sum of the kinetic energy and the potential energy. In case of translation and rotation, the potential energy is zero, therefore only 1/2 kT energy is assigned to the three translational directions, and also 1/2 kT energy for the rotation about an axis with moment of inertia I>0.
O2 and N2 also have vibrational degrees of freedom. But the vibration contributes to the heat capacity at high temperatures only, when kT exceeds the excitation energy of these modes.