Hm, true, voltage is impossible without current flowing because of how
voltage is defined. No current <--> no voltage, and voltage is
something that doesn't exist. I never thought of it like that before.

Incorrect. Voltage is strictly an excess or shortage of electrons.
Current flow is not necessary for voltage to exist.