I disabled the fuse circuit and replaced the kitchen ceiling light fixtures. I turned the power back on and the light stays on all the time, even when the switch is turned off. I replaced the switch and it still stays on.

ThingMaker3 is correct. An electrician will usually wire from the circuit breaker/fuse to the fixture and from there to the switch. This saves on the amount of wire that needs to be used. But be aware that this hot may be from a seperate switch to control a fan in a ceiling fan/light circuit, or may be an entirely different circuit.