In many realizations of electron spin qubits the dominant source of decoherence is the fluctuating nuclear spin bath of the host material. The slowness of this bath lends itself to a promising mitigation strategy where the nuclear spin bath is prepared in a narrowed state with suppressed fluctuations. Here, this approach is realized for a two-electron spin qubit in a GaAs double quantum dot and a nearly ten-fold increase in the inhomogeneous dephasing time \(T_2^*\) is demonstrated. Between subsequent measurements, the bath is prepared by using the qubit as a feedback loop that first measures its nuclear environment by coherent precession, and then polarizes it depending on the final state. This procedure results in a stable fixed point at a nonzero polarization gradient between the two dots, which enables fast universal qubit control.