I have a galaxy s6 and it works 100% perfectly with my phantom 4. No lag at all (except for when it loses signal, of course)
The galaxy S4, as dronedestination said, is 3 years old now. It cannot support modern apps and power-hungry apps like DJI GO and litchi and other apps

My dinosaur S4 with a shattered screen is working fine for me. It's not ideal (screen) but I get smooth FPV and no app issues. Is it relatively cleaned out and do you have any memory hogging apps closed?