Not sure about granting people immunity... people do use it in pk, and even if you restrict their attacking ability like a poster above said, it could basically be used to cheat death in panic moments. People also use it to anger explosion town or nova town. A rare strategy but fun to do.
Maybe the stealth fade-out duration could be made so it takes at least 1.5 seconds to lose the invisibility as opposed to normal stealthing effects, but I think that might break regular rogue stealth.
The best solution is probably to prevent genie from locking attacks into players or monsters if you are stealthed.