It should be fine. You may not even be able to taste the sheng in the pot. If you can, since it was only the few infusions one good shou session would probably cover it back up.

I use sheng in my shou pot from time to time. I also know a lot of people have a relatively broad spectrum of teas they'll use in the same pot. If these leaves taste good with this pot, they'll use it, despite what they may usually brew in the pot. From my experience you'll always predominantly taste the tea you're brewing. Any seasoning in the pot will just be background.