Removing the Adhesive Backing Residue After Stripping Jersey

I am in the middle of stripping a jersey and I stripped part of a number and started peeling it off slowly only to see an annoyingly noticeable adhesive backing residue. What is the best way to get it off? Any help would be much appreciated!

Here is the pic of the annoyance! Maybe hard to see in the pic, but easy enough in real life!