rear sight on my STI

So, after the first couple trips to the range with my new STI Ranger II I noticed the rear sight seemed a little far to the right. I blew it of because the weapon was hitting in the spot I aimed for. I went again a couple weeks ago and now the rear sight is starting to protrude bast the right side or the slide. So, do I need a tool to adjust this sight, or if I loosen the set screw on top and use a wood dowel and tap it back will that work. I would like to adjust it at the range to sight it back in. i guess the set screw was not tight enough from the factory.