When the Idle changes, does it go up or down from where it normally is?

It goes down, like it's struggling. I noticed it at a red light as if using the brakes was straining the motor. Could it have to do with the belt? That's the one thing I've actually fixed since the mash-up.