Lowering the buffer size only helps mitigate some audible latency within the software. There are other factors to consider that can play a role in latency, as in the driver you have installed and the type of sound card you are using as well as your computers hard drive speed, processor and ram. 512 is typically middle ground for lowering the buffer on a USB interface, if it's not allowing you to lower further then I would look into the other factors I mentioned.