will the Koss solution work for in-call volume? that is, can it make the output volume actually be less than the minimum volume provided by the phone?
–
AlexMar 15 '14 at 3:00

Also, what if I don't wan to lose the functionality of the controls on my headphones.
–
Seanny123Oct 3 '14 at 16:19

The answer to both comments is yes. This is a "dumb" physical device that reduces the signal going to the headphones. It doesn't care what mode your phone is in and shouldn't interfere with the functionality of other devices.
–
octernDec 11 '14 at 16:05

For me the app Fine Volume Control was the solution.
It does not need root, unfortunately is not free ($ 2.50) but provides a 7 day trial.

However it does a great job: it subdivides the media volume range in 100 steps, allowing a much finer resolution for your volume choice including enabling to shut the volume to almost nothing. In the setting there is also the option to enable extra low volumes.

Overall it solved the problem for me perfectly and made my phone a much better music player, thus it is worth every penny. To my knowledge this is the only app with this functionality at the moment.

On rooted phones, it's possible to install custom kernel which can lower the volume setting lower than is available stock. You might want to look around for custom ROM for your device, if it's a common problem it's highly possible someone would make a custom ROM to modify the volume setting. You'd generally want to try other solutions first though.