I agree completely – it is much easier to get a quiet mix that translates well to loud, than to mix loud and get a mix that translates well to quiet.
I’ve added a small wrinkle to this idea. Since many (or most) music is listened to in environments with fairly high ambient noise, I’ve found that mixing *very* quietly and adding in pink noise at -10 or -15 dB relative to the music RMS level allows one to make sure everything is heard even with masking noise present. It could be argued that this will reduce dynamic range, but most music has less than 10 or 15 dB range anyway.
There are exceptions, of course. Classical, small ensemble jazz, opera really do not benefit. But most other genres do, in my humble opinion.