I have been unable to anticipate any case where this is the most desirable behavior.

Am I missing something?

This is causing me considerable frustration as I try to specialize a library, since if the user asks for a single type that I omit, everything defaults to the slowest possible case. So the user (e.g. me) has to remember exactly what is specialized and what is not, and wrap the non-specialized ones by hand in order to gain the benefit of the arguments that are specialized.