Help with Voice Actions / Car Mode

There is one thing that has been keeping me from really making the best (and probably safest) use of my car dock for my Droid, and that is the voice search does not seem to search my contacts.

To give an example, with voice search I try saying "navigate to [contact name]" instead of looking through my contacts it does search via Google Maps for the phrase I gave it, which is basically useless. Why does it not try my contacts first and use Maps/Web as a backup? Is there a way to force it to do this? Do I use a different command phrase to have it navigate to a contacts address?

Interestingly it knows to search my contacts for other voice actions, like telling it to dial or send a text to a contact name works just fine.