Contactless payment card security

I've just been asked something interesting about the security of contactless payment cards.

In short, what's to stop scammers simply scanning your card while it's in your wallet? I guess much the same could be asked of NFC-enabled mobile phones, particularly those relying on the SIM?

I must confess it's a question I don't have any ready answers to, so would appreciate any views on it. We've had hundreds of cases of thieves attaching card scanners to ATMs to glean card details, so understandably people are a little concerned about this new development.

I'm pretty sure existing security mechanisms will still work, and would be shocked if no-one has addressed this when deploying contactless cards/handsets for payments. Either way, be good to get some definitive information on the matter.