Wow, what nice spammers you meet: give you real addresses. Mine all
use fake sending addresses, so would never receive any challenge I
sent.

If challenge response ever becomes ubiquitous, then spammers will
trivially be able to verify the responses without providing their own
email address. They will simply do what the currently do - open up
millions of backdoors on cracked computers, go through the address
books to look for email addresses, then send using a From: of the
current computer. An MTA running via the backdoor will pick up an CR
attempts, respond to them, and voila, send spam to a verified email
address.

I didn't say everyone was doing this, but as you suggested, enough of
them were doing it already that it became ineffective. I was able to
get fewer spam delivered to my INBOX through standard filtering means
than I was able to block through Challenge-Response techniques.

The most annoying problem is that if you have one spammer who sends you
a real address then he will promptly send you hundreds of email in a
day. One little leak results in a flod of activity and usually of the
very worst kind imaginable.