It may be annoying, but from their point of view... If the advertiser were to know how much traffic is coming from one specific url, they may get a better deal by visiting that site and advertising directly.

Advertiser would not use adwords, but go directly to the profitable site!

Perhaps, but in the majority of cases the advertiser would have to form and maintain relationships with dozens-hundreds of sites to match what adwords provides. And on the publisher side, their bulk of revenue is probably coming from dozens-hundreds of ads, so they would have work hard to maintain a profit over what adsense gives them.

Advertisers come and go. The overhead for maintaining all this is high, likely more than what initially appears to be savings by circumventing adwords/adsense...