Ah fair point. Surely there'd still be some amount of drip feeding going on if the site only gets crawled every 2 weeks or something though? Or I could ping the links when I remove them from robots.txt?

Note that adblockers might block our captcha, and other functionality on BHW so if you don't see the captcha or see reduced functionality please disable adblockers to ensure full functionality, note we only allow relevant management verified ads on BHW.