Help! Bingbot is crawling sectors of my site too much!

I'm new to this kind of stuff so I need your help. I have published website and manage to get it indexed in Google and Bing. However, Bing bot is crawling private sectors of my site and i want to know how to stop it. It also keeps on over-crawling my site and I'm afraid that it might eventually cause an increase on my server load. Please help, any advice will be really helpful! TIA

Help! Bingbot is crawling sectors of my site too much!

Okay here's the thing, if Bing bot determines that your website has a slow connection, it automatically adjusts how often it crawls your website. However, you can increase the delay between crawls by using the Crawl-delay parameter in the Robots.txt file to specify a minimum frequency in seconds. Since you mentioned that you're new to this business, try asking for assistance from a SEO or other webmaster on how to configure this. Look at the example below:

User-agent: MSNBot/Bingbot
Crawl-delay: 120

Individual crawler sections override the settings that are specified in * sections. If you've specified Disallow settings for all crawlers, you must add the Disallow settings to the MSNBot/Bingbot section you create in the Robots.txt file. For example, your Robots.txt file might have the following:

User-agent: *
Disallow: /private/

If you add an MSNBot/Bingbot section, you must add any Disallow settings to the MSNBot/Bingbot section. For example:

User-agent: MSNBot/Bingbot
Crawl-delay: 120
Disallow: /private/

This is are the same robots.txt configuration we use on our website so I hope this helps.

Techyv is one of the leading solution providers covering different aspects of Computers and Information Technology. We have a hardworking team of professionals in different areas that can provide you with guaranteed solutions to a blend of your problems. We have a dedicated and devoted team of professional writers with multi-dimensional experience of several years. As a result, we produce quality content on a variety of subjects.