Hey friend! Have fun exploring Q&A, but in order to ask your own
questions, comment, or give thumbs up, you need to be logged in to your
Moz Pro account.
You can also earn access by receiving 500
MozPoints
from participating in YouMoz and the Moz Blog!

Bing Won't Index Site - Help!

For the past few weeks I’ve been trying to figure out why my client's site is not indexed on bing and yahoo search engines.

My Google analytics is telling me I’m getting traffic (very little traffic) from Bing almost daily but Bing webmaster tools is telling me I’ve received no traffic and no pages have been indexed into Bing since the beginning of December.

At once point I was showing ranking in Bing for only one keyword then all of a sudden none of my pages were being indexed and I now rank for nothing for that website. From Google I’m getting over 1200 visits per month. I have been doing everything I can to possibly find the culprit behind this issue.

I feel like the issue could be a redirect problem. In webmaster tools on Bing I’ve used “Fetch as Bingbot” and every time I use it I get a Status of “Redirection limit reached.”. I also checked the CRAWL Information and it’s saying all the URL’s to the site are under 301 redirect. A month or so ago the site was completely revamped and the canonical URL was changed from non www to www. I have tried manually adding pages to be indexed multiple times and Bing will not index any of the sites pages. I have submitted the sitemap to Bing and I am now at a loss. I don’t know what’s going on and why I can’t get the site listed on Bing.

I did check a small handful of pages and i did see <link rel="canonical" in the page source code. I only saw it on product pages and category pages etc. blog and other pages not advertising product i didn't see it.

But like i said i only checked about 8 pages and 6 out of the 8 had that tag on it.

It looks like the site was down for a long period of time and was redirecting to another site: http://web.archive.org/web/20090615041707/http://dunninggolf.com/

It also then looks like there was a holding page up for a while.

I can't see the complete history on archive.org and nothing since Feb 2011 but it looks like these historical redirects are your problem.

Few questions

How long has the new site been up?

Have you won any new links?

It seems that Bing still believes this site is redirecting somewhere else so you may just have to wait a while till they update their index. You could ask them to remove all of the links that they do currently hold.

If Bing webmaster tools is anyting like Google Webmaster tools the information there is horribly out of date so it may just be a matter of waiting this out till they recrawl the site and refresh everything.

A good use of your time may be just getting your on page ducks in a row and making sure everything else is okay.

The canonical looks fine.

There are no 301's on the pages I visited

You have no robots.txt file

Google is indexing it okay

Ultimately, I think this is a case of waiting it out. If you want to speed things up, try to get some fresh links to the site from well ranking sites to help drive the crawlers your way.

I have been working on getting some new links and have gotten a couple but nothing substantial as of yet.

This client is new for me and i am only aware of this site being around since June or July of last year.

I've been waiting for bing to show search results for the site for almost 5 months now and nothing has changed. I am getting to the point where i feel the culprit lies somewhere else besides saying bing indexing slowly as being an excuse.

If so, what does that tell you? It should detail the HTTP response codes that bing is seeing.

One possiblity is that there is something whacky going on for the bing user agent but fetching the page as bingbot also returns a 200 status code.

There is no robots.txt so I would put that straight first up as suggested and at the very least go with:

User-Agent: *Allow: /

Without access tp webmaster tools and digging further it is hard to see what is going on but 5 months seems too long. That said, there are no issues preventing crawling or indexation in so far as a 15 minute look can identify.

I also have submitted the URL in the "submit URL toolbox page" a couple times.

I will get working on the robots.txt and make sure it's put in place.

I don't do any of the development work on the site i just perform the SEO duties. Seeing those redirect errors is part of what im talking about with it saying there are to many redirects. Any idea what that's all about if its showing everything should be working properly?

I just scanned it again and based on the URL i enter i get different results.those redirect errors come when i analyze dunninggolf.com. But when i analyzed www.dunninggolf.com I only 2 errors on homepage for missing alt tags on images and no H1 tag.

Okay, so the crawler is seeing it for the www. version but not for the non www version.

If we look in a browser at:

http://dunninggolf.com it redireects to www.dunninggolf.com

If I look in webbug at the headers it does not redirect and returns a 200 Okay status code for the non www version of the URL.

I think this could be the issue here and that for some reason, when bing visits the site is giving out some strange 301 response which then sends out a loop of 302 redirects.

Okay, there is a tool here that lets you visit the site as if you were bing bot and get the HTTP headers:

http://view-http-headers.gishan.net/fetch-as-bing-bot-tool.jsp

You have to select bingbot rather than IE6 from list of user agents.

If you visit the www.dunninggolf.com version then it returns a 200 OK

If you visit the http://dunninggolf.com version then it returns a 301 redirect (which in turn seemingly issues the loop of 302 temp redirects that never go anywhere).

Okay, there is your issue. The site has some issue with bingbot (and possibly others) when visited via the http://dunninggolf.com URL. This issue is not visible in webbug or a browser. That said, we do seem to get some redirection from http://dunninggolf.com to http://www.dunninggolf.com.

So, taking a stab in the dark, I would say there is some kind of meta refresh or something going on after the page has been returned as a HTTP 200 result. I am not on my windows PC so can't examine the headers more closely but this is certainly your issue here.

You need to find out the following.

1. How are they doing the redirect?

2. why are they redirecting this way?

3. why are they serving up these 301 / 302 loops via http://dunninggolf.com

Ultimately, this is an issue for the developers. You should advise them on correct protocol (a 301 via the HTTP headers from http://dunninggolf.com to http://www.dunninggolf.com and not via this whacky redirect they currently have in place).

Hey friend! Have fun exploring Q&A, but in order to ask your own
questions, comment, or give thumbs up, you need to be logged in to your
Moz Pro account.
You can also earn access by receiving 500
MozPoints
from participating in YouMoz and the Moz Blog!
Learn more.