Senior Member

joined:June 22, 2001
posts:3805
votes: 2

I wouldn't worry at all. Listings vary depending on which datacentre you look at. Also, they can come and go as the results are updated.

Google is much fresher nowadays than it was originally, but this is partly at the expense of short term index stability. Although frustrating for webmasters wanting to see exactly what's happening at Google, it gives searchers a better chance of finding some relevant content (even if not yours or mine!).

Preferred Member

joined:Apr 18, 2003
posts:618
votes: 0

>How does completely removing a page from the index give users a better chance of finding relevant content?

Exactly correct. My personal home page which was placed very high (mostly #1) for many keyword searches including my own name vanished yesterday. I had a few useful articles on a special and unique topic and visitors were finding me from the search for that particular keyword and/or word-of-mouth. Yesterday my traffic from Google came to an end, and basically it is hurting the visitors.

Senior Member

I realize this topic has been discussed at great length, but in this case I think it is only fair that we admit that this is a flaw, however temporary, with Google.

If Google ranks a site on the first page of their results, they must find it somewhat useful. So, to remove the page entirely the next day (not just from the first page but gone completely) suggests one of the following:

1. The SERPS were originally flawed in displaying that page.

2. The SERPS were not flawed originally and a relevant page is no longer in the index for users to find.

Senior Member

joined:June 22, 2001
posts:3805
votes: 2

I'm not suggesting that temporal SERP instability is a useful feature, I'm suggesting that (from a searcher's point of view) it's a price worth paying for having those results available for at least some of the time before the full crawl/update cycle finishes.

Senior Member

we had a site listed last week for the first time ... now we have gone from google?

It is quite usual for Google to add new pages (from a new site or not) only for a short time after freshbot [webmasterworld.com] visited the page.

To be able to return quickly the SERP, the number of pages added by freshbot but not yet indexed in the 'monthly' update cannot be too big. That would explain that after some time the pages found be freshie drop out of the SERP to make place for other new pages found somewhere on the web.

New User

joined:Oct 19, 2002
posts:30
votes: 0

I'm sure it has something to do with this radical new update.

I had a site built in March show up for the first time EVER in searches yesterday. Never in google before. Got about 150 hits and at 7:45PM EST last nite, it disappeared again. Can't be found in google anymore.

Something changed last nite at around 7:30-7:45 and stuff that had been added that morning disappeared again. It's acting like it wants to be added but then doesn't hold on as google tweaks the formula.

And that traffic came from AOL and Yahoo also. Not just google. and now that site isn't in any of those 3 indexes.

Senior Member

joined:Nov 19, 2002
posts:2139
votes: 0

<<It is quite usual for Google to add new pages (from a new site or not) only for a short time after freshbot visited the page. >>

Yes, but since the backdate, old pages/sites that are fully indexed and assigned PR have also started disappearing and reappearing. Hopfeully, the next update will work these glitches out. If not, it would be quite annoying.

Preferred Member

joined:Apr 18, 2003
posts:618
votes: 0

Am back with a vengence. My 50+ pages are showing as well as my personal home page, all in better than before positions! Just a few hours ago my home page was not to be found. Will it stick? Shows June 13 date stamp by freshbot.

Junior Member

joined:Apr 12, 2003
posts:144
votes: 0

Yes, but since the backdate, old pages/sites that are fully indexed and assigned PR have also started disappearing and reappearing. Hopfeully, the next update will work these glitches out. If not, it would be quite annoying.

I find it pretty damn annoying already! :)

I was just hoping it would settle in and then add some backlinks...but noooo, now a whole bunch of Freshbot data takes over and bumps my (heavily crawled by freshbot PR5+)page from the serps entirely...

GG, do us all a favor and please tell these guys to stop giving us freakin ulcers over here! :)

New User

joined:June 12, 2003
posts:25
votes: 0

I agree with dunnthat and mfishy that these fluctuating SERPs have much more to do with the ongoing, painful saga of "Dominic" than simply a deepbot/ freshbot situation. I had the same problem at the same time yesterday as dunnthat -- with Google's cache for my entire site reverting back to a March 29 deep crawl date. This is despite the fact that my pages have been visited by both deep (216...) and freshie (64...) numerous times since then.

Problem for me, as with IITian, is that traffic dies off for me when my cache reverts to something ancient. It's only been in the past month that I have optimized a number of pages to have keyword-targeted titles. For those short periods that Google has made available to searchers these newer versions, my position in the SERPs has jumped into the top 5 or 10 for many relevant keywords. With the old pages in the index, my site doesn't do nearly as well. But at least my site hasn't dropped out altogether. I feel sorry for those of you with phantom indexes :((

New User

joined:June 12, 2003
posts:25
votes: 0

I forgot to add that when the cache for my site reverts to March, as is the case at this very moment, none of the new pages and content that I have added in the past two and a half months is searchable via Google's index. Another reason for drops in traffic. BUMMER. :(

To cheer me up at times like this, I cruise over to FAST and can always see my latest and freshest pages coming up high in the SERPs. Too bad that less than 5% of my traffic comes from FAST.

Preferred Member

joined:Apr 18, 2003
posts:618
votes: 0

Stefan, I think you are right. But it is nice to have traffic back at least temporarily. Surprising how much difference Google makes.

truth_speak, My pages too have been visited by both deepbot and freshbot many times. My feeling is that the deepbot lost my pages during the April crawl - my first. However, seems like results are better now(at least for me - exceeds my expectations!) Hope the new Google will be better than ever.

Senior Member

joined:Nov 19, 2002
posts:2139
votes: 0

I also have a site that is about 18 months old with strong PR and backlinks that is currently not in the index (started not showing yesterday). I wouldn't go so far as to say I'm not "stressed" about it, but I do realize there is nothing I can do on "my end" to fix this.

Senior Member

joined:Nov 19, 2002
posts:2139
votes: 0

<<That's not true. Newer sites will flash in and out. >>

Err...I added that I was not speaking of fresh sites. And this has been the case long before dominic - fresh pages dropping in and out of SERPS. What has many perplexed is that older, established sites have been dropping in and out of the index since Dominic.

Everflux was around before freshbot even existed. MANY older sites have, for the first time, been popping in and out of the index, and no, it's not a penalty.

here's what nutandbolts said,

"Don't stress about it. I have sites over 2 years old doing this in this month's Google "update" so it's not just the newish sites affected by spotty Dominic. "

Read about 1000 others saying the same thing.

Everyone here knows that freshbot will pick up a site for a few days or so and then it will vanish. This is the first time I have ever seen completely indexed (showing backlinks/PR frp months or years) sites drop in and out of the index on a regular basis.

New User

joined:June 2, 2003
posts:39
votes: 0

Heres one for you! I have a site that if you go to it via the url it has a page rank of "0" however if you bookmark the site then go to it via that bookmark the site has a page rank of "1"? can anyone explain that?

Junior Member

Senior Member

joined:Feb 9, 2003
posts:806
votes: 0

mfishy...welcome to the NEW Google where sites will appear and disappear at whim. Kind of Google saying to us that they are in control as to who shows up where and not us SEO webmasters. Unfortunately, it's Google's game to play, not ours. I know that it affects our businesses drastically and hopefully over time this index will become stable.