Is AltaVista's Stale Index The Last Blow?

Earlier this year, it emerged that AltaVista had not refreshed any of its country-specific web page indexes for months. Now, it turns out that the main global database is also several months old.

It's a PR black eye that the service didn't need. AltaVista has been losing its audience steadily over the past year, with Google looking to be the big winner. To regain that audience, AltaVista has continued to proclaim for over a year that it is focused on search. Unfortunately, it's failing to deliver on that promise.

Back in March, AltaVista made a lot of noise about a new crawling system that was going to revisit each page in its index on a monthly basis or even sooner, if a page required it. Here we are in November, and that system doesn't seem to be working. Such a long delay doesn't promote confidence that AltaVista will get its act together.

To be fair, some parts of AltaVista's index were fresher than you could find at Google. For example, in mid-October, the listing for the CNN home page was only a few days old. This could easily be spotted by looking at the date of the listing. In contrast, Google's listing for CNN was still dating back to Sept. 13 -- though you would only know this by viewing Google's cached copy of the page.

That's more the exception that the rule, however. The vast majority of AltaVista's content out of date, which the company itself admits. There is one exception. Anyone who stumped up money over the past few weeks got their pages either included or revisited in a current fashion.

It's worth recalling that when AltaVista rolled out paid inclusion, it promised that regular spidering of "free" content would continue, albeit at a slower monthly refresh rate. That pledge has been broken.

Freshness is a key component for any crawler-based search engine, but verifying freshness can be difficult. One standard that all the crawlers should immediately implement would be to list the date that they last visited each page. Then, at a glance, any user could easily tell exactly how fresh -- or how old -- a search engine's listings are.

AltaVista also will no longer gather its own shopping search information but instead plans to outsource for this feature, in the near future.

"We have been approached by several leading shopping sites who are interested in becoming a strategic partner for us in this space," said spokesperson Kristi Kaspar.

AltaVista gained its own shopping search ability via the Compaq acquisition of Shopping.com back in January 1999. Despite outsourcing shopping search, the company does plan to continue producing its own web-wide search index.

CMGI still owes Compaq $220 million, apparently for the purchase of AltaVista. A new deal reduces this to $75 million in cash, $7 million in CMGI stock and a 49 percent stake for Compaq in CMGI's B2E Solutions.

In a new program aimed at earning revenue from web site owners, AltaVista will now allow listings to be "enhanced" with icons and other customized features. Basic details can be found here, directly from AltaVista. Chris Sherman is also planning a longer look in the SearchDay newsletter, later this week.