>>>>>First, it is likely that URLs submitted through Inktomi programs are already being picked up by the Yahoo! Search Technology crawler and are represented in the index separate from participation in a paid inclusion program.>>>>>>>>>

Does this mean there are diferent spiders for diferent purposes?

Actually I would appreciate if you could clarify which spiders are used for what purpose,,,,

Scooter normally spider my sites every day, havnīt been in since 14/4.

yahoo slurp stopped spider my robots.txt the 22/3 and started again the 6/4.

Allthe web stopped spidering my robots.txt for weeks ago, but suddenly came back for some days ago.

yahoo shop spiders comes in know and then for my robots.txt and my index page.

It sounds as if what I had mentioned previously might be correct - that as of Monday evening, I was seeing Inktomi listings appear with either / or "cached" or "more pages from this site" beside the listing. Obviously this implies you have been spidered, and pulled into a "cache" somewhere ......

I had never seen it before.

All of my Inktomi URL's showed this change by Tuesday night. And as of today, each and every one of my URL's still remain.

Now - two questions.

First does anyone have Inktomi feed URL's that remain in the index as of 4/16 which do not show one or both of these flags?

And also,

Has anyone as of yet lost a Inktomi Feed URL - which yesterday they know for certain did in fact show one or both of these flags?

Well it has happened. Our INK feeds, paid for 4 months ago are no longer in the SERPs.

It will be interesting to see how long it takes to get in the organic results. Our very authoritative site which has hundreds of links and has been in the Yahoo! directory for several months, ranked #8 out of over 49,000,000. I state the position only to show the relevance of the site. Now gone. Quite a loss to searchers.

The question on my mind is, (and a hot topic of discussion) is Yahoo! willing to sacrifice SERP quality for the $$$. The loss of other sites in that same 49,000,000 contribute to a lesser quality of results.

We will see how long it takes the Yahoo! crawler to include these lost and very relevant results.

>>>> The question on my mind is, (and a hot topic of discussion) is Yahoo! willing to sacrifice SERP quality for the $$$. The loss of other sites in that same 49,000,000 contribute to a lesser quality of results. >>>>

I wanted to reiterate that Yahoo! will *not* sacrifice quality of results in order to make more money. From what I've read on the boards since 4/15, more people are finding their Search Submit pages have remained in the Yahoo! SERPs (ie. their pages were crawled, besides being included via SS). There will no doubt be cases where the pages have not yet been crawled and therefore will drop out of the SERPs on Yahoo! (as we stated would happen on 4/15). A reminder that you can also use the free AddURL status for any pages that were longer appear on Yahoo and that it's likely our crawler will eventually find your pages.

>>>> Does this mean there are diferent spiders for diferent purposes? >>>>

Yes, we do have different spiders for different purposes. For example, there are different spiders that look for news stories, images and multimedia, etc. From time to time we also use some test crawlers. Your site may be visited by multiple crawlers. Yahooslurp is our main crawler for getting information into the organic results.

Yahoo_mike Can i ask you take this opportunity to clarify what actually happens when a site is flagged as you call it! You say many factors can 'flag' a site up for review and i find it hard to imagine how this actually works. What actually is a flag? Where does it show? Who see's it?

Our site Inktomi PFI have been purged. Still showing up in MSN, and organic serps in Google. Wonder how long it will take for web searches to decide to leave Yahoo so they can find what they are looking for.

Hey, I mispoke in the message above. Our Inktomi PFI urls are still in the index. If go to Yahoo advanced search, and enter "" and limit the search to my domain, all the urls are there. But if I just do a normal search on keywords for which I had good position, the urls are nowhere to be found.

>> Can i ask you take this opportunity to clarify what actually happens when a site is flagged as you call it! You say many factors can 'flag' a site up for review and i find it hard to imagine how this actually works. What actually is a flag? Where does it show? Who see's it? >>

Please reserve this forum for questions related to the 4/15 update in Yahoo! search results. Questions like these should go in the "Questions for Yahoo" forum. I will post replies there given that the Answers from Yahoo forum has been closed.

>> Mike, doesn't Yahoo follow links to find quality sites? Why does Yahoo require urls be submitted using the submit url form in order to receive a "free" listing? >>

We don't require you to use the AddURL service. The goal of the crawler is to find all of the quality pages on the web, but there is no guarantee that all pages will be found. AddURL is simply a way to help direct the crawler to your site.

helenp >> Can i ask you take this opportunity to clarify what actually happens when a site is flagged as you call it! You say many factors can 'flag' a site up for review and i find it hard to imagine how this actually works. What actually is a flag? Where does it show? Who see's it? >>

Thread answers for yahoo has been closed down, and Btw, I didnīt ask that question, but donīt matter who did, itīs still an interestion question :)

>>>> Can i ask you take this opportunity to clarify what actually happens when a site is flagged as you call it! You say many factors can 'flag' a site up for review and i find it hard to imagine how this actually works. What actually is a flag? Where does it show? Who see's it >>>>

I didn't realize the "Yahoo Questions" thread was closed as well, so...

A site might be more closely reviewed if there is something suspicious about it. For example, if it has a lot of incoming and outgoing links, we may examine the site more in order to determine if it's trying to spam or if the site makes good / valid use of the links. We have internal systems that can alert us to these situations, or a reviewer may notice these issues when they look at new sites.

We do take the registries into consideration, but do not index the whois pages currently. We are looking for new content, but also want to make sure the pages are unparked and have unique content before we index them.