If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Robots.txt Question

Okay, new to the forums (thank you) and I have a simple, or seems simple question regarding the robots.txt file for web sites.

Based on my sample below, is it necessary to list the last user-agent that is to have access to all of the site in order for the whole site to be indexed? I mean, is the robots.txt file's contents hierarchical? Anyone that can answer this question for me I would appreciate. I have found nothing on the web that supports my question.