We appreciate the enormous support that our ABestWeb community has experienced over the many years it has served its members and sponsors. We have decided to exit this business and have placed the property up for sale and we are actively entertaining interest.

In the meantime, community members will be able to read but not post to ABestWeb beginning on Jan. 18.

We want to thank you for your numerous contributions and your ongoing support. If you have any questions, please let us know.

Assuming that the spider obeys the /robots.txt file (some spiders don't), that line will stop URL's that start with /cgi-bin/ from being spidered, such as /cgi-bin/ae.pl?PARAMETERS

If you are using AE virtual directory (mod_rewrite) so that products show up in a virtual directory (e.g.: mydomain.com/amazon/) then you would need a Disallow statement for that virtual directory directory, e.g.:

User-agent: *
Disallow: /amazon/

DySE always uses virtual directory (mod_rewrite) so you would need a Disallow for the virtual directory where the DySE store is located. Disallowing /cgi-bin/ would have no effect for a DySE store. For example, mydomain.com/calendars/ would require:

Assuming that the spider obeys the /robots.txt file (some spiders don't), that line will stop URL's that start with /cgi-bin/ from being spidered, such as /cgi-bin/ae.pl?PARAMETERS

If you are using AE virtual directory (mod_rewrite) so that products show up in a virtual directory (e.g.: mydomain.com/amazon/) then you would need a Disallow statement for that virtual directory directory, e.g.:

User-agent: *
Disallow: /amazon/

DySE always uses virtual directory (mod_rewrite) so you would need a Disallow for the virtual directory where the DySE store is located. Disallowing /cgi-bin/ would have no effect for a DySE store. For example, mydomain.com/calendars/ would require:

User-agent: *
Disallow: /calendars/

Yours truly,
Cusimano.Com Corporation
per: David Cusimano

Having it as a subdomain how would one deal with that in the robot txt file?