(12-14-2008, 07:49 PM)EviLito Wrote: Anyone here can confirm this? If we disallow showthread.php the bots still can read the threads using friendly urls?

I haven't tested it, but the rename thread-xyz.html -> showthread.php?tid=xyz is something that the webserver does internally, i.e. the client (be it google bot or browser) should have no way of knowing that what really is done is a call to showthread.php. So a thread-xyz.html link would still work.

It would be nice to ship a recommended robots.txt with MyBB. It could be done in a similar fashion to .htaccess... the file is not named .htaccess so people can have a look at it and rename it to .htaccess if they like it or make changes to it. Something similar could be done with a robots.txt.example that users could look at and rename if they decide they like it.

Specifying canonical pages can be a better solution than blocking threads and posts with robots.txt since Google then will still follow if someone linked to showthread.php?tid=x instead of thread-x.html.

This can be done with a plugin, for example my Google SEO (which can also do all sorts of other stuff, such as meta descriptions or sitemaps), or labroccas canonical plugin.