The eCommerce site I work for has some adult content but it's not our primary focus (hundreds of thousands of adult only "pages" vs millions of regular.) For logged out users we prevent users from reaching the page altogether. When the user is logged in they have to manually set a preference that they wish to allow adult content to be accessible.

For SEO purposes we want the adult content to be crawlabe without inadvertently allowing access to logged out users. What is the best way to go about doing this? We're not only concern about google but any bot that might be crawling the site.

We've had discussions about allowing logged out users to visit the page but give an opaque overlay, however, if the image still downloads users can potentially get in trouble with their employer or government. If the image doesn't download but is fetch in an ajax request the image may not show up in the search engine's results. Google does provide a secure access but that doesn't satisfy the need to allow other bots through as well.

1 Answer
1

FCF is designed to allow search bots and search visitors to access content that would typically be available only to logged in users. To implement it you would need to:

Allow search bots (based on user agent) to crawl the content without being required to log in.

Allow visitors from search (based on referrer) to see the content without being required to log in.

If you implemented that, you would have to allow some users that have not specifically opted to see adult content to see the page. On the other hand, if that user had searched for the content, that might be considered enough of an opt in for you.