I have a SharePoint 2010 site collection based on a custom Publishing site definition. I'm trying to figure out why sub-sites and pages underneath a certain site (/Foo) are never included in my search results.

There is nothing obvious configured on my /Foo site or its sub-sites that would prevent them from appearing in the search results - the Allow this site to appear in search results setting is most definitely set to Yes. The site and its sub-sites are created using a sub-site Publishing site definition which is also used for all the other sites in the site collection.

Here's a description of my Search setup and configuration in my development environment.

In the configuration of the Search service application:

Content Source

I created a new content source and included http://localhost to the collection of Start Addresses for the content source. I configured a full and incremental search crawl schedule.

Scopes

I created a custom scope and added a rule of type Content Source, and set it to the content source created in the previous step.

In the configuration of the site collection:

Search Settings

I set Site Collection Search Center to Enable custom scopes and set it to the URL of the Search Center: /Search/

I also set Site Collection Search Dropdown Mode to Do not show scopes dropdown, and default to target results page

Search Scopes

In the Search Scopes configuration of the site collection, I configured the custom scope to be the Default Scope under the Search Dropdown and Advanced search groups.

Troubleshooting

Here are things I've looked into and tried while troubleshooting tis:

Tried using the out-of-the-box Local SharePoint Sites content source and defining my scope to only include content from http://localhost.

Examined the crawl logs. None of the sites under /Foo appear in the crawl logs. There are no errors in the log which would indicate that the sites and pages aren't being crawled.

Reset the index and ran a full crawl.

Created an out of the box SharePoint Publishing site and search center and used the default Local SharePoint Sites content source to search for the content. None of the sites under /Foo appear in the results.

Thoroughly stumped; would appreciate any guidance on this.

Edit - 3/37/2012
Ran stsadm -o setproperty -pn portalsuperreaderaccount -pv DOMAIN\user -url http://webappurl to make sure that the search account is also configured as the super reader account. It's a site collection administrator, and also the service account, so I don't think this makes a difference.

I will confirm tomorrow, but I'm pretty sure that in my development environment, the search account is also the system account. Thanks.
–
George DurziMar 27 '12 at 2:58

foocode, yes, the account does have unrestricted access. The site and its subsites is inheriting permissions from its parent (the root site)
–
George DurziMar 27 '12 at 12:38

Glad you found you solution!More often than not, search crawling omissions due to permissions issues manifest themselves in file share crawling. Particularly if the service account is one created specifically for the SharePoint farm (as is recommended).
–
BeeblebroxMar 29 '12 at 23:17