Apple says it will respect the customary robots.txt rules and robots meta tags. AppleBot currently originates in the 17.0.0.0 net block. If you do not mention AppleBot in your robots.txt directive, Apple will follow what you mention for Googlebot. So if you want to block AppleBot and GoogleBot, you can just block GoogleBot, but I’d recommend you block each individually.

If you notice unusual AppleBot activity, you can reach Apple about it at Apple-NOC “at” apple.com.

It is unclear if Apple plans on building and competing with Google on search but this is one step closer to that.

Postscript: Apple Insider reported later on today that Apple is “rapidly-expanding internal search group” to build their own version of a web search engine via Spotlight. So it does appear Apple is venturing into Google’s search space – at least based on these early reports.