Apple Confirms Their Web Crawler: Applebot

Barry Schwartz, writing for Search Engine Land:

After much speculation around an Apple Web Crawler, Apple has finally posted a help document confirming the existence of AppleBot, their web crawler.

Apple said, Applebot is the web crawler for Apple. AppleBot is “used by products including Siri and Spotlight Suggestions,” the company said.

It’s the confirmation of Applebot’s existence that’s new, not Applebot itself. Developer Jan Moesen wrote about it back in November, noting that it was apparently written using the Go programming language. Looking at the logs here on DF, it’s been calling itself “Applebot” since February.

(For the curious, here’s what the last 24 hours of Applebot traffic to DF look like.)

★ Wednesday, 6 May 2015