0121 31 45 374
Qoute Icon

Is Google using Analytics data to crawl additional pages?

Tim

I've been wondering for a while how Google has managed to find a couple of hidden pages. Although they were securely locked down we noticed a few rejected GoogleBot requests in the audit logs. We put this down to the users having a Google toolbar installed but today we got an error from the new Avant Garde hair salons site that's just gone into beta testing which got me thinking.

This particular link is hidden behind a form post and within a jQuery call (to track an action) so not something the GoogleBot has easy access to. I know they're getting more clever but not *that* clever! We started getting the errors shortly after adding the final Google Analytics code so the only conclusion I can come to is that they're not just registering the URLs for reporting purposes but they're also using them to crawl additional pages.

Does anyone know if they use the URLs tracked in Google Analytics to find new pages? All I can say is if this is the case, you better make sure your "secure" pages check the access permissions on a page level!

Liked this post? Got a suggestion? Leave a comment