Let's imagine that you're the AdSense Crawler. You're bouncing along all over the Internet, visiting publishers' sites, and letting the AdSense system back at Google know what the pages are all about. Then one day, you run into a site that asks you for a login and password. "Huh? I don't have a username and password to this site. How am I going to crawl the pages behind that login?"This is something our crawler sees every day here at AdSense. The result is that your users end up with poorly targeted ads and the AdSense Crawler ends up with sleepless nights, wondering what could have been -- if only it had crawled those pages.
We've recently launched a new feature called Site Authentication to take care of this problem. Using Site Authentication, you can give our crawler access to your login-protected pages by passing it information to log into your site. For example, let's say your news site has a premium content area, with articles that only paying members can access. To get ads on those pages, you can use Site Authentication to provide our crawler with a test username and password. It's an easy process that starts just by logging into your AdSense account and finding the 'Site Authentication' link under the 'AdSense Setup' tab. Once you've supplied us with a username, password, and a few other details, all you have to do is verify that you own the site through Google Sitemaps.
If this sounds a little complicated, don't worry -- just check out Site Authentication in your account and follow the instructions on the page. Please note that you will only have access to this feature if you've updated your AdSense login to a Google Account. We appreciate your patience as we roll out this feature to additional publishers.
If you need additional help, feel free to visit our Help Center. Once you've set up your authentication rule and verified ownership, it may take 1-2 weeks for our crawler to visit your site again. Your users will thank you, and so will the AdSense Crawler.
3 comments:
ok...
...so what if I want AdSense to be targeted but the content to not be indexed in Google Search ?
can i let the spider in JUST to serve context-sensitive ads, but NOT spider the content for Google Search ?
please advise - thanks !
- srini@metanotes.com
Site authentication is a cool feature! But how should I configure it to allow the crawler to access e.g. multiple paths of my forum platform?
The number of paths is too big for manually inputing a separate authentication rule for each.
http://mysite.com/forum/Planets/
http://mysite.com/forum/Stars/
... and the list goes on ...
Can I use wildcards or regexp or something?
And how about subdomains? E.g.
http://user1.mysite.com/
http://user2.mysite.com/
... etc etc ...
It's not feasible to create an authentication rule for each subdomain manually.
Suggestions would by appreciated!
thanks,
juppiter
@Juppiter -- I got the same problem too.
I have multiple communities and each community has their own log in. Does this mean I have to supply thousands upon thousands of username and passwords to adsense to spider each of these unique and potentially highly distinct communities?
Google, do you have a solution for this? I've tried the forums and other places and there doesn't seem to be a solution.
Post a Comment