We at Live Search are pleased to announce another collaboration with Yahoo and Google aimed at making webmasters’ lives easier. Webmasters have long used the Robots Exclusion Protocol (REP) to control how search engines access and display their content. The REP offers an easy and efficient way to communicate with search engines, and is currently used by millions of publishers worldwide.

Over the past few years, we have been working with Yahoo and Google to agree on common ways for webmasters to communicate with search engines. Our previous efforts include support for the Sitemaps protocol (see autodiscovery directive and cross-host sitemap submissions for more info). While most search engines already comply with the REP, this is the first time the three major search engines have come together to detail how we actually implement the protocol. This effort makes it easier for webmasters to know how REP directives will be handled by search providers.

You can view the details of how we implement the REP at Documentation for the Robots Exclusion Protocol.

If you are a publisher with feedback or suggestions for future enhancements to the REP, feel free to contact me at nbuggia@microsoft.com. We value your feedback as we continue to evolve Live Search.

Nathan Buggia, Live Search Webmaster team

Has this ever happened to you? You search online for information and click a promising link only to get an error page staring back at you:

These generic 404 error pages leave you stranded, with no option but to click the back button and start over, or just give up altogether â€” not very helpful.

We thought about this problem and realized that we had all the pieces to offer something much more useful. We could help website owners create error pages that actually suggest help even if the exact page you’re looking for isn’t available.

The Web Page Error Toolkit is a customizable Web application that extracts keywords from your search and gives you relevant search results in a custom error page.

So while you may not get the page you were originally looking for, maybe you’ll find something even better.

You can find more details, including information on how to download the Toolkit, at Customize your 404 error pages with the Web Page Error Toolkit.

Go find the page not found!

Alessandro Catorcini, Lead PM for the Live Search API, and Amy Michaels, Group Product Manager for Live Search