It seems every second client we have has a WordPress site or blog. We do a ton of custom development and design on WordPress – everything from building plugins for companies (example: Webtrends WordPress Plugin) to developing a video workflow application using Amazon cloud services (Candidio). WordPress isn’t always the right solution, but it’s quite flexible and we’re pretty good at it.
Many times, we stage sites so that our customers can preview and critique the work before we put it live. Sometimes we even import the clients current content so that we can work on a real site with live content. We don’t want Google to get confused as to which site is the real site, so we block the search engines.
To do this within WordPress is quite simple. In the Settings > Privacy menu, you can simply check a box:
Additionally, if you have access to the root web directory your site is in, you can also modify your robots.txt file to:
User-agent: * Disallow: /
The robots.txt modification will actually work for any website. If you’re developing an unfinished application, staging software at a different domain or subdomain, or developing a duplicate site for some reason – it’s good to block search engines from indexing your site and taking search engine users to the wrong location!