Technology

How to Get a New Website Crawled by Google Tomorrow

Recently, I’ve been launching a lot of new websites.  As AddressTwo has grown and my time has freed up, it’s created a perfect storm of new ideas and free time to execute, so I’ve bought dozens of domains and implemented micro-sites left and right.  Of course, I’m impatient, too.  I have an idea on Monday, build it on Tuesday, and I want traffic on Wednesday.  But it can take days or weeks before my new domain shows up in Google searches, even when I’m searching for my own domain name.

So, I’ve started tinkering with a formula for getting the spiders to come faster.  If SEO is an alechemy, then this is my home-brew for accelerating the time from launch to index.  It’s simple, but proven effective.  Some of my recent experiments have been crawled and appearing in search results in under 24 hours.   I just follow these 8 simple steps.

  1. Set up your on-page SEO first, at least minimally.  Admitedly, this has nothing to do with getting crawled, but if you don’t do this first, the next 7 steps are in vain.  Specifically, make sure that your title tags are optimized.  Why is this so important?  Because, even though we can get a spider to your page quickly, that doesn’t mean they’ll return quickly.  So, if your initial launch has poorly-written title tags, then you could be stuck for the next several weeks with less-than-ideal cached content in Google’s index.  Make sure that what you’re hurrying to be seen is worthy of being seen as-is for a few weeks while you wait for the next crawl.
  2. Install Google Analytics.  Do this before the sitemap for one simple reason: it saves time.  One of the ways you can verify your new website with Google Webmaster is via the analytics script.  So, save a step and do this first.  To do this, visit www.google.com/analytics.
  3. Submit an XML sitemap to Google Webmaster Tools.  You can install any one of dozens of WordPress plugins to create this sitemap automatically, or make one manually.  This is essential for getting an accurate and thorough crawl, however many novice webmasters believe that it is the end-all for getting crawled.  It is not.  If you stop at this step, as 99% of new webmasters do, then you’ll wait weeks or months before Google gets around to crawling your site.  What follows will accelerate that process.  To complete this step, visit www.google.com/webmasters
  4. Add the URL to your LinkedIn Profile.  When you edit your LinkedIn profile, you have the ability to include up to 3 website URL’s.  If you have already used all three of those slots, it’s time to sacrifice temporarily.  Pick one of the URL’s to remove for the next few weeks and replace it with a URL to your newly published website.  Don’t worry, you can change this back later.  Put a note in your calendar for NO SOONER than 14 days later to return to your LinkedIn profile and restore your URL list to what you had before.  In the span of those 14 days, Google will presumably have found the new link and followed to your website.
  5. Add the URL to your Google Profile.  Google is much more lenient with the number of links they permit in your profile.  When signed in to Google, from any Google page (including their home page) you can click on View Profile in the upper-right corner, then click Edit Profile.  On the right, you should find a section called “Links”.  There, you can add a custom link.  Here, you can even set the anchor text to your preferred keyword as you add your new URL to your Google profile.
  6. Cite your website on Wikipedia.  That’s right, I spam on Wikipedia.  You can send your hate-mail to [email protected]  Your goal here is to cite a source on your website (a blog article or some other informational page) in a relevant article on Wikipedia.  There’s an art to this.  Here is the goal: to have your citation survive at least 72 hours before someone deletes it on Wikipedia.  To accomplish this, look for a not-so-popular article.  If the article receives multiple edits per day, it’s likely that your addition will be quickly deleted before Google’s bots have the chance to find it.  But, if you find an article that receives edits about once monthly, that’s your golden ticket.  Add an intelligent and relevant (yes, even academic and factual) new sentence and append a CITE_WEB reference at the end.  Don’t be too bold by adding an entirely new section or paragraph.  Your goal is to be seen by bots, but NOT get noticed by humans.
  7. Publish a Google Knol article.  Once Google realized that Wikipedia was so popular, they tried to compete with it.  Google Knol (www.google.com/knol) is far less academic and there is no vigilante system for deleting self-interested articles.  You’re almost guaranteed to have your Knol survive indefinitely.  This means: it’d better be good.  Don’t publish something that will survive and reflect poorly on your name and brand for life.  Write a short, relevant article much like a blog entry and link it back to your newly published URL.  As with anything, including keywords in the title of this Knol and in your anchor text is advisable as well.
  8. Publish a YouTube video.  A few months ago, I published an article detailing how to get link juice from YouTube.  Without repeating that content here, I’ll simply tell you to follow the instructions here and your 8th step will be complete.

All totaled, when I launch a new website I have at most 30 minutes of work to do before I’m done.  If I do all eight of these steps, I can be confident that my site will appear in Google searches in a matter of days, if not hours.  How?  Because I’ve given Google every opportunity to discover the new URL.  If you have discovered any other methods that you use your “alchemy” please feel free to share in the comments below.

12 Comments

  1. 1
  2. 2

    Great write-up, Nick. I’m in the process of filling of filling in my own website with bullet points, key words, and other items related to our business. It’s not what I want everything to look like in the end, but it’s a start, and it’s having an effect. I think it was you who asked me once, “what’s keeping you from starting?” to which I answered, “nothing, I’m going to start next week.” To which you answered, “No. What’s keeping you from starting, TODAY?”

    Chet
    http://www.c2itconsulting.net

    • 3

      That sounds like something I’d say 😉

      Are you holding off on submitting for crawl until you have your website copy adequately keyword stuffed… er… I mean “optimized”?

  3. 4

    Good article filled with some solid examples, but with two surprising omissions; Twitter and Facebook pages. Since Google is paying for access to the Twitter stream, an account with decent authority sharing a link and maybe a few people retweeting it for good measure will get Google to investigate. Even if no followed. If you want to ensure the no-follow will have no impact, RSS feed it into a freindfeed account.

    Similarly a Facebook page, not profile, will get crawled quickly as well. For good measure tweet a link to the Facebook page status, and Facebook post a link to the Twitter status. The reason I recommend these is because more people already have one or both setup.

    This is by no means contradictory to what you are doing of course, just a few more that have worked very well for me. Either way, putting links where Google is frequently crawling while the crawl rate and crawl budget is low on a new domain or revised site will speed indexing and is a good idea.

    • 5

      Kevin, you’re right. Those two should be in there. Here’s why they didn’t make my “alchemy” recipe:
      – With twitter, the whole premise is that you need to have a high-klout account to tweet it from. If you go create a new twitter account, I’m convinced that’s of little value for achieving crawler’s attention.
      – With Facebook, the problem is that a page takes quite a while to do well, and to do poorly can be more harm than good.

      If you have the resource, another good idea that I often do but can’t necessarily prescribe to everyone is to place a link to your new website from a high-authority domain that you already own. Of course, many people don’t already own one, so again, I omitted that from the alchemy.

      Thanks for adding these.

      Nick

  4. 6
  5. 8

    Great ideas Nick. This is a solid formula. I hadn’t thought about Wikipedia or Knol, but that’s mainly because of the constant edits. I’ll have to remember that in the future.

  6. 9
  7. 11
  8. 12

Leave a Reply