The Importance of Crawlable Site Architecture

9 Apr 2010 | written by AJ Wilcox for the SEO section(s)

The Importance of Crawlable Site Architecture
ExperiencePR

You may have heard of the term “SEO-friendly” describing site architecture. You may have said to yourself, “What does ‘SEO friendly’ mean? I thought SEO was all about keywords!” You are correct; however, there is an exception.

Search engines are constantly crawling pages trying to find out which search queries they could show up for. The point of SEO-friendly architecture is whether or not a search engine is even able to categorize that page. If your site is in flash, frames, hidden with wonky coding, in an image, or a variety of other crazy ways of doing it wrong, the search engines are usually not able to even see what each page is about.

Let’s go through these examples so we can understand why these are not cool to the search engines.

Flash

The major search engines and Adobe got together to share enough code to be able to read the content on Flash websites. The problem is that the Flash has to be coded in a way that is search engine friendly. My experience says that the majority of flash designers have not taken the time to program their sites properly so they can be crawled by search engines. Also, from my experience, all other elements held constant, a static html website will perform better in the search engines than even an SEO-friendly Flash site.

Frames

iFrames used to be a pretty slick way of getting content to stay in a certain part of your webpage…back in the late 1980’s. Now, they are antiquated, deprecated, and useless to SEO; it’s pretty much a horrible way to include your content. If iFrames were just ugly coding, it wouldn’t be such a big deal. The largest problem lies in the fact that they pull content from another file that search engines can’t access. When you put content in a frame, a search spider will crawl the page, see no content, and leave – and that’s exactly what you don’t want.

You can accomplish the same thing with PHP include statements as with frames, and there’s less code, and it’s all crawlable to search engines.

Text in Images

I often see designers that create text in Photoshop and plop that image right onto the site. This is just fine for users, but when the search engine spiders come to crawl, they don’t read text in your images. When you look at the site, it will look complete, but to a search engine, you will have less content and look less attractive.

Wonky Coding

I came across a client site a few months whose content was written entirely in javascript.  Seriously. Not only is javascript coding more complex than plain html, but webpages don’t expect it, and the page ends up looking strange (expect your wording to hang off the side of the page, and overlap with other words).

There are lots of complex ways to do the simple stuff, and my guess is that programmers love to do things in the most complex ways possible to show their prowess. For search engines, though, simple is always the best solution. Google is pretty advanced, but I would be incredibly surprised if Google spiders were equipped to handle javascript text.

KISS – The Best Acronym for Crawlable Architecture

Kiss stands for Keep It Simple Stupid. Simplicity is always the best policy. Don’t do something in Flash, javascript, or any other kind of “creative coding” that can be done more simply in html. Once your site is easily crawlable by search engines, you will be able to reap the benefits of your keyword optimization.

You may have heard of the term “SEO-friendly” describing site architecture. You may have said to yourself “What’s SEO friendly mean? I thought SEO was all about keywords!”You are correct; however, there is an exception.

Search engines are constantly crawling pages trying to find out which search queries they could show up for. The point of SEO-friendly architecture is whether or not a search engine is even able to categorize that page. If your site is in flash, frames, hidden with wonky coding, in an image, or a variety of other crazy ways of doing it wrong, the search engines are usually not able to even see what each page is about.

Let’s go through these examples so we can understand why these are not cool to the search engines.

Flash

The major search engines and Adobe got together to share enough code to be able to read the content on Flash websites. The problem is that the Flash has to be coded in a way that is search engine friendly. My experience says that the majority of flash designers have not taken the time to program their sites properly so they can be crawled by search engines. Also, from my experience, all other elements held constant, a static html website will perform better in the search engines than even an SEO-friendly Flash site.

Frames

iFrames used to be a pretty slick way of getting content to stay in a certain part of your webpage…back in the late 1980’s. Now, they are antiquated, deprecated, and a horrible way to include your content. If they were just ugly coding, it wouldn’t be such a big deal. The largest problem lies in the fact that iframes pull content from another file, but search engines don’t have access to that file. When you put content in a frame, a search spider will crawl the page, see no content, and leave – and that’s exactly what you don’t want.

You can accomplish the same thing with PHP include statements as with frames, and there’s less code, and it’s all crawlable to search engines.

Text in Images

Often I see designers that create text in Photoshop and plop that image right onto the site. This is just fine for users, but when the search engine spiders come to crawl, they don’t read text in your images. When you look at the site, it will look complete, but to a search engine, you will have less content, and look less attractive.

Wonky Coding

I came across a client site a few months back that all the content was written in javascript. Seriously. Not only is javascript coding more complex than plain html, but webpages don’t expect it, and the page ends up looking strange (expect your wording to hang off the side of the page, and overlap with other words).

There are lots of complex ways to do the simple stuff, and my guess is that programmers love to do things in the most complex ways possible to show their prowess. For search engines, though, simple is always the best solution. Google is pretty advanced, but I would be incredibly surprised if Google spiders were equipped to handle javascript text.

KISS – The Best Acronym for Crawlable Architecture

Kiss stands for Keep It Simple Stupid. Simplicity is always the best policy. Don’t do something in Flash, javascript, or any other kind of “creative coding” that can be done more simply in html. Once your site is easily crawlable by search engines, you will be able to reap the benefits of your keyword optimization.

9 Comments

9 Comments

  • Jason |   Apr 9th, 2010 at 2:27 pm

    Great article! Another big NO NO I have noticed is people using copy and paste directly from a word document and loading that into the body of the page. What some don’t realize is that word uses a lot of its own coding that will show up to a search engine killing your seo. If you must copy and paste your content from word, be sure to paste it into notepad first, then copy it to the body.

  • AJ Wilcox |   Apr 9th, 2010 at 2:32 pm

    Absolutely, Jason. Word has some funky formatting, and if you paste from word, you often take it’s crazy HTML coding with it. All that unnecessary code can cause slow load times, and a low content/code ratio, which can negatively affect your status with the search engines.

  • Dan Garfield |   Apr 9th, 2010 at 2:36 pm

    If you’re using WordPress and even some input types with Drupal there’s a paste from word option.

  • dayalive |   Apr 10th, 2010 at 11:22 pm

    Great ~ WordPress seo is good.

  • Chris Dayley |   May 5th, 2010 at 2:45 pm

    Great article AJ. I wish more people would read this or related articles.

  • AJ Wilcox |   May 6th, 2010 at 7:41 am

    Thanks Crhis – I wish more startup web deisgners would read this. Then we’d have less headaches later when their customers come to us and we require some code changes.

  • Micah Lauret |   Jul 26th, 2010 at 8:52 am

    Another good read as usual. Flash can do some awesome things, but you’re right, if you can accomplish the same things in a simpler fashion by using HTML, then do it.
    However, if flash is the only answer, what can flash developers out there do to make their flash SEO friendly?

  • AJ Wilcox |   Aug 2nd, 2010 at 7:05 am

    @Micah When you require Flash, and it’s your only option (although it rarely is), make each page on your site in html, and then put each flash page inside each html page. Make sure that your text is highlightable within the content (meaning the text is in HTML). Those things will ensure that your content will get read, but most flash developers overlook the fact that a site can look and feel like flash when it is just an html page with flash elements embedded into the pages. I’m ranting again, but you can’t put alt image attributes in flash images, and load times are bad. I’d avoid when possible.

  • How Do I Learn SEO? | OrangeSoda Blog |   Oct 12th, 2010 at 12:23 pm

    [...] helps to pick a specialty. If you are a techie programmer, it may make sense for you to tackle the crawlability and indexing aspect of SEO. If you love networking and are ultra creative, you may consider the linkbuilding [...]

Leave a Reply