Search Engine Optimization, aka SEO, is the technique of cleaning up your web page code to maximize your “search engine friendliness” and search engine coverage. The core purpose of search engine optimization is:
- Help search engines search your entire website or blog.
- Provide key words and phrases to help a search engine determine how to categorize and prioritize your content in their search results.
- Help the searcher use the search engines to find information on your website or blog.
The latter two points are aspects of search engine optimization you control through the content you provide on your site. It’s what you write, how you write, and the words you write with which get picked up by the search engines’ web crawlers or spiders.
The first point is the most important. If there is borked code in your web page code or programming language, a search engine is stopped at that point, unable to go further. If your website is designed using tables or old fashioned coding, search engines are often hindered or may skip right over it. Anything that gets in the way of a search engine, hurts. With good web design and coding practices, you put out the welcome mat that says “This site welcomes you. We hope you enjoy your visit and have a pleasant stay.”
There is an often overlooked sub-point in the first point. The phrase “search your ENTIRE website”.
Having a well-designed, coded, and keyword packed front page is fine, but what if the search engine crawler can’t get past the first page of your blog? Is the web crawler really finding every page on your site? Or only a few of them? The key to the first point is helping the search engine gather information about every page on your website.
To a web crawler, a link is an open door to pass through and investigate. If it finds content on the other side, it stays, gathering information and looking for more doors to pass through.
A search engine web crawler is often known as a “web spider”, which is a more visibly graphic way to describe what it does, spreading out through all open links, gathering information and “laying tracks”. The more interconnected your website or blog is, the more doors and pages a web crawler finds to add to their database.
Try this experiment. Make sure you are viewing the front page of your website (or any website) in Firefox with the Firefox Web Developer Extension installed.
- From the Firefox Web Developer Extension Toolbar, choose the Information tab.
- Click View Link Information near the bottom of the drop-down list.
- A new page will be generated with a list of all the links found on your site. Count them. Pay attention to where they are going.
How many links did you find? For the front page of Lorelle on WordPress I found 763 links. While this number changes every time I post a new article, let’s examine what my 763 links represent. Then use the following examples on your own blog to get an idea of how a search engine might be moving through your blog.
Tracking Down the Path of Search Engines Through Your Blog
I copied the list of 763 links from the front page of Lorelle on WordPress and pasted it into a text editor, cleaned up all the tabs and extra spaces, then copied and pasted the list into a spreadsheet program. Make sure to remove all the list numbers in front of the links and clean up the copied list before you transfer it over. HTML tags and other unwanted code can transfer over to the spreadsheet program, making this process more complicated than it needs to be.
In the spreadsheet program, I sorted the data alphabetically. This grouped together all the links starting with lorelle.wordpress.com. This totaled 514.
When a search engine visits the front page of this blog, it will find 514 open doors to other pages on this blog. In theory, it should follow every link to all these pages on my blog, gathering information from each one to store in its database.
Now, let’s do the same experiment with a single post page. I randomly picked Do You Need Permission to Use Feeds, a fair example of the typical writing and links I use in an average article. There were 180 links from the article. Of these, 95 were intrasite links within this blog.
Not all of these links go somewhere a search engine crawler can go. For instance, all WordPress.com blogs feature a login link on most pages. Without permission to access, this is a dead end for a search engine. Your blog may have similar dead end links. You can remove those from the list if you want. They don’t count.
Scrolling down the list, you will find links to posts you’ve written. Where do these come from?
| “As I mentioned recently, you know how worried I am about Beverly…”
You’re worried about Beverly? Now, I’m worried about Beverly. Beverly who? Why should I be worried? What happened to Beverly? Without a link to your previous post, how can I find the answers to these questions? This is one of the most annoying things I find on blog posts. Don’t assume I know what you are talking about. I might have just jumped into the middle of the conversation. If you are writing about something you talked about before, link to it. Help us keep up with the conversation. It also helps search engines find that page by creating an intrasite link to it. This technique helps everyone. Do it. |
On most blogs, the links come from within the content. If you link to another post you’ve written from within the post content area, that’s an intrasite link source.
If your blog’s design or WordPress Theme features code which generates article links in the sidebar, such as Recent Posts, Related Articles, Most Popular Articles, or Post Highlights, these also count as intrasite links.
On my blog pages, I often include intrasite links to other articles I’ve written within the post content. I also provide a list of related articles at the bottom of most posts. These lead the reader and search engine crawler to other articles on my site. In the sidebar, I have links to Most Recent and Most Popular Articles on this blog. These act as gateways to more articles for search engines to follow.
The more links to posts I can provide (without overwhelming the visitor), the less chance I have of leaving a page “unfound” by a search engine. These undiscovered pages are called orphan pages. They are overlooked and ignored by search engine crawlers and don’t make their way into a search engine’s database because no one linked to them.
Want to know if you have any orphan pages? The recent blogging challenge, Who Is Linking To You?, offers a few tips and techniques to find out who is linking to you, and it will help you find out if you are linking to yourself. For example, in Google type site:example.wordpress.com to find how many pages Google found from that site address.
To really get an idea of which pages are missing in action, you’d have to go through and check everything Google has found in their database against every post you have posted. That’s a lot of tedious work. I recommend you do a random check of your blog posts against Google’s list. You may also want to check your traffic levels. If there is a post which is never in the traffic list, check to see if Google, Yahoo, Technorati, or other search engines have that post listed. If you have a post you think is really worthwhile and deserving of attention, check to see if that one is listed. If not, start building intrasite links to help connect the dots between the rest of your blog and that post, providing a gateway of access.
For Lorelle on WordPress, Google found over 55,5000 pages in the site search. For the most part, because of the SEO built-in capabilities and search engine friendliness of WordPress and WordPress.com, I found most of my posts.
Still, do you think I’ve written 55,500 articles on this blog? You honestly believe that? 😉
Let’s look at what Google really found, and explore more of the deeper, hidden intrasite links a search engine finds when they go prowling around a WordPress blog.
Exploring Deeper, Hidden Links Within Your WordPress Blog
Search engine web crawlers follow links to other pages to find more pages to add to their database. Intrasite links which list other articles and posts on your blog help lead search engine crawlers to these pages, but other links also take search engines down different paths, as found in the list of 55,500 pages Google found on this blog.
There are many deeper, and sometimes hidden links, a search engine crawler can find as it prowls a WordPress blog, including:
- Navigation links
- Feed links
- Category links
- Tag links
Navigation Links
At the top or bottom of post content sections in many WordPress Themes are links which say “Next” and “Previous”. They can lead anywhere, as WordPress and other blogging tools use programming code to generate which post is next or previous from the current post.
Click on a next or previous link and you will be taken to the next or previous post, or, if used on a multi-post view page like categories, you will end up on a “page two” listing of the posts within that category. All of these next or previous post(s) pages are considered unique pages by search engines. Each can be added to a search engine’s database.
Using Next and Previous links avoids orphan pages. Blogging tools like WordPress post chronologically, so you will always have a previous post, and that post will always have a previous and next post. Exponentially, for every single page a search engine crawler finds, it automatically finds two more pages to crawl.
Monthly archives are another hidden source of links for search engines to follow. I don’t feature a calendar on this blog. Many people do. While monthly archives and calendars help somewhat with blog navigation, search engines treat their resulting page of posts as another page to add to their database. WordPress automatically includes monthly and date oriented links as part of blog navigation, whether or not a calendar or date archive list is visible.
For example, while I do not have such links within my blog’s pages, WordPress’ programming and structure helps search engines find the following links and pages, which Google found and added to their search results:
- https://lorelle.wordpress.com/2005/10/
- https://lorelle.wordpress.com/2005/09/
- https://lorelle.wordpress.com/2005/08/27/
- https://lorelle.wordpress.com/2006/03/31/
On each of these “pages”, you will find all the posts I published in October 2005 and September 2005, as well as all the posts I posted on the days of August 27, 2005, and March 31, 2006. Each holds information that can be found on individual posts, but they are also unique groupings of posts all on one page, and considered a “unique page” by Google.
Thus, this date-oriented page includes multiple posts, helping to eliminate the possibility of an orphan page.
A site map is another site navigation aid, helping readers find their way through the maze of your posts. To a search engine crawler, this is website candy. A link on every page to your site map is like opening a giant pantry to a search engine crawler. The web crawler will go through the site map list like a hungry teenager, exploring every cupboard for food.
If you don’t have a site map, consider making one. The site map for this site is created manually, a painful task, but if you want to get every post you write found by search engines, think about starting a site map of your own.
Think about all the navigational aids and links you have on your blog that help the visitor move around and find more or related content. You may have links in your header or footer, or just in your sidebar, which take the visitors on a tour of your blog through Pages, categories, tags, and more. Each of these are open doors to search engines to track down more posts on your site. Take time to check to see if your doors are open.
Feed Links
Recently, search engines started to view feeds as “web pages”. In a way, they are. Modern web browsers can read feeds as web pages.
Test your web browser’s ability to read feeds as web pages by clicking this link to the feed for this blog. Can you read it like a web page? Or is it a bunch of strange codes? If you can read it, your browser can handle it as a web page, and so can search engines. If you can’t, then update your browser or test your feed because it might have a problem.
Since a search engine treats a feed as a web page, then these feed “pages” are added to the search engine database. Feed links aren’t limited to your main blog feed. You can have feed links for comments only and for each category of posts.
Examples of feeds found within WordPress blogs are:
- http://example.wordpress.com/comments/feed/
- http://example.wordpress.com/feed/
- http://example.com/?feed=rss2
- http://example.com/?feed=comments-rss2
- http://example.com/categoryname/feed/
- http://example.com/category/categoryname/feed/
- http://example.com/tag/categoryname/feed/
Since WordPress Blogs often feature feeds for categories as well as the whole blog or just comments, more feed pages can be generated and discovered by search engines. You can see an example of the feed for the WordPress Tips Category for this blog, listing all the posts within that category in the feed.
Category Links
Just as category feed links create more pages for search engines to crawl, so does regular category links. If you click on one of the post category links typically found in the sidebar of a WordPress blog, you are taken to a page listing all the posts within that category. To a search engine crawler, this is a new page.
| The Uncategorized Category
WordPress doesn’t allow for uncategorized posts. Let me say this more clearly. If you do not categorize a post, it is automatically listed in the “uncategorized” category. It’s categorized, just within a category called “uncategorized”. I recommend you change the “uncategorized” category name to another name so it doesn’t hang there on your blog like unfinished business, or set another category to be your default category, so “uncategorized” doesn’t show up on your’s category list. |
Category pages are classified as multi-post views in WordPress because it showcases more than one post on the page. Since the goal is to get a search engine crawler to find as many pages as possible on your blog, category pages offer hundreds of links to other posts on your blog. If you have categorized your posts, then search engines will find them. No more orphan pages.
If you are blogging with a program that doesn’t allow for categories, then you are missing a valuable page for search engines to find. From a category page, many posts are found by search engines. A category page with “next” and “previous” links means that not only will the first page of a category’s collection of posts be found, but these navigation links may lead the search engine crawler even deeper into your post category collection, finding a new page for every next and previous link, and more links to more posts within each page. That’s serious SEO coverage.
Tag Links
Tag links are similar to category links, but even better, depending upon how you and your blog uses them.
If your tags link to an offsite location, then skip this section. Your tag links won’t help you with search engines. Your offsite tag link opens a door to invite the search engine web crawler to leave your site. However, if your tags link to onsite pages or generated content, then pay close attention, as you’ve just uncovered another powerful tool to help search engines find their way through your blog.
Like category links, if you click on a tag that links to onsite content, it will take you to a generated page similar to a category list. In WordPress, the URL may change. For a category, the address may be example.com/category/wordpress-tips/, and for a tag it might be example.com/tag/wordpress-tips. For WordPress.com blogs, the URL for a tag and category are the same (tag). If your WordPress blog generates two different URLs, then the search engine has two pages to investigate for each tag and category.
Either way, an onsite tag link acts as another page. Search engines collect these as unique pages in their database, and crawl through them looking for more links.
If your blog features a tag heat map or cloud linking to onsite content, search engines will crawl through those, treating them like new pages.
Still wondering about Google finding 55,500 pages on my site? It thinks that every site search tag in the bottom of every post on this blog is a unique page. They are. Click one and it generates a unique grouping of posts, all with that tag word found within them like a keyword. Each of these pages are added to Google’s database as a searchable page.
I’ve talked a lot about the importance of tags and tagging for blogs, but this is the biggest secret and benefit of all from tagging. Intrasite tag links generate more discoverable pages than anything for search engine crawlers to find and add to their database, increasing your SEO coverage.
Helping Search Engines Find All The Posts Within Your Blog
Without any effort from you, WordPress generates a lot of links which lead to pages search engines can find and add to their database. To review, these are the things you need to do to increase the number and range of pages on your blog that a search engine can find.
- Link often to other posts on your blog from within your post content.
- Include a list of related articles in your blog post or within the blog’s design.
- Make sure site navigational aids such as “next” and “previous” are available and working on your blog.
- Check all site navigation links such as Pages, categories, tags, and date archives are working.
- Put category feed links in your blog’s code.
- Categorize your posts.
- Develop or feature a site map of your blog posts.
- Set your tags to search your blog for onsite posts, generating unique pages.
- Check regularly to see what pages are gathered by search engines and directories, just in case some are missed.
- Validate your web page code and design to ensure there are no bugs or errors which may stop a search engine in its tracks.
Related Posts
- DIY Search Engine Optimization
- Extreme Site Optimization For Speed Loading
- Tags and Tagging in WordPress and Other Blogs
- Secret Out – How Google Ranks Websites
- Site Optimization: Optimizing Bandwidth and Cleaning Out the Code Closet
- Meet Them: Benefits of Compliance with Web Standards
- Site Optimization: Checking Loose Links
- Linkability – Link Popularity
- The Power of the Link
- How People Search the Web and How They Can Find Your Blog
- Are You Missing Out By Not Practicing SEO Techniques?
- Search Engine Friendly: Helping Googlebot Crawl Your Blog
- See What Search Engines See When They Visit Your Website
Site Search Tags: seo, search+engine+optimization, optimization, search+engine, web, web+crawler, spider, wordpress, wordpress+blog, wordpress+seo, seo+and+wordpress, links, search, web+pages, database, search+results, test, pagerank, page+rank, page+ranking
Copyright Lorelle VanFossen, member of the 9Rules Network
Lorelle on WordPress 
20 Comments
Hi,
I really enjoyed reading this article on search engine optimization and it is an eye opener. Thanks for sharing your experiences. I would like to add that IMHO, blogging communities tend to do better than blogs hosted individually. For example, a blog hosted on wordpress.com, Livejournal.com and so on will do significantly better than a blog with its own domain. I have found a lot of wordpress.com hosted blogs having a google page rank of 7 and more. Ofcourse, I agree that google page rank in itself is not the only criterion for the popularity of a site.
Hi, Lorelle!
Excellent article! Your list of 10 things to do is great and I am going to get to work on implementing the ones I don’t already do.
I have a question that I’m hoping you or your readers can help me with. I get a good portion of my traffic from Google and MSN, but when I do a search for my site on these engines, the only pages that come up are the category pages, the tag pages, the index pages, and some of the print pages (I have installed the wp-print plugin). There are 5 blogs on my domain and they are all interconnected, but the post pages on only one of the blogs show up in a site search. There are about 50 posts on each of the other blogs and none of them show up.
Each post has at least one intrasite link. I use the keyword generator plugin. The blog that has the posts showing up is structured in the exact same way as the others. Am I doing something wrong that makes the spiders overlook my pages?
Thanks for your help!
Dana
Lorelle, I’ve suppressed the “related posts” feature on my main page, but the related posts show on the individual post pages (permalink pages). Am I hurting myself by not having the related posts listed in both places? I didn’t want it on the main page because it seemed to add too much clutter–but I want to do what is going to make my blog most useful and searchable.
Dan: If you are finding most of your posts within the search engines, then this is fine. If you aren’t, then I’d add it and experiment with them. It’s up to you. A good site map will make up every difference, too.
Dana: I recommend that you validate your code and make sure there are no errors. Make sure you have a good site map and that all your navigation links are working. Sometimes what we see in search engine results are the “summary” of the posts they’ve found, not everything they’ve found. Click the “show all posts” link when you come to the end of the list in the search results (if available) to see what they aren’t showing you.
And not all posts within a database get listed. Some blog posts are visible in Google Blogsearch, but not within the main Google, and others are in the main Google search but not Google Blogsearch. I can’t figure it out, but if you are worried, check the code and do these other tests to see what you may find. It takes time for search engines to disseminate all the information they gather. So repeat these tests in a few months to see if there are any changes.
Remember, many search engines ignore meta tags today. Write with solid keywords and you will get further in helping search engines categorize your posts than you do with meta tag keywords.
Thanks for the insight, Lorelle!
I used the HTML validator on my newly installed FF Web Developer Toolbar (thanks for that tip!) and I found a ton of errors. They were mostly easy to fix, but I am wondering if I really have to avoid using center tags and tables (I know, I should learn how to do this in CSS…)if I want my posts to be indexed. The validator also does not seem to like the code I put in to generate Google Sitemaps. I guess it is time for me to start working on the other type of site map…
I’m not too hung up on getting my posts in the indeces – I seem to be able to get people doing searches to the pages that have the info they are looking for. But it is nice to understand the Google/Google Blogsearch distinction. Thanks again!
I have a question: For all those “search tags” you put at the bottom of all your posts. Do you create those manually or do you use a plugin? I tried searching for a plugin, but couldn’t find one.
badMike: Manually. A Technorati Tag Bookmarklet for WordPress and WordPress.com Users
Lorelle, thanks for the reminder about the site map. I’ve bookmarked some info about that (probably from this blog!) and I’ve found the Google sitemaps plugin, but I’ve not taken the time to implement it.
Dan, the Google sitemap Plugin is not the same as creating a site map. A site map is a published list of the posts, categories, etc., published on your blog or website. The Plugin generates a file (which might be able to be turned into a real site map…hmm, have to look at that again) for uploading to Google to get max search engine coverage.
Hey Lorelle,
I was about to recommend the Google Sitemaps plug-in, but got beaten to it. As part of your research, you might check out the creator’s excellent FAQ at http://www.arnebrachhold.de/2006/04/07/google-sitemaps-faq-sitemap-issues-errors-and-problems which speaks both generally about how Google sitemaps work and specifically how the plug-in works with Google. In addition, I notice that the Google Sitemaps beta (http://www.google.com/webmasters/sitemaps/siteoverview) also has tools for finding web crawl errors — although my sitemap is apparently too new right now to test them. 🙂
Very impressive. I’ve been longing to know how search engines really read web contents. And this one is very useful. Thanks a lot.
i think Crawl-Page WordPress plugin helps search engine find all the post links and other links available in the blog .
Please reply me , of what do you think of it .
It may not help with google , but yahoo , i think it helps .
Thanks . Nice post .
Good post, I have just strated a wordpress blog and I have found this post very informative, thanks alot for the tips.
Hi, I just want to say thanks for taking the time to post this, I found it very interesting. I’m currently looking into starting my own wordpress blog and I’ll certainly use the information from this page.
Great article however my own feelings are there is more to SEO than sorting out code issues, however this is important.
Wow. that’s really in depth reviews. Thanks for the tips, I know this would be useful someday 🙂
Hai lorelle,
Hey lorelle i havent seen such a article till now..my eyes are really opened..You explained each and everything how to optimize each post,how to optimize websites and how to choose keywords…I got each and everthing..But i am having one doubt..please help me out..
You explained each and everything that how to do seo for websites having content..Well i can bring my post in top position only if i wrote big article that is having huge number of keywords..But i dont know how to bring my website which is related to songs and wallpapers….
I dont know how to seo for those websites.
There will be no content inthose websites…then how can i bring those websites in top position?and how to do seo for those websites?
There is no specific tricks to specific blog topics for SEO. And size isn’t important, as long as there is consistent content across the whole site. If money is your objective, then blog on money making topics, which I admit I don’t know much about songs and wallpaper (I assume for cell phones) but I do know you have tons of competition. So look at what the successful ones are doing and do it better. It’s marketing and business techniques of old that will make or break you, not SEO games. Think business not web specific.
If you have no content on your sites, then you are stuck as search engines only see words, not pictures. Get words on your site.
How about me? Because I’m blogspot user? Can you help me please.. Thankyou.
@ Garden:
If you asked a question, I might be able to help you, no matter what blogging service you are on. It depends upon the question.
7 Trackbacks/Pingbacks
[…] SEO Secret: Exploring How Search Engines Explore […]
[…] SEO Secret: Exploring How Search Engines Explore […]
[…] good use of intra-site links. Linking to other articles or blog entries on your site is a great way to increase page views and to improve your search engine rankings. Not […]
[…] Lorelle on WordPress – SEO Secret: Exploring How Search Engines Explore […]
[…] SEO Secret: Exploring How Search Engines Explore […]
[…] SEO Secret: Exploring How Search Engines Explore […]
[…] a Sitemap on your site, take time to make sure your site is as navigation friendly as possible. Search engines follow your intrasite links to find their way through your site. The Sitemap is used as a form of checks and balances, helping […]