I have been trying to optimize this blog over the last few days,  so I thought I would share exactly what I have been upto

Well,  first off let me tell you that this is a WordPress Blog.  This is important because I used some WordPress plugins to help my site get discovered by the internet at large.

First plugin that is simply a must is:  Meta SEO Pack

It allows you to control the meta tags on your blog.   What is a meta tag?

Meta elements provide information about a given Web page, most often to help search engines categorize them correctly. They are inserted into the HTML document, but are often not directly visible to a user visiting the site.

They have been the focus of a field of marketing research known as search engine optimization (SEO), where different methods are explored to provide a user’s site with a higher ranking on search engines. In the mid to late 1990s, search engines were reliant on meta data to correctly classify a Web page and webmasters quickly learned the commercial significance of having the right meta element, as it frequently led to a high ranking in the search engines — and thus, high traffic to the website.

As search engine traffic achieved greater significance in online marketing plans, consultants were brought in who were well versed in how search engines perceive a website. These consultants used a variety of techniques (legitimate and otherwise) to improve ranking for their clients.

Meta elements have significantly less effect on search engine results pages today than they did in the 1990s and their utility has decreased dramatically as search engine robots have become more sophisticated. This is due in part to the nearly infinite re-occurrence (keyword stuffing) of meta elements and/or to attempts by unscrupulous website placement consultants to manipulate (spamdexing) or otherwise circumvent search engine ranking algorithms.

I mainly use it to control the Keywords and Description meta tags.  The keywords need a little tweaking, but generally by using them will see your page ranked higher  by the search engines.   With regards to a good investment of time,  I would say YES! It should take less than 30 minutes to setup.  On a time cost reward scale I think it is a pretty good return.  Important to note that as of September 2009 Google no longer uses Keywords in its page ranking engine.   The Description meta tag settings can be changed,  however I personally found that the default settings were fine.

The second plugin that is pretty important is:  Simple Google Sitemap

It creates a Sitemap for your blog.  What is a Sitemap?

Google introduced Google Sitemaps so web developers can publish lists of links from across their sites. The basic premise is that some sites have a large number of dynamic pages that are only available through the use of forms and user entries. The Sitemap files can then be used to indicate to a web crawler how such pages can be found. Google, Bing, Yahoo and Ask now jointly support the Sitemaps protocol.

Since Bing, Yahoo, Ask, and Google use the same protocol, having a Sitemap lets the four biggest search engines have the updated page information. Sitemaps do not guarantee all links will be crawled, and being crawled does not guarantee indexing. However, a Sitemap is still the best insurance for getting a search engine to learn about your entire site.

XML Sitemaps have replaced the older method of “submitting to search engines” by filling out a form on the search engine’s submission page. Now web developers submit a Sitemap directly, or wait for search engines to find it.

XML (extensible markup language) is much more precise than HTML coding. Errors are not tolerated, and so syntax must be exact. It is advised to use an XML syntax validator such as the free one found at: http://validator.w3.org

So,  yes its a pretty good thing to have on your site   Simple-Sitemap generates this for you!  You do  not have to do anything to get this XML file.     Once the file has been created the sitemap can be submitted to Google here (under site configuration) and also to Yahoo here.  Please note that both Google and Yahoo will require you to authenticate that you own the site.  This can either be done by adding a certain meta tag to your home page or uploading a simple html file onto your server.  I opted to upload the file,  but either is fine.

Okay,  so all the SEO stuff is great,  but some pages I don’t want to be indexed.

Something’s you do want easily accessible on the web.  For example you may have a directory where you keep testing files.  You can stop Google and Co ,  so don’t panic!  You will need to create a robots.txt file and upload it to the root directory of your site.  Thankfully tools are available to help you with this.  I suggest using the Google Webmaster services.

Once you are logged in you should see a menu on the left.  Click on: Site configuration.  This will display a few other options.  The one we are interested in is Crawler access.  You should then see the option to generate robots.txt.  This will tool will allow you to create your file and block certain directories from indexed   Once you have the file simply upload it to the root directory of your website.  Simple as that   Not only will this stop Google,  but also the other major search engines.

Related Article: Setting Up Google Analytics.  This will allow you to see how your SEO is faring.