TJantunen.com

Rick Regan asks:

I have a two-part question about how Google indexes sites:

1) Is Google supposed to index an entire site at once or does it do so incrementally? My blog (on my own domain) appears to be getting indexed incrementally, to the point where it’s taken a month to index all 30+ of my pages. Most of those pages were present before I manually submitted my URL to Google. I’m wondering if I’m doing something wrong or not doing something I should be doing. Does this have anything to do with my blog being new, or having no external links pointing to it?

2) Does Google eventually drop noindex and 404 pages? I have archive pages that got indexed but I have since added to them the “noindex” tag. I also deleted an empty category that Google now gets a 404 on. Will Google eventually remove those pages as it re-crawls my site?

I will answer to each question individually.

1) Most of the times Google will index a new website gradually, yes. That is at least what I have observed with most of my websites. The speed at which Google will index all your internal pages will depend on different factors though.

If you get some very trusted and relevant backlinks, and on top of that you also have a very efficient internal link structure, all your internal pages will get indexed fast. If, on the other hand, you have very few backlinks and a poor link structure, it might take a while before you get to see all your pages indexed.

Keep in mind that using the manual URL submission to Google will have a small impact upon the speed and breadth of your indexation. In fact many people recommend that if you want to get a site indexed fast you should NOT use that feature, and rather focus on getting some trusted backlinks to your site.

2) Yes, eventually Google will fix those issues. New sites don’t get crawled very often, so that is certainly the reason. As soon as Google finds out about the “noindex” tag, for example, it will remove those pages from their index.

Go to Source

The little buttons in my post-editing toolbar in WordPress disappeared and no alt text showed up in its place. The images were all valid and viewable individually in their respective folders, so I couldn’t see a problem. I searched for references to the issue with respect to wordpress post editor tinyMCE. No luck. I even installed the tinyMCE Advanced plugn to see if that would solve the problem, but that just gave me the opportunity to stare at even more blank buttons.

Meanwhile, I’d noticed an issue with my web-hosting stats program. For some reason clicking a visited link that had no www. in front of the domain, in the logs, was redirecting to the root folder for the site and adding the www. This meant the links were wrong and it was spawning a 404 missing page error. Very frustrating.

The solution to both problems came when it occurred to me that it might be due to some faulty redirects in the Apache server .htaccess file. So, I checked that configuration file in the root…all seemed okay. I then took a look at the .htaccess file in the WordPress folder. Eureka! I’d found it. The WordPress version of .htaccess is meant to handle permalink rewrites and any mods required by plugins, but in addition to those it seemed to be a carbon copy of its namesake in my root folder.

As my root folder .htaccess had some rewrites for adding the www without duplicate content penalties that explained the 404s with the stats file. It also had some code to direct people reaching the site via an image search to an appropriate page. It seems that this was the underlying cause of the missing button. Having those rewrites in the WordPress version of .htaccess meant every time I opened the editor it was looking in the wrong place for the images to draw the buttons. Problem solved.

Go to Source

The Blog Traffic Exchange website released a WordPress plugin back on March 15th called Related Websites. The plugin taps into membership sites, that is those who have signed up to the Related Websites service who have been approved and uses a specially crafted algorithm to display links from those member sites to articles deemed to be related. It didn’t take long for me to think of BlogRush when reading the description of how this plugin/service works. However, unlike BlogRush, their are no credits and it seems like everyone has an equal shot at having their links displayed as long as you play by the rules.

BlogTrafficExchange explains how the plugin works here:

Links are randomly rotated both locally per page load and globally every 24 hours. A proprietary traffic balancing algorithm rewards blogs that send diverse traffic into the exchange more frequent placement. Relevancy is never compromised.

While a lot of people are raving about how well the plugin is working for them so far, I hate the fact that I have no control over who shows up in my related content box. I love the idea on how Related Websites work, I said the same thing when BlogRush was launched, but instead of showing websites from across your internal database of URLs, I’d rather be the one adding specific URL’s to my own internal database. Then, through a contextual algorithm, it determines from those URL’s to show related content. I believe this would make the related websites 10 times more relevant than what BlogTrafficExchange or any other traffic exchange system does. You know, algorithms can’t solve every problem and this is one of them. I as a human being know which specific sites are most related/relevant to the content I publish, I only need the algorithm to scan the content of those sites.

I’ve come to the conclusion that Sphere, or any other related content network which displays ‘relevant‘ content from across a wide spectrum of sites is useless. But, if you’re looking to really open the flood gates in terms of coverage or visibility, only then do I see these types of systems being valuable to a blogger. While you could make the argument that being part of a related content network which is used by big name sites such as CNN provides the opportunity for a link from your blog to show up on their site, don’t kid yourself. It’s happened to a few people but it’s not an everyday occurrence. Just about anytime I’ve read the CNN tech section and reviewed the related content section, a little blog called TechCrunch always seems to be featured their.

If a traffic exchange network works for you, great. It’s just my opinion that you represent the minority.

Go to Source

The Teens In Tech blogging network which is operated by CEO Daniel Brusilovsky who only happens to be 16 years of age has merged with the Youth Bloggers Network. According to a post written on the Teens In Tech blog the two networks should compliment each other nicely:

We figured that YBN and TinT complement each other very nicely. TinT offers a place where teens can get set up with a free blog, while YBN offers a community for young bloggers to collaborate, communicate, and grow their blogs and projects.

Teens In Tech is currently still in private beta while Youth Bloggers has over 150 members. YBN looks like it will be used as a means of turning the entire company into a center point for teen bloggers to use not only for publishing content, but for establishing the site as their social network. I’m impressed by what Daniel has been able to accomplish with regards to running a company at just 16 years old. It wasn’t too long ago when Daniel was on stage at his own Teens In Tech conference discussing teen trends and basically, all things teen. This kid has a bright future ahead of him.

Go to Source

If you are not familiar with it, Darren uses to run a very useful series on his blog titled “31 Days to Build a Better Blog.” He basically takes one month out of the year and packs it with practical articles that people can use to improve their blogs.

This year he added an interesting twist to the thing: you can signup to get a closer interaction with the project. Here is how Darren described it:

The idea behind this is simply to have a group of bloggers setting aside a month of their time to work at improving their blogs. While we all want to have better blogs sometimes it becomes one of those things that we’re going to do…. one day.

I personally find that I improve (in all areas of my life) when I’m more intentional and set aside a specific time to make the improvements. That’s what this project is about.

Yesterday on Twitter I mentioned that I was thinking of running this again and had 50 people express interest in 5 minutes. I quickly set up a sign up page just for my Twitter users and within 24 hours had well over 1000 registered participants. I guess people want to do it.

Interested in joining us? If so, read on…

If you are looking to improve your blog during April, I would recommend signing up for this. First of all it will give you an extra motivation to actually take action instead of just sitting around reading your RSS feeds…. Secondly, Darren will also pack some bonuses for the people who join, so you can’t lose.

Go to Source

If you have ever come across a dynamic website, you may have noticed not many of them use SEO friendly URLs. You will see URLs like this:

http://www.domain-name.com/friendlyurl.php?id=6&cat=productname

These URLs are complex and not very user friendly and although search engines can crawl these dynamic pages they are, in most cases, not relevant to the pages they display. They can also pose a security risk as hackers can perform front door query string injection attacks on the site. SEO friendly URLs contain human-readable and easy-to-remember names such as:

http://www.domain-name/friendlyurl.php

these seo friendly URLs can be accomplished by using mod_rewrite for Apache or ISAPI filters for IIS, depending on which server platform you are using. The implementation of Apache mod_rewrite is fairly simple to do by placing the code in a .htaccess file on the root of the server. For example, the above URLs would have the mod_rewrite rule of:

RewriteEngine on
RewriteRule ^([^/\.]+)/([^/\.]+)/?$ / friendlyurl.php? ?id=6&cat=productname [L]

This tells the Apache server to read the URL and remove the query string (everything after the?) from the URL. This is all done server-side so the search engines only read the friendly version of the URL allowing for better website usability, relevance and page validation.

Using the same method on an IIS server will not work as IIS servers do not support mod_rewrite, but they do have the next best thing “ISAPI filter” plug-ins. These filters are DLL (dynamic link libraries) written in c# or vb.net and can be applied to all windows based server platforms using the “internet services manager”. These filters can perform the same functions as .htaccess for example mod_rewrite, 301 redirects etc.

I personally like using ISAPI filters as I find them easy to use and implement. The only downside to using these filters is quite a lot of them are commercial, whereas using mod_rewrite on Apache is free and open source.

Adam Wood
SEO Programmer

Go to Source