http://www.webpronews.com/expertarticles/2009/07/10/the-google-shuffle#comment-88027

• New sites aren’t getting spidered – or not as quickly as earlier this year. Webmaster tools gives a generic message about the website not being listed in Google’s index, along with a link to a video that seems to mostly be about websites that get themselves banned for violating Google’s terms of service. Also existing websites that are growing aren’t always having the new content added as quickly as before – or rather it happens inconsistantly lately.
• Google PageRank tools don’t seem to be working any more. I’ve tried a number of them lately.
• While we’re on the topic of PageRank, it seems to be even less relevant than before. In one controlled scenario where we have many listings showing for a specific search term, a PR5 page is showing on the third page, while much lower PR websites are showing on the first page. Sorry, I can’t be more specific, but it is a fairly controlled scenario. All of the pages involved are similar in size with similar numbers of occurrences of this keyword.
• Searches are often slow. As far as I can tell, this isn’t just my internet connection. Its been years since I’ve had the Google homepage time out on me.
• Search listings sometimes change dramatically in short time periods.

All of the above seems to indicate that Google is gearing their entire system up for something big. Speculation among my staff says they’re going to try to make everything realtime (or close to it) in order to compete with Twitter. That means that they’re going to try and reorder all of their indexes very quickly (rather than weekly or possibly daily) in order to try and provide something closer to the immediate zeitgeist that one can obtain through Twitter.

Having some idea of the size of Google’s indices, and a vague notion that the number of servers in their demesne is in the low millions, the scale of this boggles my mind.

Image via CrunchBase

Over the past few months, we’ve averaged around one new blog setup per day.

Recently, Nathan (one of our SEO experts – see his article on Buddy Press) and I started putting together a list of the standard things that we do after we install a WordPress blog.

The following assumes some familiarity with WordPress. We’ve started playing around with the latest version (2.8), and I suggest that you use that unless there’s a pressing reason not to (i.e. incompatible plugins).

1. General Config

• Make sure that you have configured clean URLs in Settings -> Permalinks.
• Under Settings -> Writing, put in additional locations to ping whenever you update your blog. There is a decent list here.

2. Themes

• We try to make small changes to all stock themes that we use. This means that search engines are less likely to group your site along with every other blog that is using the same theme.
• Even better: use a premium theme, or make your own one.

3. Plugins

Our objective with plugins is to automate the process of creating quality meta information for blog entries to the largest extent possible, and to make sure that our blogs talk nicely to search engines. We install the following set of plugins:

• TagThePress
• TagMeta
• PingPressFM
• Google Sitemap (there’s a few good options)
• Ultimate Google Analytics

Make sure you configure all of the above. You may need to create some accounts in various places in order for some of the above to work.

If you’re running Firefox, we highly recommend installing the Zemanta plugin.

We used to put tag clouds into the sidebars of all new blogs, but if Sitemaps is working correctly that isn’t necessary (and it can take up a lot of important real estate).

4. SEO Stuff

• Make sure you have accounts for Google Analytics and Google Webmaster Tools. Use them. Play around with them. Learn how to use them inside and out.
• Make sure every new blog has one or two posts containing YouTube videos.
• Getting the right number of tags per post is critical – we try to hit a sweet spot between 10 and 15 tags for each post. This may change depending on search engines.
• Make sure that your blog is configured to use different page titles and meta tags for each page. Use HeadSpace if necessary to automate this process.

There’s probably a ton of important things I’m missing here (please let me know!), but this is a minimal list of things that you should be doing whenever you setup a new blog (if you want it to perform well).

To reiterate something that Evan Carmichael frequently talks about, the amount earned from Google Adwords is equal to the number of click-throughs, multiplied by the dollarvalue of a click-through. Sounds obvious enough, but there’s a huge divergence in the quality of ads, and that is somewhat dependant on the blogger themselves, since Google tries to place ads topically. You’ll see what I mean below.

Let’s talk about traffic quickly first. Building traffic to a website takes a lot of hard work and tremendous patience, which is why many website owners simple throw up their hands and accept whatever comes their way (or try to drive revenue by paying for traffic themselves – which is a tricky proposition for a blog). I’ve seen many websites that have built up to the low thousands of unique visitors per day though, through a ton of sweat equity. Anything beyond that may be a black swan event, so let’s set that as the upper bar of what the average individual can achieve through hard labour.

The value of an ad on a website is largely driven by topic and industry. There are people making higher than average rates using other ad placement systems (or by selling ad space themselves), but Google AdSense is the most accessible system to the average blogger, so let’s use some examples from there. The majority of click-throughs that I get on this site (and others I’ve run in the past) varies between $0.10 and $2.00. In one extreme example, I think I once received $5 for a single click-through. I know of specific topics that pay significantly higher (life insurance being one such).

Click-through rates tend to depend a lot on where people place ads on a page. Having high quality ads can help as well, but since Google tries to tie ads into the contents of a page, bloggers have some control over the sorts of things that generally appear. Spending some time experimenting with placement can have a large payoff. Editor’s Note: I’m guilty here; I do have ads, but I really can’t be bothered where they show up, since ad revenue isn’t what I’m after.

Therefore, the expected average earnings for a statistically significant number of hard-working bloggers could be calculated as being in the following range:

Low End: Assume 1000 visitors per day, 3% click-through rate and $0.10 per click = $3/day or $90/month.

High End: Assume 5% click-through rate and $1 per click = $50/day or $1500/month.

Bear in mind that the above figures are for somebody with average knowledge of how search engines work, a good work ethic, a willingness to experiment, and the patience to build things up over time. I don’t know how many people this covers.

Like I said before though, there’s a black swan or power law effect that’s at work here. What will typically happen is that the vast majority of bloggers will earn next to nothing through ad revenue, a small but well defined set will make enough to make it worthwhile to do full time, and a tiny (and exceptionally well-known) group will make a fortune. Similar to other kinds of creative efforts right? Think authors or musicians.

Disclaimers (I think they’re needed here):

a) I use Google AdWords on this site. I’ve made $10 in the past 6 months. I’m too busy with other things to care too much. I’ve run sites that made $50 to $100 per month in the past, with minimal effort on my part.

b) I know of several people who make a decent living blogging (by decent I mean more than I make!). There are some interesting differentiators between them and other bloggers. They all approach it as a business. Most of them seem to have found ways to make other people do the hard work for them. They also all find real-world outlets (i.e. seminars, consulting, selling product) that neatly tie in to their blogs, in such a way as to create a reinforcing upward spiral of activity. Believe it or not, only a few of the ones I know are “famous” or are active mainstream journalists. The people I know aren’t a big enough set to be statistically significant.

Image by shinyai via Flickr

While I’m on the topic of helpful websites, here’s another simple but useful one: Wefollow.

Its a user-edit Twitter directory, that Nathan pointed out to me a few days ago.

I picked up a few followers just by signing up.

I find it remarkable how many followers some of the top users of Twitter have accumulated. Yes, some of them are leveraging off of some form of celebrity status (real-world or online). Its quite amazing how large the reach of some dedicated tweeters is though. It takes a lot of hard work to scale up a following like that, regardless of where it is.

For those of you who aren’t familiar with it, I suggest taking a look at Grader.com, a tool provided by web marketing gurus HubSpot.

Their website grading tool provides a host of useful information that can help you fine-tune your site.

I’ve been playing around with their tools for the past few months, and they’ve been extraordinarily useful in terms of tweaking things to make them more search engine friendly. Also useful is their Twitter profile grader.

If you use WordPress then you might of heard about their sister project BuddyPress. BuddyPress is the open source social platform that promises to be the WordPress of social platforms. To be scalable and the same time light, to have a variety of features and be user friendly… Just as WordPress made it possible for anyone to have a nice professional looking blog without even touching the code, now you could have your own social network too with very minimal interaction with code.

Now that the version 1.0 is officially out I decided to test. So I downloaded the file only to find out that its not a program by itself but a plug-in for WordPress.

I installed WordPress on my Linux machine on localhost. When tried installing BuddyPress I was notified by the program that I have to run WordPress-mu (multi user version) to be able to install BuddyPress. I tried installing the mu version, but guess what, WordPress-mu can’t be installed on localhost.

At that point I just decided to buy a domain name and get a proper hosting for it. I installed WordPress-mu, activated the BuddyPress plugin that I inserted into plugin folder. Hooray! Looks like its working, I found few nice web 2.0 themes installed them, it looked really nice, until my friend who uses Internet Explorer tried to signup and for some “unknown reason” got a 404 error, page doesn’t exist. I have got a new challenge why does it show this error when I know that the page is there…

One day later the problem is solved: you can’t use a regular WordPress theme for the main blog, what you need is two themes one specially designed for BuddyPress “home” and the second one for the  BuddyPress “member” page. So I put them in the appropriate folders and I have finally got it working.

Or did I?

Wait! What about the forums capability? Why don’t I see it?

Oh! In order to have forums you have to install BBpress which is a stand alone application but has the built-in options for WordPress integration.

So I download the latest stable version, install it, integrate the user info with WordPress, but for some “strange” reason even with the key master account I can’t start a new forum…

I almost gave up for now on forum capability, when I by mere coincidence found a folder in the BuddyPress installation that had the BBpress integration files; there I saw a readme document with detailed instruction of how to integrate them. I found what I did wrong – the latest stable release is too old, what I need is the latest Alpha release. So I uninstalled the stable release and installed the unstable one. Got through the installation by following every line in the readme. And guess what – its finally working.

Moral of the story: I am part of generation that doesn’t like reading the manuals and readme files, but sometimes reading them could save you a lot of time.

Some resources:

WordPress MU – http://mu.wordpress.org
BuddyPress – http://buddypress.org
BBpress – http://bbpress.org
Working Example: http://postnonstop.com

I keep an eye on the Google Analytics reports for this site. The past couple of months, there were an unusually large number of hits from people searching for info on Twitter. I just tried a couple of searches on Google.com and Google.ca this morning, and for “twitter purpose” (and a variety of other combinations), we’re showing up near the top of the first page. In the past, we’ve also had searches for people looking for info on Askme.com, DandyId and other specific topics that I’ve written about here.

I think the general principal, one that has relevance for SEO (search engine optimization), is that it’s just about impossible to determine in advance what the zeitgeist of the moment is going to be. As a result, place content on your site that covers a wide range of related topics, and there’s a good chance that something that you write will be relevant to somebody, somewhere, at some point in time.

This is otherwise known as the spaghetti principal – when you don’t know what precisely will work, try a bunch of different things, and record your results. This isn’t a new idea by any means. Bloggers, internet marketers and SEOs use this tactic all the time, in a variety of different ways.