Category Archives: Uncategorized

When will virtual currencies be useful?

I currently have small amounts of money floating around in a variety of virtual currencies. In some cases, I can convert those currencies to other virtual currencies or to real world money (i.e. there’s a slow process to move paypal money into my bank account).

It occurs to me though that it would be very useful if I could pay real world bills (think groceries or mortgage) directly using virtual currency.

Before that can happen, there would need to be a lot more transparency (i.e. no grocery will accept magicbuxx if they don’t know how much they are worth, or whether they can in turn get value out of them), and a whole lot of big institutions like banks and payment portals would need to sign on too. There would also need to be physical mechanisms that can transfer the payments (i.e. the new mobile payment technology that is slowly being adopted by cellphone manufacturers would be helpful).

I wonder how we can make that happen. It would be very nice to be able to go to a restaurant with a pile of Facebook credits, or Bitcoins.

HP Needs a BHAG (Big Hairy Audacious Goal)

Flickr Creative Commons - Taken by kevindooley

A number of years ago, I read a book called “Built to Last”, by Jim Collins and Jerry Porras. The book, a classic of the genre, discusses a number of companies that the author feel to be “visionary” in nature. One of those companies is HP. The founders of the company built not just a company, but also a coherent internal culture, commonly called the “HP Way”. This has lead to the company being greatly admired in business circles.

The last few years have been rough on HP’s external image, in large part – I believe – unfairly. The past week has seen the second sudden departure of its CEO in only a few years. Both were due to scandal. While profitability has not been hurt (in fact HP is doing better than ever, with much credit to its recently departed CEO!), the stock has lately been pummeled in the markets.

Some of the commentary that I’ve read describe the most recent tenure as being one of building a solid financial foundation for the company. Which leads me to my point. What HP needs in its new leader is a vision for where the company should be moving technologically. Not just a specific set of goals, but something that is going to put fire in their bellies (and enthral their customers). In short, they need a BHAG – a Big Hairy Audacious Goal.

One possibility that comes to mind: HP is one of the largest manufacturers of electronics (both consumer and business) in the world. If they made a decision that in 3 to 5 years time, every single device that they manufactured would contain a wireless mesh device, they could theoretically blanket the entire world with free, decentralized, high speed internet connections. And by implication, free telephony and broadcast media. Yes, there are still big technical issues to address, and wireless mesh networks are still very much the realm of techy enthusiasts (and the US military, and also to some extent Google). But that’s the point of a BHAG. Yes, the telecom industry would scream (including likely some of HP’s board members – hey, I’m just sayin’) as their entire business model evaporated. Oh, and Apple might be in trouble as well – they make money on the telecom contracts for iPhones, not on the hardware. But imagine the sales pitch to consumers – buy our printers, our laptops, our telephones, and never pay for internet, telephone or cable TV ever again. Nice, eh?

Here’s another possibility: HP is already widely known for its environmentally friendly policies, and especially for its experience handling and recycling plastics. Imagine what effect a Fortune 500 company (with $100 billion plus per year in revenue) could have, if it would back a project like WHIM Architecture’s Recycled Island project? WHIM are trying to gather all of the waste plastic floating in the middle of the Pacific Ocean, and turn it into habitable land. I don’t know with any certainty if their economic projections are feasible, but there’s a potential for large profits from this type of venture.

There’s no doubt that there’s any number of highly talented people that can step into the chief executive role at HP. Let’s hope that whomever they chose will bring this kind of vision to the table, and that this remarkable company can quickly move beyond this temporary setback.

Heavy Traffic – Lessons Learned

In the past 15 or 16 years, I’ve worked on a number of websites that had fairly significant traffic (mostly in the form of unique daily visitors – there’s many ways to measure traffic). In one specific case, the traffic on a well-known author’s website spiked significantly (several thousand unique visitors per day) after his appearance on a television talk show. The website, although database driven, primarily consisted of articles, along with a store – and even on shared hosting, this wasn’t a problem.

Recently, my company built an online “live auction” website for a customer, a project which posed a number of interesting challenges and learning experiences (the hard way, of course) regarding how to build a site that has heavy traffic. In this case, the nature of the website requires that all users see information that is current and accurate – resulting in a need for AJAX calls that run repeatedly on a per second basis per user. This project is the first one that I have worked on that required serious optimization work; typically even the heaviest custom development that my team works on is primarily focused on business use cases rather than things like speed or algorithm design; not so here.

The “coming soon” page, long before the site was launched, already received several hundred unique visitors per day (based on Google Analytics). The site launched with more than 500 registered users (pre-registration via the coming soon page), and traffic spiked heavily following launch. The initial traffic spike actually forced the site to close for several days, in order for our team to rework code. The re-launch was preceded by several Beta tests that involved registered users. Bear in mind that a registered user on most sites isn’t individually responsible for much server load. On this particular site, each user is receiving at least one update per second, each of which may involve multiple database calls.

The following is a description of some of the issues we encountered, and how they were addressed or mitigated. In some cases, work is ongoing, in order to adapt to continued growth. In many cases, the challenges that we encountered forced me to revise some assumptions I had held about how to approach traffic. Hopefully the following lessons will save a few people the weeks of sleep deprivation that I went through in order to learn them.

Project Description:

  • Penny Auction website
  • Technology: PHP (Zend Framework), Javascript
  • Server: Various VPS packages (so far)
  • Description of traffic: All users receive one data update per second; there are additional data updates every 3 seconds, and once per minute.

1. Don’t Rely Too Much On Your Server

Many web developers build code that simply assumes that the server will work properly. The problem is that under heavy load, it isn’t at all uncommon for servers to actually not function in the way that might be expected. Examples include things like file resources dropping, database calls dropping – sometimes without intelligible error codes, and even things like system time being unreliable. The following are a couple of specific examples we encountered:

a) PHP time() – When developing in PHP, it is very common to rely on function calls such as time() (to obtain system time, in UNIX timestamp form) for algorithms to work properly. Our setup involved a VPS with multiple CPUs dedicated to our use, and the ability to “burst” to more CPUs as needed. As it turned out, whenever our server went into burst mode, the additional CPUs reported different system times than “our” CPUs did. This is probably an issue with the underlying VPS software, but we didn’t have the luxury of investigating fully. This meant that rows were frequently (as in: about one quarter of the time) saved in the wrong order into the database, which is a serious issue for an auction website! When possible, use a timestamp within SQL code (i.e. MySQL’s TIMESTAMP() function) instead. Fixing the system time on the other VPS partitions wasn’t feasible, since they “belonged” to a different customer.

b) Not every database call will work. Under heavy load, it isn’t at all unusual for a SQL insert or update statement to be dropped. Unless your code is designed to check error statements, and handle retries properly, your site will not work.

2. Pick Your Hosting Company Wisely

We launched the project on one of our hosting company’s smaller VPS packages. We quickly went to one of the middle-range packages, discovered it was also insufficient, and then switched to the largest package that they offer.

In the process, we also entered a number of second tier or higher tickets into their system, including serious operating system level problems.

Luckily, we chose a hosting company that responds quickly to issues, and whose staff are familiar with the types of issues we encountered.

This isn’t something to take for granted. Not every hosting company has the ability to quickly and seamlessly transition a site through different packages on different servers, nor do they necessarily have tier 3 support staff who can address unusual support requests.

In this case, our conversations with the company seem to indicate that they have never seen a new site with this level of load in the past; they still worked valiantly to assist us in keeping things running.

3. Shared Hosting, VPS, Dedicated, Cloud Hosting?

In our previous experience, when a hosting company sells somebody a dedicated server, the notion is that the customer knows what they are doing, and can handle most issues. This occurs even where a SLA (service level agreement) is in place, and can seriously effect response time for trouble tickets.

As a result, our first inclination was to use a VPS service. Our decision was further supported by the level of backup provided by default with VPS packages at our chosen vendor. A similar backup service on a dedicated server of equivalent specifications appeared to be much more expensive.

One of the larger competitors of our customer’s site currently runs under a cloud hosting system. We are continuing to look at a variety of “grid” and cloud hosting options; the main issue is that it is extremely hard to estimate the monthly costs involved in cloud hosting, without having a good handle on how much traffic a site will receive. It isn’t unusual for hosting costs to scale in such a way as to make an otherwise profitable site lose money. That said, we will likely have to transition over to a cloud hosting service of some kind at some point in time.

4. Database Keys Are Your Friend

At one point, we managed to reduce server load from > 100% load, down to around 20%, by adding three keys into the database. This is easy for many web developers to overlook (yes I know, serious “desktop” application developers are used to thinking of this stuff).

5. Zend Framework is Good For Business Logic – But It Isn’t Fast

We initially built the entire code base using Zend Framework 1.10. Using Zend helped build the site in a lot less time than it would otherwise have taken, and it also allows for an extremely maintainable and robust code base. It isn’t particularly fast, however, since there’s significant overhead involved in everything it does.

After some experimentation, we removed any code that supported AJAX calls from Zend, and placed it into a set of “gateway” scripts that were optimized for speed. By building most of the application in Zend, and moving specific pieces of code that need to run quickly out of it, we found a compromise that appears to work – for now.

The next step appears to be to build some kind of compiled daemon to handle requests that need speed.

6. Javascript

Our mandate was to support several of the more common browsers currently in use (mid-2010), including Firefox, IE7-9, Opera, and – if feasible – Safari.

The site is extremely Javascript-intense in nature, although the scripting itself isn’t particularly complex.

We used Jquery as the basis for much of the coding, and then created custom code on top of this. Using a library – while not a magic solution in itself – makes cross-browser support much, much easier. We’re not very picky / particular about specific libraries, but have used Jquery on a number of projects in the past couple of years, to generally good results.

Specific issues encountered included IE’s tendancy to cache AJAX posts, which had to be resolved by tacking a randomized variable onto resources; this, unfortunately, doesn’t “play nice” with Google Speedtest (see below).

We also had a serious issue with scripts that do animated transitions, which resulted in excessive client-side load (and thus poor perceived responsiveness) in addition to intermittantly causing Javascript errors in IE.

Javascript debugging in IE isn’t easy at the best of times, and is made more complex by our usage of minify (see below) to compress script size. One tool that occasionally helped was FireBug Lite, which essentially simulates Firefox’s Firebug plugin in other browsers (but which also sometimes can change the behaviour of the scripts being debugged). The underlying issue is that IE does a poor job of pointing coders to exactly where a script crashed, and the error messages tend to be unhelpful. The debugging method in IE basically boils down to a) downloading a copy of the minified resource in the form that the browser sees it, b) using an editor with good row/column reporting (I often use Notepad++) to track down roughly where the error occurs, and c) put in debug statements randomly to try and isolate the problem. After working with Firebug for a while, this is an unpleasant chore.

7. Testing Server

Long before your site launches, set up a separate testing server with as close to a duplicate of the live environment as possible. Keep the code current (we usually try to use SVN along with some batch scripts to allow quick updating), and test EVERY change on the test site before pushing the code over to the live server. Simple, but frequently overlooked (I’m personally guilty on occasion).

8. CSS

Designers and web developers often think of CSS purely in terms of cross-browser compatibility. Building sites that actually work in major browsers goes without saying, and based on personal experience, CSS issues can lead to a lot of customer support calls (“help, the button is missing”) that could be easily avoided. In the case of this specific project, we actually had to remove or partially degrade some CSS-related features, in order to provide for a more uniform experience across browsers. Attempting to simulate CSS3 functionality using Javascript is not a solution for a heavy-traffic, speed-intensive site; we tried this, and in many cases had to remove the code due to poor performance.

An often overlooked CSS issue (which Google and Yahoo have started plugging – see below) has to do with render speed. Browsers view documents essentially like a multi-dimensional array of elements, and specifying elements in an inefficient way can actually have a significant effect on the apparent page load time for users. It is well worth your while to spend some time with Google Speed Tester (or Yahoo’s competing product) in order to optimize the CSS on your site for speed.

9. Why Caching Doesn’t Always Work

Caching technology can be a very useful way of obtaining additional performance. Unfortunately, it isn’t a magic bullet, and in some cases (i.e. our project specifically), it can not only hurt performance – but can actually make a site unreliable.

High traffic websites tend to fall into one of two categories:

On the one hand, there are sites such as Facebook, whose business model is largely based on advertising; what this means is that if user data isn’t completely, totally current and accurate, it is at most an annoyance (“where’s that photo I just uploaded?”). Facebook famously uses a modified version of memcached to handle much of their data, and this kind of caching is probably the only way they can (profitably) serve half a billion customers.

On the other hand, financial types of websites (think of your bank’s online portal, or a stock trading site) have business models that pertain directly to the user’s pocketbook. This means that – no matter how many users are available, or the volume of data – the information shown on the screen has to be both accurate and timely. You would not want to login to your bank’s site and see an inaccurate account balance, right? In many cases, sites of this nature use a very different type of architecture to “social media” sites. Some banks actually have supercomputers running their websites in order to accomodate this.

Underlying the dichotomy above, is the fundamental notion of what caching is all about – “write infrequently, view often”. Caches work best in situations where there are far fewer updates to data than views.

The initial version of our code actually implemented memcached, in an attempt to try to reduce the number of (relatively expensive) database calls. The problem is that our underlying data changes so rapidly (many times per second, for a relatively small number of resources, that are actively being viewed and changed by many users), that caching the data was happening extremely frequently. The result in practice was that some users were seeing out of date cached data, at least some of the time. Abandoning caching in our specific case helped resolve these issues.

10. Speed Optimization

We used Google Speed Test in order to optimize our project. There is a similar competing product from Yahoo as well. These tools provide a wealth of information about how to make websites load faster – in many cases significantly faster.

Among the many changes that we made to the site, based on the information from the tester, were the following:

a) Use minify to combine and compress Javascript and CSS files. No kidding – this works. Not only that, but if you have a large number of CSS files that are loaded in each page, you can run into odd (and very hard to trace) problems in IE, which appears to only be able to handle approximately 30 external CSS files on a page. Compressing and combining these files using minify and/or yui can save you more than bandwidth.

b) Use sprites to combine images into large files. This does not work well in some cases (i.e. some kinds of buttons), but this technique can save precious seconds of load time. We used a Firefox plugin called Spriteme to automate this task, although we didn’t follow all of its suggestions.

c) Validate your HTML. Again, another “no brainer”. The load time saved by having valid HTML will actually surprise many readers. The process of validation is a nuisance, particularly if your site serves up dynamic, user-contributed content. Set aside a few hours for this process, and just do it though. It makes a difference.

11. Don’t Forget Algorithms 101

I took several courses on algorithm design at university, and then did nothing with that knowledge for more than a decade. Surprise, surprise – a complex, multi-user site actually needs proper thought in this regard.

One example from our experience – the data that tracks the status of an auction (i.e. whether it is currently running, paused, won etc etc) can be “touched” by 9 different pieces of code in the site, including “gateway” code that responds to users, and background tasks.

It took significant effort to build a reliable algorithm that can determine when an auction has actually ended, and the task was complicated by the fact that some of the code runs relatively slowly, and it is quite possible for another operation to attempt to modify the underlying data while the first task is still operating. Furthermore, “locking” in this case may have negative ramifications for user experience, since we did not want to unduly reject or delay incoming “bids” from users.

Conclusions

  1. It is very hard to plan ahead of time for growth in a web environment. Sometimes steps taken specifically to try and address traffic (i.e. caching in our case) can actually be detrimental. The process of adapting to growth can actually involve a surprising amount of trial and error experimentation.
  2. Using frameworks can be very helpful for writing maintainable code. Unfortunately its sometimes necessary to work around them, when specific optimization is needed. Proper documentation and comments can help – I try to write as if I’m explaining to somebody really dumb, years in the future, what is going on in my code – and then I’m often surprised when I need my own comments later on…
  3. Work with the right people. Not just your internal team, but also your hosting company etc. This can make a big difference when you are under pressure.
  4. Prepare yourself for periods of high stress. Not much you can do about this, unfortunately. In most cases, it will be unlikely that you will actually have access to the resources you really need to get the job done. Make sure you schedule breaks too. Its hard. Burnout is much harder though.

Long time…no update

We’ve recently launched several projects on behalf of customers. The Beta process, as you can imagine, can have a negative impact on the time available to blog! Anyhow, we should soon be resuming regularly scheduled writing.

“Whole Life” Approach to Website Development – Part 3

Continued from Part 2 – http://lichtman.ca/uncategorized/whole-life-approach-to-website-development-part-2

In Part 2, we covered some of the stages during development and just after the launch of a website.

8. Operations

Image of a logistics provider's warehouse of g...
Image via Wikipedia

I’ve frequently seen websites launch successfully, only to lose steam rapidly, once the obvious difficulties in actually running a web-business come into play.

Operating a successful web-based business can be a lot of work, no matter how much effort is placed on automation.

A website that sells a product can have a major time committment required in order to fulfill orders (i.e. package them up and mail them), or to answer queries.

Content-based sites can be even harder to maintain, particularly when the general public has the ability to participate in the content creation process – large social media sites often have a significant number of staff dedicated to removing content that contravenes the terms of service.

I’ve recently worked with several customers who have partnered with fulfillment centers in order to offload a portion of the order fulfillment process onto specialists. The websites have been designed to integrate with the fulfillment center’s systems in order to keep the order statuses current. I’ll eventually post an update here when I’ve seen how well this works in practice.

Creating a plan – possibly even as early as the business plan stage – for how operations will work, the amount of time needed to keep a website running, and whether any staff are required for this purpose (and particularly how much that would cost), is an effective way of making sure that the resources will be in place to run things after launch. It also gears owners of web businesses up for the long, ongoing task of running an online business.

9. Maintenance Cycle

Car Repair
Image by akseabird via Flickr

Most web developers approach the ongoing maintenance of a customer’s site as a form of additional revenue after a project has completed.

Few owners of websites appear to budget for ongoing maintenance related issues – which could include bug fixing or building additional functionality – before or during development.

For developers, there’s a risk associated with the maintenance cycle, given that an ongoing series of invoices (unless well justified) could result in the customer leaving for another development company.

Ensuring that new owners of web businesses understand what is involved (particularly with regards to costing) in maintaining a website over time will go a long way towards forging a better partnership between developer and customer.

10. Creative Destruction – When to Scrap It and Start Over

Wrecking Ball
Image by Editor B via Flickr

I’ve worked on a number of websites where the site had some initial success, which dropped off over time. Unfortunately some products are faddish in nature, which means that there is a very well defined life cycle for the product.

The products or services sold by a website may not be the only things subject to this inherent life cycle.

A website built today may not work as well in the future. Technologies change, people’s design taste changes, and websites do become outdated over time.

As some point in time, a decision may need to be made to scrap a site and start over.

This often seems to happen on an ad-hoc basis, where a customer becomes infuriated with a developer and leaves for another web development company; the new company will typically try to sell a complete start-over, as opposed to modifying somebody else’s work.

By ensuring from the start that customers understand that there are inherent lifespans involved in a web business (just like any other), developers can potentially develop a longer term relationship with their customers that lasts beyond a single project.

Conclusions

There are a few quick conclusions that can be drawn from all of this:

  • A web business involves a lot more than just throwing together a website and putting it up on the internet. If you ignore the fact that its a business like any other, you may be setting yourself up for failure.
  • Web development companies are only part of the “supply chain” involved in building a successful web business. Other companies that are often involved include: business consultants, accountants, lawyers, loan companies or investors, PR people, SEO/SEM companies, supply chain management or fulfillment centres, phone support companies and many more. It makes a lot of sense for web developers to cultivate relationships with each of these kinds of companies, in order to be able to provide a full “package” to their customers.
  • It takes time and extended effort to build a successful web business. Both you and your vendors need to have tenacity, because a lack of staying power won’t cut it. Despite anything you’ve seen in the news, most successful online businesses are the result of many, many years of hard work.

“Whole Life” Approach to Website Development – Part 2

Continued from Part 1 – http://lichtman.ca/uncategorized/whole-life-approach-to-website-development-part-1

In Part 1, we discussed that websites need to be viewed from the perspective of the business cycle, rather than as a simple project.

4. Design / Implementation

Travelstart Design Studio is officially 3 of u...
Image by *spo0ky* via Flickr

There isn’t all that much I can add regarding the design and implementation phases of a web project. This is the technical portion of the project, and the business-owner is somewhat at the mercy of the skills of the people that have been hired to build the site.

Web developers can use this phase as an opportunity to obtain feedback from customers early and often, which may reduce the amount of time involved in the testing and “Beta” shakedown period after launch.

5. Testing

Vince and Larry the crash test dummies, who ap...
Image via Wikipedia

I know of few web development companies that have a formal testing process in place.

Part of the problem is that testing is labour intensive, and requires a particular nitpicky mindset that developers seldom choose to acquire.

If web projects properly budget for the testing phase (including adequately estimating the amount of time involved), then more options become available, including hiring staff specifically for this purpose, or utilizing a third-party testing company.

This avoids the prevalent (I’m guilty of this too sometimes) practice of pushing a large chunk of the testing process onto the customer.

6. Launch

Launching of flight Mercury-Redstone 3 (MR-3),...
Image via Wikipedia

Typically the launch of a website tends to be greeted with a lot of fanfare.

If the owner has engaged a PR company, there can be a significant buzz attached to the initial launch.

The critical things to bear in mind are that a) the buzz may not correspond to a significant number of sales, and b) there needs to be a committment to work at building the business after the initial buzz wears off (which it usually will).

I’ve seen a lot of sites launch well, only to founder later on.

Its critical not to confuse the big launch with the lengthy hard work of building an online clientelle, which may take years (just like with a regular “real world” business).

Tenacity pays off! Do not be discouraged if things don’t immediately work on launch!

7. Building Traffic – PR, SEO, Organic Growth and Networking

Stock chart showing levels of support (4,5,6, ...
Image via Wikipedia

There are quite a few factors involved in building traffic to a website over time.

A quick point before discussing this important issue: the “conversion” or “closing” ratio of your website – it isn’t any good to obtain a large amount of traffic on your website if you aren’t turning that traffic into sales (or some kind of revenue anyhow).

Keep a close eye on the closing ratio (i.e. percentage of visitors that result in revenue) over time, and don’t be afraid to change aspects of your website if things aren’t working.

I find that tools like the “Funnels” system built into Google Analytics do a great job of showing where a website is losing people.

I’ve seen a decidedly mixed bag of results in the past from both PR companies and SEO (search engine optimization) people. I’ve written a bit in the past on both topics, so I won’t go into a lot of detail here.

Generally the process of building a significant amount of traffic on a website boils down to continually promoting the site over a long period of time; this is particularly the case where a business is looking for so-called “organic growth”, which basically means people just finding the website randomly, due to it being frequently referenced elsewhere.

The process of building organic growth often is a result of “networking” with the owners of other websites. This takes both hard work and a willingness to give and take. Note: I’m not talking about lame “link exchanges”, but rather the more “blog oriented” approach where people discuss each other on their sites. An example would be somebody discussing how much a like a particular site, or how an interaction with somebody at a web-based company went. You can’t beat that kind of press.

As mentioned above, I’ve worked on a number of projects over the years where various PR companies were engaged in order to promote projects.

My experience has been that PR companies tend to have specific areas where they shine, and promoting websites isn’t something that all of them do well. Partially this is because PR is oriented towards short term promotion of products, or finding ways to plug something newsworthy (and hence time-limited).

If you can find a PR person who has staying power, and can keep doggedly plugging away at something long after it becomes boring – stick with them, as they are rare.

Its also been somewhat evident that there are many PR people out there who are primarily good at promoting themselves. I don’t have much ability personally to see through “BS” of this nature; my recommendation would be to get some references for previous jobs that are similar to yours, and take the time to call them.

SEO is also something of a mixed basket.

Search engine optimizers and marketers tend to specialize in particular areas, for instance Pay Per Click, On Page SEO, Organic SEO, Link Building, Social Media Marketing etc. I don’t want to go into detail on the topic of SEO, despite its importance to newly launched websites, since I’ve written frequently on this topic before.

The key with hiring an SEO/SEM company is a) check refences, and b) start with a small, limited scale project first, and don’t be afraid to drop them quickly if they don’t deliver.

As somebody who has worked in this field, I’m also aware that there’s a similar effect at play to PR people – over time the effectiveness of a specific technique may wane, resulting in a loss of interest on the part of the SEO people involved. When this happens, it may be time to move on.

To Be Continued…

“Whole Life” Approach to Website Development – Part 1

In my experience, small business owners tend to view websites as either a business card that happens to be online (i.e. excessively low expectations), or as something that is supposed to miraculously provide revenue the instant it is launched.

Master List of Websites
Image by anselm23 via Flickr

The problem is partially one of not taking a broader “whole life cycle” view of the website as a business in its own right. It may also be exacerbated by web developers who may be more comfortable with creative or software development, as opposed to a hard-headed business-oriented view of a site.

The issue is that a website is a business like any other, and that implies that there is more going on than its appearance or functionality alone.

Developers also need to resist the temptation to take the easy way out and just implement whatever the customer asks for. This may require developers to pick up some business-oriented skills in order to provide a more professional level of service for their clients.

A look at the full cycle involved in building a website – or more accurately a web business – gives a wider picture that may suggest both areas for improvement on the part of developers, as well as a more realistic set of expectations from business owners.

The “Whole Life” development cycle starts long before anybody discusses what a website is going to look like, and continues (often for years) after the site has been launched.

1. Concept

Lightbulb
Image by Martin Charbonneau Photographe via Flickr

Frequently, the initial concept for a website originates with the business owner.

In many cases, they take their flash of brilliance and immediately start trying to build it.

A quick feasibility study may be in order though.

This may consist of a quick Google search to see if there are competing products, or bouncing the idea off a trusted friend (I’ve encountered people who are incredibly paranoid about protecting their ideas, regardless of whether the idea is actually valuable or not).

Where web development companies come into this is in the area of professionalism: it isn’t easy to tell a potential customer that their ideas are terrible, or to try and make them modify their concepts in order to allow them to work better online.

Part of that is that developers and designers are by nature creative people, and we don’t like raining on somebody’s parade.

Part of it is also the risk of losing a possible customer.

Yes, this should be done tactfully (that could be a complete article on its own), but if we don’t address this up front, it creates a larger likelyhood of the project blowing up in our faces later on.

Attempting to adjust expectations mid-course isn’t usually much of a solution.

2. Business Model / Business Plan

Modern blueprint of the French galleon La Belle.
Image via Wikipedia

This is a neglected area with new web businesses, and one that could provide additional revenue for development firms that are willing to learn new skills – or work with other companies that have the skillset.

Frequently new websites are built and launched without a hard look at how they are going to make money. “Oh I’ll just build something and throw ads up on it” or “I’ll have a shopping cart so people will buy things” don’t hold water any more – if they ever did. I’m still amazed at how many people I talk to still think along these lines.

Want a successful web business? Start with a well defined model for how it will make money. Then put together a business plan (and possibly a marketing plan).

A well-crafted business plan can take a long time to put together (Factor at least 35 hours for a simple plan, and 6 weeks or more for a complex one).

If the web development company you are working with doesn’t provide this service (hint: my company does about 3 or 4 per month) then hopefully they have a relationship with somebody else who does.

The business plan may help later on as well, if you are seeking financing for your website.

3. Financing

If you are building a small website, the issue of financing probably boils down to giving the designer an initial deposit and then whatever you owe once its all done.

Money Back Guarantee
Image by Roby© via Flickr

Larger web projects, however, can be expensive, particularly when you factor in the labour involved in custom graphic design or software development. This means that you could potentially need to look at financing options, in order to complete your project.

Most web companies are far too small to be able to provide sophisticate financing options for web development projects, so it is possible you’ll be looking around for a different way to put together financing.

The current financial situation has resulted in difficulties in obtaining traditional sources of financing (typically loans) for web development projects. As of the time of writing, (tail end of 2009) the people I’ve spoken to have had little success obtaining bank financing for business development of any kind, including web development. It is unclear whether this source of funding is still viable for web projects.

I’ve recently spoken to people in the business of brokering small (i.e. under $100,000) loans, which are typically unsecured. My understanding is that this type of financing still exists, if one is willing to hunt around to find it, and it one is willing to accept the terms as-is. Secured loans on a larger scale can also be obtained for development projects, but again the terms may or may not be acceptable. Personally I haven’t had any experience actually taking out a loan of this nature, so this is mostly based on discussing the topic with people in that industry; I haven’t spoken recently to anybody who has obtained this kind of financing for a project.

“Angel” or “Venture Capital” funding is another traditional (at least in a Dot Com sense) way of putting together financing for web development. I also don’t have personal experience with going this route, although I’ve had “nibbles” from local angel investors that I know regarding specific projects in the past; none of those has resulted in anything to date… Usually people in either of those areas are looking to buy a piece of the equity of whatever it is that you’re building, so your track record is going to be important to them (hence the business plan!).

One unusual source of funding for web development (or software) projects can be equipment financing companies. Many of the “captive” financing companies for computer hardware manufacturers (one example would be IBM Global Financing, but all of the large equipment manufacturers have them) will finance software or web development projects, provided that you purchase equipment from them as part of the deal. So for instance you could buy the servers required to run your site, finance them, and then roll other startup expenses such as off the shelf software and custom development that are also required, into a single loan.

By cultivating relationships with potential sources of funding, web development companies can add a powerful sales tool into their “kit”, which will allow them to land larger projects. This kind of relationship could also be extremely beneficial for lenders and investors as well, since web developers are frequently the first line of vendors to see newly launched concepts. I’ve actively cultivated relationships with companies in these areas for years as a result, and I’m often puzzled when a VC doesn’t “get it” – in their position who wouldn’t want a steady stream of pre-qualified applicants?

To Be Continued….

PingPress back up

Thankfully the new server has a current version of PHP 5. As a result, I’ve been able to get my posts to ping.fm running again, via PingPressFM. That was an ongoing low-level annoyance, the past couple of months.

Update on Server Issues

The server on which our site is hosted was switched last week, resulting in some issues. I believe most of them have been sorted out at this point. If you notice anything broken on the site, please let me know though.

Google Wave Releasing Beta

Google Wave is releasing in Beta to about 100,000 people today. I have at least one staff member who is excited about this.