Photo by Bondidwhat

I agree with Fred Wilson that sometimes the comments add a lot more value than the original article. Stefan from Permessa just added a comment I recommend reading to my post on the email world’s storage limitations. My argument was that disks are dirt cheap, and so the days of email quotas should be numbered. He points out that there’s an awful lot of secondary costs associated with email storage, such as backups and bandwidth.

This reminds me of a pattern that I’ve seen repeatedly, one that I’ve seen cause a number of large projects to fail. You take a long-term trend, like increasing CPU power or storage size, and use that to argue that the future belongs with some approach that’s currently too expensive to contemplate, like doing all your image processing on 32 bit floats rather than 8 bit integers, or writing everything in an interpreted language. The tough part is, these arguments are usually correct, but they don’t specify a time-frame. I’m fairly convinced that nuclear fusion will be the ultimate power source in the future, but it’s remained 20 years away since 1945. It’s a very seductive pattern for engineers, because everyone likes to be visionary and completely new systems, making it a perfect petri dish to grow architecture astronauts. The leaders who authorize them get to dream big, rather than dealing with the boring details of upgrading existing projects.

The usual outcome is for the delivery date to mysteriously keep slipping, as the barriers to achieving acceptable performance prove more enduring than expected. Surprise, there’s actually strong reasons why everybody else is still using the old way.

So what’s the answer? You should be able to specify a hard time-frame for that nebulous future, and demonstrate a compelling prototype that proves your assertions about relative performance. In my case I’m building a system that takes 500,000 emails from the Enron set, and indexes and analyzes on a commodity Linux box. I’m convinced that clouds of this sort of hardware can open the door to innovative email processing tools today, in a way that Exchange’s single-server model never will. I think Gmail is a demonstration that it’s possible, and I’m trying to show that it can be done even without the Google Operating System.

If somebody tries to sell you on a vision of the future, ask them when exactly that is, and demand a prototype.