Photo by Eye2Eye

Our inboxes contain deeply private messages, so I'm used to hearing an initial reaction like this from Pascal Van Hecke

Watching http://web.mailana.com/demo/ don't understand anyone can believe employees will ever let their employers analyse all of their email

I need to create a full video to address why that first impression is wrong, but here's the short version:

Employees have complete control. The system suggests a profile for every person, but they're free to alter it, or even not publish it at all. There's a white-list of expertise keywords so all the suggestions will be safe, and nothing is revealed but your profile.

Employees benefit. I'm passionate about building this thanks to my experiences at the Apple coal face. Ask anyone working in a large company, they desperately need better ways of finding expertise. My goal is to save some unneeded late nights and hair loss and help people get things done faster.

Big Brother doesn't pay. There is a small market out there for business intelligence and  'compliance' monitoring of email, but very few companies want to alienate their employees by making them feel spied on. The big opportunity is in building a tool that is eagerly adopted by users, and that means building something they can trust to protect their privacy.

Similar projects were popular. I've spoken directly to a number of users of both Microsoft Knowledge Network and Tacit, earlier attempts based on similar ideas. Everyone who'd used them was very happy with the results.

I know I'll have to fight hard to win people over, but this really is an innovation that can make the world better, not another step towards an Orwellian nightmare. As @ev said at TED "when you give people better ways to share information, great things happen".

I released Mailana running on public Twitter data today! Tim O'Reilly kindly retweeted a link to it, so it's had a good load test. Fingers-crossed it's handling it so far.

If you're a new reader, here's a bit of background. My names's Pete Warden, I'm the founder of Mailana, and up until 6 months ago I was an engineer at Apple. I can't imagine a more impressive corporation to work for, but I knew there had to be better ways to collaborate within big companies. I spent a lot of time tracking down internal experts and the right people to talk to at external organizations by word of mouth. There was a lot of wasted work simply because employees in one department had no way of telling that their problem had already been solved by another team.

One day, I realized that just by analyzing my inbox, I could create a pretty decent profile of what projects and areas I was involved in, by looking for how often keywords show up, and who I talk to both within the company and at external customers and suppliers. Being able to easily build these expertise and contact profiles would let companies build directories to allow employees to easily find the resources they need. The biggest problem is ensuring that the privacy of people's email is respected, so the system I designed only publishes profiles after each employee has reviewed and edited their own.

You can see an example of how the system works here:
http://web.mailana.com/demo/

To build these applications, I designed a cloud-based system that imports from Exchange, Gmail accounts via IMAP, Outlook PST files and Twitter, offers a simple REST API to the stored data, and allows you to build apps as web widgets that can be deployed through the browser, on Sharepoint and as native menu entries and windows within Outlook.

Since there's not much public email out there, I decided to demonstrate what it's capable of using the massive number of public messages available on Twitter. My system internally processes a standard XML format, so I just wrote an importer that converted tweets into the right form, the rest of the system is identical to the Exchange version. I tidied up the web app that displays your immediate contacts, and had a Twitter social graph viewer up and running.

I'm hopeful that http://twitter.mailana.com/ will demonstrate how much interesting information there is sitting unused in your personal communication data. I think there's some amazing opportunities to build really useful tools, as long as we can design systems that preserve privacy. Imagine a version of LinkedIn that knew how close you were to all your contacts! Let me know what you think, and if your organization is interested in giving Mailana a try.

Photo by Matt Watts

The only way to figure out what your users want from your software is to release it and get feedback. As an engineer, this is something I constantly struggle with, because at any given time the actual product is only a pale shadow of what I know it could be if I had just one more week! How can I possibly release it in this state?!

Having a release deadline forces you to prioritize and fix the boring problems you know the user will hit, rather than doing something more fun and arcane. Once it's released, you'll also rapidly discover that users don't care about the things you expect, and drive the product in completely unexpected directions.

A couple of articles this week were good reminders to keep focused on getting releases into the wild. Tom Evslin blogged his advice for entrepreneurs, drawing on Jeff Jarvis's new book What Would Google Do? When Google News was about to be released, the programmers were warring over whether to put their resources into sorting by date, or location. They couldn't decide, so it was released with neither, and 300 of 305 emails from users asked for date.

In a darker scenario, Eric Ries talks about a project he'd been involved with that waited too long to release. It was a well-funded, multi-year project, staffed with smart people with a clear plan that they executed on. I've seen a lot of these at engineering-driven companies, where a powerful figure with a technical background will spin tales of the wonderful software he can build, if he's only given enough resources. They always feel like the Apollo program, full of idealism and shooting for audacious goals. The trouble is, you don't find out if your goals match the customer's needs until the end. No matter how many market surveys and focus groups you run, you'll never get a clear idea of what people will actually pay for. Building it in one shot like that also means you've got a massive foundation of instant legacy code that makes it hard to adapt once you do hear from real users. That's exactly what happened to Eric's project, after pouring years and millions of dollars into the software it flopped on release and the company failed.

Have a strong vision of what you wanted to achieve and big hairy audacious goals, but stay on course by releasing early and often.

While they have plenty of limitations, tag clouds are excellent for outlining the relative importance of many concepts at once. They're also visually very attractive, a lot more eye-catching than a uniform paragraph of text or a bullet-point list.

I'm having trouble coming up with any arguments on what a 3D tag cloud is actually good for (if pushed I'd mumble something about using the extra dimension to show time information) but boy, is it it pretty! Awesome job by Roy Tanck, and thanks to Logic Nest for finding it.

My friend Mrinal just pointed me towards someone I should have found long ago. Rob Cross is a management professor at the University of Virginia who's applied social network analysis to improve all sorts of companies. The part I like the most is that his case studies are strongly focused on concrete results and money saved. Here's some of the things that he's used network analysis for:

– Cloning a successful team by duplicating its structure with another set of people
– Fostering innovation by identifying brokers who spread improvements across the organization
– Spreading best practices by integrating groups within the company working on similar problems
– Building connections between sales teams so they can boost revenue by cross-selling

I've ordered his two books, Driving Results Through Social Networks and Hidden Power of Social Networks:

If you're looking for a layman's guide to what he's able to do for companies, his newspaper articles are a good place to start.

All of his analysis, in common with other practitioners like Ronald Burt, is based on self-reporting and surveys. I'm pretty convinced that many of these same techniques will be even more interesting and detailed if you apply them to implicitly generated communications data. Your email inbox is going to be a more accurate and comprehensive record of who you communicate than your recollection.

Photo by Irish Typepad

Dr John Wang [Update- sorry, wrong John Wang!] has just started a new site called EnronData.org, dedicated to developing and refining the Enron email dataset. It’s off to a cracking start, offering all the Enron emails as 148 PST files, one for each ‘custodian’ (informally each mail user). I did my own PST conversion, but it was primarily so I had a large data set to load onto an Exchange server and test Mailana against. John’s version is much closer to the original source data, and so will be more of a real-world test for applications.

I’m really pleased John has put this together, it will be a boon to anyone looking at doing heavy-duty email data-mining. I can’t wait to see what else the project produces.

Photo by Cayusa

Founder of ClearContext, Deva's been pushing the boundaries of email productivity for years with his Outlook plugin. There's too many clever ideas in there to list, and who can argue with a guy who got his seed-funding playing online poker?

T.A.'s behind Gist. Along with Steve Newman, they're building a startup focused on linking external information to email contacts, breaking down the silo that surrounds email, a cause dear to my own heart. I'm jealous of the name too, very appropriate and memorable.

Former Engineering VP of Xobni, Gabor's now beavering away on a new email startup. Not many details have emerged yet, but I know it will be worth watching.

Co-founder of Postbox with Scott MacGregor, Sherman's building an entirely new email client, with a radically new interface for working with all the information we've accumulated in our inboxes. The dominant UI we have at the moment for mail messages is essentially a DOS directory listing, Postbox (and Xoopit with some similar ideas) gives you a truly graphical user interface for mail.

Joshua is the founder of OtherInbox, another company trying to move email away from being a flat list. Rather than reimagining the core interface, they're using multiple email addresses and automated-email recognition to present all the information that Facebook and other services send via email in a much simpler form. And by removing that bacn from your messages, they return email to its roots as a person-to-person medium, one of the big reasons that Facebook mail is so pleasant to use.

CEO of Inboxer, Roger's one of the few people I know innovating in email on the server side. Since administrators make the purchase decisions there, not end users, a lot of the product is driven by regulatory and security concerns, but they also manage to offer some interesting options for searching and recovering attachments.

Who am I missing? Let me know…

Photo by Giorgio Zanetti

My friend Howard Cohen, one of the few people to complete the 70 mile Backbone Trail in a day, is off to North Africa next month to take part in the Libyan Challenge. Running 120 miles across the desert, unsupported, he's part of the first American team ever to enter the race.

I like to think I'm fairly fit, but Howard and the other ultra-runners are in a class of their own. It's awe-inducing to see how far they can stretch a human body, and the mental toughness they need to get through their ordeals is amazing. It takes some inspired insanity to tackle challenges like these.

Stop by Howard's site and wish him the best, and keep an eye on the offical website if you want to follow their progress starting February 24th.

The comments on Micah's post about the Boulder tech scene got me thinking about the difference between a community and a clique. On one level, a clique is any community you're not in. All social groups offer advantages to their members and have barriers to entry, which can make them infuriating to outsiders.

Communities work because they distribute the cost of evaluating strangers across all the members, and magnify the cost of offending any member by spreading information about transgressions. In other words, social networks generate reputations. Reputations aren't perfect, but they're harder to fake than any other signal we've got about trustworthiness.

So, we need groups, but we all know from high school how they can be used for evil. What distinguishes a good community from a clique?

– Meritocratic. Inclusion and status should be determined by something more or less objective.

– Open. There should be established, well-known ways to become part of the group.

– Positive. The community should be defined by who they include, not who's excluded.

In my experience, Boulder's tech scene is all of these. Sure, there's well-connected people, and others who are less in the loop, but the central people tend to be there because they're heavily involved in events, or have a long track record in startups. There's a lot of events and programs like TechStars, boulder.me, new tech meetups, hackspace and Ignite Boulder that are good entrypoints to the community.

If you want to understand how VCs really decide which startups to fund, or how entrepreneurs arrive at their sales forecasts, watch this stunning series from the San Diego Venture Group. If nothing else, you’ll learn the best way to approach VCs (hint- it involves assembling a massive spreadsheet of hundreds of email addresses and sending nightly emails to them all until they agree to give you money).

Thanks to Startup Lawyer and Brandon Zeuner for alerting me to this fine example of investigative journalism.