Five short links – friction reduction edition

Photo by J-ster

Try Gnip – My friends at Gnip have just launched a free trial service. Making it all self-serve radically lowers the barriers to just giving it a try, so if you're at all interested in sucking down masses of data from Twitter, Flickr, etc, you really should give it a shot.

The Hotlist – There's tonnes of information about what our friends are up to flowing around us these days, but it's far too time-consuming to actually make sense of it all. That's why I love this mashup of your social graph, Facebook events and a map to help you discover events you should care about.

Jumppost – It's really hard to uncover the information you need to find good apartment rentals, especially if you're looking more than a few weeks ahead of your moving date. These folks have figured out that existing tenants know when their lease is up, and are willing to pay them up to $500 to pass that information along to apartment hunters.

Everlater books – My old Techstars colleagues Nate, Natty and Ryan have been doing an awesome job with their service that makes it much simpler to share your travel memories than the alternatives of email, blogs or Flickr albums. Now they've made it easy to create books from your trip diaries, and the quality of the examples I got to handle was impressive. If you order one today, you'll get free shipping too.

Super-simple Storage Service – I've been frustrated by how much complexity is introduced into my processing pipelines by actually having to read data, so S4's new write-only storage engine promises to radically simplify my code. At $12 a year for a terabyte of data the price is right, and AT&T's adoption of it for their customer complaint database proves it's ready for enterprise use.

Are you in danger from Facebook’s privacy changes?

Photo by Johnny Grim

"How am I in danger? Do people really care about what I post – like and
dislike on a social networking site? If so, what are they going to do
with the information? I don't get it.

This question came up in the comments of my blog, and though it's very simple, the answer's surprisingly complex and brings up much deeper philosophical questions.

The short answer is that you're in no danger right now, despite all the gnashing of teeth and wailing in the tech community. There's no evidence that anyone's using this information for malicious purposes, just as I've seen no actual burglars using the information in Please Rob Me.

So why are the geeks so upset? They're looking down the road and imagining all the bad things that the people wearing Black Hats will be able to do once they figure out what a bonanza of information is being released. Do you remember in the 90's when techies were hating on Windows for its poor security model? That seemed pretty esoteric for ordinary people because it didn't cause many problems in their day-to-day usage. The next decade was when those bad decisions about the security architecture became important, as viruses and malware became far more common, and the measures to prevent them became a lot more burdensome. The geeks were proved right, you can't start with a shoddy security model and just patch it into something secure.

I think the inelegance of Facebook's approach is what makes engineers' skin crawl. The model they use to prevent your information leaking out is a mess, both from the API side and in the UI. This makes it almost certain that there's unintended holes that leak information that even Facebook aren't aware they're revealing, and ensures users have no clue about what they're opening up to the world.

Fueling the anger is the feeling that Facebook is being deceptive in how they change their privacy model. They appear to believe there's a simple trade-off between making money and keeping users happy, and have apparently decided that they're in a strong enough position to ignore user complaints in order to increase their revenue. They're making information public because they want Google Juice. The more user-generated content they have on the public web, the more visitors from search engines they'll get, and the more important it will be for companies to have Facebook pages and advertising.

In practical terms, why is the information they're revealing important? Here's some of the scenarios that dance through geek's heads:

Embarrassment: There's a lot of personal information we'd rather keep to ourselves that might be revealed by our fan choices or friendships. You fan a gay club, and a homophobic potential employer spots that. Your ex-partner's divorce lawyer spots you're a fan of 'partying', and uses that as evidence against you in a child custody battle. Someone with a grudge targets your friends and family for harassment.

Big Brother: Social tools played an important part in the Green uprising in Iran, but you can bet your bottom dollar that there's now people within the regime using the same tools to track down dissidents. There's a lot of people within Iran who are fans of Mousavi, and since people generally use their real names on Facebook they could easily be found. I actually removed detailed data from FanPageAnalytics for Iran, Burma and North Korea because I was worried about this sort of usage.

Criminals: I'm skeptical that social network information will help traditional criminals, but there's a massive world of phishers, scammers and identity thieves I can see learning to use what's being revealed. If you got an email that appeared to be from one of your friends, said hello by name and giving you a link to something you were interested in, wouldn't you be a lot more likely to click on it? Facebook's starting to reveal the information criminals need to personalize social engineering attacks like phishing emails, it's just that the bad guys don't have the sophistication to use it yet.

So, don't panic, but pay attention to what Facebook's doing. In the short term the biggest security issue on the site is still the spread of traditional Windows viruses and malware so keeping your virus checkers up to date should be your first priority. Long term, we need to figure out what information we want to reveal, rather than letting Facebook decide for us.

How to lure people to your startup with analytics

Photo from Marquette University

I'm fascinated by statistics, and a lot of my work has revolved around trying to analyze and visualize online activity, whether it's Twitter conversations, email behavior or Facebook friendships. These have generated a lot of interest, but it's been hard to see how to convert that into enough revenue to create a real business.

The fundamental problem is that in most cases the statistics aren't solving a painful problem for anyone. We all love to look at ourselves in the mirror, and services that analyze our online behavior satisfy that craving, but there's seldom enough to justify a subscription or purchase. There's also the problem of driving repeat visits. Many meaningful statistics are very static, so the second visit to a site will often show exactly the same information as the first, discouraging continued engagement.

As a practical example, look at Xobni. I love what they're doing, and they initially launched with a lot of in-depth statistics about your email behavior, but in subsequent releases de-emphasized those in favor of productivity enhancements to Outlook. That's been successful for them and they're expanding back into analytics again, but it shows what a problematic driver statistics can be.

There is a tried-and-tested path to building a business around analytics though. I think of it as the Feedburner model, since they're the first company I encountered using it. They offered analytics as the carrot to give users a reason to sign up to the service, and then monetized by inserting ads into the RSS feeds they now controlled. I'm addicted to my Feedburner stats, but I'd never pay for them, so this was a great way of getting revenue out of a free service.

More generally it doesn't have to be ad-based, you just need a business model that benefits from access to a large audience of engaged users. Can you up-sell premium services that appeal to the same market? Can you get permission to pass their contact information to other companies they might be interested in, and get paid for sales leads? Is building an appealing analytics package just a good marketing investment, driving traffic to your site more cheaply than conventional advertising?

If you are interested in that approach, here's a couple of tips. First, try to show users something as soon as possible. In an ideal world they arrive at your page and immediately see a graph that tells them something interesting about themselves or something they relate too. Typically this isn't achievable, but at the very least have a single step where they enter an email address, twitter name, etc and then within a few seconds get some information. You should also show an example of what they will get on the landing page. These techniques reduced my bounce rate massively, never overestimate people's patience, you constantly need to be convincing them to spend time navigating your site.

The second key is presenting your statistics in an actionable way. If you can not only tell a user something interesting, but cause them to do something based on that information, then your chances of a repeat visit shoot way up. Feedburner has an 'Optimize' tab that guides you through ways of increasing your traffic. I found that changing from just showing your most-frequently-contacted friends to sending a report of the people you used to talk to and haven't for a while ('Losing touch report') and giving them a link to email each person alongside the list turned it from an 'oh, that's nice' to a must-have.

If you're as addicted to statistics as I am, but despairing about turning it into a business, it's worth thinking laterally. Analytics are a great carrot, can you use them to super-charge another business model?

Five short links

Photo by Ucumari

The Institute for Unnecessary ResearchArt, science and magic, makes me wish I was at NetSci to hear about Cybernetic Bacteria

Pentaho – Some friends have been producing amazing results with this open-source business intelligence package, it's a really impressive framework for exploring your data

An analog video synthesizer – I'm fascinated by the outer reaches of video technology, and I love this application of old-school analog technology to video stream by LZX – I've ranted about this before, but if you're in the US and want your money to be lent out to local businesses rather than pumped into financial speculation and prefer customer service from actual humans, you should check out this site to find a nearby credit union

Karmasphere – I haven't spent much time with it, but so far this friendly interface to Hadoop looks very promising. Elastic MapReduce does a good job of making Hadoop available to the masses, but this goes a long way to make it a lot less intimidating too

From pub to pub on the West Highland Way


One of the joys of hiking in Britain is that you can end every day with a pint of beer. Even the remotest hamlets will have a pub, even if there's barely enough inhabitants to staff it, let alone provide customers. To take advantage of this, we set out on the mother of all pub crawls – 40 miles from bar to bar along the West Highland Way.

Sitting in Lauders pub in the center of Glasgow, I savored my first pint of '80 shilling', the national beer that I've never seen outside of Scotland. It's closest to what you'd call a 'bitter' in England, dark, rich and not fizzy like the typical 'lager' beers in the US. Every brewery has its own version and you'll rarely go wrong in Scotland if you ask for 'a pint of 80'. As we enjoyed our pints, we were entertained by a group of 12 year-olds smoking cigarettes and spattering each other by stamping on ketchup packets.

The next day, we caught the train to the start of our hike, Crianlarich, about half-way along the Way from its beginnings in Glasgow. The station itself had the world's tiniest tea-room squeezed into what appeared to be a converted toilet cubicle on the platform, but fortified by visions of a luke-warm pint at the end of the day we started out on our hike.

Though our first stage was only six miles, we stumbled over the sort of history that tends to accumulate in a small country that's been inhabited for 8,000 years. We passed a pond that Robert the Bruce had thrown away his sword whilst being pursued, and slightly later where he stopped and defeated the pursuers. From my memories of Dundee's nightlife, I'm guessing he just head-butted them into submission.

We rapidly realized that most people weren't carrying 30 pound packs, and instead had taken advantage of the van shuttle services to take their gear between each stop. That just seemed like cheating, but then we weren't the real hard-core, carrying all their camping gear in even heavier packs, or even running the whole 95 miles in under 24 hours!

Just before we arrived in Tyndrum, a large patch of bare earth accompanied a cheery sign explaining that the are had been used for hundreds of years to crush lead ore using child labor, leaving it poisoned and lifeless, and pointing just off the trail to our B&B. Happily it was upstream from the old leadworks, but it was yet another reason to stick to refreshments other than tap-water.

Wandering bedraggled into Glengarry House, we were immediately greeted by the owners Ellen and Pat. Wonderfully helpful, they magicked up a welcome cup of tea as we got settled in our room, and we started to regret our plans to eat out that night as we smelt the meal they prepared for their other guests. Steeling ourselves, we wandered half a mile down the way to Tyndrum proper, and made ourselves at home in Paddy's Bar. My McEwans 80 shilling hit the spot, but the place itself was fairly anonymous, a bit like a converted portakabin. Moving across the road for food, The Real Food Cafe was stupendous, with ana amazing menu of lovingly prepared versions of British classics like fish and chips, all using local, organic and even gluten-free ingredients. The seating was at two long bars, which led to a lot of socialising between the parties.

Next morning we chatted to our hosts a little more (apparently April is a much quieter month than May, with just as good weather) we strapped on our packs and, after a brief diversion to The Green Welly Stop for postcards, we headed for our next stop at Bridge of Orchy. Again, this was a short stage, only 7 miles, but it still left our feet aching as we came down the hill into the village. I'd booked us into the main rooms at the Bridge of Orchy Hotel (they also have a bunkhouse), and the staff treated us like kings. Almost entirely staffed by South Africans, both my pint of Belhaven Best and the following meal of salmon were superb and delivered with great ceremony. The barmaid offered me a tasting glass of some of the other beers while we were browsing, and Liz was handed an impressive data sheet on the different characteristics of the whiskies after she asked the bar tender's advice. The receptionist enthusiastically told us "That's sensational!" after she heard that we'd done an extra reconnaissance hike up the hill that evening, which became our phrase of the day.

Day 3 was the first of the tough hikes, 12 miles to the Kingshouse pub with 1500 feet of elevation. We had a slog up and over the ridge we'd explored the night before, passing the Inveroran hotel and campground, then a seemingly never-ending march up a sloping old drovers road, headed to Ba Bridge. We were apprehensive about this section because we'd seen signs warning that the track was going to be used as part of the Six Day Trials motorbike event, and we did get passed by a few, but they were remarkably quiet, more like mopeds than the Harleys we used to live next door to in California. It was fascinating to watch 'guys trying to ride a motorbike up a waterfall' as Liz put it, and I was actually glad we caught them up close.

The Kingshouse pub appears as you come over the final ridge, and it looks deceptively close. When we finally arrived, we both nursed a Calder's 80 Shilling and called for a taxi to our B&B, since Kingshouse is alone in the middle of a stunningly beautiful stretch of moorland and they'd been booked up even 5 months ago! The taxi was going to take a while to get to us, so we ended up having a second pint which led to a very chatty ride.


Kinlochleven itself is in a valley at the head of a loch, surrounded by steep hills wooded with beech trees that looked golden with the sun on their spring leaves. The view took my breath away, and our room at The Highland Getaway had a back window looking out onto a wide stream. Wandering around the town, we stopped at the Tailrace Inn for dinner and another pint. It felt like a traditional British pub, my Tennant's Ember 80 shilling was almost spicy, but Liz's request for whiskies was met by the barman with a scowl and a gesture at the shelf behind him. A final whisky at the Harlequin Cafe attached to our B&B was a lot more pleasant, with the dark wood and red wallpaper that every pub needs.


We skipped the Kingshouse to Kinlochleven stage of the way, missing out on the Devils Staircase (Beelzebub gets all the cool geography), but our final day was the real test. 16 miles and 2,000 feet of elevation to make it to Fort William, with our heavy packs on. It was just as painful as we'd imagined, but we finally stumbled into Fort William and the end of the way. With astonishing lack of foresight, nobody had built a pub there, and we ended up walking all the way to the town center until we found liquid refreshment. The Crofter gave me another chance to try McEwans, while Liz fell back to the old favorite of Guinness. Wandering down the High Street, I discovered that the little gift shops selling figurines had apparently progressed onto Stripper Fairies since my last visit:

I had high hopes for our final lodging, the Cruachan Hotel. As Liz said when she saw its website "it looks like a castle!", and I'd gone crazy and spent $200 for the night. I had my first misgivings when the sign out the front advertised vacancies and $75 rooms for two. That first impression was borne out by the pokey room that looked like a time capsule from 1973, complete with no toilet lid, a single pillow each that was flat as a pancake, mould in the bathroom and strange stains on the wall heater.

We really didn't care though, we'd made it! It was an amazing trip, I just wish we could have had more time and managed the whole 95 miles. The West Highland Way is a hidden treasure, I was surprised at how few people we encountered, and almost everyone was local. If you're considering a visit to Scotland and want to experience that amazing landscape up close, I can't think of a better vacation. Just don't forget to check out the '80'!


Five short links

Photo by Jeff the Trojan

I'm flying home from the UK today (volcanoes permitting) so if you're waiting for a reply to your email I should be caught up soon.

Facebook files criminal charges against startup for violating terms of service – I think Arvind nails it with his analysis of Facebook's real problem. They're scared that they're losing the trust of their users, and they've fallen back to legal bullying rather than trying to solve the thorny issues that caused it.

Top 10 reasons you should quit Facebook – The incredible popularity of Dan Yoder's article makes it clear  there's deep concern about Facebook in the tech community. I'm still amazed by everything they've achieved, and don't want them to jump the shark, but if this starts resonate with people outside the bubble then they're in deep trouble.

Amazon S3 file deletion fail – Amazon make it very hard to delete large amounts of data once you've uploaded it. You can't even delete a bucket without removing all of the contents first, which makes no sense at all. Of course, you could always delete you account, and somehow they can remove the data then!

Location-based art: Audio Graffiti – Art is an incredibly underrated driver of technology, artists with a technical bent often come up with amazingly innovative uses of new tools, and this presentation made me miss my days of hacking on VJ software.

Jedi vs Tescos – I spent three years working for Tescos on the checkouts, and learned an amazing amount about customer service from one of the few UK stores to care about it. I thought the spokesman's comments at the bottom were pitch-perfect, "If Jedi walk around our stores with their hoods on, they'll miss lots of special offers". (via Overlawyered)