Photo by Jesse Bell

I have middlebrow tendencies, but over the years I've learned that the struggle with difficult work can pay off. I grew to love Infinite Jest, once I figured out Wallace was boring me deliberately, that he cared about the mundane details that make up people's real lives. As a teenager I figured out that subtitles on BBC2 late at night meant nudity, and I wound up appreciating French cinema despite my base motivations. My favorite play from last year was a production of Beckett's End Game, which left me heartbroken for characters who should have been unrelatable, screaming at each other from trash cans.

On Sunday night I made it to the Cutting Ball's production of Eugene Ionesco's The Chairs. I was ready to put some work in but I hoped it would pay off. I left a little disappointed. There was a lot to chew on intellectually, and the performances were fantastic, but I never cared about what was happening. It was a puzzle, but a cold one, and I never felt there was anything at stake, despite it being set at the end of the world. The basic plot (do existentialists care about spoilers?) is that an old married couple, apparently the last people on earth, begin to host a party full of imaginary guests, and the husband prepares to give his inspiring message to the world. The 'orator' who will deliver the message appears as a flesh-and-blood person, and the couple commit suicide, and then the orator delivers what turns out to be nonsensensical gobbledegook. I could imagine a play that made this pack a punch but when the couple threw themselves out of the windows, all that was going through my mind was "How long until we hear the splash?". It didn't feel like the company's fault, the translation was strong and the acting was up to the high standard I've come to expect from the Cutting Ball. The barrier I hit was Ionesco's writing. I know he was demonstrating how the "language of society" breaks down and how hard it is to communicate, but I'm bourgeois enough to want something more than an intellectual thesis from a play. I wish I'd caught The Bald Soprano by the same director a few years ago, one of my friends told me that worked much more effectively for him, so maybe that would have helped me connect with Ionesco?

I had very different expectation for last night's entertainment, The Taming of the Shrew by TheaterPub at the Cafe Royale. I discovered the group last month when they did multiple interpretations of a short experimental play, and I knew they had attracted a team of talented and enthusiastic actors, so I was excited to see how they'd tackle Shakespeare. Shrew isn't an easy play to produce, modern audiences are going to struggle to swallow the central plot, that an opinionated woman needs to be psychologically tortured until she submits to her husband. Shakespeare can't help but write fleshed-out characters though, so there was usually enough wiggle room in the interpretation to make them sympathetic to us. The only exception was Kate's final speech, even with the emphasis that she'd only bow to her husband's honest will it was hard to see as a happy ending. Despite that quibble with the source text, the whole evening was a massive amount of fun. I loved the relish and gusto that the whole cast showed. Kim Saunder's Kate and Paul Jenning's Petrucio appropriately stole the show with big performances that had me laughing and completely believing in their tricky-to-swallow relationship. Paul seemed to be channeling the best of John Goodman and Jack Black as he played the crazy suitor, and Kim's obvious enjoyment of the tongue-lashings Kate gives to the world played perfectly. I'd also single out Shane Rhode for the energy and imagination he brought to the tough part of Grumio, playing up to the audience as he witnessed the ridiculousness unfolding along with us. Ron Talbot, Jan Marsh, Vince Faso, Brian Martin, Sam Bertken, and Sarah Stewart all deserve credit for their work too, everyone was throwing themselves wholeheartedly into their roles, and generating a lot of laughter.

There are three more performances coming up, one tonight (Tuesday March 19th), and then Monday March 25th and Wednesday March 27th, all at 8pm. If you're looking for an experience that's all theater, with an audience and cast that are all there for pure enjoyment of a play, try to make it along. The venue is incredibly relaxed (thanks to the director Stuart Bousel for gently handling a couple of folks who were ahem, a little too relaxed, when the play started), you don't need reservations, and they have great beer!

Photo by Earl

Want to live somewhere nice? Be prepared to work longer – How an area's living costs affect poor and rich workers differently.

Moving towards an identity and patient records locator – As Ben Adida points out, a one-way hash of a cell phone number with less than a billion possible inputs is not very useful. The flipside of de-anonymization being easy with enough dimensions is that it should be possible to perform entity disambiguation using the same data, so just store the messy redundant information you get as input, and do joins when you need to. The problem of matching entities is hard because defining an entity is hard, don't fight it.

Open Access Coalition – We're going to look back on the '00's as a golden age when data was open on the web if we're not careful.

DIY McDonald's Recipes – This is a maker project I can get behind! Fast food is my guilty pleasure, thankfully only occasional these days, but I have an engineer's appreciation for the thought that has gone into designing their recipes.

Save yourself from Reddit, Hacker News, Slashdot – A neat little productivity hack from Steve Coast!

Photo by Kevin Collins

I used to be an image processing engineer. I'd be handed a picture, and I'd have to do something useful with it. To do that I had to take a big mental leap. Instead of seeing it as an image I had to picture it in my mind as a grid of measurements.

At first this was intensely frustrating, because they were deeply crappy measurements. A million factors introduced noise or errors, everything from lenses to sensor noise to encoding software. Gradually I began to make progress, despite all these problems. Decades of engineers before me had figured out inelegant but effective methods of getting value from an unpromising soup of pixels, and I was able to learn from their approaches.

Interesting algorithms in image processing are almost comically domain specific. Thousands of man years of work have gone into detecting and correcting the distinctive reflections that occur when peoples' eyes are caught in a camera flash. Compressing photos effectively requires an exhaustive knowledge of the human perception system, and very clear ideas of the likely subject matter for photos. The process behind facial recognition is a like a game of Mouse Trap, with a whole series of steps that have been empirically proven to work, but which could never have predicted from any theory.

The computer science I was taught at college grew out of mathematics, and assumed that you have a minimal set of clean inputs. Provability and understandability were prized values, and so messy ad-hoc algorithms were seen as dead ends, even if they worked for the problem at hand. Image processing taught me to value them instead, as long as they could be proven to work across the kind of inputs I was likely to encounter in practice.

Once I'd learned that, the world began to look very different. My image processing training gave me the mental tools to tackle problems that other people shied away from. If I have a large enough set of data, I know how to search for the signal, even if the noise is deafening. I'm happy to rely on correlations that aren't guaranteed to hold for all time, as long as I can test it holds in the cases I care about now, and have instrumentation to spot if the prediction stops working. I know that getting 80% of the way there and having a human fill in the blanks is often good enough.

I wasn't the only one to discover how effective this mindset can be, and it has come to be known as Data Science. It's an approach to solving problems that's light on elegance and heavy on pragmatism. It doesn't care about proofs but relies on experiments. Entirely new things are possible once you have massive amounts of data, so even if you're a grizzled old engineer like me and instinctively shy away from trendy new labels, give Big Data a try. Amongst all the marketing hype, there's some powerful techniques for building algorithms that have no right to work, but do.

Photo by Yersinia

The Deleted City – A spatial reinterpretation of the old Geocities sites. Having data in a single large dump instead of behind an API makes it possible to do things like this with it, things that the creators could never have foreseen.

Asteroid Discovery from 1980 to 2011 – See how our knowledge about the world around us has grown with this amazing animation. At the start new asteroids appear as discrete pinpricks days or months apart, by the end they're being discovered so fast they're a solid mass, it's like a lighthouse beam hitting fog. It's not only that we're finding out more, but that the rate of discovery is accelerating.

Open data on depression treatment in London – I love seeing mass adoption of data technologies, it's this sort of democratization of the tools that makes the real difference to the world. What's special about this approach is that it's so ordinary, what used to be elite techniques are now available to people in every walk of life.

BitDeli – I haven't used this yet, but I love the idea of being able to program custom analytics code, without the hassle of having to host it myself, and with the benefit of being able to reuse other people's approaches too.

Silicon Valley poverty – Even after twelve years here, I'm still shocked by how wide the gap is between the rich and the poor in the US. 

Photo by Michael Scott

People often think I'm a privacy researcher, thanks to the Facebook and iPhone stories. The truth is I'm just curious about undiscovered data. Because a lot of it is about people's behavior, and that's an inherently creepy area, I blog about what I'm doing to keep myself honest. It might look like I'm on a privacy crusade, but that's just a by-product of my attempts to figure out ethical ways to use these sources of information. I'm a data hacker, and I'm trying to keep my hat clean.

This has been on my mind a lot recently as I'm looking around at all the information that's publicly available about exactly where people have been. Facebook, Google+, Instagram, Flickr, Twitter are all making rich streams of location data available, especially around photos. My vision is a world where I can make those digital footprints visible to ordinary users. Who comes to this bar? Any of my friends? What sort of people take photos at this hotel?

The raw data to do this is already out there in multiple places, and you can do some of it by going to individual sites like Foursquare, but there's something different about merging together scattered information, even if it's all theoretically public already. You have to make a choice before your activities are publicly visible from these services, but the implications of that choice aren't clear until somebody aggregates the data and demonstrates why the sum is greater than its parts.

I wish I could pretend I was only worried about the privacy implications, but the truth is I'm excited about how fun and useful the applications could be!

Photo by Flood G

BetaShapes – Using geotagged Flickr photos to define San Franciscos neighborhoods as a crowd-sourced 'folksonomy'. I'm entranced by how many useful things emerge from the clouds of data exhaust we're all generating.

Bacteria farming and software design – Code is an incredibly useful tool for artists, I love this behind the scenes look at how an amazing visualization was built.

Ang Lee and the uncertainty of success – The acclaimed director spent six years of his career with no visible signs of making any progress, and this post does a fantastic job highlighting how hard that must have been.

Founders and dysfunctional families – Growing up in chaos is good preparation for working in a startup.

Common Crawl URL search – Thinking of crawling the web? Check out this web interface to see if Common Crawl already has what you're looking for sitting in a handy S3 bucket!