The Best Park in San Francisco

Buenavista0
All photos © Heather Champ, with permission

My biggest worry when I moved to San Francisco in December was that my dog Thor would find urban life tough, without the wide-open spaces we'd got used to in Colorado. I found an apartment with a wide sunny window-sill for him to lie on and next to the great off-leash Duboce dog park, but what I didn't realize until I started exploring was that there was another gem just half a mile away. The first signs I saw of Buena Vista Park were the trees at its peak looming over the neighborhood, their tops wrapped in fog. Following Duboce Avenue uphill to its end, and then meandering through 37 acres of woodland, I found myself 575 feet high, looking out over the city as the clouds cleared.

It's now become our morning walk, and we can make it to the peak and back in 45 minutes if I'm in a hurry. That's not often though, because catching up with the other regulars has become part of the pleasure. In the peculiar way of the dog-walking world, I often feel like I know the canines before I've properly met the owners, especially since I can't compete with Thor's natural charms. A great case in point is Bug and Chieka's owner, or as she's better known in the tech world, Heather Champ, the pioneering former Flickr community manager who furnished these photos.

Though it's not on the same scale as Golden Gate or Presidio, it's actually the oldest official park in the city, and is full of corners and history to explore. It was only when I was chatting to one of the gardeners that I realized the marble chunks lining some of the paths were actually fragments of gravestones from the cemetery that was razed by WPA workers in the 30's. At night its nooks can make it less welcoming, with hoboes camping out and casual hookups, but in the morning it's a slice of heaven.

Buenavista1

If you live anywhere near the Lower Haight, Upper Castro or Noe Valley areas, you really should check out this urban garden. There's nothing like wandering through groves of Eucalyptus and Redwoods, watching the fog blow through the canopy, to refresh your soul after working through painfully obtuse YouTube comments (just to pick a random example, ahem). And if you see a cute Chihuahua mix with a snaggle-tooth, be sure to make a fuss of him.

Buenavista2

Location Tracking as Art

Livingbrushstrokes

I just discovered Maria Scileppi’s Living Brushstroke project that uses iPhone location tracking to create artistic views of people’s movements. The picture above is from the Chiditarod, “Probably the world’s largest pub crawl/food drive”. Almost everyone’s first reaction to seeing the trace of their movements retrieved from the iPhone logs is the same as ours, “Cool!”. If we can give users control over their own data (asking permission to record as Maria’s app does), there’s so many amazing projects like this we can build.

Anthem from Maria Scileppi on Vimeo.

Five Short iPhone Tracking Links

Tinfoildog
Photo by Evil Science Chick

Some of you may have noticed a weekend hack I put together with Alasdair Allan for visualizing location data on your iPhone. Here's some random links related to the project:

Tell-All Telephone – View a German politician's life as a visualization, after he agreed to have detailed recording software added to his phone. [Update, I misread the story, and it was actually information gathered without his knowledge, but that he agreed to share afterwards to raise awareness. As commenter Seve says "The movement data of the german politician Matle Spitz were collected by law and not with his agreement. The movement and call data of everybody connected to a german mobil phone network were stored for 6 months. Finally the german supreme court stopped that law and the data were deleted. "

Geoloqi – A fascinating system that lets you volunteer to track your own movements, and share certain aspects of them with people you trust.

Location tracking on Android too – Detailed and fair technical analysis of the way that Android monitors your location, and how it differs from Apple's approach.

A Cryptographic Approach to Location Privacy – There are ways to get a lot of the benefits of location services without recording or revealing your position. Arvind's proposal shows how one application could work in a secure way.

22 Free Tools for Data Visualization and Analysis – The navigation is tough without a table of contents, but this is a guide overview of a lot of the tools you can use to turn your data into a visual story.

Data Science Toolkit 0.35 released

Toolkit
Photo by Wonderlane

I've just completed deploying a new version of the toolkit. It contains quite a few bug fixes and improvements, along with two new features:

UK Support 

You can now enter British postal addresses into the street2coordinates geocoder, and it will return the coordinates. With post codes included, it's normally accurate to within a couple of hundred feet. You can then run those positions through coordinates2politics to get the parliamentary constituency, county, council district and ward, NHS area and post code.

Time and date extraction

The new text2times method will scan through the text that you pass it, and pull out any strings that it can understand as times or dates. These include both a variety of formal date/time combinations like '10/28/01' as well as informal descriptions like 'next Friday'.

You can try it out by going to http://www.datasciencetoolkit.org/ , the command-line tools are at http://www.datasciencetoolkit.org/python_tools.zip , the new AMI is ami-f6e11d9f, and there's a new VM at http://static.openheatmap.com/dstk_v0.35.vmwarevm.tar.bz2

You should still be able to use your existing code with no changes, I've done my best to ensure everything's backwards-compatible, but let me know if anything breaks.

Five Short Links

Tetrahedra
Photo by Nicolas Suzor

Newscoop – A content-management system designed by and for journalists. It’s been used in conjunction with Ushahidi to interesting effect.

Crisismappers extends UN capacity in Libya – As the power of crowd-mapping becomes more obvious, there will be pressure to use it in more ambiguous situations than natural disasters or the toppling of tyrants. Mapping military airbases in Libya gives a hint of what crowd-sourced warfare could look like.

StarCluster – A simple way to run and manage clusters of EC2 machines for scientific computing, complete with AMIs pre-loaded with useful software and sensible defaults. If this is your thing, you should check out Infochimp’s impressive ClusterChef too.

Fathom – A design firm with a portfolio of clear, crisp and beautiful infographics,

Trunkler – Link curation for your iPhone, built on top of the powerful Trunk.ly service. Shows why having a third-party API can pay off, even in the early days.

The cock-up theory of technology news

Brachiosaurus
Photo by Joshua Mellin

A few times over the last few weeks I've been talking to friends about big tech company news, and one of the hardest struggles I have is to convince them that the latest Twitter or Google happening could just be a random cockup, rather than a sign of hidden plans by the company's management. I could well be wrong, but Google releasing a group-messaging app that doesn't run on Android reminds of the time I was involved in launching a GPU image processing API at Apple that competed against a similar interface created by another team at almost the same time. We mostly managed to keep that sort of thing from reaching the outside world, but every team and department in a large corporation is competing against all the other teams for a slice of the budget pie, and can have very different goals. Good management will keep the worst excesses in check, but any large organization is a massively distributed system where the communication overhead of keeping everyone totally in line would be crippling.

I was sorry to learn that large dinosaurs are no longer thought to have had an extra brain in their buttocks, as I'd memorably learned as a child from the Dinosaur Club. It's still a pretty good image for the decision-making apparatus in big companies though. Upper-management's most precious resource is time, and so attention has to be rationed. Especially at a company like Google that prizes experimentation, that means lower-level people can release projects into the wild that don't fit into the grander strategy. As another example from Apple, I know several teams that still hadn't ported their code over to the Cocoa framework even by the time I left in '08. Eating your own dog food is a fine goal, but if it comes down to that or shipping, sometimes pragmatism wins.

Think of my approach as the cock-up theory of technology news. The next time one of the big firms does something that makes no sense at all, consider taking it at face value. As General Nasser said in the 50's – "The genius of you Americans is that you never make clear-cut stupid moves, only complicated stupid moves which make the rest of us wonder at the possibility that we might be missing something".

Save the world with data

Globefloat
Photo by Daniela Hartmann

I was talking to someone recently who at first I thought was an angel investor interested in my work, but it turned out he was keen on funding charitable projects. It shows I need to improve my messaging – while I am technically running a non-profit, that's not by choice. I do love supporting that world, but I'm always looking for revenue opportunities where I'm adding value for users who can afford it, even for my open-source projects.

It did get me thinking about some of my favorite non-profit teams though. They're all primarily focused on using data to do good, even if they're not purely non-profit in structure. Here's the people I'd trust to put my money to good use, if I was a social investor.

Ushahidi/SwiftRiver

With uptake worldwide, Ushahidi has been doing valuable work spreading information around disasters and political unrest, everywhere from Egypt to New Zealand and Japan. The SwiftRiver part is the brainchild of Jon Gosier, and while I'm a bit biased as I'm contributing some time and code, it's a really powerful use of technologies that are usually only focused on ad-serving. There's so many practical ways this kind of filtering and routing of information can help in an emergency.

Global Virus Forecasting Initiative

With feet on the ground in 23 countries, the GVFI has roots in the traditional non-profit world, but a clear understanding of how modern data processing can help its mission. Lucky Gunesekara, Lalith Polepeddi and the rest of their team are building a cutting-edge information infrastructure to make sense of the masses of information gathered by the teams, matching it against social media and traditional news organizations. The results are then used to track and predict human viral outbreaks and save lives. 

Media Cloud

Ethan Zuckerman is a legend in the non-profit world as a driving force behind projects like Global Voices and Geekcorps, but with Media Cloud he's applying his skills to data analysis on an ambitious scale. As the Arab revolts show, one of the biggest problems around the world is the way people are denied a voice. By tracking and quantifying media coverage in unprecedented detail, the project hopes to draw a picture of the problem, as a first step to offering solutions. It's an important project in a lot of ways – just ask yourself when you first heard about the Tunisian uprising? It took three weeks for coverage to enter most of the Western media (though France was a bit ahead of the curve, thanks to their colonial history). Those sort of gaps in our collective vision mean we're missing out on vital information.