Team Jetpac from JetpacApp on Vimeo.

Here's a timelapse of thirty minutes in the life of the Jetpac team. You may notice I spend most of my time talking, which is an accurate reflection of my life as a CTO!

Photo by Sanchtv

How Zappo's user agreement failed in court – A lot of people have an almost-mystical belief in the power of terms of service and license agreements. Courts generally look for some minimal evidence that people are reading and understanding what they're agreeing to before they'll agree to enforce those terms. They also don't look kindly on agreements that allow one party to change the terms at any time, without notice or assent. The law may be an ass, but there are humans at the controls and you can't get too far from common sense before they'll intervene.

Free Foundation Trilogy Audio Books – Every now and again I have to stop and be amazed at the cornucopia of art that the internet has made instantly available to me. I remember struggling to piece together book series from libraries and stores, in a way that the web has made obsolete. I was reminded of that when I ran across this eight hour epic production by one of my favorite childhood authors, though I'm slightly afraid to listen in case Asimov hasn't aged well.

The Evil Bit RFC – The perfect antidote to endless angels-on-a-pinhead debates around web specifications (*cough* webfinger *cough*). It's way more fun to sketch the rules of a perfect world than to wrestle with the tradeoffs of an ugly one.

Unwise Microwave Experiments – Who knew you could use your microwave to create lava? I'll be trying these shortly before I move out of my current apartment.

Mind your nanoseconds – It's pretty thrilling to know that we're now doing things so fast in computing that the speed of light is becoming a significant barrier, and Admiral Grace Hopper does an amazing job turning very abstract ideas into something very concrete.

Photo by Lovestruck

There are lots of examples of big companies A/B testing their way to greater success, but it's hard to figure out how to get started with event data as an early-stage company. For one thing, you probably don't have very much data to go on! I'm going to talk about a few of the things I've learned at Jetpac as we've built something from nothing.

Sample size matters

We're still focused on getting a user experience people love, and a value proposition they understand, before we attack distribution. That means we have hundreds of users a day, not thousands or tens of thousands. My initial approach was to try a feature experiment, look at the results after a couple of days and then decide which was more successful. I rapidly discovered that this wasn't working, as a week later the statistics on a feature's usage might be much worse. Looking at measures like the number of slides viewed, it became clear how big the natural day-to-day variation was, often plus or minus 50%!

It's easy to forget basic statistics like I did, but the sample size is really, really important. There are robust methods to figure out the confidence of your results, but for my applications I've found anything less than a hundred is useless, a few hundred becomes indicative, and over a thousand is reliable.

This doesn't often come up because optimizing based on event tracking doesn't usually happen until you have large numbers of users, so it's easy to gather big samples quickly. For us, it's meant that we can only run a limited set of experiments, so we have to be very careful about how we choose the most important hypotheses. It also means that we tend to try out new features on 100% of our users, and compare against historical data, since that's the fastest way to gather samples and time is our scarcest resource!

What people do doesn't always reveal what they want to do

One of the most productive outcomes of having a rich set of data about how our users behave is that we've learned to argue about data, rather than opinions. When somebody has a product idea, we can dig into the existing data and see what evidence there is to support it. A lot of the time this is effective, but the approach has some subtle flaws. We still try to prototype some of the ideas on users, even when existing behavior seems to rule against them, and a few of them work despite the data. Sometimes we've figured out that what the old data is really showing that people didn't use a similar feature because its placement in the interface was poor, or they didn't understand what the text meant, or the icon was unattractive.

People are funny beasts, and looking at how they're using an application and saying, "they want to do X and Y, but Z isn't popular" is helpful, but not sufficient. Sometimes a small variation on "Z" will make a big difference, they may actually want to do it but be discouraged by the way it's been presented. At a big company there's probably a lot of institutional knowledge about what's worked in the past, but you won't have that as a small startup.

There's no substitute for talking to users

So if data's not the whole answer, what can you do? I've been lucky enough to work with a very talented user experience designer and one of his super-powers is sitting down and watching new users try out the app, and talking to them about what they're thinking. A classic example of how this can work is mis-swiping. Our whole app is based around touch-swiping through hundreds of travel photos your friends have shared with you, and he noticed that a lot of people seemed to be having trouble moving to the next picture. He mentioned this at a standup, and I was intrigued. I queried our activity data, and discovered that over 8% of swipes that were started in the app weren't seen as vigorous enough to result in a slide advance! This matched what he was seeing, so we redesigned the swipe gesture to be more sensitive. After we made the changes, we videoed a couple more new users and saw that they had a lot more success. A few days later I had a big enough sample size to be confident that the percentage of mis-swipes had dropped to just 2%!

Guerilla user-testing

User testing sounds awesome, but don't you need a lab to do it properly? I'm sure that would help, but we've managed to get a long way with some simple approaches that anyone can adopt. Bryce's killer technique is tilting a laptop screen down so it's looking at an ipad on the desk, and then having the user play with the app while he records the video of their hands and the audio. I know bringing folks into the office can be very time-consuming though, so we've also ended up getting a lot of value out of in-app surveys from Qualaroo, and simple user tests from UserTesting.com. The videos you get out of the latter can be incredibly useful, and it's a godsend being able to put in a request and within an hour have several fresh users work through any part of your application you want tested.

Knowing in real-time how people are using Jetpac has been amazingly helpful, it's been an incredible learning experience. Even if you're a small fry why not dive into your event data yourself?

Photo by Yesica

I'm embarrassed to admit that I've lost the compiled versions of my open-source visual-effects plugins! In a recent overhaul of petewarden.com I managed to overwrite the binary versions, and while the source code is still available, I no longer have the After Effects and other SDKs and compilers I'd need to rebuild them. They're over twelve years old now and don't work with recent AE versions, but I still get regularly get requests and questions about them, so could anybody who has downloaded binary copies of either the FreeFrame, VisualJockey or AfterEffects plugins drop me an email to pete@petewarden.com? Any help much appreciated!

Hello World in Common Crawl – I've been very excited to see the wider developer community starting to realize how many awesome projects Common Crawl makes possible. Here's a simple guide to getting started with its four billion web pages.

Remember the memristor – A great explanation of an intriguing new computing component, along with a clear-eyed look at why so many of these early-stage experiments never make it into commercial production.

ZIP code data hacking – It's amazing how many of the fundamental facts about our world are kept in closed-license silos. Anybody who wants to link US data sets pretty quickly runs into the translation issues between different area designations, so this project to convert between the common ZIP codes and the governmental FIPS designations is a fantastic idea.

PlacePugs – The ultimate image placeholders, as featured above.

Sorting one million eight-digit numbers in 1MB of RAM – So much of coding is about spotting exploitable patterns in your input data and requirements.

Photo by Joey Rozier

One of the most fun parts of working on Jetpac has been following Cathrine’s stellar posts on the company blog. She’s had some amazing finds, including surprises like the hammock tent! We’ve had hundreds of thousands of visitors and a lot of links, but until recently I didn’t realize that having a separate subdomain at blog.jetpac.com meant that we didn’t get the full search ranking benefit. We’re hosted on Tumblr, so I assumed moving things over to a subdirectory would be fairly straightforward, but it turned out to be surprisingly tough. In the end I not only had to manually proxy all /blog content through our frontend servers, I even had to rewrite the page content on the fly to remove hard-coded links! I found a couple of examples in PHP, but nothing that worked with our Ruby and Sinatra setup, so here’s what we ended up with:

https://gist.github.com/petewarden/3950261

In brief, this code intercepts all URLs that start with /blog on a site, and pulls the content from the equivalent yourblog.tumblr.com. If the page is HTML, it also does some quick-and-dirty regex alterations to change any root URLs in links and resource references to point to the new /blog versions. Not my most elegant bit of code, but it seems to be doing the job.

Photo by Rennett Stowe

Save vs Death – As a gamer, and especially as a role-playing gamer, I've spent a lot of time learning to viscerally understand probabilities at a gut level. I'd never made the connection, but that perspective has informed so much of what I've done in my life, from optimizing products to making life choices. It's sobering to read Jim's meditation on how that approach affects your world view when you're facing mortality head on.

It’s not what you know, but who you know: The role of connections in academic promotions – A natural experiment that demonstrates how much having a personal connection to a job candidate affects decision makers, with some thoughtful analysis of why that sometimes makes sense at an individual level, even when it results in overall unfairness.

Mapping with Geocommons and OpenHeatMap – Out of all the open-source projects I've done, OpenHeatMap has had the longest life. It's a painful mess of PHP code, but it seems to meet a real need, so I try to keep it up and running. The hardest part is that as more and more people use it, the mean time between server failures shrinks, so I end up having frequent down time. I've been working on strategies to reduce the flakiness (the usual failure mode is the postgres server hitting memory problems and dying) but apologies to anyone who's been affected recently.

How location technology can change publishing – My friend Drew Breunig always has interesting things to say about practical applications of location data.

Zombie.js – After the shame of having the Jetpac login system down overnight without noticing, I've been investing heavily in automated testing. So far desktop Selenium has been my favorite approach, but someone pointed me at Zombie as an alternative, and it looks very impressive. I've used PhantomJS before for screenshots, but from what I've seen so far Zombie doesn't require X windows or other heavy dependencies. I'll give a full report once I've had a chance to use it in production!

Photo by Supermuch

One of the hardest things about the recruiting process is having to turn down great potential hires because they're not a good fit with the positions we have. Most of my time at Jetpac these days is spent doing recruiting, so I've ended up with quite a few job-seekers I've been really impressed by, but that I can't hire myself. I've been sending them on piecemeal to friends, but I decided to set up something a bit more organized, so here's a mailing list you can join to see them:

https://groups.google.com/d/forum/youreallyshouldhire

Here's how the group works:

 - You must have a candidate's permission to post their details!

 - Anyone can post candidates, I just ask that you've at least emailed or spoken to them on the phone, to do the initial "Is she a serial killer?" screening.

 - There's no blind resumes, every post should give the full contact details for the candidate. 

 - This is focused on Bay Area tech startups. Feel free to start your own for other areas and industries, with my blessing.

 - This will only add value if it's mostly folks who are actively hiring at their companies, so I'll be moderating the list, hopefully with a very light touch.

If you're a jobseeker and want to be shared with some interesting tech startups, contact me through our jobs page and mention 'youreallyshouldhire'!

The 'Freedom Burger' I ate in Sitka, AK last week – a bacon cheeseburger between two grilled cheese sandwiches!

Heritrix – The open-source web crawler created by the Internet Archive, source is here. It's easy to get started writing a crawler, but there are a lot of deep issues you have to wrestle with if you want sophisticated features, so it's great to have production-tested code to reference.

The animals of O'Reilly – A wonderful initiative to highlight worthwhile wildlife projects, a lot of them involving fascinating technology hacks.

What makes Paris look like Paris? – Automatically extracting the visual elements that define a place.

MangoDB – MongoDB has been fine for the applications I've used it on, and the support has been top-notch, but some frustrated person has put way too much thought into this open-source parody.

Solr vs ElasticSearch – A good overview of how the two big open-source search frameworks stack up against each other. This quora thread has some informed opinions too.

Picture by Pete Kaminski

Dancing with handcuffs – the changing geography of trust in China - There's a lot to chew on in Tricia Wang's talk and notes, but it's fascinating to watch state censorship collide with social media, even if technology isn't a silver bullet against repression.

An archive of quotes – My friend Drew has put up a collection of quotes he's gathered by scraping news for download. I'm fascinated by hacks like these that rely on human conventions around the way we produce text, in this case looking for the formatting signatures that indicate that something's a quote.

Why are Americans so… – Another beautiful hack, this time using the data from Google's auto-complete to map the adjectives that are paired with different US states.

The free CDN – A little-known feature of Google's App Engine setup that gives you a content-delivery network for no extra charge!

Craigslist blocks search bots – I've been following the PadMapper/CL case closely, because it's putting the implicit bargain that publishers make with search engines under the spotlight. It's still unclear what's going on with the latest developments, but obviously Craig wants to keep the site showing up in search results but avoid handing over all his data to third parties. There's a whole long post I need to write with my thoughts about this.