Facebook’s new application statistics

Addstats

I’m a statistics junkie. Picking some significant metrics, and sticking with them to measure performance is the only way to figure out what’s working and what isn’t. I usually try to design in some measurement tools, but that’s hard with Facebook apps, since their setup hides the referring address/previous page and other useful information.

Luckily, they recently introduced a new statistics page for every app, which you can access from the More Stats link below the application name, from the main developer page.

The first big innovation is the ability to see how many people added and removed you during the previous 24 hours. Before, you could only guess at this by comparing the user totals from day to day, but this wouldn’t tell you how much turnover you had from people removing your app. The most useful part of this, and one that’s a bit hidden, is that the total number of adds is actually a link. If you click on it, you’ll see a bar graph like the one above, showing you exactly where your new users came from.

The top picture is for Funhouse Photo, and it tells me a lot. I was suspicious that my app was very non-viral because the growth in users was very linear, but this confirms that I’m getting the majority from the directory and direct searches, rather than feed stories or other friend-to-friend communications. To improve growth that needs to change, and I’ll be able to tell very quickly if alterations to the app help by looking at those stats.

Less exciting, but still very useful, are the response metrics. I’ve had a recurrent problem with time-outs on my facebook apps because they’re doing heavy processing on the server. It seems like any page request that takes more than 8 seconds to complete results in a Facebook error screen for the user, so to work around that I had to implement asynchronous Ajax loading of page elements that might take a while. Looking at the response statistics shows that both my apps that use this aren’t returning error pages for any users, something I couldn’t verify before.

The final interesting feature is a selection of the URLs that were requested from the app recently. This sampling is a great way to figure out how people are using your app, which features they’re accessing and how often. My apps generally encode a lot of information in the URL using GET rather than POST, so I’m able to get quite a fine-grained look at my users’ interactions with them.

Funhouse Photo User Count: 1,723 total, 95 active. As I mention above, I’ve got new insight into the growth pattern from the add statistics. It explains why growth is so linear, there’s little friend-to-friend spreading of the app.

Event Connector User Count: 71 total, 11 active. The add source statistics show that most of the trickle of new users came from the product directory, which is what I’d expect since I don’t have a conference signed up yet.

Facebook app submission

Checks

There isn’t an official guide on how to submit your app to Facebook’s directory, so here’s what I’ve found after going through the process several times.

You don’t have to submit your app to the directory. I never did for Defrag Connector, since it was distributed directly by the conference promoter, and wouldn’t have been helpful to the general public anyway. The strength of the directory is that it gives you access to the early-adopters, people who are actively looking for a new app to try. What you want long-term is an app that’s viral, so friends spread it to their friends to reach a much wider audience, but the directory is a good way to start the ball rolling.

In preparation for submission, you need to go to your Developer application and click on My Applications. Below each of your apps is an ‘Edit about page’ link. This is where you can provide a short description, a screenshot, and importantly, set the two directory categories you want your app to appear in. I’ve got no statistics to back this up, but it seems like "Just for Fun" is one of the most popular categories, so I’d recommend that if possible.

Next, make sure your app has an icon, by going to ‘Edit Settings’ for your application. Even though the icon is part of the optional fields section, Event Connector was initially rejected for not having one.

Once you’ve filled out all the information on those pages, you can click on "Submit Application" to the right of your app. There, you’ll need to fill out another description, and supply a larger icon for the directory listing. Once you’ve done that, your app is almost ready to be submitted.

The final hurdle is making sure you have enough users. You need a minimum of five to demonstrate you’ve been testing your app, and that you can personally persuade five people to use it. Hit up your friends and get them to give it a test run, and let you know what they think.

Once you’ve met all those conditions, you can submit the app for review. I’ve found it’s taken me two to three days, and as far as I can tell the inspection is fairly perfunctory. I can’t imagine it’s a sought-after job reviewing the apps, and I’ve had a spotty experience with the quality of the examination.

I submitted Event Connector four times in total before it was accepted. The first rejection was for the lack of an icon, which I fixed, but then the next two were random and confusing, claiming that my app had no content, and then that it violated the ToS because it gave access to secret events. Since there was little information in the rejections, and after research I couldn’t reproduce the problems, I resubmitted the app unchanged both times, and it was eventually accepted. Unlike the Firefox directory, there’s no two-way communication between the reviewers and the app developers, so I wasn’t able to get any additional guidance.

Once you’re in, you should get an immediate spike in the number of users, as the app shows up in the ‘Just added’ section of the directory. Then, I’d expect a steady flow of users, the size depending on the categories and appeal of your app. Hopefully you can gather a good base to build on with some viral distribution.

Funhouse Photo User Count: 1,697 total, 76 active. Still ticking upward, with fairly slow growth.

Event Connector
User Count
: 67 total, 6 active. I’m getting the Facebook equivalent of cosmic background radiation, in terms of new users through the directory. Still an uphill battle winning a new conference.

Privacy and the implicit web

Securitycamera

Implicit web apps rely on access to information you don’t want everyone to know. Unlike traditional server driven web sites, implicit web apps often run on the user’s own machine. This gives them access to all the user’s data, whereas a web service can only see a small slice covering what the user did whilst visiting that site.

The only difference between an implicit web client app and spyware is intent. Fred Wilson has a quote "If someone’s going to spy on you, it’s probably best if it’s you." I think "If someone’s going to spy on you, it’s probably best if it’s us" is a better reflection of the current state of the implicit web scene. We aren’t empowering users by letting them own their information, and control exactly what is revealed. Instead at install time we’re asking them to sign over the right to pull all their information onto our servers.

This isn’t a big issue yet, because there’s not much awareness amongst users of the dangers. But it would only take one big privacy breach to start people worrying. We need to plan ahead to make sure we don’t get classed as spyware by zealous blockers.

I think the model for the future is something like the Attention Trust. Set up to provide a standard for the treatment of user’s web-browsing behavior, they mandate a set of principles their members must follow. In return, organizations that meet those principles can display a badge demonstrating their trustworthiness.

It’s not perfect, there’s not a rigorous inspection or application process to join, it’s mostly self-regulated, and the rules are focused on web-browsing. But it is an organization I expect to grow and mature as the demand from legitimate implicit web companies to avoid being labelled as spyware gets stronger. They also offer a very interesting Firefox extension for tracking user’s web-browsing, I’m tempted to try a port over to IE.

The trickiest practical part of this is that providing the sort of fine-grained user control will take a lot of extra engineering, and some smart UI to avoid baffling the user with a space shuttle control panel of options. Most services allow you to temporarily disable information capture, but I think one of the requirements is going to be the ability for users to remove data from your server after it’s been captured, and that’s going to be a lot harder to implement.

As I was researching this post, I ran across an article by Alex Iskold on ReadWriteWeb that was really helpful. I guess I wasn’t the first to spot Amazon as the ur-Implicit-Web-App!

Funhouse Photo User Count: 1,686 total, 73 active. The stats have moved at least, but still seem a little flakey, showing data from three days ago.

Event Connector User Count: 65 total, 10 active. No progress on signing up a conference, I will be chasing this up again, and considering some different approaches to reaching organizers.

Hiking trails on Santa Cruz Island

Montanonridge

Santa Cruz Island sits about twenty miles off the southern California coast, and is an amazing place to camp and hike. Never inhabited except by the Chumash Indians and a few sheep ranchers, visiting it is like a trip back in time. Completely undeveloped, without paved roads, power lines or pretty much any other reminder of modern life, it’s impossible not to relax!

There’s daily scheduled boat trips from Ventura, it takes about an hour, and you’ll often see some dolphins or whales as you cross the channel. You cross the shipping lanes, so you may be lucky enough to see a container ship up close too. Call me crazy, but I think they’re one of the wonders of the modern world, and always inspire a sense of wonder. Go to IslandPackers.com to reserve your boat ride.

With over one hundred square miles of back country, there’s a lot to explore. I’d recommend picking up a commercial map before you go, such as National Geographic’s, but I’ve also created a Google map showing the trails I’ve hiked on the eastern National Park side of the island. You can check out the map to build your own trips, but I’ll describe a few of my favorites here.

Cavern Point Loop

The landing point for the eastern side of the island is at Scorpion Harbor. This is the quickest and easiest loop, at 1.6 miles, though it does have some steep sections of uphill. You start off up the cliffs on the north side of the harbor, on a tough uphill climb. If it’s clear, at the top you’ll be rewarded with some great views of both the mainland, and the rest of the island. You then follow the trail along the top of the cliffs to Cavern Point, a bare patch at a cliff-top, where you can look west along the rugged coast. You can then take a fire road back, past the ranger’s huts, but I prefer going a little further along the trail, and then following a trail down a gully to the lower campground. When you’re going down the gully, check out the stone dams built to check erosion. They’ve lasted for decades, despite being dry stone construction, and were built by Italian stone-masons all over the island.

Once you’re at the lower campground, follow the main fire road east back to the pier. Breathe deeply as you pass through the eucalyptus planted by the ranchers. They may be non-native (like me!), but that smell always brings me back to the island.

Potato Harbor

A 5 mile out-and-back, this is a good intermediate hike. The route takes you up to Cavern Point first, but then takes you along the cliffs to a beautiful sheltered cove. Here’s a photo showing the view you’ll get if you make it:
Potatoharbor

From Cavern Point, you can just follow the cliff-top path until it joins with a fire road. Take that for another mile, and you’ll be there.

There’s a route known as the Quad Trail that branches off near Potato Harbor overlook. This is mostly unmaintained, and unmarked on most maps. Me and Liz followed it for a couple of miles, it had a lot of punishing uphill, but was generally clear and in good shape. I wouldn’t recommend it unless you’re an experienced hiker.

Smugglers Cove

You can head over to Smugglers Cove by following the fire road up the hill on the south side of Scorpion Canyon, visible from the pier. You won’t see it whilst you’re hiking on it, but there’s some magnificent rock walls supporting the road, built by the Italian stone-masons. It’s around four miles to Smugglers, but there’s a lot of uphill, so it’s harder than you might expect.

There’s a lovely grove of Cypress trees visible from the fire road. You can take a short-cut through this grove on a single-track trail, and it’s a perfect spot to recover from the struggle up the first hill. Planted for the rancher’s daughter to enjoy, it adds to the Mediterranean atmosphere.

Turn to the east when you get to a junction with another fire road, and you’ll be headed to Smugglers. Once you arrive, you’ll see a grove of olive trees planted by the ranchers, which always makes me wish I’d brought the rest of a martini. Smugglers Cove has an old Ranch House, but the best part is the beach. Exposed to the Pacific, you often get majestic surf rolling in.

Scorpion Canyon Trail and the Old Oil Well

A 4 mile loop with plenty of climbing. You start off heading west along the Scorpion Canyon fire road, and then continue along Scorpion Canyon trail once the road ends, after the upper campground. There’s a trail heading uphill near the end of Scorpion Canyon. It’s possible to continue up the canyon too, rather than taking this Old Oil Well trail, but it’s extremely rocky and technical, so I wouldn’t recommend it. Here’s a photo Liz took as we were exploring up the canyon a couple of weeks ago:
Scorpioncanyon

The Old Oil Well trail is very hard work, you gain a lot of elevation over a short distance. At the top, it joins up with a fire road, and a few hundred feet away is an old oil drill, with a drilling head left next to the trail. This is from an exploration effort many years ago, they dug down several thousand feet, but all they found was water! It was cheaper to write off the equipment than ship it back to the mainland, so here it sits, slowly rusting, good fodder for future archaeologists.

You can take the Old Oil Well fire road back down to get back to Scorpion Harbor.

Montanon Peak Trail

This is the longest and hardest trail we’ve taken on Santa Cruz. It’s 4.5 miles each way, and takes you to the highest point on the eastern island, Montanon Peak, at 1808 feet.

You start off by heading to the Old Oil Well, either by the fire road, or the Scorpion Canyon trail route. Once you’re at the well, continue west along the fire road. The trail itself is pretty well marked, with some signs added recently. Along the way, keep an eye out for the aircraft crash site. I don’t know the story behind this one, but there’s some old wreckage still visible as you pass over a rise.

The trail heads up to a local landmark known as High Mount. There, a trail splits off and heads off 12 miles west to Prisoners Harbor, a trip which I’ve never done myself, but hope to soon. The area gets fog rolling in from the ocean, so make sure you dress warmly. There’s some spectacular lichen and moss, here’s another photo from Liz:

Mossyoak

Stick to the ridge, and head south and uphill, and you’ll find yourself at Montanon Peak. You’ll know you’ve arrived when you see a small hut, and some solar panels, for the radio equipment that’s housed up there.

I recommend taking the same trail back, but some maps do show a partial connector to Smugglers. We tried to make a loop down that way on our last trip, and it was a really horrible experience. The trail is non-existent, and getting down involves some scary rock scrambling. Liz had a nasty fall, and was lucky not to be injured beyond cuts and bruises. I’ve included the rough route we took in my map, but marked it in red, and I highly recommend not following in our footsteps!


Funhouse Photo User Count
: 1,638 total, 77 active. I’m wondering if the stats are having a problem, since this seems to be exactly the same as last night?

Event Connector User Count: 61 total, 8 active. I just refreshed, and went from 48 total to 61, so there’s definitely something wild and whacky going on with Facebook’s statistics.

Busy weekend!

Yawn

Me and Liz led a crew at the COSCA annual trail work day today, and we’ve got the CORBA Fat Tire Festival tomorrow. They’re both 6am starts, and we’re not exactly natural morning people, so we’re knackered!

I was hoping to complete a mini-guide to hiking on Santa Cruz Island, but bed is definitely calling. You can check out the Google map I finished showing the trails, but you’ll have to wait until later for photos and a description. I also had some fun learning about Outlook extension writing, I feel an article brewing on that too!

Funhouse Photo User Count: 1,638 total, 77 active. Not many adds, and almost as many removes, not sure why!

Event Connector User Count:
55 total, 8 active. A few more people finding it through the directory, no progress on new conference sign ups.

The implicit web and Clippy

Clippy

"It looks like you’re writing a blog post. Can I Help?"

Clippy was an implicit non-web application. He’s was built on a really clever piece of implicit analysis technology, and deliver a nightmarish user experience. It was so bad, there’s even a research paper devoted to exactly why he was so hated.

I try to remember Clippy when I find myself getting too deep into arcane algorithms, and too far from the user experience. One of the best pieces of advice I’ve had about application design is to imagine an actual concrete user in as much detail as possible, including a name. Then describe, step-by-step, her thoughts and actions as she tries to achieve her goal with the software. It’s amazing how many potholes you can discover with that simple process.

Fundamentally, the implicit web is about magic. It’s about weaving disorganized information into something useful. The danger comes because you’re filling in the blanks for the user, guessing at what information they want to see rather than relying on them asking for something explicit. The user pays a price every time they have to process the information you give them. If you throw irrelevant data at them, they’ll kick you to the curb, to join Clippy in his retirement.

Funhouse Photo User Count
: 1638 total, 77 active. A friend just pointed out that it’s unclear if your profile picture will get replaced when you pick from the app for the first time. That seems obvious now he says it, so I’ll be looking at making that clearer. A good example of the sort of thing it’s easy to miss when you’re focused on the technical problems.

Event Connector User Count: 46 total, 8 active. Still working on getting an ‘anchor’ conference for the app.

Build a social graph from your mailbox

Envelope
The implicit web is all about analyzing information a user has generated as part of some activity, and giving them a new way of looking at that data, generating insights they wouldn’t otherwise see. The most common source of information is web browsing, which sites are visited, which links are clicked. There is a source that’s just as interesting, but nobody’s using it; your mailbox.

Who sends mail to you, and who you respond to, how often and at what length. If you take that raw data and plot it in a graph, with links between you and people you correspond with, you end up with a pretty damn accurate graph of your relationships, limited only by the extent to which you use email to interact with your friends and colleagues. What’s more, by using the frequency of correspondence, you can approximate the strength of each relationship, and by seeing who else is included in emails, have a sketchy idea of the links between your friends.

As well as a social graph, you can also look through mail for links to web pages. Each one of those can be treated just as if that friend had voted for it in a service like Digg, and can be added to a list of the sites recommended by your local network.

Why’s no one doing this, if it’s so wonderful? Because it’s really, really hard to get to that data. Web-based email services like Google Mail or Hotmail would be able to do it on the server side, but that approach requires a large existing user-base. An alternative I’m investigating is writing an Outlook Add-In to gather some of this information. This would restrict me to business users, and involves wrestling the COM beast to the ground to implement, but I should be able to reuse some of my Internet Explorer BHO work at least.

Funhouse Photo User Count: 1,552 total, 123 active. This is encouraging, another peak in the growth rate, and no public holiday to account for it this time. Perhaps the graph is becoming slightly less linear?

Event Connector User Count
: 41 total, 10 active. Not much change. I’m still talking with promoters, trying to arrange official support for another conference.

Limitations of the Facebook API

Redlight

Facebook is walking a tightrope with its API; they need to expose enough functionality so we can develop compelling services, but guard against malicious applications that could degrade the Facebook user experience, for example by flooding people with spam.

The API is a compromise between these two conflicting goals, and I’m going to cover what you can and can’t do. Overall, I’ve been able to see some patterns in the decisions they’ve made about what to expose, and how to expose it:

  • Apps can only see what the logged-in user can see.
  • Getting access to any information held by Facebook requires the user to go through a screen where they temporarily authorize, or permanently add, the application.
  • The Facebook team are very conservative about letting applications change data held by Facebook. Most of the API is focused on reading data, there’s only a few specific places where you can alter data:
  1. Adding an application box to the user’s profile. This gives the app a small sandbox to draw something interesting, but the content has to be statically set by the application, and then is stored by Facebook. The only time you can update it is if the user takes an action that involves your application, there’s no way to fetch it dynamically. If there’s some scripts within the markup you place in the box, they aren’t run unless the user clicks on it in the profile.
  2. Publish an item on the feed. You can only publish to the current user’s feed, and the app is has time limits on how often it may call it, once every 12 hours for stories, 10 times a 48 hour period for actions.
  3. Send notifications or emails to friends. Again, there’s limits on how many you can send in a day, up to 10 emails and 40 emails and notifications. The user also has to go through an additional screen to authorize emails.
  4. Photo upload. An app must get additional permission from the user before it’s allowed to upload photos. Each application only has to do this once per user, the permission is granted permanently.
  5. Setting the user’s status text. This is another operation that requires an additional step of seeking permission from the user.
  • There’s no way to use the API to affect anything not covered here, such as adding information to group or event pages.

There is an alternative way to perform some actions that aren’t covered by the API, by hand-crafting Facebook internal URLs, and redirecting to them. There’s actually a bit of official documentation on this here.

Funhouse Photo User Count: 1493 total, 123 active. Back to the more typical growth rate, which is strong evidence the strong growth of the last few days was caused by Columbus Day boredom.

Event Connector User Count: 40 total, 10 active. Not much happening here so far, I’m reaching out to some more event promoters to get them to give it a go.

Funhouse photo gets a lot more users

Graph
After growing so linearly it was almost creepy, Funhouse Photo gained over 200 users since yesterday. And I have no clue why! I’ve been searching to see if it got mentioned or reviewed somewhere, but I’m drawing a blank. I don’t get much information from Facebook about how people found the app, I’d need to manually build in some tracking myself, and I’d probably not be able to get much information about the referring page anyway since Facebook does so much redirection.

Oh, I did just have a thought. It was Columbus Day yesterday, I bet there was a lot of bored people looking for something fun to do on Facebook. Interesting, probably won’t translate into a long-term trend if that’s the case.

I wish I could have made it to Graphing Social Patterns, the Facebook developer conference. From the notes, it looks like Danny Sullivan is thinking along the same lines as me with social search; wanting to use your friends’ browsing habits to give you more focused results.

Funhouse Photo User Count: 1466 total, 344 active. Probably a Columbus Day blip, but nice to see a growth spike.

Event Connector User Count
: 39 total, 5 active. Still working with Emile and Tim from New Media Expo to arrange some distribution of their connector, but not much activity otherwise.

Amazon’s already built an implicit web app

Cobweb

In preparation for Defrag, I’ve been trying to organize my thoughts on the implicit web. Previously, I gave an example of how implicit information could improve search. I realized as I was thinking about this last night that there was already a successful implicit application being used by millions of people every day; Amazon’s recommendation system.

It fits well with my idea of the implicit web; it’s using the data passively collected from users behavior to offer up recommendations, there’s no user data-entry required, it’s a behind-the-scenes servant offering up useful information. So if the implicit web’s been in use for years, why do we even need the term?

Amazon is in a very rare position; they have a massive set of trusted data to work with, and they know a lot about their users. This gives them enough information to chew on and produce something useful. Very few other sites have enough breadth (number of users) and depth (information about their user’s behavior) to do anything similar. To do something comparable without owning a site like that, services have to sit on the client side and collect information as the user browses the web.

me.dium is an example of that sort of service. At the moment, they’re offering a very simple service; show me where my friends are surfing, and related sites strangers are visiting that are similar to my current page. The second part is pretty similar to Amazon’s recommendations. The really exciting bit is that once users trust you with the data about where they’re surfing, and what their friends are, you can build some really compelling services. Here’s a few examples of what I’d do with that information:

  • Build an implicit list of favorites based on how often a user visits sites, and how long they spend there. Let them use it as a bookmark toolbar, or even publish it to their friends as their current favorites.
  • Highlight links that other people took most often to leave a page, and show pages they came from most often, giving friends a higher influence.
  • Let the user ‘stumble-upon’ pages that are popular with their friends right now.

I’m really excited about the possiblities, and I’m looking forward to some interesting conversations at Defrag!

Funhouse Photo User Count: 1,251 total, 60 active. Weird, a big jump from yesterday, almost 90 users, but the stats page claims only 22 adds in the last 24 hours. I’m not sure what’s up, but I’m not complaining!

Event Connector User Count: 36 total, 9 active. Not much change in the numbers, but there has been some progress. Tim from New Media Expo has set up a Facebook event for their 2008 show, and has around 100 guests so far. You can try out the connector here, I’m hoping that it will be as popular as it was for Defrag.