Defrag Connector

Defragbuttonpete
I’m getting ready for the Defrag conference in November, and wanted to see which attendees were  friends, or even friends-of-friends. The organizer, Eric Norlin, has created a Facebook event, and a lot of the attendees have signed up.

In Facebook, the only way to interact with that information is by browsing an alphabetical list of the attendees. There’s no way to sort or filter them by their closeness in the social graph, or by shared networks.

This seems like an opportunity to build something better using the Facebook API. My goal is to create a simple application that lets me see attendees who I’m socially related to. That seems both technically straightforward and useful to other people who are either attending, or deciding whether to attend.

I’ll be spending my spare time putting that together, I’m hopeful it won’t take too long to get it functional. So far, I’ve added the application information to Facebook, and I’ve started with Funhouse Photo’s code as a template. The challenging bit is going to be efficiently querying the graph, since it will require a lot more database code than I’ve written before.

Funhouse Photo User Count: 839 total, 98 active. This is a nice increase in the total count, and the active users are up a bit. It’s too early to say if this is related to my changes or just noise, since they’ve been in for less than half a day, but it’s good to see nonetheless.

Funhouse Photo improvements

Scarymonkey
I spent a lot of today working on Funhouse Photo. My first goal was to add notifications to user’s feeds when they updated their status. This was top of my list because it would improve the user experience by spreading the pictures and captions they created more widely. It should also help spread word about the app, so I’ll be keeping an eye on the user figures over the next few days to see if it has an impact.

My old picture choice used a two-step process; first picking the picture, then the caption. I had to cut that down to a single stage so I could add the feed item at the end. To do that, I integrated the caption setting with the picture choice, which left the picking page looking a bit more cluttered, but simplified the flow.

While I was working on that, I discovered a bug that must have been lurking for some time. I was always calling require_login() at the start of my PHP page generation, but for most of what I was doing (adding items to the profile, etc) it made more sense to call require_add(), since that gives the needed privileges. This meant that all the links designed to lure in new users would fail to produce any results, and instead the only way to get the application was to manually add it! This actually made me feel a bit cheerier about my user figures, since they were jumping through more hoops than I thought before they could install the app.

While I was researching that (there’s no documentation on those calls) I also ran across fb:profile-action. This lets you add a menu item below a user’s picture, which is perfect for Funhouse Photo, so while I was working I also put one of those in for folks who’d installed the app. Now when you’re browsing a profile, you will see a ‘Funhouse this photo’ link below the portrait, which I’m hoping will help the engagement level with the app.

As always, I’ll be watching the figures to see what difference this makes.

Funhouse Photo
User Count: 798 total, 59 active. A little worrying, since I think this was the exact total I saw a few hours ago. It seems a bit unlikely that nobody added it in the last few hours, so I’m hoping that it’s a glitch in the stats reporting rather than a bug in the app that’s preventing users from adding. The active total is a lot lower than I’d like, we’ll see if my recent changes help that.

Job Satisfaction

I just returned from a day leading a trail crew with the Santa Monica Mountains Trails Council. Here’s one of the problems we tackled:
Toestubber1

And here’s a close-up:
Toestubber2

The trail had washed away, leaving only a few inches for people to walk on!

Toestubber3

Here’s the solution, after me, Ed and Liz spent half an hour with pick-axes, saws and loppers. A whole new trail section. Ed’s standing where the old trail was, everything to the left is new.

I spend all week hunched in front of a computer, and it can take months or years before I see the final results of a project. That’s why trail work’s so much fun for me; it’s outdoors, physical, and you see instant results.

Funhouse Photo
User Count: 780 total, 70 active. Much the same as before, hopefully I can spend some more time hunched over my laptop tomorrow, adding feed notifications and some new effects.

Developing an IE plugin compared to a Firefox extension

Goatvsgoat

Yvonnick Esnault mailed me to ask how long it takes to develop an extension for Internet Explorer compared to one for Firefox, and where he could learn more about the differences between the two.

I don’t know of many resources that will help, which is why I started my series on porting a Firefox plugin to Internet Explorer. That doesn’t include a summary of the differences though, just an examination of the details, so I’ll try to sum up what I know.

Compiled versus interpreted

IE plugins are Browser Helper Objects, which are effectively special DLLs. These require the use of a compiler, and creating a new plugin or modifying an existing one takes a lot longer than if you’re writing in Javascript for a Firefox extension. It does have one advantage though; you have the option of keeping the source code closed with IE, whereas it’s inherently open-source with Firefox’s Javascript.

Writing an installer

Firefox plugins have a dedicated installer system that you can write some simple scripts to interface with, and use very quickly. You don’t get any help from Internet Explorer, instead you have to write a standard Windows installer executable, which can be pretty time-consuming.

Performance

One other benefit of being compiled rather than interpreted is that processing-heavy operations tend to run a lot faster. I notice this when I’m doing things like string searching within a document.

Firefox development is a well-trodden path

There’s a lot of people creating Firefox extensions, there’s very few creating IE plugins. That has a lot of consequences:

  • There’s much more documentation for FF extensions, both from Mozilla and developers themselves, and it’s easier to get help.
  • There’s fewer obscure bugs in Firefox than IE, because the interface is heavily tested in use.
  • Firefox has addons.mozilla.org to distribute extensions. The IE equivalent, Windows Marketplace is not as well-known or promoted.
  • Since there’s fewer IE extensions than Firefox ones, there’s less competition for users.
  • Unfortunately, many users who want plugins use Firefox rather than IE already because of the lack of IE extensions!

Overall, developing plugins for IE is a lot harder than developing for Firefox, and it took me a couple of months of weekends and evenings to convert mine over. If you do decide to do it, I’d recommend looking at and adding to the BHO documentation wiki.

Funhouse Photo User Count
: 761 total, 68 active. Growth a little slower today, still within the rough range I’ve been seeing for the past couple of weeks.

Camping in the Santa Monicas – Circle X

Mishemokwa

Moving east from Sycamore Canyon and La Jolla Valley, the next campground I know of is Circle X. It’s owned by the National Park Service, since that part is NPS property, confusingly even though there’s no National Park in the Santa Monicas. A ranger friend once told me there’s over a hundred different public and non-profit agencies who own land in the Santa Monica Mountains National Recreation Area, as it’s officially known!

One nice thing about it being a National property is that they have a bigger budget than the cash-strapped State Parks, and so actually have a good map and infomation web page up. The camp site is another group one, for 10 to 75 people. There’s water and toilets, and you can drive up to the camp ground. There’s also a ranch house that’s available for day-use functions, but not overnight.

Some of my favorite trails start nearby; Mishe Mokwa, with its great views of hanging rock, and seasonal springs; the Grotto with some lovely rock-scrambling, and Sandstone Peak, the highest point in the Santa Monicas at 3,111 feet that offers some amazing views.

Funhouse Photo User Count: 747 total, 80 active. Much as before, there seems to be a steady stream of new people adding the app, and a smaller number using it each day. Some questions I’d like to figure out are if new users are arriving via messages sent by their friends, or just spotting it on profiles, and whether people tend to change their photo, or just pick one and leave it. I shall need to do some database-crunching to pull out that information.

The way forward for search

Search

Most of the focus on improving search is around the algorithms for producing the results. Even this article asking Is Search Broken? mentions that the holy grail is producing the right result with a single click.

I think that’s rubbish, for a few big reasons:

  • Most of my searches aren’t for one perfect page. There’s a lot of different types of searches. Usually there’s a set of pages that will each either answer my question fully on their own, will answer the question when read together, or will help me refine my search.
  • Search terms don’t provide enough information. There’s plenty of techniques to infer extra information from things known about the user, but they’re all guesses. In my experience, they can really get in the way since they can’t read minds, and block out results they think I’m not interested in. With the inherent fuzziness of search results, what’s really needed is a better way for me to do the final winnowing, interactively.
  • I don’t always know what I’m looking for! Often, I’m using search results as a way to learn more about a general area, or get an idea of what’s out there so I can focus my thinking. Since I don’t have a specific goal in mind, there is no single right answer.

Machine-powered algorithms are necessary, but once they’re good enough, they start giving you diminishing returns. What’s missing is support for the final user-powered step of rapidly sorting and eyeballing those search results. There’s a workflow that we all go through when searching, that involves clicking on a link, scrolling down through the page, clicking back, hunting for the next interesting link in the results. It makes no sense that there’s no good tools to speed up this process.

Google Hot Keys
is a start for addressing this, but there’s so much more that could be done. I want a visual representation of my search results that I can truly browse through, like an OS X dock full of pages. I want to be able to eliminate a page, and have all similar pages disappear from my results. I want context menus. I want my search results page to be easily accessible, even visible, as I browse through pages I’ve found. I want to be able to focus on pages that are in my browser history, or are linked to ones in my history, since I’ve implicitly voted those as trustworthy to me. I want to know which pages people clicked on most, and spent most time on.

Funhouse Photo User Count: 731 total, 86 active. Much the same story as yesterday, I want to make some changes to freshen it up, but I won’t have time for at least a few days.

Using Ajax in Facebook Apps

Ajax

I was unhappy with the user experience that my asynchronous loading changes introduced. It could take up to thirty seconds, and two pauses with blank browser windows due to reloads, before the user would see a set of photos.

So, I took a step back and tried to analyze the problem I was attempting to solve. I made these changes primarily because Facebook will show an error page if the server takes too long to produce a page. My first solution attempted to solve that by showing multiple intermediate loading pages, with artificial pauses between them to give the server time to do the time-consuming image processing.

The approach I’ve taken now is different. There’s a single page that’s loaded, but that then fires off an Ajax-style request to my server, asking for all the photos. The key thing is that this request will not produce a time-out error the way loading the same page directly through a Facebook frame will. It also avoids the blank page/reloading problem, instead you just see a loading message until the pictures are ready, and then that area is populated with photos. Another advantage is that there’s no artificial pause, instead the server returns the photos as soon as they are ready.

This doesn’t tackle the bigger problem of making sure that multiple image processing requests are handled efficiently, but the median response time still seems fast. The big problem before was that the less-common multi-second response times would be catastrophic, with an error page appearing instead of the expected result. Seeing the loading message for a few seconds instead is a lot less painful.

Here’s the actual code I use:

$ajaxurl = get_serverurl() . 'index.php?action=geteffectpicker' .
    '&user=' . $user .
    '&pickactionurl=' . urlencode($pickactionurl) .
    '&ismessage=' . $ismessage .
    '&sender=' . $sender .
    '&sourceurl=' . urlencode($sourceurl) .
    '&firstname=' . $firstname .
    '&senderfirst=' . $senderfirst;

?>
<div id="delayedpicker" style="padding: 10px;">

<a href="http://apps.facebook.com/funhousephoto/">Funhouse Photo</a> is loading the portrait for <?=$\
firstname?>, you should see their pictures in a few seconds

<script><!--

    var ajax = new Ajax();
    ajax.responseType = Ajax.FBML;
    ajax.ondone = function(data)
    {
      document.getElementById('delayedpicker').setInnerFBML(data);
    }
    ajax.requireLogin = true;
    ajax.post('<?=$ajaxurl?>');

//--></script>

This is all using Facebook’s MockAjax functionality, since real XMLHttpRequest calls are not allowed for security reasons. It took me a while to get working, mostly because I was calling $facebook->require_login() at the start of the PHP code that created the picture content. I discovered that this would try to redirect to the Facebook login page, and since the request is headless and you can’t actually log in, an empty result would be returned.

Instead, I ended up encoding all the information I needed from the Facebook API into the request URL, such as the first names of the users, and the URLs to use to pick a picture. This meant I was able to create the pictures without logging in to Facebook, and then it all worked swimmingly.

Funhouse Photo User Count: 701 total, 66 active. Still growing slowly, and I haven’t had time to add any more effects, or implement feed functionality.

Camping in the Santa Monicas – Sycamore Canyon

Danielson

A few miles to the east of La Jolla Valley is Sycamore Canyon. There’s a fire road that runs along the floor of the canyon all the way from the beach up to Satwiwa/Rancho Sierra Vista in Thousand Oaks, and some glorious side-trails that head off it into the back-country. The road is great for a fairly gentle bike ride, and very popular, and there’s some great technical trails on the west side of the canyon. The east half is designated wilderness, so there’s no biking. Some of my favorite hiking trails go through the wilderness there though, like Old Boney and Chamberlain trails.

There’s a big campground right at the south end of the Canyon, just off the PCH. Sycamore Cove is a large paved site, suitable for RVs, and very, very popular! I don’t think there’s any time of year that it isn’t fully booked in advance. It’s in a great location for exploring the mountains, but I’ve been put off by how busy it is. Probably most suitable if you want a family getaway with access to the beach, it has full facilities including showers and a campground host.

Much less used is Danielson campground, about four miles north up the Canyon fire road. It’s a group site, so you’ll need to have a large group, a dozen or more, to reserve it. Its location is amazing though, you can drive up the fire road to get there, but it’s in a small side-valley, in the center of a grove of oaks. There’s a central outdoor fireplace and picnic area left over from when the park was a ranch, and it will comfortably hold a couple of hundred people.

Perhaps the best way to check it out is the annual Santa Monica Mountains Trail Days camp-out, where a hundred or so volunteers camp Friday and Saturday nights, do trail maintenance during the day, and are rewarded with a lovely barbecue on Saturday evening, as well as some entertainment from the rangers. The heading photo is from this years barbecue. It’s in the last week of April, keep an eye on the Santa Monica Mountains Trails Council site for details.

Funhouse Photo User Count: 681 total, 81 active. Good to see the total still growing, I still need to find time to update the effects, and maybe add news-feed notifications to see if I can boost growth.

madKast Review

Madkastlogo

After commenting on a post by Brad, I was contacted by Josh Larson of madKast, asking if I’d like to try their widget.

As background, madKast is one of the offspring of the TechStars incubator program, which was founded by David Cohen. Coincidentally, I’d just finished "All Drive, No Vision", the story of David’s first startup, which I found fascinating, and deserves a blog entry of its own.

Once you add madKast to your blog, a small icon appears next to all your blog entries. Clicking on that icon lets users email or text message the blog entry to their friends. It’s a very clean user experience, they see an additional icon, and clicking on it brings up a modal dialog that allows them to email, text-message, or share the entry through a social tagging service like Digg or Facebook Sharing:

Madkastscreenshot

It doesn’t seem to slow down the loading of my blog, which is a major problem with a lot of other widgets, and it doesn’t add much visual clutter. I really like the widget’s focus, it’s aimed at doing a great job at helping users share blog links, and succeeds. I’m planning on keeping the widget on my blog, it seems like it will improve my reader’s experience. Congratulations to Josh and the team, it’s a great product!

Here’s some things I’d love to see as it grows in the future:

  • Site statistics. I’d love to know which posts users were sharing, and anything that offers usage feedback is like catnip for bloggers, so it seems like it would help spread the widget. Anonymous would be fine for me if there’s any privacy concerns. Maybe the most popular shared links could be shown on the madKast main page? Those would all be great incentives for sites to install it.
  • Client-based. There’s a small but vocal minority of people who like to share links they find, and a larger group of passive consumers. This is my bias towards client-based solutions showing, but I think it could be a big win to persuade the avid link-sharers to install a Firefox extension (or IE BHO) that implemented the same functionality. That’s drifting more towards a delicious or digg model obviously, but it seems an obvious reuse of the existing infrastructure.
  • Customization. I’d love to be able to alter the appearance and positioning of the icon, and maybe even the look of the dialog that appears, maybe using some CSS options.

Funhouse Photo User Count: 665 total, 83 active. Last night I implemented Ajax loading for the photo pages, so that the user immediately sees a loading page, and then the photos are created synchronously on the server. The result is that the photos appear quickly, and there’s no painful full-page reloading as there was with the old method. I’m hoping this will increase the growth rate. Now I’ve got this going, I really need to add a few more effects to draw people in.

How to keep adding features while you rebuild

Balancedstones

Brad Feld had a recent post about a classic scenario in software; you have an existing product that needs a complete strip-down and rebuild to meet new needs, but you can’t afford to stop releasing new features while the work’s being done. His advice is that you just need to grin and bear the pain until the rework is done. As an engineer, that’s definitely my preferred option. It’s much more efficient to keep the whole team focused on the rewriting.

Often though, engineering efficiency is beaten out by market, management and investor demands. So then what do you do?

The first time I ran across this was in games. Sports games demand annual releases. There’s no way that you can rebuild a whole rendering engine and art pipeline in a year, so what EA did was plan for an under-the-hood rewritten release every two years. In the off year they focused on adding things like championship simulations, updated graphics and new statistics. What amazed me was users were just as happy with the off-year releases!

A few years later I was working on an image-processing product. We had a very short time to ship a second release, and we were struggling because most of the features we looked at required big rewrites. For fun, I’d written a MIDI interface to the program, even though it wasn’t music-related. It took me a weekend to write it, everyone on the team considered it an amusing curiosity, but users loved it. In the end it turned out to be one of the most-talked about features of the release, and really helped the release.

So, what’s my advice? I try to follow the EA strategy, and identify engineering-light features that we can develop with minimal disruption to the rewrite. Try to keep an open mind and don’t automatically dismiss these as cosmetic. As long as they improve the experience of using your product, your customers will thank you!

Funhouse Photo User Count: 634 users, 51 active daily. I’m definitely seeing growth and active use tail off, and the obvious suspect is the async loading changes that cause the initial screen to take longer. I’m definitely hitting scaling problems myself here! I have some new effects in mind that I’d like to add, but it seems like that’s less urgent than fixing the loading.