Facebook API documentation continued

Face2

In the first half, I covered the initial steps to getting your first facebook app running, and got as far as unpacking the standard sample app onto the server.

The next step is to customize the sample app with the technical information about what SQL database you’re using, and how to uniquely identify your app to facebook. To make these changes, you’ll need to edit the config.php file in the facebook-platform/footprints/ folder. I did this by logging into the server with ssh, and then loading the file in emacs within the shell.

To get the values for the $api_key and $secret variables in your config.php, you’ll need to go to http://www.facebook.com/developers/ (not developers.facebook.com as it says in the file), specifically the setup new application link. Once you’ve followed the setup process, you should see the two values you need in the My Applications section of the developer application.

Next, you’ll need to create an sql database for the sample app to store its information. Most hosting companies offer some number of mysql databases, and have a control panel to add them. In dreamhost’s case, they’re held on a different servers than the file system, so I had to create a subdomain and wait for the DNS propagation again.

Once you’ve created a database, you’ll need to create a table to hold the sample app’s data. Luckily the SQL code is included in the config.php file, so you should just be able to locate a console or mysql shell in your control panel, or through running mysql after ssh-ing into the server, and paste the code right in there.

You’ll also need to create a user, and make a note of the database name, user name and password so you can add them to config.php. The last thing you’ll need is an ip address for your database. This was problematic for me, since on dreamhost the sql server is on a different machine than the PHP is running on, so I couldn’t just use the local host address, 127.0.0.1. The IP address of the SQL machine is not guaranteed, so I should probably use DNS lookup to resolve it dynamically, but for testing I just checked what the current IP address of it was using ping and then statically set it to that in config.php.

Once all of that information was set up, I then made sure I had a callback URL set up in the facebook applications settings. This should point to the public address of your application folder, so something like http://somehost.com/facebook-platform/footprints/

Now, if you type that same address into your browser, you should see a page asking you to log into facebook. Once you log in, you should then see the default footprints application page, or a descriptive error. One really nice thing about the facebook system is that they include a bunch of developer information describing the cause of an error in more detail in the HTML comments of most error pages, if you’re the developer of the application. To look at this, view the source and you should see something near the top, very handy!

Have a play around, hopefully you should be able to plant footprints on your friends, and then view your database and see entries added to the table.

Facebook API documentation

Face

Facebook’s so promising as a solution to the distribution problem, I’m learning more about the technology so to see if there’s a way I can use it. I’m a client-side guy, I’ve done very little server programming, so I’ll be documenting what I learn for the benefit of other server-side newbies.

To start with the basics, you’ll need a facebook account. Once that’s set up, go to the getting started page on developers.facebook.com. Add the developer application to your profile, that gives you access to the control panel for registering new applications with facebook.

Facebook offers a wide range of documentation, including a step-by-step tutorial, a wiki and an API reference. If you’re like me, and learn best by doing, the best place to start is their sample application, which you can download and get started with right away.

As the getting started page says, once you’ve downloaded the sample app "open your favorite development environment, and you’re ready to go". The assumption is that you’re an experienced server-side developer, but if you’re a newbie like me you’ll have to make some choices about what to go with. My recommendation is PHP, it’s probably the most common server-side language, and it’s what the official Facebook client libraries and sample application are written in, though you can download unofficial versions for most server technologies. I’ve also hacked in PHP a bit before, and I’m fairly comfortable with it since it’s very C-like.

Now, you’ll need your own server that you can host the application on. You’ll need one that you can access the unix shell on, through ssh, so you can set up the application, and one that supports PHP 5 (though 4 is also unofficially supported).

I’ve been using WebHSP for about five years, I’m fairly happy with them, event though I have an ancient hosting package they’ve been reliable and have a web interface I’m very familiar with. A few months ago they turned off ssh access by default, which I wasn’t using too much at the time for the mostly-static web pages I host on there. So, I contacted their tech support to get it re-enabled.

While I was waiting for that, I decided to check out alternative hosts, since I knew I wasn’t getting a good deal anymore with WebHSP, and they are aimed very much at non-technical hosting. I was suffering a bout of irrational exuberance, so I also wanted somewhere that could scale up if a facebook app I wrote went viral. I first ran across dreamhost when Seth Godin posted about the way they dealt with a nightmare power outage issue that took their sites down repeatedly over three weeks. This may not sound like a promising start, but I was impressed by how well they dealt with customer communication, they’re based locally in LA, and they’re obviously run by techies, for techies, so I decided to give them a go.

I got their basic plan, with bandwidth and space limits so high I didn’t even bother to remember them, for $8 a month if I paid for two years in advance. Once I’d paid, I then had to wait a couple of hours before I could try out my domain, since they only offer access through the yourname.dreamhosters.com url, and don’t have a unique IP like my last package, so I had to wait for DNS to trickle through. For some reason, I thought DNS propagation only applied to the top-level domain resolution (eg as long as it could work out what address dreamhosters.com was from the .com servers, it would fetch the sub-domain directly from a DNS server associated with dreamhosters, rather than reading from a cache somewhere else), so I complained to tech support about the delay, but they patiently explained the issue.

Once that was working, I had to turn PHP support from 4 to 5 in their control panel, and then I was ready to upload my app. I set up an ssh shell account in the control panel, and logged in to test. Then I did an scp of the sample application up to the server, using a command line like this:
scp facebook-platform.tar.tar myusername@loginserver.dreamhost.com:myhostname.dreamhosters.com/facebook-platform.tar.tar
You’ll need to replace the italics with your own details. On windows, make sure you’ve got the cygwin package installed and in your path to use some of these useful unix commands from the DOS prompt, it makes life so much easier, and on OS X of course you get them out of the box.

Then, I logged in using ssh, went to the directory where I’d uploaded the tar file and did this to extract it into a folder:
tar -xf facebook-plaform.tar.tar

This left me with a facebook-platform folder full of other folders and files. The next step involves sorting out the config files and mysql database for the sample app itself, and I’ll describe that in the next article.

Distribution, distribution, distribution

Eye1

What location is for real estate, distribution is for businesses. This is something I learnt early in my career, when I was in the game industry. I couldn’t understand why a truly terrible game that I worked on like F198 could sell two million copies, while some wonderful independent PC games languished. Why didn’t they take over the world, since they were clearly superior?

What F198 had that they didn’t was a major-league publisher, Sony/Psygnosis. And what the publisher added was distribution; they put the game on the shelf of thousands of shops around the world.

So why didn’t those shops stock all those wonderful independent games? Because the shop’s owners did not have the time or inclination to each search through all of the thousands of games out there to figure out which ones were fun, reliable, had a high marketing budget, and were likely to sell well. Instead, they effectively sub-contracted quality control to the publishers. Psygnosis had a lot to lose financially by promoting games that didn’t sell, so the shops trusted them to be gatekeepers.

Distribution is about that chain of trust, where customers give shops they visit some trust to do part of their decision-making for them by weeding out bad products, and shops in turn chose suppliers, and then trust their judgement. The same process works with blogs; lifehacker readers rely on the editors to weed out the rubbish, and only present interesting products. In turn, the editors rely on a trusted set of sources to do some initial filtering for them, such as other blogs and social ranking sites.

What makes Facebook so interesting as an app platform is that they have a different distribution model; instead of being a global hierarchy, your friends act as the filters for you. It’s a flat local network, and the recommendation comes from someone you know personally, rather than a trusted authority. It’s hard and time-consuming for new companies to break into a distribution pyramid, because they have to persuade a whole chain of very busy people to spend time looking at their product before it is placed in front of a large audience. Facebook holds out the possibility of reaching a lot of people a lot faster.

Nice review from Bob Jomart

Thumbsup
Bob Jomart has just posted a quick positive review of GoogleHotKeys on his blog. He’s using the Internet Explorer version, which is good to see since it’s been a lot easier to reach Firefox users thanks to addons.mozilla.org. I’m in the process of getting onto Windows Marketplace, which is the MS equivalent, but I’m not sure if it’s as widely used as Mozilla’s, it’s certainly less focused on add-ons.

The beautiful Santa Monicas

Santamonica1

When I first talked to a recruiter about jobs in the US, my only criteria was ‘anywhere but LA’, since all I’d heard was traffic, gangs and Hollywood. Of course, all the offers I received were in LA. I accepted one, since I was only coming over for a year or so. That was six years ago! This picture shows one big reason why I stayed.

The first week I was here, I stayed in Thousand Oaks, a suburb of LA. I knew I wanted to get some hiking in, so I looked for big areas of green on the road map, and headed to one about 10 minutes drive away. Once I stepped out of the car, I knew it was something special. Ahead of me was Boney Mountain, and a wilderness 10 miles deep to the ocean. A mangy looking dog was trotting loose on the trail nearby, so I followed it. Pretty soon I found myself in the middle of a whole bunch of mangy dogs, making funny yelps that I assumed meant ‘I wonder if he’s edible?’, and I realized I’d wandered into a pack of coyotes.

It was evening, so that was a short hike, but as soon as I had a weekend free I set off for the ocean. I found waterfalls, cliffs, flowers, lichen, deer, bobcats. I fell in love, it was like nothing I’d ever seen. The Santa Monica mountains are an amazing secret of LA, an area around
40 miles by 8 miles of wilderness filled with spectacular trails. You can hike for hours and not see another soul, just half an hour from the 405.

I kept finding excuses to get out in the mountains, met Liz while when we both were volunteers repairing trails, and fell in love again.

The photo is of Boney Mountain from the ocean side, from a hike we just did today. If you’re ever in LA, look up Topanga, Point Mugu, Rancho Sierra Vista/Satwiwa (where I saw the coyotes), Malibu Creek, or one of the dozens of trail heads out in the mountains, you won’t be disappointed.

New review posted

Test

Sridhar Katakam has just posted a thoughtful review of Google Hot Keys. He’s a keen user of SearchKeys, so it’s good to hear he agrees they complement each other, and his overall verdict is that it’s "a useful add-on".

He also brings up some suggestions for improvements; being able to disable the term highlighting, turning off the hot-keys for moving between terms, and being able to go to the next page with a key command. I think those are all good ideas, I was planning on removing the term movement keys since it’s not working as well as I’d hoped. I’ll need to figure out a good way to term highlighting as an option, maybe in the top tool bar that’s shown on all search pages. I think it’s a good idea to offer a next-page key, I removed it to simplify the tool, but it does seem like that might have been too much of a simplification.

Thanks Sridhar for taking the time to write up that review! You should check out his blog if you’re interested in productivity and key commands. He pointed me towards Mouseless Browsing, a Firefox extension for keyboard navigation, and has a lot of other tips on power surfing using Firefox.

Why client-side mashup tools are important

Wrench
One of the things that’s driven me to experiment with Java applets, Firefox extensions and now IE BHOs, is the conviction that the only way we’ll get to a richer and more usable web is through client-side code.

There’s lots of data stored on servers across the world, data that could be used to create some really interesting applications. For example, if there’s a URL that my facebook friends have tagged in delic.ious, or dugg, or stumbledupon, I want any links to that page marked with a gold star as I browse the web. As a human user, I have access to all the data I need to do that. Ignoring the digital life aggregation issue until another post, there’s a fundamental problem that’s blocking this.

Can’t get the data!

That data is mostly only accessible as human-readable web pages, or as an RSS XML feed. This means there’s no reliable way for a computer to take those text representations of the data, and figure out what they actually mean. It’s a weak AI problem, but even weak AI problems are hairy. It’s not quite as bad as trying to turn a hamburger back into a cow, but it’s close.

This is something that GoogleHotKeys has to do, to extract the search result links from all of the other links on a typical results page. Even for the comparatively simple and uniform format used for those pages, it took some time to get right, and it’s still vulnerable to small changes on Google’s side, such as when they added Orkut to their menu.

How do we fix this?

It would be much better if all sites gave access to their data directly, in a computer-readable form through an API. Facebook has had great success doing just that, Amazon has had a flash-based API available for a couple of years, and Google used to give you access to search results. There isn’t much standardization in what these APIs return though, apart from usually being XML (apart from some JSON mavericks). They’re also usually subject to a lot of restrictions by the site owners, can be shut down like Google’s, and easily run foul of security restrictions like the cross-domain policy unless you run through a proxy.

The semantic web is a different approach to solving this problem, it allows you to specify the meaning of the text representations, eg this link is a result link, this link is a friend link. Unfortunately, there’s no point in site owners going to the extra work of embedding that information into their web pages if there’s no compelling applications that use it. And since no sites implement it, there’s no compelling applications! It’s chicken and egg.

What this means for client-side tools

This is why client tools are so important. They aren’t subject to the draconian security restrictions of server-side code. They have access to the same data as users, without the arbitrary restrictions site owners apply to APIs. This sounds like an invitatation to a DoS attack, but they can still block an individual machine if you start being obnoxious and flooding their servers. With a server-based proxy, they can easily block you on a whim, with client-side they can’t even distinguish you from a regular user as long as you’re well-behaved.

It’s a great environment for innovating in ways that just aren’t possible on the server. This is where I think we can build some compelling and practical semantic web tools, cutting through the gordian knot that’s blocking development. I’m convinced we’ll see some appear within the next couple of years.

Why aren’t there more client-side tools?

There’s two big obstacles. Client-side web tools are hard to write. Development environments are poorly documented, buggy and don’t have big user communities. This is something I’m trying to do my small bit to help with, by documenting how to develop for IE. I don’t see this changing in the near-term, but if there were more commercial demand, developers would overcome this.

More serious is the distribution problem. It’s hard to persuade users to download and install your tool, when server-side tools can give near-instant satisfaction, and don’t raise security concerns. I don’t have a good answer to this one yet, but believe me, I’m working on it!

GoogleHotKeys is Go!

Launch

I’ve officially launched Google Hot Keys. Check out the new site for the Internet Explorer and Firefox versions, help and screenshots.

There’s already been a lot more Firefox downloads since the name change, now I’m going to work hard to promote it to IE users too. So far, I’ve started the submission process for CNet’s download.com, at the confusingly named upload.com! It looks like this will take around three or four weeks to go through their submission process, though I could speed it up if I paid $250, or went for a $9 a month package. One nice bonus is that it will appear on Windows Marketplace once CNet accepts it.

I’ll be reaching out to some of the people who’ve shown a past interest in PeteSearch, since I’m finally happy that I’ve got a product that will deliver a great experience! There’s an audience out there for this, and I will be pushing hard to get it in front of them.

Keyboard navigation in Firefox

Pianokeys2

I’ve uploaded the new incarnation of PeteSearch, GoogleHotKeys, to the Firefox add-ons site, and the new name does seem to have caught people’s interest. Downloads are up, and I’ve been getting more comments from users.

One question was how it’s different from another add-on, SearchKeys. I thought I’d review that and a few of my other favorite keyboard tips here.

SearchKeys is the closest to GoogleHotKeys. After loading some search results, you can press the number keys to go to individual results, and ‘,/.’ to move to the next or previous pages of results. It’s simple and robust to use, but I have trouble remembering which search result is associated with which number key. That’s why I use a visible selection box, and up/down for GHK, for me it’s a lot more intuitive.

GHK also does a lot more than SearchKeys, with split-screen previews, page-checking, term-highlighting and support for some other engines.

Hit-A-Hint is a more general way of navigating through any web page using just the keyboard. Every link on a page is given a number, which you can see by pressing space:

Magicshot

If you press a number key with space held down, you’ll open that link. It’s a pretty quick to learn and easy to use. It’s the closest I’ve found to Lynx’s navigation in a graphical browser. I don’t use it all that much though, since I find the slowest is visually finding the right link, and moving the mouse there doesn’t take much time.

There’s also Firefox’s built-in type-ahead find. If you press the ‘ key, and then type some text, the first link that starts with those letters will be selected, and then return will open it. Again, this is a neat idea, but I don’t make much use of it.

How to promote your add-on – What not to do!

Spill

To echo Thomas Edison, I now know a thousand ways not to market an add-on.

Build it and they won’t come

My initial marketing strategy was passive. I’m used to working in niche markets like video editing, where early-adopters actively seek out new products, and then evangelize them to a wider user-base. This is generally the pattern when the demand for solutions in a market is high, and the supply of software is low. I had a lot of success in the past just by making my software available, alerting a few contacts, and letting the users find me.

Add-ons are usually aimed at a consumer market. There’s a lot more software competing for people’s attention, and the problems solved are a lot less pressing. That makes it tougher to persuade people to make the first step of trying your add-on out. I didn’t have a plan of how to overcome that hurdle, and so ended up with very few people giving it a try.

Blind to my baby’s ugliness

After all the work I’d put in, I wanted to believe the add-on was
obviously compelling. In reality, everyone’s software is ugly in the
early days. It’s important to accept that it will have a lot of
problems, people won’t get it, and you’ll need to explain your vision
of how it will grow into something beautiful. Thanks to Dharmesh Shah
for the metaphor.

On my visit to Chicago I read The Devil in the White City. There’s a quote there from the architect Daniel Burnham:
"Make no little plans. They have no magic to stir men’s blood and probably will not themselves be realized."
I should spend more time getting across the excitement I feel about what can be done with client-side mashups, there’s some amazing possibilities, and this is just a first step down that road!

Pick a confusing name

I picked PeteSearch because its initial name of SearchMash was grabbed by Google after I launched my Java applet version, I’d had success with PetesPlugins in the video world,  and I couldn’t think of a better one.

The name didn’t advertise what the add-on does, and wasn’t very intriguing. Most of the time, a bare name in a list is the first contact a potential user has with an add-on. That’s all the information they have to decide whether to invest more time in it, so I should have made it tell a story. GoogleHotKeys is a much more effective title.

Don’t track progress

There’s no records on exactly how many users I have in total, how many of them have been referred from other sites or any breakdown of user feedback. I’ve been keeping track in an informal, qualitative way, by looking at my web-server stats and chatting to users, but there’s no quantitative statistics. This is a problem because it makes it very hard to really judge what’s working and what’s not, and so to decide what the best use of my time is.

It doesn’t have to be a big report, but it’s incredibly helpful to keep track of a few key statistics like total downloads per day, and numbers of referrals from each source. I don’t have a good process for doing this automatically yet, but I’ll let you know how I get on.

Have unclear goals

I go into this in more depth below, but my fundamental mistake was not being clear about what problem I was solving. I wanted to keep my options open, and be able to experiment, but I ended up with so may features it was confusing both to use and describe.

More posts on add-on promotion.