No Vision, All Drive is the story of Pinpoint Technologies, from its origins as a startup, through to its sale to ZOLL. One of the founders, David Brown, wrote it, but it includes some great sections from the other founder, David Cohen, as well as many of the long-time employees.

The employee’s stories are a great reminder that creating good jobs is not just an abstract statistic, it can make a real difference to people’s lives. Al Thompson (aka Weird Al the Tattooed Freak) describes how he was determined not to get another coffee-shop job, and got his only interview at Pinpoint, and was offered the job even though he didn’t have any experience. Six years later he’s a highly-skilled, highly-valued old-timer, and met his fiancee there too!

Marcie Cary got started as a receptionist, and went on to become marketing manager. She really gives an insight into how humiliating she felt the admin work could be, faxing and mailing packages. I’m no stranger to low-status jobs, having spent five years stacking shelves in supermarkets, but it was a good reminder. She also acknowledges the other side of the coin, that it can be really tough for the rest of the team to cope with someone avoiding all that work, when it’s their job to get it done. I don’t know what the solution is, but it’s a scenario I’ve seen play out many times. Maybe the combination of training and advancement, together with an insistence on keeping on top of the boring stuff is the right way to go.

My favorite part of this book is all the anecdotes, they really give you a flavor of the startup life, from the brown shag carpeting in their first office, to their receptionist being afraid to be left alone in the office with their first programmer!

It’s a great read, it keeps you turning the page, and is focused on the nuts and bolts of growing the business. I’d highly recommend it if you’re interested in an honest and informative account of a startup that succeeded.

Funhouse Photo User Count: 958 total, 55 active. Much as before.
Event Connector User Count: 5 total. AdWords isn’t showing any conversions at all, so I’ll need to rethink my tactics there. It’s in the queue for the applications directory, I’ll see how that helps the totals.

"Go that way, really fast. If something gets in your way, turn." – Charles, describing how to ski the ‘impossible’ k12 in Better Off Dead.

I had a big response to my last post on web annotation. One sentence in there sticks out as similarly accurate, but useless advice: "you can build it, and they won’t come, unless you get marketing and distribution right too." So here’s some more concrete analysis of why people don’t come.

There are three reasons someone might use your product; because they like playing with new technology, because they have a painful problem they need to solve, or because it looks like fun.

Geeks are driven by a lust for technology, business customers want to stop the pain, and consumers want to kill boredom. OneNote is the only tool that’s totally focused on the business market, the others are after consumers, but actually seem to be mostly limited to geeks so far.

I did find another good summary of the annotation market from techcrunch. One related tool I didn’t mention before was Stumbleupon, and compared to the others it’s got a stellar adoption rate, with around two million registered users in April.

What’s different about Stumbleupon? It’s instant fun. All of the other services require the user to invest some time and effort before they get a reward. With SU, you install, answer a few quick questions, and it starts showing you sites straight away. That’s fun! That’s what pulls in consumers.

One of my first jobs was helping to port the PC game Diablo to the Playstation 1. Gary Liddon, a programming god from the 8-bit days, explained to me a big reason why it was so successful; every monster you kill explodes in a big, flashy, noisy shower of gold and goodies. It’s like an infinite series of pinatas, people respond to quick and frequent rewards.

Diigo is a beautifully crafted tool, but it doesn’t offer a reward until you’ve spent some time setting things up, linking with friends, and creating your own content. That’s fine for us geeks, or people who have a strong enough need to invest that time to solve a problem, but it’s not going to be something that grabs the idle masses, even though a lot of them would like it once they had put the effort in!

Trailfire takes a bit more of a consumer-focused approach, with popular trails right on its front-page, giving an immediate example of what reward you’ll get. Diigo does have a ‘what’s hot’ page, but it’s deeper in the site.

This may sound like I’m treating consumers like idiots, but the truth is that most normal people don’t enjoy playing with new technology, they need a strong reason to try it, and they find any setup or installation really annoying, even things we’d not think twice about.

PC games don’t sell, apart from a few blockbusters, and even they have tiny sales figures compared to pretty much any franchise sports game. A lot of people who buy consoles already have a PC with comparable power, so why do they bother? The difference is, you (or your granny buying your christmas present) makes sure the sticker on the game’s box says Wii in the store. Then you put the disc in the console, turn it on and play. People buy the game because they anticipate a quick and reliable reward.

With a PC, you take a notebook full of specs to the store to make sure you reach the minimum requirements, then you sit through eight screens of installation mumbo-jumbo and requests to make choices about indecipherable technical details. Then you realize there’s a DLL conflict, and Office no longer runs. There’s no certainty of a reward at the end, and it takes a long time to get there, so most people don’t bother.

Most Web 2.0 tools seem doomed to a geek ghetto, unless we can get the time-to-reward down to seconds rather than minutes or hours.

In the real world, people add their own layer of information on top of printed documents by scribbling notes on them, and highlighting passages. They can share the modified document with their friends, and pass on their opinions and insights.

This is really hard to do on the computer. Distribution formats like HTML pages and pdfs are designed to deliver information from the producer to the consumer. You can’t even add your own notes, let alone share them with others.

With the real-world analogy so glaring, there’s been a lot of attempts to fix this. The wikipedia article on web annotation alone includes twenty-two active projects! In a different direction, MS Office now includes OneNote, which is a tool for creating free-form notes, and sharing them with friends. It focuses on pulling content into a separate notebook, rather than adding notes on top of the content.

So why haven’t you heard of most of these web annotation tools? Casey has a post covering the history of web annotation, from 2004. There’s been a few successes since then, but most of the annotators have struggled to get the critical mass of users they need to be useful. One of my favorites is JumpKnowledge, which has a focus on letting you email pages with your comments on top, using their AWE Firefox plugin.

Reading their blog is pretty painful. When Yaakov describes trying to get coverage for their system as "like banging my head against a wall" I can sympathize. He’s got some great reviews, from some big names, but it seems like even they’re no longer in active development.

[Update- I just heard from Yaakov, and they have been working on some interesting new features, which is great news! See his comment on this post]

This is a good reminder you can build it, and they won’t come, unless you get marketing and distribution right too.

Diigo seems like the most successful annotator out there at the moment. It’s in active development, supports a lot of cool features like sharing with groups, and some neat search functionality for things like looking for inbound links to the current page, or searching on the same site. Trailfire is another very active annotation tool, that does some content analysis of the page you’re on to figure out if there are related pages in its database.

It still feels like the killer app for web annotation is missing. Maybe people really don’t want to scribble on web pages as much as us techies think, or maybe we just haven’t found the right way to do it. I thought that JumpKnowledge‘s focus on mailing web pages together with annotations had a lot of potential, but it doesn’t seem to have caught on. Diigo has tools for easily posting annotated pages to a blog, which also seems like it could be really popular, but I don’t see too many blogs being created that way.

Vlad also asked how a BHO can be added to the registry, outside of the compiler? This is a question I’m wrestling with as I write the installer, so here’s a dump of my current understanding.

There’s three purposes to registration; telling IE that there’s a BHO with a certain ID it should load, telling the system where the DLL is that contains that ID, and what code actually implements the interface we’ve declared.

The first part is covered by the .rgs file in the project. It handles setting up the registry so that our BHO is added to the list of ones to load. If you look at the file, it contains a description of what our DLL contains, including a UUID, and also adds that UUID to the list of BHOs for IE to load. It’s added to the DLL as a resource, and we reference that in our implementation class, by calling the macro DECLARE_REGISTRY_RESOURCEID(<resource ID>) in our class declaration. This adds an UpdateRegistry() member function to our class, that calls an ATL function that parses the rgs script, and adds the keys to the registry.

Telling the system where the DLL lives on disk is handled by calling code inside the DLL itself, in a process called ‘self-registration’. There’s a custom build step that calls regsvr32 /s /c <your dll>. This in turn calls the DllRegisterServer() function inside your dll, which in our case calls a standard ATL _Module.RegisterServer() call. We’ve set up an ‘object map’ containing our UUID, and the actual class that implements that interface, and the ATL call takes that and adds the right information to the registry. This call also adds the DLL location to the registry, and calls the UpdateRegistry() function we set up in our class, that adds the .rgs entries to the registry.

If you want to make this happen outside of Visual Studio, you can call regsvr32 yourself from the command line. Procedural installers like NSIS have script commands that do the same, like RegisterDLL.

As always, there’s a catch. WIX/MSI is a declarative installer, and MS strongly recommends against calling procedural DLL code as part of the process, since that’s a black box to the installation system, and so will be a lot harder to roll back. Instead, I’m going to have to capture a static description of all the registry changes that calling regsvr32 on my DLL causes, and add that to the installation script. Luckily, there’s a tool called tallow that looks like it may help. I shall let you know how I get on!

More posts on porting Firefox add-ons to IE

I’m still very busy with my day job, but I did have a chance to put together a Firefox extension over the weekend. It looks through your history, and puts your top ten visited sites as shortcuts in a toolbar. This is something I wanted for my own browsing, since I have a lot of sites I visit often, but I find bookmark management tedious.

You can give it a whirl at http://petehistory.com

I know I’m not alone in finding bookmarks a pain (eg http://wiki.mozilla.org/Places quotes figures that only 1-3% of page navigations use bookmarks or the history), so I believe other people will find it handy too.

Now you can move through the results just using a button, I realized there should be an indication of which result the preview is showing. Luckily, Google already used tables for their formatting, so I was able to use an existing table showing the snippet which had no border, and set its border to one to give a nice selection box.

Before I put that change in, I also tracked down the IE/youtube preview hang. It turns out some of my script cleaning code was too aggressive, and chewed up the final ‘>’ at the end of the document, so it ended with ‘</html' rather than '’. This confused explorer enough to cause a hang. There’s now a fix in for that problem.

While I was testing the selection box, I ran into some other bugs, mostly minor object access errors that caused different browsers to choke, which I was able to fix by adding more check to make sure methods existed before I called them.

I did see some problems with pdf links. Since these aren’t text, and are very large, the script doesn’t cope too well with them, and isn’t able to display them at all anyway. My solution was to check the ‘Content-Type’ header, and only let through pages with ‘text/’ in there. This should stop binary files like flash getting through too.

One of my main goals with SearchMash has been to save search time by skipping pages that don’t actually have the terms I want, despite what they tell Google. Typically, these are subscription sites that require registration and login to see the information Google’s indexed, and that’s normally too big a barrier for me.
To avoid those pages, SearchMash now checks the web pages it gets for the search terms, and tells you if it doesn’t find some of the terms.
I also upgraded the status display of results over the weekend, so you’ll see a clearer indication of any problems getting the page. I now draw a line through pages that couldn’t be fetched, and color any links that had errors as red.

I discovered MS’s script debugger whilst investigating the preview frame sticking problem in Internet Explorer. I’m used to developing on Safari with just error messages to go by, so having even a basic debugger was a pleasant surprise. I also hear good things about Firefox’s debugger. My only niggle so far is that I can’t figure out how to tell what the error was that caused it to invoke the debugger, but I’m sure that’s just an RTFM issue.

I started off by turning off “disable script debugging” in IE’s preferences, but unfortunately that didn’t do very much. I discovered you had to manually install the debugger for the option to do anything, which made sense once I realized. I downloaded it from a link in an MSDN blog.

That’s a good question.

The informal rules behind what’s acceptable use of someone else’s web server are clear if you write a new browser. Nobody complained when firefox came along, because there’s real people reading the content that the server owners are paying to send.

The rules are also well understood if you write a new robot to crawl the web, they should tread very lightly indeed, respect the robots.txt file, and keep some delays in between fetches, so as to avoid slowing down the server for the real traffic.

SearchMash is somewhere in between these two extremes. Originally, it was a pure browser. It is still entirely user-directed, so there’s a good chance that the bandwidth is going towards your target audience. On the other hand, an entire page of search results will be fetched at once, so it’s not as user directed as if they’d directly clicked on your link.

I do avoid fetching anything but the main HTML until the user requests a preview of the page, to keep the bandwidth demands as small as possible, so no images are requested.

I know not everyone will agree that it’s a net benefit, so I’ve made sure that the User-Agent header is always set to MashProxy for all requests, so servers can easily block my traffic. I considered a whitelist system too, since that would also prevent intranet access, but could see no practical way of that gaining adoption.

Hi, I’m Pete Warden (that’s not me in the photo!), and as a fun project, I decided to create a search mashup that let me search the way I wish I could.

While I was doing that, I discovered that there was a big restriction on using XMLHttpRequest and AJAX. You can only request pages from the same server you’re on, as a security measure. This obviously makes doing a mashup of pages on other servers much more difficult.

The standard ways of working around this involve setting up a way to use your server to fetch external web pages. There were several reasons I wanted to avoid this:

– It doesn’t scale with the number of users, since everything has to go through your server.
– A big goal of the project was to discover if the pages found in the search were accesible to the user. This isn’t possible if it’s a remote server doing the checking.
– Setting up the server to act as a proxy requires at least some knowledge of scripting and Apache

The main reason that client-side proxies haven’t been done before is the potential for security holes that it opens up. Chris Shiflett has a great article that covers the problems if XMLHttpRequest were opened up to allow cross-domain requests, which is equivalent to what MashProxy allows.

Julian Couvreur also has helped my understanding of the issues. He’s written something similar using Flash rather than Java, FlashXMLHttpRequest.

I’ll discuss the security policy I adopted in my next post, including the safeguards against abuse I’ve implemented and possible remaining problems.

In short, MashProxy is a Java applet that lets JavaScript request web pages, just like XMLHttpRequest, but without the same domain restriction. This let me build the SearchMash project, implementing my ideas on a better interface to search results. It’s open source, and up on SourceForge, and my hope is that other developers will use it as an easier way to create mashups. I want to see more mashing, and I think the server proxy requirements have been holding things back.