Porting Firefox extensions to Internet Explorer Part 2

Skeleton

In the first part, I showed how to set up the tools you need to build a BHO. I’ve now created a skeleton project that will compile into a very simple BHO, TinyBHO. I wasn’t able to find any good official sample code, all the articles involve starting with a template, and describe how to build on it, but the SurfHelper example by Xiaolin Zhang on CodeProject was a great help. Having a complete example I could build and run gave me the information I needed to build my own.

To build the BHO, download and unzip the project, open the TinyBHO.vcproj file in Visual Studio Express, and hit f7 to build it. The build process should compile, and then add the DLL to the registry, and the next time you run Internet Explorer, a message box should appear whenever a document’s loaded. This provides all the boilerplate to hook into browser events, and provides a stub that you can insert your  own document processing logic into.

If we want to build on this, we’ll need your own GUIDs for the interface and type library. Use a tool like uuidgen or one of the online versions to generate three IDs, and then load the TinyBHO.idl file into VS’s editor. Replace the first ID that begins with 03147ee0 with our first ID, it should only appear once near the top of the .idl file.

The second ID we need to replace starts with 7cd37f36, and appears once in both TinyBHO.idl and TinyBHO.rgs. The third begins with 00e71626, and defines the main class’s ID. It occurs four times in TinyBHO.rgs and once in TinyBHO.idl. I’d also recommend customizing the class and file names, but that shouldn’t be strictly necessary for the BHO to coexist with others the way that unique GUIDs are.

Now we’ve got the BHO personalized, we can start looking at CTinyBHOClass::Invoke(), the function that gets called whenever a browser event, such as a page load or key press, occurs. Right now, all it does is catch document load events, and pulls the document interface pointer from the event, and shuts down the event handler when the browser quits. For PeteSearch, I’ll be focused on key presses and document loads, since that’s where it does most of its work, but there’s a wide range of other events to grab if you want to implement something else, like a popup blocker. In Firefox terms, it’s pretty close to having called AddEventListener() on the browser, except that you get called on all events and have to check yourself to figure out what happened.

The trickiest thing about getting this working was that all the ATL COM handling code is such a black box. The registration part of the build phase was failing, and I eventually tracked it down to a misspelt resource name, but there was no logging or other useful debug information available, I just had to spend a long time inspecting the code looking for anything suspicious!

More posts on porting Firefox add-ons to IE

Porting Firefox extensions to Internet Explorer

Explorerlogo
Compared to writing Firefox extensions, there isn’t a whole lot of information about writing add-ons for Internet Explorer, and almost nothing about porting from Firefox to IE.  I’m converting PeteSearch over, and I’ll be sharing my notes on the process here.

There’s no way of writing a plugin using an interpreted environment like Firefox’s Javascript, instead you have to create a binary library in a DLL. These are known as Browser Helper Objects, and give you the ability to catch browser events like page loads, and work with the DOM, but don’t offer any UI access for things like toolbars. Hopefully PeteSearch doesn’t use anything outside of a BHO’s domain, since it works almost entirely on processing and altering existing pages.

Since we’ll need to compile a DLL, the first thing we need is a compiler. Luckily Microsoft has released a cutdown version of Visual Studio for free, though there are some limitations that I’ll describe. I chose to use C++ to create the DLL, because I know it well and it’s what most of the BHO examples use, but you could also use C# or Visual Basic.

After we’ve downloaded the compiler there’s some other components we need before we can build a DLL. Since the BHO is a win32 DLL, we need the platform SDK. To get it working with the express edition of Visual Studio, we’ll also need to muck about with some config files, as described in this MSDN article.

For dealing with COM code, ATL/WTL is very useful, so I’d also recommend downloading the open source WTL from MS, and following the steps in this CodeProject article.

Now we’re ready to compile a DLL, the best place to start is with some example code. There’s two good articles from Microsoft that describe writing BHOs, one from 1999 and a more recent 2006 version. Unfortunately they both start off using a wizard to create the base project, and the ATL wizard isn’t available for the express edition! They don’t have any complete examples downloadable, which is a bit puzzling. I’ve contacted one of the authors, Tony Schreiner, in the hope he can provide a complete example. If not, I may finagle a base ATL project from someone with the full studio package, then I’d be able to build it myself following the rest of the steps. Of course, I could purchase the standard studio for $300, but it seems worthwhile to work out a free way to write IE extensions.

René Nyffenegger has some example code that we can use instead, though I found I had to do some tweaking to get it to compile, such as defining a couple of logging macros, and sorting out some string literals that were expected to be wide but weren’t.

Now we should have the starting point for a BHO. I’ll cover the next steps, and show some example code, in a following article.

Edit – I’m trying this on a new Vista laptop, and there’s a few extra steps I noticed:

  • Folder permissions for editing the ATL headers in the SDK are tricky in Vista. You need to make yourself the owner, and only after that sort out the permissions.
  • You need to add the mfc include folder from the SDK too. I may have just forgotten this before.
  • The registry-setting part of the build process doesn’t work. I’ll cover fixing that in my description of writing an installer, and update here once that’s done.

Edit – To fix the registry build stage, you can run Visual C++ as administrator by right-clicking on the app and choosing that option from the menu.

More posts on porting Firefox add-ons to IE

Slow Java Startup

Snail

I knew I was taking a gamble using a Java applet for SearchMash, since Sun seems to have abandoned it as a client-side technology. Mostly it’s paid off, I’ve been able to develop a pretty rich web app that runs entirely on the client, and with no install step. There’s a big problem with using a Java applet though, it takes forever to load, often adding 30 seconds to the load time of the page.

This is a well documented problem, but it seems like a tough one to solve. On OS X with Safari it’s not as bad, since it seems like the Java environment is pre-loaded, but it still takes 10 seconds or so the first time you visit the page. On OS X Firefox it’s a bit worse, and on Windows both Firefox and IE can take a very long time.

My only ideas on tackling this are to do something interesting during the loading process (a game of pong anyone?), or switch to a different technology entirely.

I’ve been considering two different approaches, either using a Firefox extension, or writing a standalone app. The Firefox extension would require an install step, something I’ve tried to avoid, but would be an integrated part of browsing after that. A standalone app would be more work to write, require installation, and have the barrier of not being part of the browser, but it would allow me to do a lot of the more advanced things I’ve wanted, like animated thumbnail browsing of pages.

Firefox Javascript Debugging

Bug

I’ve got the week off for Thanksgiving, so I had a chance to get back to coding searchmash. I was planning on tackling the back button support, but ended up spending the day on a perplexing Firefox issue.

One great thing about debugging Safari is being able to output text to the console using “window.console.log()”. It’s not as good as MS’s free script debugger, but it’s a lot better than using alert()’s.

I eventually found a way, but it was not well documented, so here’s a quick walk-through:

  • Type about:config in Firefox’s address bar.
  • You should now see a list of names and values in the main window.
  • Right-click anywhere in the window, and chose New->Boolean from the menu.
  • In the box that comes up, put in browser.dom.window.dump.enabled
  • In the next box, chose true

You can now call window.dump(), and the string you pass in will appear in the system console. On the Mac, this is visible in the /Applications/Utilities/Console.app program.

Whois

Eye4
There’s now a ‘Whois’ link for all results, next to the similar pages and cache links after the summary. This uses the internet whois protocol to try to get information about the owner of the domain the page is on.

To use the new link, just click on it and the results of the whois search should show up in the preview pane. There’s usually a bunch of boilerplate standard information from the server holding the information, followed by the information about who the domain is actually registered to.

Since the whois servers hold the owners names, emails and phone numbers if they’ve not gone with a private listing, I had to be careful about how I offered this within SearchMash. In particular, I didn’t want to overload the whois servers providing the information, so I’m careful to only put in requests when you click on a link, rather than pre-fetching as I do for web pages. This is basic courtesy to the owners of the servers, and also complies with their terms of use designed to prevent robots from harvesting addresses for spam.

Last modified, better preview, and ask

Clock_1
The status line now shows when a site was last modified, which can be useful to know if you’re searching for time-sensitive information. Not all sites return the header I use to find that out though.

I’d noticed there were some sites still showing up with missing pictures or bad formatting in the preview. I spent some time debugging some of the problem cases, and realized there were some bugs in the code I was using to insert the tag that resolves relative URLs. I put in fixes for those problems, which made a lot of CSS based sites appear much better.
It also allowed some scripts to run that hadn’t before, so I then had to fix the bugs in my script blocking that let those through. The end result is that preview works a lot more reliably.

I’ve also been experimenting with offering ask.com as an alternative to Google. I managed to get something working pretty quickly, but hit problems with their use of Javascript in the results pages. It works fine in Safari and Internet Explorer, but Firefox throws a lot of errors. I’ll be returning to debug that some more when I have a chance, for now I’ve left the code in, but I’m not providing any obvious links to it. If you want to experiment, use http://mashproxy.com/search?ask to invoke it.

Browsing speed boost

Eye2_5
I’ve added a “Show Next Result” button to the search page. This moves the preview forward to the next link in the results, and if you’re at the end of a results page, it fetches the next one. The beauty of this is that you can go through links very quickly without having to hunt through the results page.

I was also hoping to get the up and down keys to also cycle through results, but this looks like a cross-browser minefield. I had it working in Safari, but only after you manually set the focus by clicking in the window, I couldn’t get it working with Firefox, and I haven’t even looked at IE yet!

Another addition I want is some indication of which link is being previewed. I’m thinking about setting the table border of the snippet below the result to 1, so you get a selection box.

One disturbing thing I noticed while testing my latest changes on the PC is that youtube links in the results seem to hang IE when they’re previewed. This seems to be new, I’m worried there’s been some flash changes, and it might affect a lot of sites. Firefox and Safari still seem fine, but I’ll be trying to work out what’s happening tomorrow. It’s filed as Previewing YouTube sites hangs IE.

HTTP socket code fixes

Wires

I was chasing up the cause of the bug Socket based fetches can fail, and I tracked it down to an error in the way I was forming my initial HTTP GET request. I was putting the full URL as the argument on the first line, rather than just the path part. Interestingly, most servers accepted this, but typepad.com didn’t. I’ve updated the applet to do the right thing in this case, and also added some logging code that helped me track down the problem.

Something else that was really useful was the LiveHTTPHeader addon for Firefox. It gives you a full dump of the header information that’s passed back and forth between Firefox and the hosts it contacts. I’ve tried to absorb the RFC, but nothing beats being able to see a working example. I was also impressed by how easy it was to install, it makes me wonder how hard it would be to create a Firefox plugin version of SearchMash in the future.

I also fixed another bug that would sometimes prevent pages from loading, and removed the seperate status request check I did before, that used URLConnection. Now I just make a single socket connection for every page, that should be faster and easier to maintain.

Preview polished

Eye
I’ve been talking with Philipp Lenssen from the excellent Google Blogoscoped, and he had some great feedback. In particular he called out the problems with some things not showing up correctly in the preview window.

This is something I noticed too, and have a bug filed in sourceforge’s bug tracker on, but it hadn’t made it to the top of my list.

I spent a bit of time on it this morning, and managed to greatly improve it. First off, I was changing the HTML to all lower case as part of my security measures, which messed up any case-sensitive resource paths. I’m doing all my search and replace case insensitively now, so I was able to remove that.

I also discovered that the BASE tag I was adding to resolve relative resources should actually point to the full URL of the page. I was trying to be too clever, and setting it to the URL up until the last /, assuming it had to be a folder name. Just setting it to the full URL made a lot more pages show up correctly.

There’s still some pages that have problems so I’m leaving the bug open, but these two changes seem to have fixed most of the issue.

Philipp also asked why I don’t just set the URL of the frame, rather than writing in the HTML into a local one. That’s a good question, since the current functionality would still work. I’m planning on adding something to help find search terms in the page in the future though, highlighting and scrolling to the right words, and that wouldn’t be possible just setting the location. I like to make life difficult for myself!

Prettification

Lipstick
I’ve made some cosmetic changes to SearchMash, so that you don’t just see a blank screen while it’s loading. You now see “Loading…” in the results frame, and a quick primer on how to use SearchMash in the preview. The primer is there because I had a lot of feedback that the initial screen was confusing to first-time users, since it’s just the google start page in a frame. Hopefully the extra information should make it easier to get started.

I also worked on the problem with the preview window sometimes causing script errors in IE, filed as Preview window can stop working after changing pages and Previewing missing sites on IE stops window in Sourceforge’s bug tracker. I added a try/catch around the access to the preview frame’s document, and tried to reset security by setting it back to the original local ‘src’ if there was an exception. I still see the exception occuring (I’m using MS’s very handy script debugger, which I’ll cover soon), but I haven’t been able to reproduce the problem with the window getting stuck, I hope my changes will unwedge it if it’s in that state.