In-page Search for IE

Ookiilogo
I just discovered an addon for IE I didn’t know about, Find As You Type. Even better, Sven Groot, the developer has some technical notes up on the under-documented issues he ran into while developing it, the solutions he found, and the source code to the add-on.

The add-on itself looks like it brings a lot of the useful Firefox functionality I love for in-page searching to IE, including highlighting all terms, and searching as soon as you start typing.

In a nice twist, I actually discovered this using the me.dium Firefox extension. It’s been a fun social tool, but in this case it figured out that the site was related to the MSDN extensions forum I was browsing, and showed it on the radar. Very, very useful, great job by the guys who wrote that algorithm!

More posts on porting Firefox add-ons to IE

More Search Tips…

Porting Firefox extensions to Internet Explorer – Part 4

Tractor
In my previous articles, I described how to set up the compiler, build a basic BHO, work with regular expressions and fetch pages using XMLHttpRequest’s. I’ve now combined those components into the first working version of PeteSearch in IE. The source code is available through Sourceforge, or you can download it as a zip archive.

This version detects search pages from Google, Live, Ask, Technorati and Exalead, pulls out the search result links, fetches the contents of those pages to ensure they’re still active and relevant, and updates the search page to show each link’s status. I’ll be covering the remaining tasks in later articles, which include implementing hot-keys, adding a split-screen view to IE and writing an installer.

More posts on porting Firefox add-ons to IE

Multi-threaded DOM access in a BHO causing crashes

Threads
PeteSearch relies on Document Object Model (DOM) access to analyze and update HTML pages. This is built-in to Firefox’s Javascript, but in IE you have to use the IHTMLDocument(2/3) COM object. I’ll give some code examples in a later article, but first I want to cover a serious problem I ran into, and the solution I found.   

I was seeing a lot of crashes once I started analyzing search results, and fetching pages. I discovered that the crashes all seemed to happen when multiple threads were all accessing the same document. I was passing a pointer to the IHTMLDocument3 object to the XMLHttpRequest worker threads, and after a little thought, it made sense that this object wasn’t thread-safe, and so I’d need to synchronize my access to it somehow.

This part really puzzled me. I couldn’t figure out a good way to call back my BHO on the main thread, because the BHO doesn’t control the message loop for the main thread, that’s all in IE. And since IE could be using the same object, and I don’t control that code, I couldn’t implement my own manual locking.

Luckily, I came across a life-saving post from Tony Schreiner. He explains that COM objects in IE are not thread-safe by default, but that you can get a wrapped version that adds thread-safe locking by calling CoMarshalInterface(). This gives you a version of the object that you can use on a different thread, without worrying about crashes caused by other threads accessing it at the same time.

It was only a few lines of code, and implementing it removed all the bugs I was hitting. Thanks Tony!

More posts on porting Firefox add-ons to IE

XMLHttpRequest in C++ using WinHttp

Glowflow

My decision to use WinInet to implement my XMLHttpRequest class turned out to be a mistake! I went back to the drawing-board, and reimplemented the class using WinHttp, the more modern alternative from Microsoft. Here’s the code, the CPeteHttpRequest class that implements the classic XMLHttpRequest interface together with an app to test its use. It’s available for reuse with no restrictions.

Here’s the differences between this new version and the old WinInet-based one:

  • It no longer crashes when under heavy multi-thread use
  • The interface is now entirely in wide character strings
  • WinHttp is only available on Windows 2000, XP and Vista, so use the WinInet code for legacy support

More posts on porting Firefox add-ons to IE

WinInet Thread Crashes

Crash
In my previous post on XMLHttpRequest, I explained why I chose to implement my http request class using the legacy WinInet library, rather than one of the more modern alternatives. Last week, I was able to get the DOM parsing part of PeteSearch running, and so I was able to start stress-testing my class.

I was getting a lot of crashes, and after finding and eliminating a couple of my own bugs, and paring down the code as much as I could I was left with some minimal code that still crashed, often in the wininet dll!

After digging around some more on the MSDN site, and reading between the lines, I realized this seemed to be a known problem, and there probably wasn’t much I could do about it. They recommend using WinHttp for "server applications", and there seem to be a lot of hard-coded limits in WinInet, such as how many concurrent accesses to the same server you can have.

As far as I could tell, the crashes all occurred when I had a lot of requests being processed on different threads at the same time.

I replaced WinInet in my XMLHttpRequest implementation with WinHttp, and the crashes went away, which is strong evidence that it wasn’t my code doing something whacky. I’ll be posting the updated code in my next article.

More posts on porting Firefox add-ons to IE

XMLHttpRequest in C++ on Windows Example

Swirly
In the first three parts I covered how to get a simple IE extension built. For PeteSearch I need to be able to fetch web pages, so the next step is to figure out how to do that in Internet Explorer.

Firefox lets extensions use XMLHTTPRequest objects to fetch pages. It’s quite lovely; well documented and tested since it’s the basis of most AJAX sites, and with an easy-to-use interface. The first thing I looked for was an IE equivalent.

There’s a COM interface called IXMLHTTPRequest that looked very promising, with almost the same interface, but it turned out to involve some very gnarly code to implement the asynchronous callback in C++. It was also tough to find a simple example that didn’t involve a lot of ATL and MFC cruft, and it involved using a pretty recent copy of the MSXML DLL, and there were multiple different versions. All-in-all, I ruled it out because it was just sucking up too much time, and I dreaded the maintenance involved in using something so complex.

There’s also the newer IWinHttpRequest object,  but that’s only available on XP, 2000 and NT4.0, and seems far enough off the beaten track that there’s no much non-MS documentation on it.

I finally settled on a really old and simple library, WinINet. It’s a C-style API, and lower-level than XMLHttpRequest, and is a bit old-fashioned with some situations that require CPU polling, but it offers a full set of HTTP handling functions. It’s also been around since 1996, so it’s everywhere, and there’s lots of examples out on the web. Since I liked the XMLHttpRequest interface, I decided to write my own C++ class implementing the same methods using WinINet under the hood.

Here’s the code I came up with
. It implements a class called CPeteHttpRequest that has the classic XMLHttpRequest interface, with a simple callback API for async access. I’m making it freely available for any commercial or non-commercial use, and I’ll cover my experiences using it with PeteSearch in a later article.

Edit – It turns out that WinInet is actually very prone to crashing when used heavily in a multi-threaded app. You should use my WinHttp based version of this class instead.

More posts on porting Firefox add-ons to IE

Porting Firefox Extensions to Internet Explorer Part 3

Text
In the first two parts I covered setting up our tools, and creating a code template to build on. The next big challenge is replacing some of the built-in facilities of Javascript, like string handling, regular expressions and DOM manipulation, with C++ substitutes.

For string handling, watch out for the encoding! Most code examples use 8 bit ASCII strings, but Firefox supports Unicode strings, which allow a lot more languages to be represented. If we want a wide audience for our extension, we’ll need to support them too.

C++ inherits C’s built-in strings, as either char (for ASCII )or wchar_t (for Unicode) pointers. These are pretty old-fashioned and clunky to use, doing common operations like appending two strings involves explicit function calls, and you have to manually manage the memory allocated for them.

We should use the STL’s string class, std::wstring, instead. This is the Unicode version of std::string, and supports all the same operations, including append just by doing "+".  The equivalent for indexOf() is find(), which returns std::wstring::npos rather than -1 if the substring is not found. lastIndexOf() is similarly matched by find_last_of(). The substring() method is closely matched by the substr() call, but beware, the second argument is the length of the substring you want, not the index of the last character as in JS!

For regular expressions, our best bet is the Boost Regex library. You’ll need to download and install boost to use it but luckily the windows installer is very painless. Once that’s done, we can use the boost::wregex object to do Unicode regular expression work (the boost::regex only handles ASCII). One pain dealing with REs in C++ is that you have to use double slashes in the string literals you set them up with, so that to get the expression \?, you need a literal "\\?", since the compiler otherwise treats the slash as the start of a C character escape. The regular expressions functions themselves are a bit different than Javascript’s; regex_match() only returns true if the whole string matches the RE, and regex_search() is the one to use for repeated searches.

DOM maniplation is possible through the MSHTML collection of interfaces. IHTMLDocument3 is a good start, it supports a lot of familiar functions such as getElementsByTagName and getElementById. It does involve a lot of COM query-interfacing to work with the DOM, so I’d recommend using ATL pointers to handle some of the house-keeping with reference counts and casting.

PeteSearch is now detecting search page loads, and extracting the search terms and links from the document, next we’ll look at XMLHttpRequest-style loading from within a BHO.

More posts on porting Firefox add-ons to IE

Porting Firefox extensions to Internet Explorer Part 2

Skeleton

In the first part, I showed how to set up the tools you need to build a BHO. I’ve now created a skeleton project that will compile into a very simple BHO, TinyBHO. I wasn’t able to find any good official sample code, all the articles involve starting with a template, and describe how to build on it, but the SurfHelper example by Xiaolin Zhang on CodeProject was a great help. Having a complete example I could build and run gave me the information I needed to build my own.

To build the BHO, download and unzip the project, open the TinyBHO.vcproj file in Visual Studio Express, and hit f7 to build it. The build process should compile, and then add the DLL to the registry, and the next time you run Internet Explorer, a message box should appear whenever a document’s loaded. This provides all the boilerplate to hook into browser events, and provides a stub that you can insert your  own document processing logic into.

If we want to build on this, we’ll need your own GUIDs for the interface and type library. Use a tool like uuidgen or one of the online versions to generate three IDs, and then load the TinyBHO.idl file into VS’s editor. Replace the first ID that begins with 03147ee0 with our first ID, it should only appear once near the top of the .idl file.

The second ID we need to replace starts with 7cd37f36, and appears once in both TinyBHO.idl and TinyBHO.rgs. The third begins with 00e71626, and defines the main class’s ID. It occurs four times in TinyBHO.rgs and once in TinyBHO.idl. I’d also recommend customizing the class and file names, but that shouldn’t be strictly necessary for the BHO to coexist with others the way that unique GUIDs are.

Now we’ve got the BHO personalized, we can start looking at CTinyBHOClass::Invoke(), the function that gets called whenever a browser event, such as a page load or key press, occurs. Right now, all it does is catch document load events, and pulls the document interface pointer from the event, and shuts down the event handler when the browser quits. For PeteSearch, I’ll be focused on key presses and document loads, since that’s where it does most of its work, but there’s a wide range of other events to grab if you want to implement something else, like a popup blocker. In Firefox terms, it’s pretty close to having called AddEventListener() on the browser, except that you get called on all events and have to check yourself to figure out what happened.

The trickiest thing about getting this working was that all the ATL COM handling code is such a black box. The registration part of the build phase was failing, and I eventually tracked it down to a misspelt resource name, but there was no logging or other useful debug information available, I just had to spend a long time inspecting the code looking for anything suspicious!

More posts on porting Firefox add-ons to IE

Porting Firefox extensions to Internet Explorer

Explorerlogo
Compared to writing Firefox extensions, there isn’t a whole lot of information about writing add-ons for Internet Explorer, and almost nothing about porting from Firefox to IE.  I’m converting PeteSearch over, and I’ll be sharing my notes on the process here.

There’s no way of writing a plugin using an interpreted environment like Firefox’s Javascript, instead you have to create a binary library in a DLL. These are known as Browser Helper Objects, and give you the ability to catch browser events like page loads, and work with the DOM, but don’t offer any UI access for things like toolbars. Hopefully PeteSearch doesn’t use anything outside of a BHO’s domain, since it works almost entirely on processing and altering existing pages.

Since we’ll need to compile a DLL, the first thing we need is a compiler. Luckily Microsoft has released a cutdown version of Visual Studio for free, though there are some limitations that I’ll describe. I chose to use C++ to create the DLL, because I know it well and it’s what most of the BHO examples use, but you could also use C# or Visual Basic.

After we’ve downloaded the compiler there’s some other components we need before we can build a DLL. Since the BHO is a win32 DLL, we need the platform SDK. To get it working with the express edition of Visual Studio, we’ll also need to muck about with some config files, as described in this MSDN article.

For dealing with COM code, ATL/WTL is very useful, so I’d also recommend downloading the open source WTL from MS, and following the steps in this CodeProject article.

Now we’re ready to compile a DLL, the best place to start is with some example code. There’s two good articles from Microsoft that describe writing BHOs, one from 1999 and a more recent 2006 version. Unfortunately they both start off using a wizard to create the base project, and the ATL wizard isn’t available for the express edition! They don’t have any complete examples downloadable, which is a bit puzzling. I’ve contacted one of the authors, Tony Schreiner, in the hope he can provide a complete example. If not, I may finagle a base ATL project from someone with the full studio package, then I’d be able to build it myself following the rest of the steps. Of course, I could purchase the standard studio for $300, but it seems worthwhile to work out a free way to write IE extensions.

René Nyffenegger has some example code that we can use instead, though I found I had to do some tweaking to get it to compile, such as defining a couple of logging macros, and sorting out some string literals that were expected to be wide but weren’t.

Now we should have the starting point for a BHO. I’ll cover the next steps, and show some example code, in a following article.

Edit – I’m trying this on a new Vista laptop, and there’s a few extra steps I noticed:

  • Folder permissions for editing the ATL headers in the SDK are tricky in Vista. You need to make yourself the owner, and only after that sort out the permissions.
  • You need to add the mfc include folder from the SDK too. I may have just forgotten this before.
  • The registry-setting part of the build process doesn’t work. I’ll cover fixing that in my description of writing an installer, and update here once that’s done.

Edit – To fix the registry build stage, you can run Visual C++ as administrator by right-clicking on the app and choosing that option from the menu.

More posts on porting Firefox add-ons to IE