Vlad also asked how a BHO can be added to the registry, outside of the compiler? This is a question I’m wrestling with as I write the installer, so here’s a dump of my current understanding.
There’s three purposes to registration; telling IE that there’s a BHO with a certain ID it should load, telling the system where the DLL is that contains that ID, and what code actually implements the interface we’ve declared.
The first part is covered by the .rgs file in the project. It handles setting up the registry so that our BHO is added to the list of ones to load. If you look at the file, it contains a description of what our DLL contains, including a UUID, and also adds that UUID to the list of BHOs for IE to load. It’s added to the DLL as a resource, and we reference that in our implementation class, by calling the macro DECLARE_REGISTRY_RESOURCEID(<resource ID>) in our class declaration. This adds an UpdateRegistry() member function to our class, that calls an ATL function that parses the rgs script, and adds the keys to the registry.
Telling the system where the DLL lives on disk is handled by calling code inside the DLL itself, in a process called ‘self-registration’. There’s a custom build step that calls regsvr32 /s /c <your dll>. This in turn calls the DllRegisterServer() function inside your dll, which in our case calls a standard ATL _Module.RegisterServer() call. We’ve set up an ‘object map’ containing our UUID, and the actual class that implements that interface, and the ATL call takes that and adds the right information to the registry. This call also adds the DLL location to the registry, and calls the UpdateRegistry() function we set up in our class, that adds the .rgs entries to the registry.
If you want to make this happen outside of Visual Studio, you can call regsvr32 yourself from the command line. Procedural installers like NSIS have script commands that do the same, like RegisterDLL.
As always, there’s a catch. WIX/MSI is a declarative installer, and MS strongly recommends against calling procedural DLL code as part of the process, since that’s a black box to the installation system, and so will be a lot harder to roll back. Instead, I’m going to have to capture a static description of all the registry changes that calling regsvr32 on my DLL causes, and add that to the installation script. Luckily, there’s a tool called tallow that looks like it may help. I shall let you know how I get on!