I was chasing up the cause of the bug Socket based fetches can fail, and I tracked it down to an error in the way I was forming my initial HTTP GET request. I was putting the full URL as the argument on the first line, rather than just the path part. Interestingly, most servers accepted this, but typepad.com didn’t. I’ve updated the applet to do the right thing in this case, and also added some logging code that helped me track down the problem.

Something else that was really useful was the LiveHTTPHeader addon for Firefox. It gives you a full dump of the header information that’s passed back and forth between Firefox and the hosts it contacts. I’ve tried to absorb the RFC, but nothing beats being able to see a working example. I was also impressed by how easy it was to install, it makes me wonder how hard it would be to create a Firefox plugin version of SearchMash in the future.

I also fixed another bug that would sometimes prevent pages from loading, and removed the seperate status request check I did before, that used URLConnection. Now I just make a single socket connection for every page, that should be faster and easier to maintain.