Try a secret new search engine

Mqlogo

Well, I’m not sure about secret, but it sure is mysterious. http://alpha.managedq.com/ showed up in my visitor logs, and visiting the site it looks like rather a nifty visual search interface. It’s got thumbnails of the top results, and automatically generated keywords sorted by type down the left side:

Mqscreenshot

The interesting part is that most of the site is returning 404 or authorization errors, which makes me wonder if they might still be in stealth mode? Unfortunately email messages to their public inquiries@managedq.com address bounce, they’ve got a private domain registration so I can get any contact details from that, and Google searches don’t get me any more information, so I can’t check with them before mentioning it here.

They’re using snap for the thumbnails, and I’m not sure how they’re pulling out the tags. The keywords definitely look automatically generated, rather than user driven. I’d love to know more about their work, so if anyone has more details or a way to contact them, email me or add a comment.

All-new PeteSearch

EyeI’ve just put up a new version of the PeteSearch Firefox plugin. Highlights include Technorati support, a better summary popup, control+equals will open all the results in new tabs, and the preview is now shown in a split-screen view rather than a popup window.

These improvements were inspired by some great feedback, including Ionut from Google Operating System and Phillipp from Google Blogoscoped, thanks for all your help!  One question that I’m planning to answer with a short video demo is why it second-guesses Google and checks all the pages in the results itself, but here’s a summary:

I’m a graphics researcher, so I have to do a lot of exotic queries. I got very frustrated one day after clicking on a bunch of results, and on each one waiting for it to load, only to discover that it didn’t have the term I was looking for, or the page didn’t exist at all. Computers are meant to speed up repetitive, time-consuming work like that, so I decided to write a tool to automate the process I was going through.

Here’s an example of a query that has a lot of bad results:
http://www.google.com/search?q=nsIWebBrowserFind+regexp

And here’s the top ten results I get:

customizing mozilla
regexp term missing

Mozilla Firefox 1.5 files and source code
Exists, and has all the terms

Search
Gives a 404 error when clicked on

mozilla-devel-1.6-12.5.100mdk.ia64 RPM
Gives a 404 error when clicked on

seamonkey-debug-1.1.1-2mdv2007.1.i586 RPM
Gives a 404 error when clicked on

r976 – in changes/jat/mozilla1.7.13/tools/sdk/mozilla-1.7.13 …
Exists, and has all the terms

Ž­è @ d – ¨ I \ ( ] Œ ^ ß BD < , Y Ä ,$ , : G > d ã † Ž ˜ y Ô Ó h …
All terms found –

/usr/share/comps/sparc/.discinfo /usr/share/comps/sparc/comps.xml …
Exists, but doesn’t contain either of the terms

libPropList-devel-0.10.1-371.i586.rpm …
Exists, and has both terms

bin/cat utils/coreutils bin/chgrp utils/coreutils bin/chmod utils …
Exists, and has both terms

So four out of the top ten results are bad, and without PeteSearch I’d have to waste time clicking on them, waiting for them to load or fail, and then search through for the terms I wanted. With my extension, I can focus on the good results.

If you are doing a lot of complex or obscure searches, you should try PeteSearch, it’s designed to save you time.

Firefox Search Addon

Iconbig_2I’ve finally been able to spend some more time on my quest to speed up searching. I came to the painful conclusion that the cross-platform, cross-browser, no-installation advantages of using a Java applet were outweighed by the long startup times and scary security messages.

Instead, I’ve written a Firefox extension that does everything SearchMash did, and more. PeteSearch is now live at http://petesearch.com/

It’s a low-key release at the moment, because I ran into a Firefox bug that caused getElementById to crash the browser occasionally. It turned out that this was one of the top reported crashers in Firefox 2, but nobody had a good repro case or fix.

I rolled up my sleeves, checked out the Firefox source, and after some logging, debugging and head-scratching, figured out what was going wrong. Jonas Sicking was able to give me some pointers on how to fix it, and I’ve got a patch he’s reviewing at the moment.

Since it’s a high-priority bug, I believe my fix will be in the next point release of Firefox, 2.0.4, which I hope will be out in a few weeks. So I’m going to wait for that fix before I really push it.

There’s some big improvements from SearchMash:

  • Hot-keys – You can use control+left/right arrow keys to move through the results, and up and down to move through the pages of results
  • Text preview – You can get a plain-text summary of each result, with every occurrence of a search term shown in context. It’s like the summary below the link in Google, but for every time the term occurs.
  • Internationalization – 55 national google sites are supported
  • Ask.com and Microsoft Live are supported as well as Google

SearchMash in IE7

Eye5

As I mentioned in Top Three Usability Problems, the Firefox 1.x toolbar plugin has been very popular, and I really wanted to add one for Internet Explorer too. Luckily, IE7 now supports the OpenSearch standard for search plugins, and so does Firefox 2.0. I had to write an XML description of SearchMash, then I was just able to add a couple of javascript calls and a link in the head, and SearchMash now shows up in IE’s toolbar when you click on the link in the preview frame.

It even shows up temporarily in the list if you visit http://mashproxy.com/search, thanks to some autodiscovery magic.

Out of the three usability improvements I wanted to make, that was the easiest. Getting the history working, and quicker load times are going to be harder. Work is very busy right now, and this is my ‘relax after a long day’ project, so I’ll see how much progress I make.

Top three usability problems

Hourglass
My goal for SearchMash is to get people using it as their regular search page, the way I do, by helping them find useful results more quickly, and find more of them. My firm belief is that searching can be improved, and that people will use a better search interface, and this project is a way of proving that.

I want to convert more of my visitors into regular users, and I believe there are three crucial problems I need to solve to do that:

Loading Speed

Especially on PC’s, the Java engine can take several seconds to load up, which adds a big delay to seeing your search results. There’s nothing I can do to speed up that startup time, but I could try and display something useful while it is loading, such as the plain Google search results for that query. I’ve filed a feature request with the Sourceforge system to track my work on this; Display useful information during Java loading.

The Back Button Doesn’t Work

Since I’m writing HTM directly into existing frames, rather than changing their location, the browser’s history doesn’t get updated when you got to a new page of search results within the site, and so hitting ‘back’ takes you to the page before SearchMash, rather than going through the pages of searh results. This is a surprisingly tricky thing to get working because different browsers behave differently, and you have to use subtle tricks like hidden iframe location changes to put changes into the history without actually reloading the page. This is in Sourceforge as Make page changes work with history.

Toolbar Search With Internet Explorer

The Firefox search extension is one of the most popular ways to access SearchMash. IE7 now supports a different standard, OpenSearch, but that allows similar functionality, with a few nice extras. One cool addition is that you can advertise your search engine through a link in the of your page’s HTML, and it will automatically show up in the search box for users to try out. I’ve got Write search plugin for IE 7 tracking my work on this one.

Whois

Eye4
There’s now a ‘Whois’ link for all results, next to the similar pages and cache links after the summary. This uses the internet whois protocol to try to get information about the owner of the domain the page is on.

To use the new link, just click on it and the results of the whois search should show up in the preview pane. There’s usually a bunch of boilerplate standard information from the server holding the information, followed by the information about who the domain is actually registered to.

Since the whois servers hold the owners names, emails and phone numbers if they’ve not gone with a private listing, I had to be careful about how I offered this within SearchMash. In particular, I didn’t want to overload the whois servers providing the information, so I’m careful to only put in requests when you click on a link, rather than pre-fetching as I do for web pages. This is basic courtesy to the owners of the servers, and also complies with their terms of use designed to prevent robots from harvesting addresses for spam.

Thanks John and Melanie!

Thanks_1

On John Batelle’s SearchBlog (the inspiration for SearchMash’s name), Melanie Colburn posted a roundup of recent SearchMob posts, including SearchMash with a “This is pretty cool” comment.

Since I enjoy his blog, I’d sent John an email out of the blue last week, telling him about SearchMash. He replied back almost immediately, directing me to his new SearchMob site, and suggesting I put a post up there.

John didn’t have the take the time to reply to my random email. I think it’s great that he did, and even nicer that Melanie linked from the front page. It may not seem like a big deal, but it really made my day.

Last modified, better preview, and ask

Clock_1
The status line now shows when a site was last modified, which can be useful to know if you’re searching for time-sensitive information. Not all sites return the header I use to find that out though.

I’d noticed there were some sites still showing up with missing pictures or bad formatting in the preview. I spent some time debugging some of the problem cases, and realized there were some bugs in the code I was using to insert the tag that resolves relative URLs. I put in fixes for those problems, which made a lot of CSS based sites appear much better.
It also allowed some scripts to run that hadn’t before, so I then had to fix the bugs in my script blocking that let those through. The end result is that preview works a lot more reliably.

I’ve also been experimenting with offering ask.com as an alternative to Google. I managed to get something working pretty quickly, but hit problems with their use of Javascript in the results pages. It works fine in Safari and Internet Explorer, but Firefox throws a lot of errors. I’ll be returning to debug that some more when I have a chance, for now I’ve left the code in, but I’m not providing any obvious links to it. If you want to experiment, use http://mashproxy.com/search?ask to invoke it.

Google launches their own SearchMash…

Searchmash
… and I was the first person to discover it! On Monday morning I did my usual Google search for ‘searchmash’ to see who was linking to me, and I was surprised to see http://www.searchmash.com come up as the first result. I looked at the site, discovered from whois and the privacy policy that Google were running it, though trying to be low-key about it.

I posted to Google Blogoscoped about my discovery, and it mushroomed from there. It’s fun being part of a news story, and I’m glad to see Google trying some really different looks to their interface.

I’ve also got a lot of traffic from the news, it’s been great to have so many people discover my SearchMash, I really couldn’t have paid for that sort of publicity. It’s interesting to compare their approach to mine too; they’re providing a new interface but with the same information, whereas I’ve stuck pretty closely to their original interface, but try to offer some extra information on top.

Added a firefox search plugin

Firefox

I just finished a plugin for Firefox, so you can now add SearchMash to your search toolbar, just like the standard Yahoo, Google and EBay ones. To install it, go to http://mashproxy.com/search/previewframe.html, and click on the “Add SearchMash to Firefox’s toolbar” link.

Once it’s there, you can do searches with SearchMash by clicking on the top right toolbar, that shows a ‘G’ and does Google searches by default, and select ‘S’ for SearchMash instead.