WinInet Thread Crashes

Crash
In my previous post on XMLHttpRequest, I explained why I chose to implement my http request class using the legacy WinInet library, rather than one of the more modern alternatives. Last week, I was able to get the DOM parsing part of PeteSearch running, and so I was able to start stress-testing my class.

I was getting a lot of crashes, and after finding and eliminating a couple of my own bugs, and paring down the code as much as I could I was left with some minimal code that still crashed, often in the wininet dll!

After digging around some more on the MSDN site, and reading between the lines, I realized this seemed to be a known problem, and there probably wasn’t much I could do about it. They recommend using WinHttp for "server applications", and there seem to be a lot of hard-coded limits in WinInet, such as how many concurrent accesses to the same server you can have.

As far as I could tell, the crashes all occurred when I had a lot of requests being processed on different threads at the same time.

I replaced WinInet in my XMLHttpRequest implementation with WinHttp, and the crashes went away, which is strong evidence that it wasn’t my code doing something whacky. I’ll be posting the updated code in my next article.

More posts on porting Firefox add-ons to IE

Firefox Shortcuts

Fire

More Search Tips…

Here’s some ways to speed up searching and browsing by using shortcuts for common tasks.

  • Press F7 to turn on Caret Browsing. You’ll now see a blinking cursor
    when you click on a page, which you can move around with the arrow
    keys. What’s really handy is that when you move the cursor to a link,
    you can open it by pressing Return, which allows you to move rapidly
    through web pages using only the keyboard. Control+Left/Right in Caret Browsing mode will move by whole words. It doesn’t always work well on pages with complex layouts.
  • Hold Command/Control when pressing Return after typing a word in the location bar and it will add www and .com to the start and end. For example, typing ibm and Command/Control+Return will take you to http://www.ibm.com. Shift+Return does the same, but adds .net, and Command/Control+Shift+Return adds .org.
  • If you select some text, you can instantly search for it on Google using the right-click menu.
  • Control/Command+L will let you type a URL into the location bar, without having to move the mouse and click on it.
  • Control/Command+K lets you type into the search box without mousing over to it.

Firefox Search Keywords

Keys
Firefox has a powerful feature that lets you search specialized sites from the location bar, called Search Keywords. Normally, when you type something in the location bar, and it isn’t recognized as an address, Firefox will open up a Google search page for the string you typed. The idea is you can type wiki optical flow and you’ll be directed to wikipedia’s internal search results for optical flow.

In theory, all you need to do to set up custom keywords for the sites you visit most often is navigate to the main page, right-click on the search box and choose Add a Keyword for this Search from the menu.

In practice, this GUI method doesn’t work on a lot of sites, since Firefox seems to have trouble with relative URLs in search forms. So doing this on wikipedia will send you to  http://wiki/Special:Search?search=optical+flow rather than the full address!

There’s some handy pre-packaged collections you can download, such as this one from Lifehacker, or this massive compendium, but if you’re comfortable building up URLs, I’ll show how you can create your own.

We start by doing a search on the site we want to add, and creating a bookmark of that page. In this case, we go to http://en.wikipedia.org and enter optical flow in the search box on the left, and press the Search button (not Go, since that’s the equrivalent of I Feel Lucky on Google). That takes us to this page:

http://en.wikipedia.org/wiki/Special:Search?search=optical+flow&fulltext=Search

Pressing Control+D, or choosing Bookmarks->Bookmark This Page from the main menu adds this to our bookmarks.

Now, bring up the Bookmark Organizer by picking Bookmarks->Organize Bookmarks... from the menu. Find the bookmark you just created in the list, select it, and press Control+I or click the Properties icon in the top bar.

We should now see a dialog box like this:
Bookmarkbox1

In the location text box, scroll until you see the words you entered as a search, separated by a plus sign:
Bookmarkbox2

This is the part we’ll want to replace with the terms after the keyword in the location bar, so delete just those words and any plus’s separating them, making sure not to delete past the words. We’ll delete up to the end of the URL, or to the next & sign, whichever comes first.

Once the specific terms have been removed, we can put in the placeholder for the terms we type after the keyword, %s
Bookmarkbox3

Finally, we chose a keyword to type for this search, in our case wiki, and click OK.
Bookmarkbox4

Now, we can test out the keyword by typing wiki gaussian blur in the location bar, and we should see a new page of search results.

This is a real time-saver if you set it up for reference sites you commonly search on, sometimes you can’t beat the command-line style for speed of access!

More Search Tips…

Tips on Focused Searching

Handglow
Knowing what kind of search you’re doing will help you focus on the best strategy for finding what you want.

  • Page Hunt

You’re looking for a page you’ve looked at before. Try to remember specific, distinctive details from the page. These can be unusual phrases, something in the title or the URL, or specific names. Then use operators to  narrow your search to just that page, for example putting a remembered phrase in quotes, or using intitle with part of the title.

  • Fishing Expedition

When you want to learn more about a subject, without a specific question to answer. The best place to start is on a reference site with authority, like wikipedia for general knowledge, or MSDN for windows development. Use the site operator to restrict your search to those sites, and the ‘|’ or operator to widen your search to cover many common terms that might be used in a introductory article.

  • Search for Answers

You have an exact question you need to know the answer to. Think of how the answer is likely to be phrased, and then do a quoted search for that phrase with the ‘*’ wildcard symbol where the answer should be. For example "* was the first king of england". Look for places where such questions are commonly asked and answered, such as FAQs, specialist mailing lists and forums, usenet, and answer sites.

  • Detective Work

There’s a person or company you need more information on. Try to gather as much identifying information as you can before you search. If they have a website, you can use a whois tool to get real-world contact information. Search for a name in quotes, a website address, their username, email address or any other identifying features you can find. Do a search restricted to their domain using the site operator, to find any information there. Look through blogs using something like technorati, to find out what people are saying about the person or company. Search press releases for a company name. But remember, the internet is full of incorrect information, so make sure you use a reliable second-source before you take any serious action on it.

More Search Tips…

Using Google’s Advanced Search

Brush
If you want to find something, and just using simple terms isn’t getting you there, you should try Google’s advanced search There’s a graphical interface, but to be really productive, learn the text commands you can type inside the normal search box. Commands are a word followed by a ‘:’ character, with no space before the colon, followed immediately by the setting for the command, for example:
site:petewarden.typepad.com google

  1. Site. This is the operator I use the most, because Google is better than most site’s built-in search. I’ll often pick a documentation base address and use it like site:msdn.microsoft.com xml. One lesser-known use is restricting your search to a country or other domain, all you have to do is put the top-level domain after the site operator. Here’s how you’d search for pages on opticl flow from only US education sites; site:edu optical flow.
  2. Cache. If you put a web-page address after this, you’ll see the version of the page that Google picked up when it last indexed the site. It’s the same view you see below the main link as "Cache" in the search results, and is very handy for sites that are temporarily down. They may still try to load images or other elements that stall page loading, so if you want to see just the text, add &strip=1 to the end of the Google search URL.
  3. Link. This is so nearly so cool. It’s supposed to show all pages that link to a certain page. In practice, it shows some, but not all, and there’s speculation that’s because showing them all would give too much information about Google’s database. It’s still useful if you’ve got a page you like, and want to find others in the same area, just don’t expect it to show all links.
  4. Synonyms-‘~’. Putting the tilde character, ‘~’, in front of a word will expand the search to include words that google thinks mean the same thing. Google does this automatically in a lot of cases now, and it only works for common words, for example trying ~trainers (which is a British way of saying sneakers) doesn’t bring up alternate shoe sites, unlike ~sneakers, which is recognized.
  5. Filetype. This lets you restrict your search to addresses that end with the extension you specify. Google can only understand thirteen types of files though, so doing a search for filetype:mp3 won’t give you any music results!
  6. Exclude-‘-‘. The minus sign, ‘-‘, in front of a search word, means don’t show any pages that include that word. This is really useful if you’re getting a lot of results that are from an area you’re not interested in, you can pick a term that’s common to that area, and hopefully most of those pages will contain it, and they’ll disappear from your results. For example, if you were looking for gateway computers, but meant the network technical term rather than the company, you could add -buy -inc to exclude a lot of the commercial pages, and find the ones you want more easily.
  7. Or-‘|’. Using the vertical bar symbol, ‘|’, between two terms, will find pages that contain either of the terms. This is mostly useful if you’re getting too few results and want to expand your search to similar words.
  8. Wildcard-‘*’. This is only useful as part of a quoted search, where you’re looking for an exact phrase, but there’s a section you don’t know. For example "senator * spoke against the war" will find all the pages that mention particular senators.
  9. Related. This will find any pages that are similar to the URL you enter, using Google’s criteria for similarity. It can sometimes be useful if you want to get more search results based on one good page you’ve found.
  10. Intitle. If you’re getting too many results, and you really want to restrict them to just those with a given word in the title, this allows you to limit your search to just those.

For a really good in-depth guide to searching with Google, I highly recommend http://www.googleguide.com, it contains really detailed tutorials to improve your searching.

More Search Tips…

XMLHttpRequest in C++ on Windows Example

Swirly
In the first three parts I covered how to get a simple IE extension built. For PeteSearch I need to be able to fetch web pages, so the next step is to figure out how to do that in Internet Explorer.

Firefox lets extensions use XMLHTTPRequest objects to fetch pages. It’s quite lovely; well documented and tested since it’s the basis of most AJAX sites, and with an easy-to-use interface. The first thing I looked for was an IE equivalent.

There’s a COM interface called IXMLHTTPRequest that looked very promising, with almost the same interface, but it turned out to involve some very gnarly code to implement the asynchronous callback in C++. It was also tough to find a simple example that didn’t involve a lot of ATL and MFC cruft, and it involved using a pretty recent copy of the MSXML DLL, and there were multiple different versions. All-in-all, I ruled it out because it was just sucking up too much time, and I dreaded the maintenance involved in using something so complex.

There’s also the newer IWinHttpRequest object,  but that’s only available on XP, 2000 and NT4.0, and seems far enough off the beaten track that there’s no much non-MS documentation on it.

I finally settled on a really old and simple library, WinINet. It’s a C-style API, and lower-level than XMLHttpRequest, and is a bit old-fashioned with some situations that require CPU polling, but it offers a full set of HTTP handling functions. It’s also been around since 1996, so it’s everywhere, and there’s lots of examples out on the web. Since I liked the XMLHttpRequest interface, I decided to write my own C++ class implementing the same methods using WinINet under the hood.

Here’s the code I came up with
. It implements a class called CPeteHttpRequest that has the classic XMLHttpRequest interface, with a simple callback API for async access. I’m making it freely available for any commercial or non-commercial use, and I’ll cover my experiences using it with PeteSearch in a later article.

Edit – It turns out that WinInet is actually very prone to crashing when used heavily in a multi-threaded app. You should use my WinHttp based version of this class instead.

More posts on porting Firefox add-ons to IE

Shorten your Google and Amazon URLs

Scissors
If you want to email or post a link to a Google search, it’s a pain, because they’re pretty long. Here’s an example:
http://www.google.com/search?q=some+terms+here&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a
But most of that isn’t necessary, here’s what it could look like:
http://google.com/search?q=some+terms+here

To create your own version, just take the original search URL, look for the part after "search?" that starts with "q=". Keep everything up to the next & symbol, and delete everything else. The www starting part can also be safely removed to save a bit more space.

You can also create your own simple queries by starting them with starting with http://google.com/search?q= and adding your terms, separating them with + signs.

Amazon is another offender; here’s a full URL for a book:
http://www.amazon.com/Norstrilia-Cordwainer-Smith/dp/1596872578/ref=pd_bbs_sr_1/102-4304635-3614560?ie=UTF8&s=books&qid=1181107054&sr=8-1
and here’s all you need to get to the same product page:
http://amazon.com/dp/1596872578

The important part here is the dp/ and the following number, you can delete everything else. This works with everything from books to computers on Amazon.

Once you get the hang of URL formats, it’s easy to try on your favorite sites. Try to spot the important parts, and experiment with deleting the other bits. The URL components are often separated by & signs, and you can usually delete the www part to save a few more characters.

 

Who needs tinyurl.com? 🙂

More Search Tips…

Firefox in-page search tips

Keys
After you’ve found a promising page in your search results, you’ll need to find the information you want on that page. Firefox has some great short-cuts for that end of the job!

  1. Use the / key quick find. If you press the ‘/’ key on any page, and start typing a word, Firefox will take you directly to the first occurrence of that word as you type!
  2. Press F3 to go to the next result. The F3 is the standard ‘go to next’ key on Windows, but handily it also works on OS X, since the usual Mac alternative is the less convenient Command+G. Hold down shift to move backwards.
  3. Use the ‘ key link finder. If you’re looking for a link, and know part of the address, pressing the ‘ key works just like the text quick find, but goes to the first link containing the words you type.
  4. Highlight all occurrences. Sometimes it’s easier to highlight all the places in the page a term exists, and then scroll through and do an eyeball search looking for the right one. There isn’t an instant key combo for this, but if you press Ctrl/Command+F to bring up the search bar, you can select the ‘Highlight all’ checkbox.
  5. Work on pages in parallel. Rather than sitting waiting for pages to load, try opening up several promising ones in different tabs, by holding down Command or Control when you click on the links. By the time you’ve searched through one, another should be loaded. PeteSearch lets you open up all ten results in tabs with Ctrl+’=’.
  6. Move through those tabs using the keyboard. Press Ctrl/Command+Page Up/Down to go to the next or previous tab.
  7. Case-sensitive searching. If you’re looking for a coding term you know has to be all upper case, or a name that starts with a capital, click on the ‘Case sensitive’ checkbox in the find bar. Alex from Lifespy has a neat tip for turning it on by default, and MozillaLinks has this hack to get the checkbox in the type-ahead find box too.
  8. Select some text to restrict your search to the bottom of a page. This one may be a bug, but it’s useful: if you select some text half-way down a page before you search, then only results at or below that selection are found in the type-ahead find. This can be very useful for sites like experts-exchange, that pad the top of their pages out with a lot of ads.
  9. Tweak the typeahead find options. Mozilla.org shows how to customize a couple of options that can be handy, whether the link finding matches any part of the address or just the start, and how much of a pause between letters you can have before it decides you’re done typing.
  10. Give PeteSearch a try! It automatically highlights all of the search terms in the split-screen preview of a page, no typing needed, and Control+Down Arrow moves you through them.

More Search Tips…

More Search Tips

Webworkerdailylogo
WebWorkerDaily is a full of great ideas for improving your life online, and their recent post on better searching with Firefox inspired me to put up my own tips, which were then mentioned by Anne. My goal with PeteSearch is to figure out ways to speed up searching for power users. I want to see the same time-saving tricks of hot-keys and smart previews for searching that modern desktop apps all deliver. Let me know your ideas on improving it, and I’ll see what I can do!

For more general tips on searching, Danny Sullivan has a excellent article covering the basics; it’s old, but it holds up well.

Here’s some more background on the process I’ve developed for searching. As the builders of Deep Thought discovered, the tricky bit is asking the right questions!

Filter out the noise
My process for this usually involves trying different combinations of search terms, and not clicking on any links until most of the summaries below the links look relevant.

Search to find search terms
If you’ve run out of terms to try, and you’re still not getting good results, try going to a more general link that covers the knowledge area you’re looking in, and read up on it, looking out for any words that you might narrow your search.

For example, I was trying to work out how to implement an HTTP request in C++, and using "http sample code c++" brought up a lot of results that didn’t help. I noticed a few of them mention wininet in passing though, and adding that to my search brought me to the AsyncDemo sample code, which was exactly what I needed.

Use specialized search sites
These vary for different areas, you’ll need to find ones for yours, but here’s some examples of ones I use:
CiteSeer – Comprehensive coverage of academic papers about anything imaginable
Google CodeSearch – Searches lots of open-source code, great for finding examples how to use a function
Technorati – Instant blog search, though I’m less keen on their redesign

If you’re a researcher like me, I’d recommend keeping an eye on resourceshelf.com, since it’s a great way of finding specialized sites as they come out.

Skim, skim, skim!
If you’re still stuck with too many results, start picking some at random, and eyeball through them as quickly as you can. Sooner or later you’ll see something that will give you a lead on doing a more focused search, even if it’s not quite the right result. Speed-reading really helps here. Do a Ctrl/Cmd-F in Firefox to find one of the terms (or Shift+Down in PeteSearch to find any of them).

Give something back
If you just completed a really massive, obscure and time-consuming search, share your triumph with the world by putting up a little article linking to what you found, along with a quick description. That way, the next person searching may avoid losing a small chunk of their life to it too! My approach is to add technical notes on my personal site.

More Search Tips…