How to POST an HTTPRequest in C#

Tuningfork
Photo by Daddyoh

I’ve been learning C# over the last few weeks, and I’m very impressed. A lot like the Objective C/Cocoa combination on the Mac, it’s focused on making GUI development very fast and easy. The Visual Studio integration is a lot slicker than Apple’s Interface Builder, without the complex, obscure and manual wiring together of components and code that IB requires. I still found myself hunting around in endless property windows for the right member or event, but touches like being able to double-click on an event name and have VS insert an empty handler function in your class really sped up my development. Combining that ease-of-use with Add-In Express’s environment for building Office and IE plugins has helped me make great progress.

One of the things I need to do a lot is communicate with a remote web server. The  XMLHttpRequest JavaScript interface has become the standard for web APIs, and C# has its own version, WebRequest/HttpRequest. I’ve included an example class below that implements a synchronous POST request on top of this. To use it in your own project you’ll need to add System.Web as an external reference if you don’t already have it. It’s also possible to use HttpRequest asynchronously, but I’ve left that out of this code. The interface takes an array map of variable names and values to pass as the POST  variables, and the current error logging is through a MessageBox alert, which you’ll want to change in production!

Download PeteXMLHttpRequest.cs

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Windows.Forms;
using System.Web;

namespace MailanaOutlook
{   
    class PeteXMLHttpRequest
    {
        public static string dictionaryToPostString(Dictionary<string, string> postVariables)
        {
            string postString = "";
            foreach (KeyValuePair<string, string> pair in postVariables)
            {
                postString += HttpUtility.UrlEncode(pair.Key) + "=" +
                    HttpUtility.UrlEncode(pair.Value) + "&";
            }

            return postString;
        }

        public static Dictionary<string, string> postStringToDictionary(string postString)
        {
            char[] delimiters = { ‘&’ };
            string[] postPairs = postString.Split(delimiters);

            Dictionary<string, string> postVariables = new Dictionary<string, string>();
            foreach (string pair in postPairs)
            {
                char[] keyDelimiters = { ‘=’ };
                string[] keyAndValue = pair.Split(keyDelimiters);
                if (keyAndValue.Length > 1)
                {
                    postVariables.Add(HttpUtility.UrlDecode(keyAndValue[0]),
                        HttpUtility.UrlDecode(keyAndValue[1]));
                }
            }

            return postVariables;
        }

        public static string postSynchronous(string url, Dictionary<string, string> postVariables)
        {
            string result = null;
            try
            {
                string postString = dictionaryToPostString(postVariables);
                byte[] postBytes = Encoding.ASCII.GetBytes(postString);

                HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
                webRequest.Method = "POST";
                webRequest.ContentType = "application/x-www-form-urlencoded";
                webRequest.ContentLength = postBytes.Length;

                Stream postStream = webRequest.GetRequestStream();
                postStream.Write(postBytes, 0, postBytes.Length);
                postStream.Close();

                HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();

                Console.WriteLine(webResponse.StatusCode);
                Console.WriteLine(webResponse.Server);

                Stream responseStream = webResponse.GetResponseStream();
                StreamReader responseStreamReader = new StreamReader(responseStream);
                result = responseStreamReader.ReadToEnd();
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message);
            }
            return result;
        }

    }
}

Is Google stuck in the mud?

Stuckinthemud
Photo by misfitgirl

There’s been a round of blog sparring about the state of Google, with ReadWriteWeb’s Bernard Lunn claiming that they’re spreading themselves too thin, and Tim O’Reilly firing back a defense that they’re making strong strategic moves. I think they’re both wrong.

Microsoft terrified people in the 90’s because once they moved into a market they would carpet-bomb their way to dominance, using their deep pockets and distribution with the OS to crush competitors with feature-rich applications. Once they’d won, the products would often fester, but while there was still a race they would keep improving with every release. Look at Internet Explorer vs Netscape for the classic pattern. IE 3 was awful, IE 5 was pretty darn good for its time and killed Navigator. Then nothing much happened for years until Firefox came along and goaded MS into doing good things with versions 6 and 7.

Google’s moved into several major markets, come out with an excellent initial product, and then left it mostly unchanged. Look at Gmail, which I track obsessively because I think they did such a good job at launch, and I keep expecting them to follow up with some mind-blowing innovations. Instead the recent Labs section mostly contains UI tweaks. Google Documents is the same, a great launch but years later there’s still the same bugs and limitations I expected to get fixed quickly. Blogger is stagnating too, I could go on. In almost every major area Google has expanded into outside search, they’ve implemented or bought a great initial product, and then neglected it.

Unlike Bernard I don’t think it’s inherently a problem that they’re spread so widely. Microsoft in its prime showed that it’s possible to win a lot of independent markets simultaneously. Google just can’t seem to execute on their strategy. And unlike Tim, I think you can’t say they’ve got a strong strategy and ignore their operational track record. Strategy is useless without execution. Aiming to catalog the world’s information is a great strategy. Building an open-source smart phone OS is a clever move as part of that. The trouble is that the implementation isn’t looking great. Android already alienated a lot of developers before a single phone was been released, and they won’t be supporting ActiveSync amongst other things.

Google have produced amazing innovations for years. I’m just hoping they can keep their incredible momentum going, and capitalize on their initial releases by pushing the products forward. If they don’t, Zoho may not have millions of customers today, but there’s a lot fewer barriers to switching in the web world than there ever was on the desktop.

How I got things done at Apple

Brilliantgenius
Photo by luckyfish

One of my obsessions is finding experts. Before Apple, I’d never worked at a company larger than 30 people. Once there, I rapidly realized that almost any engineering problem our team was looking at had already been solved by somebody else in the company. For example, we’d developed a fantastic image comparison command line application to use in our automated testing. Over the 5 years I was there, I ran across several other internal groups who’d written tools to solve the exact same problem, each taking between 3 to 6 engineer-months.

This drove me crazy, we were all being paid by Steve so there’s no reason to waste that effort. Since I’m very curious (some might say nosey!) and I love chatting to people about what they’re working on, I made it my mission to get to know folks across the company and keep up to date with what their groups were doing. When I heard they were hitting an issue our group had already tackled, I could hook them up with the right people on our team, and do the same for my immediate colleagues with developers in the rest of Apple.

I felt like this was helping me get things done within the company, moving the projects I was involved in to completion, but it was also a frustrating process.

Totally manual

It was completely informal, relying on water-cooler chats and building personal relationships. There was no way of finding the experts in an area without relying on word-of-mouth.

No credit

When people did help me, it was very hard to give them credit with their management. They took time out of their day to help my projects meet their deadlines, but all they were assessed on was their own department’s success. There was no mechanism to capture how much they’d done for the overall company by helping across organizational boundaries. I ended up trying to finagle ad-hoc rewards like taking other teams out to dinner or mailing them cookies from Harry & David.

Part of the reason I left Apple is that I think there’s massive productivity gains to be made by giving people within large companies better tools for this. There’s already been some work by companies like Tacit or even Microsoft’s Knowledge Network protototype, but I’m convinced that an effective expertise location service can save massive amounts of time and money for large companies. There’s two main characteristics it will need that haven’t been achieved so far:

Automatic

It must have broad coverage without much effort. In practice this is likely to mean basing expertise tags on automated analysis of emails, with employees then tweaking the generated profile. Interestingly McKinsey actively use manual versions of profiles like these, but in most companies they won’t get created and updated without McKinsey’s strong emphasis on collaboration and expertise.

Rewarding

There has to be a way to reward participants in a meaningful way. There must be some tracking of ‘assists’ that somebody offers to the rest of the company through the system, in a form that can come up at annual reviews.

If you want to hear more about how I’m solving this with Mailana, join me at Defrag in a few weeks and I’ll give you a demo!

How to get a U720 EVDO wireless broadband USB modem working with OS X

Radiodial
Photo by Infra-Ken

I’m extremely conservative when it comes to changes to my development machines. As a geek my natural tendency is to install all the whizzy new software and hardware I come across, but that’s a massive time-suck away from actual development, since inevitably there’s driver and compatibility issues I end up debugging. I try to stay firmly on the well-beaten path, so others before me will have stepped on the land-mines.

Unfortunately I had to break my policy yesterday, and I paid for it. I’ve been contemplating getting a wireless EVDO device for my laptop, primarily so I have a backup to Wifi for my product demos. As a last resort I’ve got a canned movie I can show to customers and investors, but the online version is much more effective. The large databases (eg 5GB) I’m dealing with also make it tough to set up a local server on the same machine. I’d also like to stop paying the extortionate airport Wifi fees when I’m travelling.

I did my research, chose EVDOInfo as my vendor, since they have a good reputation for Mac support, a USB devicee from Novatel, the U720, that was widely used, and Sprint since I’m stuck with AT&T on my iPhone and wanted a different carrier for this. The ordering process was painless, I actually did it online through my iPhone whilst waiting to get seated for breakfast. It turned out I’d mis-keyed the expiry date on my credit card, but a very helpful salesman phoned me up and sorted out the mixup. They also preactivated the device for my account which was very handy.

Once the modem arrived, I unpacked the box and looked through the documentation. There wasn’t an obvious quick-start guide, so I inserted the provided CD and looked through the manual. It sounded like I should have the modem connected to a USB port when I ran the installer, so I plugged it into the side of my MacBook Pro. That brought up a prompt mentioning that a new network device had been found, and asking if I wanted to install the drivers? This automatic discovery sounded perfect, so I clicked through the OS’s native installation process (this wasn’t from the CD). That’s when the nightmare began.

After that process, large parts of the OS stopped working. I could no longer open most preference items, it would just hang indefinitely when I did. I also couldn’t run Software Update, or even su from the terminal. My first reaction was to do full backups of everything important on my machine, which ate up an hour or two. Then I booted from DVD and ran a full cycle of disk and permission repairs, which didn’t solve the problems. At that point I cut my losses and did a clean OS install on a different partition, taking about 2 hours including copying over all my backups, running all the software updates and reinstalling applications.

I tried the installation process again, this time running the CD SmartView package from Sprint. This ran successfully, but bringing up the new application and trying to connect failed. Checking in the system console I saw this message:

9/23/08 5:51:27 PM Sprint SmartView[192] SERIOUS WARNING : All 3 Connection Attempts Have Failed
9/23/08 5:51:57 PM [0x0-0x17017].com.roamingclient.cell.mac.roamingclient[283] /Users/hms/Projects/pctel.1.4/Modules/MoreSCF/MoreSCF.c:1640: failed assertion `(err != noErr) || (servicesDict == NULL) || (*serviceOrder == NULL) || (CFDictionaryGetCount(*servicesDict) == CFArrayGetCount(*serviceOrder))'

A lot of Google research later, I finally found a workaround here. Ignore the initial steps for removing and reinstalling the drivers he describes, the important part for me was:

– Go to Network in the Preferences
– Select the Novatel CDMA network device in the left pane
– Click on the Advanced button
– Go to the WWAN tab
– Choose Novatel as the vendor and CDMA as the model
– Click OK, then Apply
– Click on the Connect button

That was enough to give me a wireless broadband connection. I still can’t use the SmartView software which means I can’t see my monthly usage totals, but at least I can get online.

The case against transparency

Bubble
Photo by istargazer

Eric just posted on the advantages of transparency. I’m a fanatical believer in the power of more openness to transform businesses and my whole email startup is based on the idea that there’s hidden information in our emails that’s worth revealing. The problem is, within the tech community ‘open’ is a synonym for ‘good’, and that gets my contrarian antenna twitching. As Fred Wilson says, you don’t make money by doing the same thing as everyone else, so here’s a couple of examples of transparency gone wrong.

Misleading metrics

The mortgage industry moved from a centralized business model to one where different stages were handled by separate firms. Landing clients was handled by mortgage brokers, firms like Countrywide would then write the mortgage, but the money itself was provided by investors through securitization. In the old days a single firm would handle all of this in-house, which meant they had deep and immediate access to all the information about a borrower at every stage. To make the decentralized model work, an open and standardized way of categorizing the quality of the loan was developed. Statistics and measures to cover the credit history, income and collateral offered by the borrower were passed up the chain. These were then used by the agencies to rate the loans and split them into tranches of risk. It looked like a model of transparency, increasing the efficiency of a whole industry.

The problem was the metrics were systematically false. Brokers had massive financial incentives to inflate collateral house values through friendly assessors, and help borrowers inflate their income. Unlike the old single-firm approach, there was no real accountability for the true quality of the loans, they would still get their commission. The rating agencies relied on the broker data, and had similar incentives to grade the loans favorably.

Part of the reason we’re in this mess is that the appearance of transparency made everyone complacent. A manager of a team of sales people in the single firm model would get fired if her team were fudging originations. Her management would have a strong incentive to prevent lax lending standards because that would lose the firm money. There just wasn’t an accountability mechanism to go along with the new open model, and so the apparent transparency was a dangerous illusion.

Destroying the magic

Walter Bagehot said about royalty "The monarchy’s mystery is its life. We must not let in daylight upon magic." The same could apply to Apple. One of the distinctive elements of its culture is the obsessive secrecy. This isn’t just the usual bureaucratic urge to hide information, it’s a deliberate part of their marketing strategy. The impact of any announcement is so much larger when it’s a surprise. When nobody knows what Apple’s really working on, people’s imagination runs overtime anticipating what could be coming next. Any projects that went south before release were never known to the public, helping us look far better in comparison to more open companies.

There’s massive downsides to this too, I always struggled with simple things like getting trusted developers onto beta programs because of the secrecy, but it’s hard to argue with the results.

Tarantula Hawk

Tarantulahawk

Apologies to any arachnophobes, but last night I was lucky enough to run across a really gruesome bit of nature I had to share. We often spot Tarantula Hawks flying around, but I’d never seen how they got their name. They’re enormous wasps, several inches long, and the adults live on nectar. They’ve worked out an ingenious business plan for feeding their larvae:

1- First hunt down a wandering Tarantula.
2- Paralyze it with your venom.
3- Dig a hole, shove the spider into it, and lay an egg inside its body.
4- Cover up the hole.
5- The larva hatches, first sucks all the juices from the still-living spider, and then eats it from the inside, saving the vital organs until last so it stays alive and fresh as long as possible.

The photo is from a wasp we came across that had just paralyzed its victim, and was getting ready to drag it to its lair. How cool is that?!

Tarantulahawk2

Creating an Outlook plugin with Add-In Express

Addinexpress

I’ve always written Outlook plugins from the ground up in C++, since I’m very wary of dependencies on frameworks like .Net and other components that can turn deployment and debugging into a nightmare. I recently ran across the Add-In Express suite of tools for building Office plugins, and it offers enough to change my mind.

I paid $349 for the standard version, and the first pleasant surprise is that there’s no royalties for your end-users. Another big plus is that you can upgrade to a premium version that includes full source code for just $949. This is very important if you’re creating a commercial product, it means if they go out of business you can still keep tweaking the code to deal with OS upgrades or minor bugs. There’s also various discounts available for things like blog reviews, though I didn’t take advantage of that. [Update- After posting Andrei from ADX was kind enough to give me a free upgrade to the Premium edition]

The purchase and download was very painless, and it installed itself as part of Visual Studio, offering new project templates for the various Office plugin types. The license is limited to 3 development machines, so I will have to see how that works with my frequent reinstalls of my Parallels VM. I chose ADX COM Add-In from the extensibility section of the project templates,  and then went through a couple of wizard screens choosing which language and applications I wanted to use. I went for C# (new to me, but since I needed a heavily UI-based plugin C++ was just getting too painful) and Outlook.

You choose to have an installer automatically generated when you create your project, and this is an incredible time-saver. I’ve lost countless hours fiddling with the guts of WIX installer, so this alone could be worth the price. Unfortunately I wasn’t able to get it working, and it looked like it was related to my use of UAC on Vista. I discovered a workaround I could use during development though, the ‘Register ADX’ context menu item on the project worked like a charm. It looks like UAC may be an ongoing problem for Vista deployment, but their forums are extremely active with both developers and support staff, so I feel confident I can get help on issues as they come up, and I’ll be on a well-trodden path unlike my home-brew efforts.

Their claim is that they will handle all the horrible add-in plumbing and let you focus on writing your application code, and so far they live up to that promise. I was able to get a simple window added to Outlook within a day, a lot faster than my previous plug-ins I wrote from scratch. There were plenty of head-scratching moments as I read through the documentation, but nothing that I couldn’t figure out with some experimentation and looking through the forums.

All-in-all, a big thumbs up for Add-In Express. I’ll have a more detailed review once I’ve really used it in anger, but I’m optimistic I’ve found a real time-saver.

Adding cookies to CPeteHttpRequest

Cookiemonster
Photo by Esti-s

I’ve recently been emailing back and forth with Edward Hibbert as he builds on some of my BHO work to create an Internet Explorer plugin for Freecycle moderators users. I’m a fan of the organization, it’s a real win-win if unwanted items that would otherwise go into a landfill can be passed on to someone who can use them. I was the treasurer of a vaguely similar alternative currency group when I lived in Dundee, but this is a much simpler way of unlocking untapped value in a community.

One of his requirements was that he needed to pass around cookies in his internal HTTP requests, something that my base CPeteHttpRequest class doesn’t support. He’s implemented this functionality and kindly passed me the modified code, which I’ve included below.

Download httprequest_cookies.zip

[Update- corrected a couple of details like plugin’s homepage]

Run the Backbone Trail

Backbone

I like to think I’m fit
, but ultra-runners like my friend Howard Cohen make me look like a couch potato. I just discovered a great page he’s got up on running the Backbone Trail, going the length of the Santa Monica Mountains on the western edge of Los Angeles. It’s about 70 miles with some serious hills, and he’s managed to do it all in 16 hours on a single day. That blows my mind considering how tough I find some of the sections just to hike!

It’s through a beautiful expanse of wilderness just a few miles from the city. If you’re one of those crazy ultra-runners and you make it to LA, check it out, you won’t regret it.