Monthly Archives: December 2007

Killer whales in Los Angeles?

December 30, 2007 By in Personal Leave a comment

Img_0329

On the way back from a night camping on Santa Cruz, I was lucky enough to see a pod of Orca. I’m always amazed by the natural wonders you can find on your doorstep here in LA.

IslandPackers are running special whale-watching half-day trips until April. Orcas are unusual, but there’s often grey whales, and I’ve almost never stepped off the boat without seeing a good crop of dolphins and sea-lions. The crew are always a lot of fun, especially when they go into great depth on the fauna’s sex-lives. Dolphins mate up to ten times a day, apparently! Do make sure to dress up warm. Even in the summer when it’s in the 80’s on land, I’ve got very chilly on the boat.

Img_0328

Here’s a quick way to organize Outlook attachments

December 23, 2007 By in Outlook API Leave a comment

Mapilablogo

Outlook Attachment Processor from MAPILab lets you save out all your email attachments to disk, and replaces them with links in the messages. I find it a lot easier to search and organize documents as objects on the file system than when they’re embedded in emails, and this add-in makes it painless to move them over. It’s got a large array of options, but they’re well-explained and have good defaults, so it doesn’t feel too much like the space shuttle control panel.

Mapiscreenshot

The most important options cover which messages and attachments are converted to local files, and where they end up. I like the way this addin focuses on solving a single painful problem, but with a lot of flexibility and depth for customizing that solution. It’s obviously been heavily driven by user feedback.

Of course, there’s a downside to saving all your attachments like this; the links break when you move to a different machine. There’s an ‘Update Links’ tool to change them to a new location to solve this problem, but it shows that separating your attachments from the source PST does add some complications. You can try the add-in free for 30 days with fully-functional trial version, and costs $24 to buy a single-user license.

MAPILab offer a range of other Microsoft plugins, including a couple of tools for Exchange. They employ 25 people, which shows the engineering effort that solid plugins like these require, and that there’s market demand for their solutions. They explicitly spell out their strategy as targeting narrow problems, leaving larger companies to "focus on the creation of platforms and technological foundations".

One of the problems I’m interested in solving is making document collaboration through email less painful. Attachment Processor and some of their other tools like File Fetch and File Send Automatically are solving parts of what makes it so awkward. What I’d like to see is a more comprehensive system that offers the advantages of a wiki without having to force people away from sharing documents through email. It seems like an Exchange extension that turned attachments into links to Sharepoint documents, like Attachment Processor does for the local filesystem, would be an interesting direction to go down.

Fancy a trail with oil bubbling from the ground?

December 22, 2007 By in Personal Leave a comment

Bubblingoil

Towsley Canyon is a park on the edge of Santa Clarita, just off the 5 freeway. It’s a lovely place, full of beautiful flowers in the springtime, but my favorite features are the small oil seeps scattered amongst the hills. Once a commercial oil field, there’s now just a few rusty derricks and some natural springs of oil slowly bubbling to the surface. I never believed the title sequence of the Beverly Hillbillies could be real, but you can see it here! There’s also a miniature version of the Zion narrows, with a small but spectacular canyon carved out by occasional heavy flooding.

The main trail there is a 5 mile loop, that climbs about 800 feet. It’s got sections with a 30% grade on the east side, so I recommend taking the counter-clockwise direction where there’s some gentler uphill grades thanks to some switchbacks. It’s popular with bikers and runners and is a good place to take your dogs, though you’ll need them leashed. Here’s a map showing the main loop, and a shorter variation you can take.

View Larger Map

To get there from LA, drive north up the 5 until the Calgrove exit, and turn left at the stop sign at the end of the ramp. About 1/4 mile along that road you’ll see a sign for the Ed Davis Park on the right. You can either park on the street lot which is free, or drive in a few hundred yards and pay $5 for one of the interior parking areas. Head north along the fire road, going past the visitor’s center and ranger’s accomodation. You’ll pass a concrete dam, and then go through the Narrows on the streambed, and eventually hit a spot where the trail turns uphill. After this, there’s some well-planned switchbacks, but it’s still hard work getting towards the top. Luckily, the trail is kept in great condition by a dedicated crew of volunteers. Liz and I just returned from a day working with them, and came home with some scrumptious fruitcake as a Christmas gift!

It should be fairly clear which way to go as you hike along the trail, there’s not much vegetation growing in, the tread is in good shape, and there’s few unofficial side-trails to confuse you. Be careful if you go in the summer, there’s little shade and can get extremely hot, so make sure you bring plenty of water. My favorite time to visit is the spring, thanks to the cool weather and wonderful wildflowers that appear after the rains, including some gorgeous Chocolate Lilies.

Towsleyview

You’ll reach a peak of around 2200 feet, and then start heading downhill fairly gently. After around a mile, it will start to get a bit steeper, culminating in a 30% grade section (marked on the map) that seems to head straight down the hill. Thankfully it eases up after that, and you’ll soon pass the largest oil seep, usually with lots of sticks left poked in it by curious children. After that, you’re less than a mile back to the parking lot.

Are Whitehouse emails wide open to hackers?

December 21, 2007 By in Outlook API, Personal Leave a comment

Whitehouse

When I heard about the deletion of the Whitehouse emails back in April, and Karl Rove’s use of a private email account, my first thought was ‘wow, they must really struggle to keep that secure’. It’s not often my technical research leads to a question of national security, but it turns out they don’t struggle, they just leave a large part of their email system unsecured!

Emails that travel outside of an organization to a private email account like Karl’s go through an unencrypted, plain text transport system, SMTP. In simple terms, a text document is passed from server to server until it reaches its destination. In theory, anybody who’s sitting on the network can see the contents of those messages. Normally, this isn’t a big issue, since emails are low value (typically not containing credit card numbers or other information valuable to hackers) and there’s so many flying around, just being in the right place to sniff it and picking an interesting one out from the noise is tough.

David Gewirtz, a techie who runs OutlookPower magazine, has spent months researching the technical aspects of the Whitehouse’s email use. He’s now published a book, and it’s scary reading for anyone who cares about America’s security. You can read extracts from it at this site, but I recommend looking through the original articles too. Start with "Prepare to be freaked out" to understand how serious the consequences of their poor technology decisions could be. This isn’t a partisan or crazy conspiracy book, email is something that every Executive in the last 20 years has made serious mistakes with, and David ends with recommendations on how to improve the current dire situation.

Buy the book, but here’s a full list of the related articles:

How to bake cookies your friends will beg for

December 20, 2007 By in Personal 1 Comment

Cookies

These are my secret weapon for keeping people coming back to my trail work days. The butter and heavy mixing are the keys to getting the perfect consistency, and I’ve got the science to back that up! The taste comes from the real butter and using good chocolate chips, not the generic ones packed with vegetable oil. Makes 20 to 30 cookies.

Ingredients

  • 3/4 cup white sugar
  • 3/4 cup real cane brown sugar (not the fake dyed white stuff, doesn’t have the moistness)
  • 2 sticks of butter
  • 2 eggs
  • 1 teaspoon vanilla extract
  • 1 teaspoon salt
  • 1 teaspoon baking soda
  • 2 1/4 cups white flour
  • 12 oz Ghirardelli semi-sweet chocolate chips
  1. Preheat the oven to 350 degrees farenheit.
  2. Put the sugar and butter in a bowl, and mix thoroughly. The initial mixing is crucial, you should get a mixture that seems almost whipped by the end, that will form peaks when you lift the beaters. Start off with the butter cold, straight from the fridge, and mix for longer than seems necessary, it makes a big difference.
  3. Add one egg, and the vanilla, and mix thoroughly again. Though for a much shorter time than the initial mix.
  4. Add the final egg and mix again.
  5. In a measuring cup, combine the salt, soda and flour, giving them a stir.
  6. Add the flour to the dough, about a third or a quarter of it at a time, mixing well between each.
  7. Pour in the chocolate chips, and mix them in with a spoon.
  8. Take a baking tray and place lumps of dough a little smaller than golf balls on it. Line it with foil first if you want to make cleaning up easier.
  9. Place in the oven, and cook for 11 to 13 minutes, depending on how gooey you like them.
  10. Use a spatula to take them off the tray, and put them on a rack to dry. Don’t worry if they’re still a bit soft, they continue baking and firm up as they cool off.

If blog comments are dark matter, then what’s the dark energy?

December 19, 2007 By in Defrag, Implicit Web, Outlook API Leave a comment

800pxwmap_2003
Brad called blog comments as the dark matter of the net. They’re really hard to search, and so there’s a lot of useful information that’s effectively lost to the world. What’s driving a lot of my work is my belief that email is the dark energy.

Dark energy makes up 74% of the universe, versus  22% for dark matter. There’s an estimated 200 billion emails sent every day, whereas the number of active blogs is in the low millions. I’m wandering dangerously close to Chinese math, but even assuming the vast majority of emails are low in information content, that’s a lot of untapped data that people are entering into computers.

The reason nobody’s taking advantage of this is that emails are a very personal and private medium, not intended for public consumption, unlike blog posts or comments which are explicitly published to the world. My hypothesis is that there’s a category of people for whom exposing partial information about their email, possibly to a limited audience, will solve some painful problems. JP Rangaswami is my poster child; he opened up his inbox to all his direct reports, as a way of mentoring and sharing information with them, as well as ensuring he doesn’t hear much complaining about each other! I wouldn’t go that far, but I do wish I could easily expose all of my technical discussion email threads to the rest of my team.

There’s practical steps that can be taken within a business setting to make a lot more information available, since that’s one place where you have access to a whole set of interacting email messages. I want to find subject matter experts within the organization, or people who have been in contact with an external group or person you want information on. Doing social graph analysis on an exchange server full of messages will help with that, as will statistical analysis for picking out keywords. I’m excited to see what tools I can build on these foundations. Stay tuned…

Want the average frequencies of 13 million words?

December 18, 2007 By in Implicit Web Leave a comment

Graph
Last year, Google released a list of how frequently single words and combinations appeared, based on analyzing over a trillion words on public web pages. It has over 13 million individual words, and the frequencies of combinations of up to 5 words. It’s available on 6 DVDs for just $180 from the Linguistic Data Consortium at the University of Pennsylvania.

If, like me, you use statistical analysis to pick out unusual words or phrases from documents, this is a god-send. It should be a great base-line to compare the document’s text against, and eliminate the common phrases, leaving just the distinctive parts. I’m hoping to at least use it as an uber-stop-word file. The main down-side is the restrictive license, that forbids "commercially exploiting" the data. It shouldn’t be rocket-science to reproduce similar data by crawling the web when that becomes an issue, so I’ll work within those limits for now.

The LDC has a great collection of other raw data sets too. It’s worth checking out their English Gigaword archive of millions of news stories if you want some more baseline data. Thanks to Ionut at Google Operating System for leading me to the article in the official Google Research blog covering this release.

Inboxer – An easy way to spy on your employee’s emails?

December 17, 2007 By in Defrag, Implicit Web, Outlook API Leave a comment

Inboxer
I first ran across Inboxer through their excellent Enron email exploration site. They offer a server appliance that sits inside a company’s firewall, analyzes all internal email, and offers a GUI interface to explore the messages. They have some sophisticated tools that let you see some common types of emails that management would be interested in, such as objectionable content, recruitment-related or by external contacts. They also let you set up alerts and triggers if particular conditions are met, such as unauthorized employees emailing messages that appear to contain contracts to external addresses. You can experiment with their UI through the Enron site, it seems to be pretty well laid out, and simple enough for non-technical people to use.

Inboxertimegraph

They offer graphs of important statistics over time.

Inboxerscreenshot

There’s a set of pre-packaged searches for things management are commonly concerned about. You can drag and drop any of them onto the main pane, and you’ll get a view of all the relevant emails.

They’ve done a great job technically with Inboxer, it seems like a well-rounded service. I’m a bit disturbed that the this is what the market is demanding though. Despite it being pretty clear from a legal standpoint that the company has no duty of privacy, most people don’t treat their work emails as public documents. Some of the searches, such as those for recruitment terms, are clearly aimed at catching employees doing something they don’t want management to know about, but that aren’t aimed at harming the company. I get worried that it would be incredibly tempting to use this as a technical fix for a management problem. Instead of focusing on keeping employees from job-hunting by keeping them happy, just try and punish anyone who makes the mistake of using the company system in their search.

I believe the Inboxer team has done their homework, they’ve clearly tried a lot of different tools, and this is the one that seems most successful. There’s a lot of legitimate uses, especially in regulated industries and government organizations, where there’s liability issues that require some email controls. I just wish that a less command-and-control, top-down approach was more popular. If Inboxer also offered a client-side version, I’d much rather work for a company that required that. It could make it clear which emails would be flagged and looked at, before they were sent, and help employees understand how public their work emails really are.

Roger Matus, the CEO of Inboxer, has collected a lot of useful email and messaging news in his blog, Death by Email. I’d recommend a visit if you’re interested in their work.

How to handle file dragging in a Firefox web app

December 16, 2007 By in Coding, Firefox to IE port, GoogleHotKeys Leave a comment

Drag

One of the things I miss most when moving from a desktop app to the web is the ability to drag and drop documents between programs. The default file open dialog within a form is definitely not an adequate substitute. The best you can manage with a plain web app is dragging elements within the same page.

To add the full functionality to a web application, you need to install some client-side code. In Firefox, the easiest way to do this is with an extension, though a signed JAR file containing the script is also a possibility. I haven’t tried to do it in IE yet, so that will have to wait for another post.

Here’s an example extension, with full source code and a test page demonstrating how to use it. To try it out:

  • Install draganddrop.xpi in Firefox
  • Load testpage.html
  • Try dragging some files onto the different text areas on the page

You should see an alert pop up with the file path and the element’s text when you do this. The extension adds a new event type to FireFox; "PeteDragDropEvent". When a file is dragged onto a web page, it sets the element underneath the mouse’s ‘dragdropfilepath’ attribute, and then fires the event on that element. If the element has called addEventListener for that event previously, then its defined handler function will be called, and the script can do what it needs to.

The main drawback is that you only get access to the local file path for the dragged object, and there’s not much an external web script can do with that. I’ll cover the options you have to do something interesting, like uploading a file to a server, in a future post.

This page was invaluable when I was developing the extension, it has a great discussion with examples of the mechanics of Firefox’s drag and drop events. One thing to watch out for if you reuse this extension for your own projects is that you don’t want to open up dragging-and-dropping for all pages. That would be a possible security problem if malicious sites lured users into dragging onto them. Instead you should do some URL white-listing to make sure only trusted locations are allowed, being careful to properly parse the address so that spoofing with @, etc, won’t fool the test.