Five short links


Photo by Giampaolo Macorig

Clocks are bad, or welcome to distributed systems – “If your distributed system isn’t explicitly dealing with data conflicts, any correct behavior it exhibits is more a matter of good luck than of good design.” Ten years ago, I was shooting myself in the foot with threads, now it’s distributed algorithms! I think we’re going to look back on the single-core CPU days as a golden age of simplicity.

The colors of sunset and twilight – I’m attempting to measure global pollution levels by analyzing the colors of millions of Instagram photos of sunsets around the world. The data isn’t cooperating yet, but it’s led me into a fascinating world of atmospheric science. Apparently there’s a lot more of a complex relationship between sunset brilliance and pollution than folk wisdom had me believing.

How misaligning data can increase performance 12x by reducing cache misses – Understanding how processor caches work is a secret weapon for serious optimization work. Once you’ve got rid of the obvious algorithm bottlenecks, keeping your working set of data in cache can make an order-of-magnitude difference if you’re chewing through big arrays.

Yale censored a student’s course selection website. So I made an unblockable replacement – The only truly open space for transforming and combining data these days is client-side. It’s a shame browser extensions are such a niche distribution channel, they give small players the freedom to get creative on top of large existing data sets. I really hope a killer app for client-side remixes appears, that would open the door for a lot more innovation.

My dad will never stop smoking pot – I spent my early twenties as a heavy pot smoker, and I was an unhappy mess who let people around me down constantly. I’m glad to see legalization gathering momentum, I know plenty of people who can smoke without those problems, but this story was a painful reminder of those days for me.

Five short links


Photo by Sean Lamb

Nine ways to break your system code using volatile – Even seemingly-simple constructs in low-level languages can have tremendous subtleties. I love reading explorations like these, hoping I never need to use the knowledge in production, but feeling like a little more of my in-game map has been filled in. At some point pedantic becomes sublime.

Bayes Rule and the paradox of pre-registration of RCTs – There’s a movement to declare what hypotheses you’re going to test before you start your research, to avoid the classic cherry-picking problem. Donald does a great job explaining why it should make a big difference in how much we trust study results, even though it feels counter-intuitive, and again a bit pedantic. I guess 2014 is becoming the year I try to bring pedantry back?

Are towns stuck in the wrong places? – It’s not often you can perform a 2,000 year-long natural experiment, but this look at the performance of British and French towns after the Romans is intriguing, and very relevant as we consider how to respond to struggling cities like Detroit.

The Chinese wheelbarrow – The eastern version of the wheelbarrow could carry far larger loads than the European approach, thanks to the design of a central wheel that allowed most of the weight to be taken by the vehicle, instead of the operator. This article makes a convincing case that the Europeans lost out heavily by sticking to their wheel-forward design that left half the lifting on the driver, but it also left me wanting to dig into the contrarian angle, and see what good reasons there might be for the adherence to tradition.

Empathy is a core engineering value – We have a tremendous amount of power as engineers, and time and again I’ve seen decisions that save a few hours for a single developer cost man-years of time and frustration for thousands of other people. Keith does a great job of showing why “it’s essential to at least attempt to understand the plight of users”. You can’t always gold-plate your software and deal with every problem in the depth you’d like, but putting yourself in the end-user’s shoes will help prioritize where to best spend your limited time.