Photo by Trugiaz
Mobile web pages crash when they run out of memory. Apple devices are particularly painful – they’ll die hard without even giving you a chance to record the crash! Even if they don’t crash, performance will be sluggish if you’re pushing memory limits. Large images are the usual cause, but since we display a lot of photos from social networks, I don’t have much control over their size outside of the default dimensions the different services supply. What I really needed was a zero-maintenance way to pull arbitrary sizes of thumbnails from external images, very fast. Happily, I found a way!
The short version is that I set up a server running the excellent ImageProxy open-source project, and then I placed a Cloudfront CDN in front of it to cache the results. The ImageProxy server listens for requests containing a source image location as a parameter, along with a set of requested operations also baked into the URL. The code forks a call to ImageMagick’s command-line tools under the hood, which isn’t the most elegant solution, but does provide a lot of flexibility. With this sort of execution model, I do recommend running the server on a separate box from your other machines, for both performance and security reasons.
There are a few wrinkles to getting the code running, so if you’re on EC2, I put together a public AMI you can use on US-West1 as ami-3c85ad79. Otherwise, here are the steps I took, and the gotchas I hit:
– I started with clean Ubuntu 12.04, since this is the modern distribution I’m most comfortable with.
– ImageProxy requires Ruby 1.9.x, and installing it instead of the default 1.8.7 using apt-get proved surprisingly hard! I eventually found these instructions, but you might be better off using RVM.
– I mostly followed the ImageProxy EC2 setup recipe, but one missing piece was that the default AllowEncodedSlashes caused Apache to give 404 errors on some requests, so I had to make a fiddly change to my configuration.
I now had an image proxy server up and running! To check if yours worked, go to the /selftest page on your own server. Pay close attention to the “Resize (CloudFront-compatible URL format)” image, since this is the one that was broken by Apache’s default AllowEncodedSlashes configuration, and is the one you’ll need for the next CDN step.
I’m an Amazon junkie, so when I needed a flexible and fast caching solution, CloudFront was my first stop. I’d never used a CDN before, so I was pleasantly surprised by how easy it was. I clicked on “Create New Distribution” in the console, specified the URL of the ImageProxy server I’d set up as the source, and within a few minutes I had a new CloudFront domain to use. It handles distributing its copies of the cached files to locations near each requesting user, and if a URL’s not yet cached, it will pull it from the ImageProxy server the first time it’s requested, and then offer super-fast access for subsequent calls. It all just worked!
Now, my code targeting mobile devices is able to automagically transform image URLs into their resized equivalents, eg from http://eahanson.s3.amazonaws.com/imageproxy/sample.png to http://imageproxy.heroku.com/convert/resize/100×100/source/
http%3A%2F%2Feahanson.s3.amazonaws.com%2Fimageproxy%2Fsample.png. There is a slight delay the first time a URL is accessed while the resizing is performed, but as long as you have a finite set of images you’re loading, this should go away pretty quickly as the results get cached by CloudFront.
The only other issue to think about is limiting the access to your server, since by default any other site could potentially start leaching off your proxy image processing. There are plenty of other ways of doing the same thing, so I wouldn’t stay awake at night worrying about it, but ImageProxy does allow you to require a signature, and restrict the source URLs to particular domains, both of which would help prevent that sort of usage.
Big thanks to Erik Hanson for building ImageProxy, I’ve been putting off doing this kind of caching for months, but his project made it far simpler than I’d hoped. It’s working great, improving performance and preventing crashes, so if you’re struggling with unwieldy images, I recommend you take a look too!
[Update – I forgot to mention that the ultimate easy solution is a paid service like embed.ly or imgix! After looking into them, I decided the overhead of depending on yet another third-party service outweighed the convenience of avoiding setup hassles. It wasn’t an easy choice though, and I could see myself switching to one of them in the future if the server maintenance becomes a pain – I know that’s what happened with my friend Xavier at Storify!]