Why you need a minimum-viable profiler


Photo by Antonio Rodriguez

Chances are that you'll hit performance problems on any non-trivial project, so you'll need some kind of profiling. Scripting languages have poor profiling support, with intrusive tools that require fiddly setup and lack strong interfaces. This is understandable, raw performance is much less of a priority when you control the hardware the code runs on and can scale horizontally, but the lack of casual profiling can really slow down development.

As a result I've ended up rolling my own painfully simple profilers when I'm working in PHP, Python or Ruby. The beauty of this minimal approach is that you don't have to spend time setting up the big guns like xdebug or perftools, most issues are obvious from a surface inspection and there's no external dependencies to juggle. I wrap the top-level modules of my code with profiler calls, and then output the results to stderr once the script's done. There's usually only five or six timings, but it's enough to answer most questions about 'why is this taking forever?'. If not I can dive into the offending function and manually add more detailed timing output. The key is that it has to be lightweight and easy to deploy (meaning no external dependencies) or it loses a lot of its value.

As an example I've open-sourced the Ruby version I'm using on Jetpac's data pipeline:


You bracket your whole script with MinimalProfiler.start/stop('MAIN') calls, and then wrap any significant high-level functions with MinimalProfiler.start/stop('Some function'). This is obviously a lot more manual and error-prone than a more automatic approach, but in practice it's not hard to maintain, and when a module becomes a hotspot you can add more calls within the function. At the end of the script it writes out a summary of where the time went:

Total time 3.001541 seconds

33% – Something else (1.001017 seconds)

66% – Doing something (2.000449 seconds)

I'm not suggesting my particular implementation is better than a full profiler, but it's way better than no profiler at all, which is the status quo for web development. It's only a single page of code, so if it's not to your taste rewriting it for your own needs shouldn't be a problem. Just make sure it's easy to deploy and use, and you'll be amazed at how much time you'll save tracking down performance issues.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: