I'm a big believer in the power of objective measurements as the best way to drive product improvements, and in the past I've built my own ramshackle logging systems to gather the data I needed. Unfortunately it always took a frustratingly long time to create the systems, and I never had enough resources to build a visualization and analysis interface that easily told me what I wanted to know. For OpenHeatMap I decided to be as aggressive as I could in finding off-the-shelf solutions for everything outside of the core of the service, so I gave KISSMetrics a try.
That's really the heart of the service for me. My goal is to get as many visitors as possible to create maps, so I religiously follow the ratio of people viewing the front page to those making to the end of the map building process. I started off with only around 2% making it all the way through, but now on a good day I'll see 9% building their own visualizations. Having that number to check my changes against has been essential. I've been able to tell very quickly if my changes are actually making a difference, and psychologically it's been a great motivator to work on improvements and make that dial move!
There's a lot of depth to KISSMetrics, including support for A/B testing and an API for custom events, but I have so much on my plate improving obvious problems with my service that I haven't dived in. There is a cost to all this goodness of course: $125 a month. That's a very steep price for a small-scale site like mine, but it's vital enough to my development that it's worth it. It's a good motivation to get my service to the point where I can roll out premium features too!