OpenHeatMap and DataScienceToolkit under new management

I’ve been running OpenHeatMap and the Data Science Toolkit for quite a few years now, but a few months ago I realized I wasn’t able to keep maintaining them. I know a lot of people out there are still using them, so I looked around for a partner I could transfer the ownership to. After some discussions, I arranged a deal with the myheatmap.com team to transfer the sites to them, for no charge, in return for their agreement to keep supporting the existing community. For the last few weeks they’ve been handling the servers, support, and maintenance, and I’m very glad they were able to step in. The goal is to keep the existing free services supported, but give them the ability to expand in a more commercial direction too, so that the site becomes more self-sustaining. All OpenHeatMap support requests should now go to help@openheatmap.com, which they administer.

The code behind DataScienceToolkit.org is all open-source on github, so that will continue to be available, but the DSTK site itself has an uncertain future. I’ve always tried to keep it open to anyone who wants to experiment with the APIs, but over the last year its come under denial-of-service level usage levels from a wide range of IP addresses. I spent some time learning firewall rules and attempting to block the problematic calls, but I wasn’t able to keep the levels low enough to keep the site consistently up. Since OpenHeatMap relies on the site as its geocoder, that meant the uploading there was also often unreliable. I came to the sad conclusion I didn’t have enough time to do the overhauling I’d need to deal with the problems, which is why I handed everything over to a team who can put in more time. The most common use of the DSTK was for geocoding US address, and with the Census Bureau now providing their own free API, that side of it became less essential too. The hosting of the large VMs unfortunately got lost when I shut down the jetpac.com site, so I’m afraid I don’t have those available any more.

Both of the sites were failed startup ideas that took on a life of their own, even though I was never able to make them commercial ventures. I’m hopeful that a fresh team with new ideas will be able to provide a better service to everyone who uses them. I’m grateful to everyone who’s been in touch over the years, I kept supporting the site for so long because I saw the amazing projects you were all using them for. My deep thanks go to the community that formed around the sites.

2 responses

  1. Pete,

    Looks like we can’t access the site since they took over.

    I work in the IT dept. of Bimbo Bakeries USA. Our company is a multi-billion dollar supplier of bread and cake products of a variety of brands, like Arnold, Entenmann’s, Thomas, and Sara Lee. We use the http://www.datasciencetoolkit.org/ site to acquire geocodes for our depots, etc. This has been working fine for us for a long time. However, it has not been working for us since Sept. 10. We get the message:

    Service site test HTTP error code : 503

    We try to access the site automatically at midnight every day, though we did change the time to 2:00 AM in hopes the site would not be so busy then. Unfortunately, we get the same error.

    Can you tell me if there has been any change to the site since the transfer? We would like to continue using this site

    Steve

  2. Sad to hear that you are letting dstk go. There are a lot of institutions that will miss it as it is one of the only free off-line geocoders out there. Sure there are online ones, but many institutions cannot transmit their addresses over the internet lawfully (US medical institutions for instance). Time to start the search again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: