This morning I ran across a wonderful open source project called “Crime doesn’t climb“, analyzing how crime rates vary with altitude in San Francisco. Then I reached this line, and honestly couldn’t decide whether to cry or scream: “Here’s the code snippet that queries the Google Elevation API (careful–Google rate limits aggressively)”
Google is very clear about the accepted usage of all their geo APIs, here’s the quote that’s repeated in almost every page: “The Elevation API may only be used in conjunction with displaying results on a Google map; using elevation data without displaying a map for which elevation data was requested is prohibited.”
The crime project isn’t an exception, it’s common to see geocoding and other closed APIs being used in all sorts of unauthorized ways . Even tutorials openly recommend going this route.
So what? Everyone ignores the terms, and Google doesn’t seem to enforce them energetically. People have projects to build, and the APIs are conveniently to hand, even if they’re technically breaking the terms of service. Here’s why I care, and why I think you should too:
Google’s sucking up all the oxygen
Because everyone’s using closed-source APIs from Google, there’s very little incentive to improve the open-source alternatives. Microsoft loved it when people in China pirated Windows, because that removed a lot of potential users for free alternatives, and so hobbled their development, and something very similar is happening in the geo world. Open geocoding alternatives would be a lot further along if crowds of frustrated geeks were diving in to improve them, rather than ignoring them.
You’re giving them a throat to choke
Do you remember when the Twitter API was a wonderful open platform to build your business on? Do you remember how well that worked out? If you’re relying on Google’s geo APIs as a core part of your projects you already have a tricky dependency to manage even if it’s all kosher. If you’re not using them according to the terms of service, you’re completely at their mercy if it becomes successful. Sometimes the trade-off is going to be worth it, but you should at least be aware of the alternatives when you make that choice.
A lot of doors are closed
Google is good about rate-limiting its API usage, so you won’t be able to run bulk data analysis. You also can only access the data in a handful of ways. For example, for the crime project they were forced to run point sampling across the city to estimate the proportion of the city that was at each elevation, when having full access to the data would have allowed them to calculate that much more directly and precisely. By starting with a closed API, you’re drastically limiting the answers you’ll be able to pull from the data.
You’re missing out on all the fun
I’m not RMS, I love open-source for very pragmatic reasons. One of the biggest is that I hate hitting black boxes when I’m debugging! When I was first using Yahoo’s old Placemaker API, I was driven crazy by its habit of marking an references to “The New York Times” as being in New York. I ended up having to patch around this habit for all sorts of nouns, doing a massive amount of work when I knew that it would be far simpler to tweak the original algorithm for my use case. When I run across bugs or features I’d like to add to open-source software, I can dive in, make the changes, and anyone else who has the same problem also benefits. It’s not only more efficient, it’s a lot more satisfying too.
So, what can you do?
There’s a reason Google’s geo APIs are dominant – they’re well-documented, have broad coverage, and are easy to access. There’s nothing in the open world that matches them overall. There are good solutions out there though, so all I’d ask is that you look into what’s available before you default to closed data.
I’ve put my money where my mouth is, by pulling together the Data Science Toolkit as an open VM that wraps a lot of the geo community’s greatest open-source projects in a friendly and familiar interface, even emulating Google’s geocoder URL structure. Instead of using Google’s elevation API, the crime project could have used NASA’s SRTM elevation data through the coordinates2statistics JSON endpoint, or even logged in to the PostGIS database that drives it to run bulk calculations.
There are a lot of other alternatives too. I have high hopes for Nominatim, OpenStreetMap’s geocoding service, though a lot of my applications require a more ‘permissive’ interface that accepts messier input. PostGIS now comes with a geocoder for US Census ‘Tiger’ data pre-installed too. Geonames has a great set of data on places all around the world you can explore.
If you don’t see what you want, figure out if there are any similar projects you might be able to extend with a little effort, or that you can persuade the maintainers to work on for you. If you need neighborhood boundaries, why not take a look at building them in Zetashapes and contributing them back? If Nominatim doesn’t work well for your country’s postal addresses, dig into improving their parser. I know only a tiny percentage of people will have the time, skills, or inclination to get involved, but just by hearing about the projects, you’ve increased the odds you’ll end up helping.
I want to live in a world where basic facts about the places we live and work are freely available, so it’s a lot easier to build amazing projects like the crime analysis that triggered this rant. Please, at least find out a little bit about the open alternatives before you use Google’s geo APIs, you might be pleasantly surprised at what’s out there!
Pingback: Speeding Up Geocoding on Rails with Geocoder : Pete Huang
Surely we fixed that bug in Placemaker before 2008?
Pingback: Why you should stop pirating Google’s geo APIs | Rocketboom
I’m surprised Cloudmade doesn’t get a mention!
If you are looking for a good mapping alternate checkout Leaflet.js. You can load from a variety of tile servers. I think a while back someone had a pretty decent tutorial an setting up your own tile server on AWS.
“…by its habit of marking an references to…” <- I think you meant "ANY references to".
While I agree with what you’re saying, I think the word “pirate” is quite a misnomer. For the strength of your argument, I’d suggest something more accurate.
As well, great work on the DST it looks pretty swanky! I’ll be giving it a try here shortly.
Worth mentioning that Nominatim’s use of OSM makes it possibly less permissive than the Google APIs. Or at least it’s completely unknown whether results from an OdBL geocoder are also OdBL, crushing most realistic uses.
Pingback: Crime Don’t Climb’s Elev Rip – OrthoInk
For a better resolution (30m) and better coverage at higher latitudes than NASA’s SRTM elevation data, see Aster GDEM : http://asterweb.jpl.nasa.gov/gdem.asp
The data is free of charge.
I mentionned this alternative in an earlier comment. Is there an advantage of using SRTM data over Aster GDEM ?
The Aster data has many annoying artifacts – Google for Aster artifacts.
It might be higher resolution, but the artifacts are just too annoying, at least on my shaded reliefs. I have stopped using it it’s so bad.
GMTED2010 is worth a look though.
I agree with Peter. I tried to replace srtm (100m resolution) with aster (30m res.) for microwave link calculations in South America and I found it useless. It does have more resolution, indeed, but it was pure noise. End results with aster were WAY more inaccurate than those obtained with srtm.
OK good to know, I haven’t had the opportunity to work with Aster GDEM data yet.
In 2014, WorldDEM from TanDEM-X mission will be available (12m resolution, 2m vertical accurracy for flat terrain), but likely not for free : http://www.astrium-geo.com/worlddem/
You are quite right – the open source projects utilising “Open Data” are vital to stopping the strangle hold of the large corporates.
Even your own project needs some help, a it’s not working at the moment! (going to download it though and run it…. 😉
Pingback: Data Viz News  | Visual Loop
Pingback: Data Viz News  › Soci-all
Pingback: How many people read my posts? « Pete Warden's blog
Google’s ToS, even for the business license, is a DISPLAY-ONLY license. So you can show geocodes, routes, etc. on a map. But you can’t store them in a database and you can’t do any analysis with them either. At least not without violating the ToS
Pingback: Pirating APIs « Smathermather's Weblog
Reblogged this on sonofbluerobot.
I am working on commercial in house asp.net project, we are letting users to key in vendors and inspectors address latitude and longitude values which are then displayed on google map. Latitude and longitude values are saved in database.
I have been thinking about improving this process and like to get the address geocode through API so end user doesn’t have to enter the latitude and longitude values.
I am looking for best API which is free for commercial use as google and yahoo API’s have some usage limit.
Also, is this possible to get the geocode from other API’s and then show them on google map? Does google allow this?
MapQuest Elevation API is even worse than google. No charge you have only 1500 requests per month!