Geocoding with the Quick ‘n Dirty Geocoder

The geocoder described in this post is free software, and can be used and distributed however you like.

It installs in a servlet container like Tomcat, and accesses the geocoder.us web service to translate the names and addresses you supply (in a text file) into geographic coordinates, which it then writes back to your PC and/or a database.

Download the free geocoder

For details on how to install and use this software, frequently asked questions, configuration, etc., refer to the wiki article. The short version is just drop geo.war into Tomcat’s webapps directory, start it, then follow the instructions at http://localhost:8080/geo/code. This post describes some of the Java code used to implement the communication between the Q&D geocoder and the geocoder.us web service.

Most lines of code in this application are used in IO on your own PC, rather than the web service. The GeoServlet receives requests, and uses the GeoImporter to read in and parse files from the text file on your PC. It then uses the GeoTransport class to communicate with the geocoder.us service, and finally writes the results back to a file and/or database with the GeoExporter class.

GeoImporter

LineIterator iter = null;

try {
iter = FileUtils.lineIterator(file, “UTF-8″);
while(iter.hasNext()) {
String line = iter.nextLine();
makePlace(line);
count++;
}

} catch(IOException ex) {
addMessage(”Error reading input file. Message: ” + ex.getMessage() + “.<br/>”);
ex.printStackTrace(System.err);
} finally {
LineIterator.closeQuietly(iter);
}

The importer leverages the excellent Commons IO library, which makes reading lines from a file a snap. Once read, makePlace(line) is called to parse the line and make sure it is formatted properly. Places are gathered in an ArrayList, which is passed back to the servlet.

GeoTransporter

Each Place is then translated with the transporter. The transporter uses two values in the web.xml file, a username and a password. If they are set, the commercial geocoder is used; otherwise, the free geocoder is used.

To get a username and password, head to geocoder.us and pay the nice folks $50 for 20,000 requests. If you are using it for commercial purposes, you must purchase a commercial license.

The transporter constructs the web service query, and ensures that a valid response is received back. The only tricky part to this class is the authentication used when accessing the commercial service. Geocoder.us’s documentation states that you can send the username:password combination in the web service URL, but I got nothing
but Http 402 errors, because the server couldn’t authenticate the request. The username and pasword had to be encoded in base 64, and added as a request header for it to function properly. And by properly, I mean at all.

//commercial url: http://geo-user:geo-pw@geocoder.us/member/service/csv/geocode?address=
//free url = http://rpc.geocoder.us/service/csv?address=

try {
URL url = new URL(wsUrl);
HttpURLConnection httpConn = (HttpURLConnection)url.openConnection();
httpConn.setRequestMethod(”GET”);

//only use authentication if using the commercial service
if(serviceAuth != null) {
httpConn.addRequestProperty(”WWW-Authenticate”, “geocoder.us”);
String authorization = Base64Coder.encodeString(serviceAuth);
httpConn.addRequestProperty(”Authorization”, “Basic ” + authorization);
}

httpConn.connect();
int responseCode = httpConn.getResponseCode();
if(responseCode == HttpURLConnection.HTTP_OK) {
response = IOUtils.toString(new InputStreamReader(httpConn.getInputStream()));
}

} catch (IOException ex) {
ex.printStackTrace(System.err);
}

The WWW-Authenticate header specifies the authentication realm (retrieved from the http response headers), and adds the encoded authorization string. A great encoder is available via the Apache Commons project.

If a valid response is received, the response is parsed and the Place is added back to the ArrayList.

GeoExporter

Finally, the ArrayList is written to a text file in the same directory as the source file, and the final statistics are printed to the screen.

Notes

• When using the free geocoding service, expect your request to be throttled. In a few sample runs, each geocoding request took roughly 15 seconds.
• I got a roughly 83 percent success rate on geocoding via geocoder.us.

Learn More & Download the Quick ‘n Dirty Geocoder (Swirl Wiki) | Read the Quick ‘n Dirty Google Geocoder post