Learn Linked Data

It’s been a while coming, but (finally), I’d like to present this 3rd article in the series. Together, we’re building a Linked Data application to map the English Indices of Multiple Deprivation Stats. The final source code for the app can be found on github. If you missed the last blog post in the series, you can find it here.

Note: These articles use Github gists for showing code-snippets, and if you’re viewing this in a feed reader they might not show up. So I recommend you read it on the web instead.

What the app will do:

The basic principle of the app is that the user pans or zooms or searches by postcode to change which bit of the map they are looking at. The app detects that, retrieves the required data using SPARQL, then draws it on the map. You can play with the finished app on the OpenDataCommunities site here.

Last time…

In the last article we explored how the deprivation data was stored, wrote a template SPARQL query, and drew a standard Google map centred on a starting location.

By the way, I’ve created a new Github repo especially for this blog-series, so that you can watch the application evolve through the git commits. Everything that we did in parts 1 and 2 are in commit bce302.

In this article, we’ll write some JavaScript to build and execute the right SPARQL query (based on the template from last time), to retrieve the deprivation data about the LSOAs currently displayed on the map. The code from this article is in commit 17e211.

A note about CORS

On the web server on which OpenDataCommunities is hosted, we’ve enabled Cross-Origin Resource Sharing (CORS), so that Ajax requests for the data can be made from sites not on the same domain. However, for this to be honoured, you’ll need to host the app on a web server (such as Apache), while you develop it. Just opening the html from your disk in a browser won’t work.

Map Manager

Much of the code we’ll write in this article will be in a file called map-manager.js, in the javascripts/swirrl directory. As you might expect, the MapManager will be responsible for dealing with the interaction with the map. The following code snippet explains the structure of the file:

What’s going to happen?…

We’re going to add some code to the main JavaScript closure (in the HTML file) which will listen for Google maps idle events (i.e. when the map stops being zoomed or dragged).

When we notice that the map has become idle, we’ll ask the MapManager to refresh the map.

The MapManager will report back to the main closure, using jQuery events, so that it can start listening again for idle events.

The MapManager Constructor

The constructor for the MapManager takes the Google Map object and the initial score domain (from the drop down) as it’s parameters, and sets some stuff up.

Interesting things to note:

• We assign this into a variable called self so that when this gets set to other things (in jQuery callbacks etc), we can always get a reference to the current MapManager object.
• The lsoaDataRetrieved event will be triggered by another function when the deprivation data has been retrieved and is ready for use. For now, we’re just going to log out the results (there’s a gist later on with an example log-output), and tell the calling code that we’ve finished.
• Finally, there’s a little bit of code to clean up if there are errors.

The refresh function

The refresh function (along with the constructor) will form the public API for the MapManager. Let’s add it to the prototype:

• When execution of the function begins, we’ll trigger the started event to tell others that we’re starting the process of refreshing the map.
• If the Google map’s zoom level is acceptable (if we’re too zoomed-out there’ll be too much data to handle), we proceed. Otherwise, we do nothing other than trigger a couple of events.
• the getTiles function (for brevity, not included in this article – see the source on github for details*), interrogates the google map and determines which 0.1×0.1 lat/long tiles are visible in the viewport.
• Next we remove any tiles that are no longer visible in the viewport, and add any new ones by comparing the results from getTiles with the set of tiles from the previous time refresh was called. The reason we do this is for efficiency: we don’t need to request data for tiles for which we already have data.
• Finally, we call getLsoaData, passing in the set of tiles currently visible.

Getting the LSOA Data

The getLsoaData function is responsible for building the right SPARQL query to call (based on the template query in the previous article), and executing it against the OpenDataCommunities SPARQL endpoint.

That code might look a bit complicated, but it’s not really. Let me break it down a bit.

• buildSparql is a nested function (as it wont be needed outside the scope of getLsoaData). It’s responsible for interpolating the bottom-left and top-right lat/long values of tiles into the template SPARQL query.
• callAjaxSparqlPaging is another nested function, which calls itself recursively until all the pages of data have been retrieved (the SPARQL endpoint will return the data in 1000-result ‘pages’).
• For each result of each page (in the $.ajax success callback), we call setLsoaData which just sets the data for an LSOA (such as label, centroid lat/long, score, and URI) into a nested object (with the top level properties being the tile’s corners, and the inner objects’ properties being the notation of the LSOA (e.g. ‘E01005061’). See the example log-output later on for an example of this structure.
• The code at the end of the getLsoaData function, calls the SPARQL query for each tile in our list of tiles.

Calling the refresh function

As I mentioned earlier, the main JavaScript closure in the HTML file is responsible for instantiating a MapManager and calling refresh. Let’s see what that looks like:

• The first bit of code is what we set up in the previous article to just create the Google map centred on Manchester.
• Next, we instantiate a MapManager, with the map object.
• At the end of this code-snippet, we define a bindMapIdle function, and then call it. As I described at the top of the article, this calls the refresh function when we see that the map is idle.
• The started event listener makes a note of the time, and then removes the idleListener if it exists (so that we only call refresh once at a time). It also shows the busy ‘spinner’.
• Once refresh has finished, we log how long it took, hide the spinner, and re-bind the idle listener.
• The zoomToWide and zoomOK handlers, just show and hide the zoom warning message (shown if the user zooms out too much and we can’t show all the data).

Let’s run it!

We’re now ready to see what all our code does.

If you’ve been following along, just open the map.html file (from your web server) in your browser. (If you’ve not been following along, just check out the code from this commit in Github).

Open your browser’s debug/console window (i.e. Web Inspector in Chrome/Safari, or Firebug in Firefox), and hit refresh. You should see a bunch of log-output lines including information on what requests were made against SPARQL endpoint and how long it took (“busy duration”).

Just before the busy duration message, there should be an entry that looks like this: [>Object] (in Webkit-based browsers, at least). Click the triangle to expand the object, and you can see what data we have for the LSOAs with centroids in the current viewport. (This is what is logged out from the lsoaDataRetrieved handler in the MapManager constructor.)

As I mentioned earlier, the top-level properties are the tiles. Each tile is an object whose properties which correspond to the LSOAs. Each LSOA contains data such as it’s label, the lat/long, the score for the currently selected domain, and the URI of the LSOA.

For example:

Try dragging and zooming the map, and watch what happens in the console window. The further zoomed out you are, the more tiles of data you will see.

Next time

In the next instalment (I promise not to leave it so long this time), we’ll get the boundary information for all the LSOAs and plot that on the map as polygons. If there’s time, we’ll also make these interactive.

*To be honest, I’m not particularly proud of this bit of code – it’s pretty hacky, but it does the job!