local map tiles for QGIS

Posted by: Ian on 24 April 2015

Screen Shot 2015-04-23 at 5.10.48 pm

The Need for Speed

I do all my personal GIS work on a 2010 MacBook Pro, which, although loaded with 8Gb of RAM, working with a detailed street dataset can really slow things down. In order to speed things up I hit upon the idea of creating myself a set of pre-rendered raster map tiles that could be loaded in lieu of displaying a large vector file of streets (another option would have been an SSD, but that costs a little more, probably with less benefit).

Creating My Tiles

For my maps, I planned to use OSM-bright as it is nice and clear, and familiar to most people – it looks like people think a map should look.

I used Mapzen to download the most up to date osm.pbf file for Sydney. Previously I’ve used the data from geofabrik, but wanted an easy to use rectangular area and didn’t feel like working out how to clip it down to a workable area with osm2pgsql (if that is even possible11).

I already had an install of PostGIS available as part of the Boundless Open Geo Suite so was able to pick up the OSM OSM-Bright tutorial at the ‘osm2pgsql’ stage, although I used pgAdmin3 to create the database rather than Terminal.

There was little bit of confusion of the actual folder structure for the simplified-land-polygons-complete-3857 and land-polygons-split-3857 files; the guide tells you to put them in a directory named ‘shp’ which doesn’t exist in the download. After a bit of research, I found the following worked.

OSM-Bright Structure

Fixing TileMill

At one part of the tutorial, it requires you to run TillMill (if you haven’t previously) so that it can set up a few folders which OSM-Bright needs. Unfortunately Tillmill was broken with the Mac OS X 10.10 update, so when I tried launching it I was greeted with an eternally spinning wheel.

Fortunately, as the code is publicly available, I managed to find a solution and quick guide to installing it from the source code. It doesn’t create an App, but does allow it to run.

git clone https://github.com/mapbox/tilemill.git
cd tilemill
npm install

I ran into a couple of problems with this; first off was not having NPM installed which was easily fixed with a package from Node.js. However I then found that the current version (v0.12.2) caused install errors and had to downgrade to v0.10.36 to get it working.

Once everything was built I just needed to do
./index.js and then go to http://localhost:20009 after the server was running.

TillMill Terminal

Since I had finished the OSM-Bright tutorial, I had a ‘Sydney OSM-Bright’ project which opened (a little slowly as it has to download some extra bits), and was able to pan and zoom my way down to Sydney.

TillMill OSM-Bright

From here, it was a simple case of selected the export to MBTiles option, changing a few parameters and waiting about 40 minutes while TillMill did it’s thing. Conveniently it does give you an idea of filesize based on your selections, so you can avoid accidentally tasking the creation of millions tiles and a multi-day export process.

Displaying the Results

At this point, I was expecting to have to set up a tileserver, and had already installed Tilestache to facilitate it. At that point, I found a comment that later versions of QGIS supported MBTiles directly, so fired it up and tried adding them as a raster layer.

For the first run I exported the MBTiles from TillMill for zoom levels 10-16, which is good for scales up to about 1:6,200 (since the tiles are images they get a little blurred at larger scales. I might try exporting down to zoom level 17, but that will result in a file 4x larger (as each zoom level creates four tiles for each tile above it). Render speed is fantastic though: far, far better than using vector layers for the basemap.

MBtiles in QGIS

You can see one further advantage of using OSM data here – the spiral shape of the Albert “Tibby” Cotter bridge over Anzac Parade is visible, and it was only finished a month or two ago.

  1. I’m guessing I’d have to use ogr2ogr first to clip, and then use osm2pgsql

Posted in GIS | Tags: , , ,  | 0 Comments

mapping cycling infrastructure

Posted by: Ian on 10 April 2015

I’ve seen a few ‘abstract’ cycle maps in the last few days, first by the Washington Post and then by TransportBlog, so I thought I would have a go at making one for Sydney. On the left is the road network for Sydney, while on the right is the bicycle infrastructure (I.e. segregated lanes, tracks or share paths).

CycingInfra

[click for full version]


A few caveats:
  1. The maps are based on Open Street Map so the quality of the map is only as good as that of the underlying data. From my knowledge of Sydney, the bike infrastructure looks about right; although I know that Botany Sea Wall doesn’t have a bike lane and that Hopetoun Avenue and Malabar Road do (at least in parts). Ideally I would like to find a dataset of all Sydney bike infrastructure, but have not managed to yet, so have to work with what is available.
  2. I have not shown roads only identified as being part of the ‘Local Cycling Network’ as these are often just quiet streets.
  3. I’ve focused on the CBD and Eastern Suburbs as its ‘my area'; to cover all of Sydney brings its own issues of being able to render something aesthetically acceptable and still useful.

Posted in Cycling,GIS | Tags: , ,  | 0 Comments

now you see me

Posted by: Ian on 10 April 2015

now you see me


via Instagram

Posted in Instagram | 0 Comments

spring classics

Posted by: Ian on 5 April 2015

spring classics


via Instagram

Its the 99th running of the Ronde van Vlaanderen today, so it seems like a perfect excuse to hit up the local fromagerie for some fancy cheeses. (L-R Pecorino Affienato, Brillat Savarin aux Truffle and Blu 61).

Posted in Instagram | 0 Comments

caek

Posted by: Ian on 25 March 2015

caek


via Instagram

Pre-run fuelling.

Posted in Instagram | 0 Comments

skullhead

Posted by: Ian on 10 February 2015

skullhead


via Instagram

I’ve been a fan of Huck Gee ever since finding some of his designs on my trip to San Francisco a couple of years ago. Since I neither have the space nor the funds to get everything, I settled for a book of his artwork instead. For now.

Posted in Instagram | 0 Comments

mr crackles

Posted by: Ian on 7 February 2015

mr crackles


via Instagram

Since it’s just down the road, I thought I would give it a go today. Next time I’m going for the double-meat half-salad ‘manwich’.

Posted in Instagram | 0 Comments

failure to comprehend

Posted by: Ian on 4 February 2015

Earlier this week, the Professional Drivers Branch of the GMB Trade Union, tweeted the following:

The document they link is this one.

It’s not exactly a well formatted document (it looks like the output from a rudimentary database querying tool), and one could misread the blue hyperlink as the title of the data. However, to do this also requires you to accept the concept that cyclists killed 430 and injured 25,000 pedestrians in Greater london over the space of 4 years. Yep, you have to accept that cyclist kill a pedestrian in London every three days, and not have that trigger a ‘I doubt that’ response. A stunning lack of rational thinking if there ever was one.

Now, I didn’t know anything about GMB Pro Drivers before today (and had to look them on up wikipedia), but a quick skim through their timeline rather suggests the ‘Pro’ in the name is not, as you might expect, for ‘professional’, but ‘for’, as in ‘for’ drivers, and against anything that affects drivers, and in their case, a particular dislike of cyclists.

Of course, like the majority of organisations like this, rather than retract what they said and just say they messed up, they quickly went aggressive, blocking people trying to correct them and proclaiming ‘Don’t shoot the messenger’. When the messenger delivers the wrong message because they don’t think, I’d say it is time to get a new messenger. At no point did the person looking at the information question what would sure be unbelievable to most people.

This is actually the document they should have tweeted, but since that shows bikes killing almost no-one, I doubt they would have, if they had comprehended what they were looking at.

Posted in General | Tags: , ,  | 0 Comments

kurtosh

Posted by: Ian on 26 January 2015

Untitled


via Instagram

Tried out a new cake shop today, where you pay by weight rather than per piece. It seemed only right that we got a selection of small ones to try and share.

Posted in Instagram | 0 Comments

2014 in numbers

Posted by: Ian on 31 December 2014

Rides and runs have been collated, cross referenced and matched, it’s the last day of the year and I hopefully haven’t lost any records.

My stats for previous years can be seen here: 2011, 2012 and 2013.

Year Cycling Running Hours
2014 6371km/3,959 miles – 231 hours 937km/582 miles – 77 hours 309 hours
2013 8254km/5129 miles – 300 hours 806km/522 miles – 68 hours 368 hours
2012 5044km/3134 miles – 181 hours 1025km/637 miles – 91 hours 272 hours
2011 3859km/2398 miles – 141 hours 741km/460 miles – 70 hours2 221 Hours
2010 5277km/3279 miles – 193 hours N/A 193 Hours
2009 2591km/1610 miles 114 hours N/A 114 Hours
2008 1338km/831 miles – 63 hours1 N/A 63 hours

Riding

It’s been a pretty average year for cycling, I’ve been searching for motivation for much of the year, and think it’s finally coming back.
 
I was really happy to complete this years Rapha Festive 500 challenge, more-or-less finishing in six days, giving me a super easy ride for day 7 and a rest day today. Everyone I normally ride with was away from Sydney for Christmas, or not riding, so it was 18 hours of solo riding with sun, wind and rain (though fortunately very little rain).
 
The fatigue I’ve built up over the last few days will pass soon, leaving me faster and stronger, making it all worth it.

Ride Fitness 2014

My PMC chart for 2014, with the blue representing overall fitness. Generally pretty poor compared to other years (where I was above 50 for most of 2013, with a fair amount of time above 75), but at least its heading in the right direction.

Running

Running for the first eight months of year went really well, with my weekly mileage up to around 40km, long runs of 16km and a pretty decent pace. From around September, my focus switched away from running and I wound up just doing a run or two each week through November and December. Since I have half a mind to try at least a half-marathon distance next year, I will have to try and work in a reasonable plan, at least to give me an option to try 21.1km.

Pilates

Pilates each Thursday evening has replaced part of my previous Thursday evening rides, so I’ve traded some 20km a week on the bike for some additional flexibility and increased core strength. Ironically, none of the people who persuaded me to start Pilates have been for the best part of a year, so it’s nearly always just me and another guy each week.

Overall

2014 could have been better in general terms as well as fitness wise. However, I still got in a fairly decent amount of quality hours in, and am looking forward to the future (which, at this moment comprises a bottle of Bulmers from the fridge, and a BBQ).
  1. From April, my bike was on a boat for the previous months
  2. Also from April, when I started running.

Posted in Cycling,Running | 0 Comments

post ride espresso

Posted by: Ian on 13 December 2014

post ride espresso - beans by Cafe Hernandez, new cup by @kanimblaclay


via Instagram

Ethiopean beans by Cafe Hernandez, new cup by @kanimblaclay

Posted in Instagram | 0 Comments

gigantauchocolat

Posted by: Ian on 13 December 2014

The Garmin had to be included for scale. #Gigantauchocolat


via Instagram

i needed some coffee beans so ended todays ride at the excellent Cafe Hernandez on the Kings Cross/ Potts Point border. Not only do they roast on-site, they have been open 24/7 since 1972!

Posted in Instagram | 0 Comments

Next Page »