14 replies to this topic

[frax]

Hi,

Inspired by recent discussions on using OSM, I thought I would try to see if I could use Open Streetmap for a project I was working on just now. I have been working on a map over South Sudan, and now the client decided that they wanted major roads on there as well. It is just for background, so the accuracy is not crucial.

Anyhow - I suspect that the data I have handy, like the ArcGIS Data & Maps roads layer (from Delorme) is decent for my purposes, but I have hunch that the OSM data for this region might be even better. That also depends on how it is classified though, I have never worked with it, so it is part of the exploration!

I kind of assume that the data for this region is not so detailed (the OSM data for a decent sized Western city is probably more, in terms of size!?).

But - I have problems doing simple exports - e.g. XML or the SVG formats provided by the main OSM website.

And that site with shape file exports has only a few countries in Africa... Global Mapper only loads up pre-rendered tiles (no vector data).

I tried with Maperitive now, but all I get is timeouts for this area of the world... And downloading using the REST/API url tells me that the bbox is too large...

So is there some other smart/easy way? I am downloading the 12 GB planet file right now.

(this is just for fun, I delivered the map with the ESRI roads data)

[Charles Syrett]

If you're talking about CloudMade, I've had occasions where data is simply missing while they do maintenance, and then it reappears next day. Then again, I've never tried downloading African data. Have you tried Tracks4Africa?

Charles Syrett
Map Graphics
http://www.mapgraphics.com

[Nick H]

There is an OSM shapefile for Sudan here (2.2-meg download):

http://download.clou...om/africa/sudan

Regards, N.

[Hans van der Maarel]

I can do a custom extract out of the database for you, full data model, delivered as shapefiles. May not be the best option for very large areas, as it's a very slow process and the OSM servers are prone to lags. Contact me privately if you're interested.

[Crischan]

I have had trouble using the Cloudmade OSM XML data in the past where the data would be corrupted for some reason. Since then I am more than happy with the downloads (XML, Shape) from http://www.geofabrik...a/download.html (German company, English website)
They say the update their data daily using the OSM servers.

Cheers,
C

[frax]

Thanks for all the input! I had missed that Cloudmade site, I have downloaded the file to peek at the data. The Geofabrik site does not have data for Sudan (they have a whole Africa osm-file).

Hans, thanks for the offer, but I don't think it is needed right now. I assume that you have downloaded a planet.osm file (and converted it?).

Any tips on what I would use to do extracts from the giant planet.osm file?

[Flanners]

I was just looking at this yesterday. I exported an osm XML file, then opened it in Quantum GIS- QGIS has a plugin (go to plugins menu/ manage plugins) that lets you load an osm file, which can be saved as a shape file.

That was just for a small area though, not a whole country...

[Nick H]

Any tips on what I would use to do extracts from the giant planet.osm file?

Hum, a twelve-gigabyte OSM file. The only practical way I can think of would be to load it into a PostGIS database using osm2pgsql. Once aboard the lugger, sorry, once in the database it would be possible to select the data required by extent and feature and bring it into GIS.

There are tools that are supposed to be able to subset OSM data, but I'm not sure how they would perform with a such a huge OSM file.

Regards, N.

[frax]

I am trying the "OSM Loader" python script that I found on arcscripts - it takes a osm/bz2 archive and puts it into a file geodatabase. It bailed out on the full 13GB planet.osm though (some size limitation?) and I am also getting an error from bzip2.exe on that file (but the checksum is correct, so it shouldn't be corrupted!).

But that loader script seems to work fine on the 183MB africa.osm.bz2 file that I picked up... (still running though, so i haven't seen the result yet!)

[Hans van der Maarel]

Hans, thanks for the offer, but I don't think it is needed right now. I assume that you have downloaded a planet.osm file (and converted it?).

I haven't dared doing that yet... No, my process queries the OSM database directly, which is why it's not exactly suited to doing large regions.

I might give the planet.osm-to-database a go. Once it's in there I can get by with just the diff files and I'd be able to get extracts faster since it'll be coming out of my local database.

[frax]

That OSM loader script seemed to work ok - running overnight for the entire africa planet file, so now I have that data in a file geodatabase for ArcGIS...

[frax]

I haven't dared doing that yet... No, my process queries the OSM database directly, which is why it's not exactly suited to doing large regions.

My understanding is that there is a maximum bbox of .25 degrees by .25 degrees for the REST api - so one would have to split into tiles and query for them separately, and then merge them...

[Hans van der Maarel]

My understanding is that there is a maximum bbox of .25 degrees by .25 degrees for the REST api - so one would have to split into tiles and query for them separately, and then merge them...

I thought it was limited to a certain number of objects. A fixed size would be too easy...

[frax]

FYI - I am not able to work with the 13GB planet file in the OSM loader from arcscripts - I suspect the file is too large and that it reaches some limit in the bz2 lib for python...

[redbeard]

And that site with shape file exports has only a few countries in Africa... Global Mapper only loads up pre-rendered tiles (no vector data).

So is there some other smart/easy way? I am downloading the 12 GB planet file right now.

(this is just for fun, I delivered the map with the ESRI roads data)

I know this thread is a little old...but

Actually globalmapper can read the OSM xml based file - just not through the menu that deals with online sources.

I will warn you, if you extract the osm file for some reason, it's huge. I suspect that the arc method might have run out of memory. I'm also interested in getting the entire planet file into a geodatabase. Has anyone since been able to do this?

Also, has anyone written a sqlite or spatialite importer for the Planet file? I ask as I wouldnt mind making my own tiles, but wish to symbolize using QGIS or ArcMap (or something similar). Editing the mapnik xml files gives me a headache.

Maybe I'll have to take a look at tilemill.