Openstreetmap Geodata Coming...

Programming Genius Nieko is known for his extra features onto the Mapthis program, but he has more upon his sleeve !

I pointed Nieko on the openstreetmap.org geodata which can be extracted and converted to CSV.

Since it's a very very large file it will take a while before it's converted.

Nieko is working his hands of trying to make the Netherlands geodata work with GMDL now.

England, Belgium, Luxemburg and even Germany and Denmark are there too, but aren't converted since they aren't complete (only the Netherlands is complete). When finished it will surely have a very very nice addon to Mapthis in general.

Stay tuned for more updates :)

congrats, nice work dudes

A short update:

• The original planet.osm file was 15.5GB;
• I've extracted the Netherlands from this, netherlands.osm is now 1.0GB;
• The osm2csv.pl script from OpenStreetMap stagnates after it's produced 2MB of CSV;
• I'll try to write my own script that produces the CSV file;
• There may be several versions, because the AND data also contains supermarkets, parking lots, etc.
• I have no idea yet when this will be done, but hopefully in the upcoming days :).

Quote:

---

Originally Posted by Nieko

A short update:

• The original planet.osm file was 15.5GB;
• I've extracted the Netherlands from this, netherlands.osm is now 1.0GB;
• The osm2csv.pl script from OpenStreetMap stagnates after it's produced 2MB of CSV;
• I'll try to write my own script that produces the CSV file;
• There may be several versions, because the AND data also contains supermarkets, parking lots, etc.
• I have no idea yet when this will be done, but hopefully in the upcoming days :).

---

Nice to Hear ;)
Sad thing was me living in Sweden and not getting any Geos but that hopefully comes after a while :)
:thumbup: to you

Quote:

---

Originally Posted by Codaz

England, Belgium, Luxemburg and even Germany and Denmark are there too, but aren't converted since they aren't complete (only the Netherlands is complete). When finished it will surely have a very very nice addon to Mapthis in general.

Stay tuned for more updates :)

---

Oh thats nice news,but i hope the germanydata are complete in future.


I wait for next news from you;) good luck.........

this is great ,i hope you can get it to work

i see on openstreetmap.org that you can download the nl AND file
only 84,4 mb

http://www.opengeodata.org/images/20.../AND-NL.tar.gz

johoo

I saw that file, but it contains mainly binary data, which didn't work well yet with MapThis. I prefer textual data (XML from OpenStreetMap in this case) :).

Also an update: I've parsed the nodes, segments and ways from the OSM file and am now writing a first CSV version containing street names. There is however one big problem, the OSM data doesn't indicate what city a street belongs to. On the OSM interface, it simply says that a street is closest to a particular city, so I'll have to make a script that does the same.

Problem is, I have ~900.000 streets and ~4.000 places+zip codes. That'd mean 900.000*4.000 = a lot of comparisons. Does anyone have a good idea on how to bring this number down?

guys,

how do i make my geodata work? i converted it frm csv to geodata.dat and 2 other files together now i placed it in the main folder of mapthis in my psp. how do i know if its working or not then?

Do i need the geodata.dat to locate me on the map, when i have the psp290?

jakekoh: first make sure the entire CSV file is uppercase, otherwise it won't work (took me a while before I figured that out). You can then check out the Address lookup and Reverse address functions in MapThis. It should work if you only fill in a streetname.

Natanael: no, with the PSP-290 working it'll just show you where you are. You can use geodata to lookup which street is closest or where a particular street is.

As for my progress, I've got everything figured out, and it's now just a matter of time :).

I now have a CSV file with all the streets with their coordinates in it, and another CSV file with all the zipcodes with city and coordinates in it.

Today I'll run the script that will merge these two files, so then I'll have one CSV file containing coordinates, street, city, zip code and province. (Turns out Perl has a nice module that figures out the closest zip code for a street.)

After that, I need to split and sort the streets by zip code, compress them and write them to a geodata.dat.

The scripts for this entire process I will put on my server.

(Btw., I'm not using GMDL's CSV import cause I couldn't find out (and didn't want to spend too much time on finding out) how to include zip code and province in the geodata.)

So for all the Dutch users, be patient, in a couple of days I'll be done :).

Nice Nieko keep it coming :)

When i pointed a street on and want to get lead there, is that able without geodata?

Natanael: I don't understand your question, please rephrase.

I have a mark set on the map.
Map!This shows me the distance to the mark.
I`m going to get to this target.
Will Map!This lead me to my target, or do i need geodata for this?

If you have a GPS dongle, MapThis will show you where you are on the map. If you have geodata, you can look up where a specific street is on the map. If you want to have street-to-street directions on how to drive, then you can use either GMDL (click the tab 'Directions') or my script, which can be found at http://getroute.nieko.net/. This will generate a POI file, which you can load in MapThis.

And I'm done! :) For those who can't wait, Dutch geodata can be found here:
http://stuff.nieko.net/psp/mapthis/geodata/geodata.dat

Put it in the same folder where MapThis' EBOOT.PBP is, or in a map folder.

On http://stuff.nieko.net/psp/mapthis/geodata/ you can also find the scripts I've made to convert the OSM data to MapThis compatible geodata (process.zip). Try it out if you want :)

There, you can also find a _ZIPCODE POI file, which you can use to lookup where a specific zipcode is (POI Lookup => circle button), or just to show the zipcodes around the current location on the map.

Be sure to also check out the readme.txt!

in7ane: with these scripts, do you think you could take a look at geodata for entire Europe? My version heavily relies on Dutch zip codes, so it won't work directly.

HURRAY for Nieko :)

England is also available on openstreetmap geodata so anyone can import that too on Mapthis.

Get Nieko's perl scripts, rebuild them for England and convert the planet.osm to geodata.dat :)

Easier said that done but ok :)

Nieko needs to check the poi file hahaha :)

There are indeed some anomalies in the data... If anybody has better data for zip codes, cities, provinces and their coordinates, feel free to contact me ;).

Hey Nieko is it possible for germany? can i use your tut. to?

Yes you can use Niekos script, BUT:

-Germany isn't complete (See Dortmund).
-You have to rebuild the perl scripts (thus you have to know how to code perl).
-You have to extract planet.osm which is 15,5 gb extracted. Germany is 1 gig ripped from that i think.

hoi nieko
works great man ,a big thnx

i noticed that i only can search for the streetnames and not the city or zipcode
it says incorrect values entered
but i see the city and zipcode when i find the street so it must be possible to search for city ,state or zipcode

i don't know how to correct that
i've tryed it before with the city csv file and got it working as geo but also only under street search
on the internet i couldn't find the answer for it
also on wiki etc.

anybody how knows the answer
haha if you understand what i wrote

how to search under other topics ?street,city,state,zipcode

If you're using geodata, you always have to fill in the street name. If you want to search for a zipcode or a specific town, you can do a POI Lookup + search (circle button) on the _ZIPCODE POI file.

Can anyone upload a Geodata for Germany or Western Europe, because downloading the whole World takes so a long time!
That will help me and some others alot.

Quote:

---

Originally Posted by Codaz

Yes you can use Niekos script, BUT:

-Germany isn't complete (See Dortmund).
-You have to rebuild the perl scripts (thus you have to know how to code perl).
-You have to extract planet.osm which is 15,5 gb extracted. Germany is 1 gig ripped from that i think.

---

Thanks Codaz;) but i see that I`m not able to rebuild perl scripts,i`m not a pro *grins*

I wait for userfriendly things;).I hope.


greets

With the new geodata, it seems we've gone from 734512 streets to 886259. That's over 20% more! :) Enjoy the new geodata at: http://stuff.nieko.net/psp/mapthis/geodata/geodata.dat.

Hi.
Can anyone upload geodata for poland? 15gb its really huge for me and downloading that would be hard.. Thx

v1ck

It's only 900 mb, extracted it's 15.5 gb :) Very good compression :)

Quote:

---

Originally Posted by Codaz

It's only 900 mb, extracted it's 15.5 gb :) Very good compression :)

---

i have HDD 20gb :> ~5gb left ;)

Quote:

---

Originally Posted by v1ck

Hi.
Can anyone upload geodata for poland? 15gb its really huge for me and downloading that would be hard.. Thx

v1ck

---

Whats the diffrence of generating it yourselve and downloading it ?
Its still the same size, but if you download a zipped file you must have twice the space than if you generate it yourselve ;)

+ If you generate it yourselve you can check off some zooms and stuff ;)

Quote:

---

Originally Posted by PDG

Whats the diffrence of generating it yourselve and downloading it ?
Its still the same size, but if you download a zipped file you must have twice the space than if you generate it yourselve ;)

+ If you generate it yourselve you can check off some zooms and stuff ;)

---

15gb is for planet.osm, i just need Poland extracted and that would be much lower, right?

Quote:

---

Originally Posted by v1ck

15gb is for planet.osm, i just need Poland extracted and that would be much lower, right?

---

Makes sense :)

On openstreetmap.org, Poland looks pretty empty to me. Warschaw has some detail, but I think "geodata for Poland" isn't too useful, because most streets won't be in it yet anyway.

Quote:

---

Originally Posted by Nieko

On openstreetmap.org, Poland looks pretty empty to me. Warschaw has some detail, but I think "geodata for Poland" isn't too useful, because most streets won't be in it yet anyway.

---

ehh.. too bad ;/ Btw, what about south Ireland? Is it empty as well?

You can check that for yourself :).

Nieko:

would you be able to make some map-editor that whe could edit streetnames on generated googlemaps ? And then save it to geodata.dat? you can edit maps on
http://openstreetmap.org/ but after that you still have to download planet.osm and export what you need.. its usless..

I think that you could add some options to GMDL where you can put names for streets and generate geoadata..

its just an idea :) What you think?

I don't think that's a good idea, or I'm not quite getting it. The idea of geodata is that you can look up streets in a particular area, and that's not much use if that data isn't anywhere near complete (you might as well make a POI file with those few streets you need).

I only started working on Dutch geodata when the AND import was done, and basically the entire Netherlands was available.

AND provided Dutch geodata and is a provider of various map and navigation solutions, like Map24 and mapquest.com

Maybe you can email them on the status of geodata and/or maps of Poland.

Donation to Openstreetmap won't happen that often since that material is licensed and very very expensive (think in amounts of 200-500 dollars).

woohoo, awesome!
Works quite well, it's actually able to find my street, the postal-code is reported incorrectly though :P