Hello, I make some extensions to my streetlist-script so that it gets nearly the functionality of the old query-to-map script but is independent of XAPI and GoogleMaps.
It now support point, line and area objects. As example you can see all churches near Dresden: http://cassini.toolserver.org/~kolossos/qtm2/queryinmap.php?BBOX=13.5333,50.... If you click on the objects you can get more informations about the object. And you can get offcourse also a list of it: http://cassini.toolserver.org/~kolossos/qtm2/featurelist.php?key=amenity&...
More documention you can found in the wiki: http://wiki.openstreetmap.org/wiki/Query-to-map
Before I want to promote it for to OSM-community I want to ask, if we can get an update of the database? And could we get some additionally columns to the database? The following keys would be really usefull and are not too much: cycleway, shop, lit, operator, surveillance, website and wikipedia. Who can do this?
Greetings Kolossos
BTW: The OSM-Toolserver get a kind of competition: http://wiki.openstreetmap.org/wiki/FOSSGIS/Server This can be good for OSM and we can concentrate us perhaps to the knowledge aspect of OSM and the interface to wikipedia. http://dict.leo.org/ende?lp=ende&p=DEdPgA&search=competition
Before I want to promote it for to OSM-community I want to ask, if we can get an update of the database? And could we get some additionally columns to the database?
The following keys would be really usefull and are not too much: cycleway, shop, lit, operator, surveillance, website and wikipedia. Who can do this?
Can we do it ourselfs? If one of the admins give the go we could import an up2date planet dump on our own, i think.
Peter
Peter Körner schrieb:
Before I want to promote it for to OSM-community I want to ask, if we can get an update of the database? And could we get some additionally columns to the database?
The following keys would be really usefull and are not too much: cycleway, shop, lit, operator, surveillance, website and wikipedia. Who can do this?
Can we do it ourselfs? If one of the admins give the go we could import an up2date planet dump on our own, i think.
Hello Peter, I hear no veto of an admin in the last 5 days and if I see the low load-avg of cassini I think you should feel free to do so. Kolossos
P.S.: What's going on on Ptolomy? Is the data import running?
Peter
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Tim Alder tim.alder@s2002.tu-chemnitz.de wrote:
Hello Peter, I hear no veto of an admin in the last 5 days and if I see the low load-avg of cassini I think you should feel free to do so. Kolossos
P.S.: What's going on on Ptolomy? Is the data import running?
Yes, I have just decided to give up on full OSM API database and prepare DB for mapnik plus the update processes to keep it up to date.
Sorry it took so long.
I hear no veto of an admin in the last 5 days and if I see the low load-avg of cassini I think you should feel free to do so.
I started with it. I'd also like to play a little with the replication-diffs, but no promise on that. after the import using osm2pgsql, is there any special action neccessary to make mod_tile flag all tiles as dirty?
Peter
Peter Körner wrote:
I started with it. I'd also like to play a little with the replication-diffs, but no promise on that. after the import using osm2pgsql, is there any special action neccessary to make mod_tile flag all tiles as dirty?
In render_config.h:
// Planet import should touch this file when complete #define PLANET_TIMESTAMP "/planet-import-complete"
touch that file and that should be it.
The server on osm.org first gets current with all diffs before touching that file.
The import has started, with diff support.
As I neither have permissions on /planet-import-complete nor on /sql I won't be able to invoke a re-render nor I can update the shorelines or the mapnik style. I'll open a Ticket about it in jira but I'm unsure If this can be done without giving root perms, what is not really what I want.
I'm going to put all tasks from [1] into a script [2].
Peter
[1] http://wiki.openstreetmap.org/wiki/Minutely_Mapnik [2] /mnt/user-store/mazder-cassini-planet-reimport/task
Lennard wrote:
Peter Körner wrote:
I started with it. I'd also like to play a little with the replication-diffs, but no promise on that. after the import using osm2pgsql, is there any special action neccessary to make mod_tile flag all tiles as dirty?
In render_config.h:
// Planet import should touch this file when complete #define PLANET_TIMESTAMP "/planet-import-complete"
touch that file and that should be it.
The server on osm.org first gets current with all diffs before touching that file.
Peter Körner wrote:
The import has started, with diff support.
As I neither have permissions on /planet-import-complete nor on /sql I
I forgot to tell you that that file is relative to the tile_dir (HASH_PATH).
I forgot to tell you that that file is relative to the tile_dir (HASH_PATH).
Ok, I'll have to take a look into the mod_tile sources so this won't happen again :)
I just asked on jira for permissions on /sql [1], as everything mapnik/osm important happens on this partition (tiles_dir is located there).
Import is at Node(136000k) Way(0k) Relation(0k)
Peter
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009. Unfortunately not everything is perfect:
1. When going through the logs I saw the "intarray contrib module not installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
2. I seems group user now has permissions on most dirs on /sql, including /sql/mod_tile, so i did "touch /sql/mod_tile/planet-import-complete" but nothing happened. It seems that there is an issue with mod_tile and/or renderd, as cached tiles are not updated and no new tiles are rendered (see e.g. [2]). As this is the first time I'm working with mod_tile/renderd and -for the sake of privacy- got no access to the apache error log (/var/log/apache2/error.log) I got no clue what's going on.
3. The new mapnik-styles are splitted in a bunch of xmls and the l18n process has to be adapted to this
Any ideas? Peter
[1]https://jira.toolserver.org/browse/TS-380 [2]http://cassini.toolserver.org/tile-browse/browse-de.html?zoom=13&lat=49....
Peter Körner wrote:
I forgot to tell you that that file is relative to the tile_dir (HASH_PATH).
Ok, I'll have to take a look into the mod_tile sources so this won't happen again :)
I just asked on jira for permissions on /sql [1], as everything mapnik/osm important happens on this partition (tiles_dir is located there).
Import is at Node(136000k) Way(0k) Relation(0k)
Peter
[1] https://jira.toolserver.org/browse/TS-381
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Peter Körner wrote:
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009. Unfortunately not everything is perfect:
Well, at least it is a good start and it will be great to see the internationalised maps uptodate again soon. :-)
...
- I seems group user now has permissions on most dirs on /sql,
including /sql/mod_tile, so i did "touch /sql/mod_tile/planet-import-complete" but nothing happened. It seems that there is an issue with mod_tile and/or renderd, as cached tiles are not updated and no new tiles are rendered (see e.g. [2]). As this is the first time I'm working with mod_tile/renderd and -for the sake of privacy- got no access to the apache error log (/var/log/apache2/error.log) I got no clue what's going on.
It looks like the touch /sql/mod_tile/planet-import-complete worked. At least all of the tiles whos status I have looked at say they are due to be rerendered, i.e. marked dirty. (e.g. http://cassini.toolserver.org/tiles/osm-like/de/5/1/1.png/status ) So mod_tile is correctly handling things it seems. However, they never get rendered. Also tiles that are missing e.g. http://cassini.toolserver.org/tiles/osm-like/de/6/1/1.png immediately return with a 404, rather than waiting 3 seconds and then returning 404 after a timeout on waiting for renderd. So my guess would be that renderd is not correctly working and either mod_tile has no connection to the render daemon at all, or it returns errors all the time. Do you have access to renderd's error log?
Not much help, but perhaps it gives some hints where to start looking.
Kai
...
Any ideas? Peter
On Wed, Nov 4, 2009 at 7:18 PM, Peter Körner osm-lists@mazdermind.de wrote: (Sorry, don't have time to look at all of this)
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009. Unfortunately not everything is perfect:
- When going through the logs I saw the "intarray contrib module not
installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
Don't know how to fix this;/
- I seems group user now has permissions on most dirs on /sql,
including /sql/mod_tile, so i did "touch /sql/mod_tile/planet-import-complete" but nothing happened. It seems that there is an issue with mod_tile and/or renderd, as cached tiles are not updated and no new tiles are rendered (see e.g. [2]). As this is the first time I'm working with mod_tile/renderd and -for the sake of privacy- got no access to the apache error log (/var/log/apache2/error.log) I got no clue what's going on.
Renderd isn't actually running. When I was debugging this I just ran it from a shell, but I didn't have the know-how to get the debian package for renderd to work (which would have installed an init script).
- The new mapnik-styles are splitted in a bunch of xmls and the l18n
process has to be adapted to this
There's a preprocessor to merge them all, but the script may need munging regardless.
- When going through the logs I saw the "intarray contrib module not
installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
Don't know how to fix this;/
It seems to be related to bug [1]. There is an open Jira-Ticket for this one (see [2]).
- I seems group user now has permissions on most dirs on /sql,
including /sql/mod_tile, so i did "touch /sql/mod_tile/planet-import-complete" but nothing happened. It seems that there is an issue with mod_tile and/or renderd, as cached tiles are not updated and no new tiles are rendered (see e.g. [2]). As this is the first time I'm working with mod_tile/renderd and -for the sake of privacy- got no access to the apache error log (/var/log/apache2/error.log) I got no clue what's going on.
Renderd isn't actually running. When I was debugging this I just ran it from a shell
I can't do this because I got no acces to the socket :)
but I didn't have the know-how to get the debian package for renderd to work (which would have installed an init script).
can't we just copy /usr/local/src/osm/applications/utils/mod_tile/debian/renderd.init to /etc/init.d and use /usr/sbin/update-rc.d to activate it?
sudo cp \ /usr/local/src/osm/applications/utils/mod_tile/debian/renderd.init \ /etc/init.d
sudo /usr/sbin/update-rc.d renderd defaults
- The new mapnik-styles are splitted in a bunch of xmls and the l18n
process has to be adapted to this
There's a preprocessor to merge them all, but the script may need munging regardless.
I think leaving them divided is better, as the script has to be adapted in both cases. I'd look into this but I don't have the perms to write to the files on /sql. There is a Jira-Ticket open for this one, too [3].
Peter
[1] https://bugs.launchpad.net/ubuntu/+source/postgresql-8.3/+bug/250483 [2] https://jira.toolserver.org/browse/TS-380 [3] https://jira.toolserver.org/browse/TS-381
On Wed, Nov 4, 2009 at 8:30 PM, Peter Körner osm-lists@mazdermind.de wrote:
- When going through the logs I saw the "intarray contrib module not
installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
Don't know how to fix this;/
It seems to be related to bug [1]. There is an open Jira-Ticket for this one (see [2]).
- I seems group user now has permissions on most dirs on /sql,
including /sql/mod_tile, so i did "touch /sql/mod_tile/planet-import-complete" but nothing happened. It seems that there is an issue with mod_tile and/or renderd, as cached tiles are not updated and no new tiles are rendered (see e.g. [2]). As this is the first time I'm working with mod_tile/renderd and -for the sake of privacy- got no access to the apache error log (/var/log/apache2/error.log) I got no clue what's going on.
Renderd isn't actually running. When I was debugging this I just ran it from a shell
I can't do this because I got no acces to the socket :)
> but I didn't have the know-how to get the debian
package for renderd to work (which would have installed an init script).
can't we just copy /usr/local/src/osm/applications/utils/mod_tile/debian/renderd.init to /etc/init.d and use /usr/sbin/update-rc.d to activate it?
sudo cp \ /usr/local/src/osm/applications/utils/mod_tile/debian/renderd.init \ /etc/init.d
sudo /usr/sbin/update-rc.d renderd defaults
I copied renderd to /usr/bin/ and installed /etc/init.d/renderd from mod_tile in /usr/local/src and did "sudo /usr/sbin/update-rc.d renderd defaults" It's now running as www-data and appears to be rendering. It's writing out *.meta tiles at least, but it's very slow ...
- The new mapnik-styles are splitted in a bunch of xmls and the l18n
process has to be adapted to this
There's a preprocessor to merge them all, but the script may need munging regardless.
I think leaving them divided is better, as the script has to be adapted in both cases. I'd look into this but I don't have the perms to write to the files on /sql. There is a Jira-Ticket open for this one, too [3].
I created a new "mappers" system group on cassini which currently has you, me and aude in it. And I did:
ravar@cassini:/sql$ sudo chown -R root:mappers etc/ mapnik-stylesheets/ misc-data/ planet.osm/ world_boundaries/ ravar@cassini:/sql$ sudo chmod -R g+rw etc/ mapnik-stylesheets/ misc-data/ planet.osm/ world_boundaries/
The mappers group also has access to /var/log/apache2/
This should work. Tell me if this isn't enough or if you need any additional permissions.
I also put you in the postgres group.
[1] https://bugs.launchpad.net/ubuntu/+source/postgresql-8.3/+bug/250483 [2] https://jira.toolserver.org/browse/TS-380 [3] https://jira.toolserver.org/browse/TS-381
Ævar Arnfjörð Bjarmason schrieb:
I copied renderd to /usr/bin/ and installed /etc/init.d/renderd ..
I created a new "mappers" system group on cassini ..
I installed all of these.
Ævar, you're cool!
Thank you! Peter
but it's very slow ...
Just a guess: maybe the debug level is too high /var/log/apache2/error.log is aroung 3GB already..
Can we change "LogLevel debug" to "LogLevel warn" in /etc/apache2/sites-enabled/000-default
and cp /dev/null /var/log/apache2/error.log
Is it normal that mod_tile is botherd for each and every http-request? The rror_log contains lines like
[Thu Nov 05 09:..:.. 2009] [info] [client ...] tile_storage_hook: handler((null)), uri(/~mazder/multilingual-country-list/index.php), filename(/home/mazder/osm_public_html/multilingual-country-list/index.php), path_info((null))
but why should mod_tile bother with /home/mazder/osm_public_html/multilingual-country-list/index.php
Peter
On Thu, Nov 5, 2009 at 9:16 AM, Peter Körner osm-lists@mazdermind.de wrote:
but it's very slow ...
Just a guess: maybe the debug level is too high /var/log/apache2/error.log is aroung 3GB already..
Can we change "LogLevel debug" to "LogLevel warn" in /etc/apache2/sites-enabled/000-default
and cp /dev/null /var/log/apache2/error.log
I'll try to have that fixed before /var overflows again.
Is it normal that mod_tile is botherd for each and every http-request? The rror_log contains lines like
[Thu Nov 05 09:..:.. 2009] [info] [client ...] tile_storage_hook: handler((null)), uri(/~mazder/multilingual-country-list/index.php), filename(/home/mazder/osm_public_html/multilingual-country-list/index.php), path_info((null))
but why should mod_tile bother with /home/mazder/osm_public_html/multilingual-country-list/index.php
AFAIK yes. It's not designed to run on anything but a stand-alone tileserver.
It's probably relatively trivial to change the apache plugin to bind to something other than / though.
AFAIK yes. It's not designed to run on anything but a stand-alone tileserver.
It's probably relatively trivial to change the apache plugin to bind to something other than / though.
The easiest thing would be to have another VirtualHost for the tiles, listening eg. on port 8000. If you want to I can write you the config-files and put them somewhere.
I'm goint to start another re-import this evening, this time *really* with diff support and with the additional fields for Tim & Colin.
I won't get to the diff job until sunday, but I think we could have a minutely updating database on sunday evening.
Just a question: would it be ok for all if we start to accept additional mapnik styles? e.g. Colins hike'n'bike? Can we open this somehow to the toolserver users, so they can work on their styles?
Peter
Peter Körner schrieb:
AFAIK yes. It's not designed to run on anything but a stand-alone tileserver.
It's probably relatively trivial to change the apache plugin to bind to something other than / though.
The easiest thing would be to have another VirtualHost for the tiles, listening eg. on port 8000. If you want to I can write you the config-files and put them somewhere.
Or maybe even better: another subdomain..
- tiles.cassini.toolserver.org - tiles-dev.toolserver.org
or sth. like this
Peter
On Thu, Nov 5, 2009 at 12:18 PM, Peter Körner osm-lists@mazdermind.de wrote:
Just a question: would it be ok for all if we start to accept additional mapnik styles? e.g. Colins hike'n'bike? Can we open this somehow to the toolserver users, so they can work on their styles?
Yes. Once we have the mapnik database running we want to support something like this. Having cool tools like the hiking map on our servers is one of the points of this project :)
But as discussed elsewhere we're going to have to find out exactly how we want to manage that, but it should definitely be done.
Ævar Arnfjörð Bjarmason schrieb:
On Thu, Nov 5, 2009 at 9:16 AM, Peter Körner osm-lists@mazdermind.de wrote:
but it's very slow ...
Just a guess: maybe the debug level is too high /var/log/apache2/error.log is aroung 3GB already..
Can we change "LogLevel debug" to "LogLevel warn" in /etc/apache2/sites-enabled/000-default
and cp /dev/null /var/log/apache2/error.log
I'll try to have that fixed before /var overflows again.
It's still not done.. I'd do it for myself but I don't have perms on /etc/apache2/sites-enabled/000-default and I'm also unable to restart apache2 after clearing the log (unsure if this is necessary).
I created two config files in /sql/etc - one creates a virtual host listening on port 80, serving the user-dirs and /var/www with mod_suphp, the other creates a tile-server on port 8000. As I included the userdir and suphp config into the virtual host, we should remove
/etc/apache2/mods-enabled/suphp.conf and /etc/apache2/mods-enabled/userdir.conf
I was *not* able to test them and I don't know if it will work like this. I opened a Jira-Ticket for those two things: https://jira.toolserver.org/browse/TS-383
The htmls at /var/www/tile-browse will need to be rewritten for this.
Peter
On Thu, Nov 5, 2009 at 9:16 AM, Peter Körner osm-lists@mazdermind.de wrote:
but it's very slow ...
Just a guess: maybe the debug level is too high /var/log/apache2/error.log is aroung 3GB already..
Can we change "LogLevel debug" to "LogLevel warn" in /etc/apache2/sites-enabled/000-default
and cp /dev/null /var/log/apache2/error.log
Done, now logging at warn & restarted apache
Ævar Arnfjörð Bjarmason wrote:
- The new mapnik-styles are splitted in a bunch of xmls and the l18n
process has to be adapted to this
There's a preprocessor to merge them all, but the script may need munging regardless.
If you really want a monolithic file, you can remove all entities and collapse it all into a single file:
$ xmllint --noent osm.xml >osm-noent.xml
Peter, what is it that you do for the i18n? Are you only replacing the name fields with the appropriate name:xx?
Peter, what is it that you do for the i18n? Are you only replacing the name fields with the appropriate name:xx?
I think it's even easier: we got views for each language that selects the right name:xx-tag and puts it into a virtual name field, so the only thing that needs to be changed is the table-prefix (from planet_osm_ to view_de_planet_osm_ or so). This can be done in the settings.xml, so we'll need a bunch of settings.xmls and a bunch of osm.xmls, right?
Peter
On Thu, Nov 5, 2009 at 12:21 AM, Peter Körner osm-lists@mazdermind.de wrote:
Peter, what is it that you do for the i18n? Are you only replacing the name fields with the appropriate name:xx?
I think it's even easier: we got views for each language that selects the right name:xx-tag and puts it into a virtual name field, so the only thing that needs to be changed is the table-prefix (from planet_osm_ to view_de_planet_osm_ or so). This can be done in the settings.xml, so we'll need a bunch of settings.xmls and a bunch of osm.xmls, right?
Correct. Generating the stylesheets is much easier now that mapnik supports config inclusion. We should do this as you suggest.
Peter Körner wrote:
thing that needs to be changed is the table-prefix (from planet_osm_ to view_de_planet_osm_ or so). This can be done in the settings.xml, so we'll need a bunch of settings.xmls and a bunch of osm.xmls, right?
Correct. Also, the few settings includes (which are in svn with the .example suffix) won't be touched by svn, so updating the stylesheet from svn should be transparent*.
Probably easiest is to checkout the stylesheet in a directory, and have a script that copies all osm.xml and inc/*.xml to various other directories, one per wiki language, and customize the settings.xml.inc with the correct &prefix; entity for your view.
* Barring any future changes to the settings, such as the definition of newer settings entities. You could test the structure of the stylesheet after an svn up, before you copy it over to the language dirs.
2009/11/4 Peter Körner osm-lists@mazdermind.de:
- When going through the logs I saw the "intarray contrib module not
installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
http://weait.com/content/build-your-own-openstreetmap-server has the part:
Enable intarray: psql gis < /usr/share/postgresql/8.3/contrib/_int.sql
Maybe that's it already?
Cheers Colin
http://weait.com/content/build-your-own-openstreetmap-server has the part:
Enable intarray: psql gis < /usr/share/postgresql/8.3/contrib/_int.sql
Hummmm.. this seams sensible to me. I'll try it, soon (see my answer to your other mail)
Peter
On Wed, Nov 4, 2009 at 11:40 PM, Colin Marquardt cmarqu42@googlemail.com wrote:
2009/11/4 Peter Körner osm-lists@mazdermind.de:
- When going through the logs I saw the "intarray contrib module not
installed" message. I was really sure that this has been fixed (see [1]) but it seems I missed sth. I'll reopen that ticket, but maybe so. of you already knows what went wrong.
http://weait.com/content/build-your-own-openstreetmap-server has the part:
Enable intarray: psql gis < /usr/share/postgresql/8.3/contrib/_int.sql
I executed this. intarray should work now.
2009/11/4 Peter Körner osm-lists@mazdermind.de:
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009.
It looks like you imported the columns that were present the last time? If so, it's no use to me yet for the Hike&Bike map (some additional columns are in http://mapnik-utils.googlecode.com/svn/sandbox/cascadenik/hike_n_bike/defaul...).
Cheers Colin
Colin Marquardt schrieb:
2009/11/4 Peter Körner osm-lists@mazdermind.de:
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009.
It looks like you imported the columns that were present the last time? If so, it's no use to me yet for the Hike&Bike map (some additional columns are in http://mapnik-utils.googlecode.com/svn/sandbox/cascadenik/hike_n_bike/defaul...).
Yes, you're right. This is the first time I do a planet import and also the first time I struggle with mod_tile, renderd and not-beeing root on cassini :)
So i tried to reproduce Ævars steps and get everything going. As I missed the intarray thing it's likely that I'm going to do a re-import soon, maybe over the weekend. This time I'll use the combined style containing the additional tags.
Peter
2009/11/5 Peter Körner osm-lists@mazdermind.de:
So i tried to reproduce Ævars steps and get everything going. As I missed the intarray thing it's likely that I'm going to do a re-import soon, maybe over the weekend. This time I'll use the combined style containing the additional tags.
That would be great, thanks.
I poked a bit around in the meantime and discovered that I need some more packages installed on cassini, namely python-imaging and python-cssutils, both for Cascadenik. Mercurial/hg would also be nice to have (for http://bitbucket.org/springmeyer/tilelite). Let's see how much whining here helps or if I need to create a JIRA account. :)
Cheers Colin
On Thu, Nov 5, 2009 at 12:37 AM, Colin Marquardt cmarqu42@googlemail.com wrote:
2009/11/5 Peter Körner osm-lists@mazdermind.de:
So i tried to reproduce Ævars steps and get everything going. As I missed the intarray thing it's likely that I'm going to do a re-import soon, maybe over the weekend. This time I'll use the combined style containing the additional tags.
That would be great, thanks.
I poked a bit around in the meantime and discovered that I need some more packages installed on cassini, namely python-imaging and python-cssutils, both for Cascadenik. Mercurial/hg would also be nice to have (for http://bitbucket.org/springmeyer/tilelite). Let's see how much whining here helps or if I need to create a JIRA account. :)
I installed all of these.
Please have also my wish-list in mind for the time of re-import: shop, lit, cycleway, operator, surveillance, website, wikipedia
Additional: I have a problem with key "natural" in my scripts and can't find the reason, all other keys in the database works fine. An idea?
Thanks, for your work. Kolossos
Peter Körner schrieb:
Colin Marquardt schrieb:
2009/11/4 Peter Körner osm-lists@mazdermind.de:
The Import is done, it took 20,8 hours. The current data is of 28-Oct-2009.
It looks like you imported the columns that were present the last time? If so, it's no use to me yet for the Hike&Bike map (some additional columns are in http://mapnik-utils.googlecode.com/svn/sandbox/cascadenik/hike_n_bike/defaul...).
Yes, you're right. This is the first time I do a planet import and also the first time I struggle with mod_tile, renderd and not-beeing root on cassini :)
So i tried to reproduce Ævars steps and get everything going. As I missed the intarray thing it's likely that I'm going to do a re-import soon, maybe over the weekend. This time I'll use the combined style containing the additional tags.
Peter
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Tim Alder schrieb:
Please have also my wish-list in mind for the time of re-import: shop, lit, cycleway, operator, surveillance, website, wikipedia
I just started the new import. I used this style definition: http://cassini.toolserver.org/~mazder/wikimedia.extended.style
I think I included the wished tags and also added some on my own (phone, fax, opening_hours, url/website will give a nice POI map or some yellow-page-like thing)
I also increased the osm2pgsql cache from 800MB to 10GB (cassini had 17GB free when I started). I want to see how this speeds up the import. If you see any negative impact from this feel free to contact me here (or in urgent cases to ask a root to kill the osm2pgsql process).
intarray is working now, at least i don't see the warning..
Peter
Peter Körner schrieb:
Tim Alder schrieb:
Please have also my wish-list in mind for the time of re-import: shop, lit, cycleway, operator, surveillance, website, wikipedia
I just started the new import. I used this style definition: http://cassini.toolserver.org/~mazder/wikimedia.extended.style
The import was canceled after 6 hours when it started to generate the indexes. I'm doing an import with a new prefix (--prefix planet_osm_new) and afterwards rename the current tables to planet_osm_old and the new ones to planet_osm.
ALTER TABLE planet_osm_line RENAME TO planet_osm_old_line; ALTER TABLE planet_osm_new_line RENAME TO planet_osm_line;
Unfortunately in Postgresql this does not rename the primary keys or the indices, so i had to do this manually:
ALTER TABLE planet_osm_line_pkey RENAME TO planet_osm_old_line_pkey; ALTER TABLE planet_osm_new_line_pkey RENAME TO planet_osm_line_pkey; ALTER TABLE planet_osm_line_index RENAME TO planet_osm_old_line_index; ALTER TABLE planet_osm_new_line_index RENAME TO planet_osm_line_index;
I missed some. Now I added all the renames to the planet-update script (/sql/planet.osm/planet-update) and it's running again.
With the increased Cache level I expect it to be much faster than before, but we'll see.
Peter
Peter Körner osm-lists@mazdermind.de wrote:
Peter Körner schrieb:
Tim Alder schrieb:
Please have also my wish-list in mind for the time of re-import: shop, lit, cycleway, operator, surveillance, website, wikipedia
I just started the new import. I used this style definition: http://cassini.toolserver.org/~mazder/wikimedia.extended.style
The import was canceled after 6 hours when it started to generate the indexes. I'm doing an import with a new prefix (--prefix planet_osm_new) and afterwards rename the current tables to planet_osm_old and the new ones to planet_osm.
Did you run into constraints/internal consistency problem?
OSM dumps are never consistent, so one should expect problems.
That's one of the issues we've run on ptolemy into.
Marcin Cieslak schrieb:
Peter Körner osm-lists@mazdermind.de wrote:
Peter Körner schrieb:
Tim Alder schrieb:
Please have also my wish-list in mind for the time of re-import: shop, lit, cycleway, operator, surveillance, website, wikipedia
I just started the new import. I used this style definition: http://cassini.toolserver.org/~mazder/wikimedia.extended.style
The import was canceled after 6 hours when it started to generate the indexes. I'm doing an import with a new prefix (--prefix planet_osm_new) and afterwards rename the current tables to planet_osm_old and the new ones to planet_osm.
Did you run into constraints/internal consistency problem?
No I just forgot to rename the indices :) After 6 hours the import was already nearly done and osm2psql instructed postgres to create the indices and then it realised that they already existed and rolled back the changes.
The speedup (6h instead of 20h) came from the increased cache size (800MB -> 10GB)
This is the script I'm currently working on: http://cassini.toolserver.org/~mazder/planet-update
The diff-releated actions are not done right now and the urls of the world_boundaries are wrong, but the rest seemed to work..
Peter