New subject: Maps-l Digest, Vol 27, Issue 3

27 Jul 2011

      Dear all,
The mapmakers using Inkscape / photoshop on wikipedia (WP:GL/M) to generate
the articles' maps currently work file after file, which generate scores of
background / data duplication which are increasingly hard to manage. Also,
we are willing to know : is there any ongoing project/system/extension to
create a kind of free google maps where we —graphists— may **create and
edits** encyclopedic maps in the clouds ?
I noticed :
* http://toolserver.org/~osm/styles/ - display many layers, no edit
functions.
* http://www.mediawiki.org/wiki/Category:Map_extensions - don't seems to
have such extension.
* the online editing tool is missing ?
Regards,
--
Yug
On 27 July 2011 21:37, maps-l-request@lists.wikimedia.org wrote:
...
Send Maps-l mailing list submissions to
       maps-l@lists.wikimedia.org
To subscribe or unsubscribe via the World Wide Web, visit
       https://lists.wikimedia.org/mailman/listinfo/maps-l
or, via email, send a message with subject or body 'help' to
       maps-l-request@lists.wikimedia.org
You can reach the person managing the list at
       maps-l-owner@lists.wikimedia.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Maps-l digest..."
Today's Topics:

Re: Discussion: is bw-mapnik useful? (Peter K?rner)
Re: ptolemy-performance (Peter K?rner)
Re: ptolemy-performance (Colin Marquardt)
Re: ptolemy-performance (Peter K?rner)
ptolemy-performance  -expire.rb (Tim Alder)
Multiple Marker Layers ? (Christoph Holtermann)
Re: Tile expiry is working again? (Kay Drangmeister)
Current ptolemy performance measures (Kay Drangmeister)
What happens with tiles that timeout? (Kay Drangmeister)
Re: Current ptolemy performance measures (Peter K?rner)
Re: Multiple Marker Layers ? (Tim Alder)
Re: Multiple Marker Layers ? (Jeroen De Dauw)

Message: 1
Date: Tue, 26 Jul 2011 19:25:46 +0200
From: Peter K?rner osm-lists@mazdermind.de
Subject: Re: [Maps-l] Discussion: is bw-mapnik useful?
To: maps-l@lists.wikimedia.org
Message-ID: 4E2EF89A.5020703@mazdermind.de
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
I'm currently talking to my boss about open-sourcing the
background-style used at
http://karriere.thyssenkrupp.com/jobmap/
It has been designed to be used as background-layer for overlays in the
range between z0 and z14 with only very basic information.
It could be used to replace mapnik-bw, which suffers in some points like
contrast in favour of details.
Peter
Am 26.07.2011 15:05, schrieb Kay Drangmeister:
...
Hi all,
while we are at deploying/upgrading those styles: I did an upgrade for
bw-mapnik and bw-noicons, they now both derive from the latest osm-mapnik
style.
However, I think that bw-mapnik, i.e. the style that's a 1:1 copy of
osm-mapnik, may not be needed. This is because the maps I am thinking
of can better use bw-noicons, the osm-mapnik style without many icons
(e.g. shop icons).
What's your opinion, I think we can save quite some space/render time
on ptolemy if we remove bw-mapnik in favour of bw-noicons. To ease
migration it may be useful to just deliver bw-noicons tiles in case
someone requests a bw-mapnik tile.
Kind regards,
Kay

Maps-l mailing list
Maps-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l

Message: 2
Date: Tue, 26 Jul 2011 19:30:01 +0200
From: Peter K?rner osm-lists@mazdermind.de
Subject: Re: [Maps-l] ptolemy-performance
To: maps-l@lists.wikimedia.org
Message-ID: 4E2EF999.1060708@mazdermind.de
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Am 26.07.2011 15:26, schrieb Kay Drangmeister:
...
Am 26.07.2011, 15:16 Uhr, schrieb Peter K?rnerosm-lists@mazdermind.de:
...
The indexes seem to be back up now, so I'll restart
tirex with the new deployed styles.
We seem to have the following problem again: (nothing is rendered):
https://jira.toolserver.org/browse/TS-1100
There's another thing we should talk about. Currently we have at max. 8
render threads which is a lot. I'd suggest reducing this to 4, to reduce
load on the db as well as the time required to render a single tile.
Peter

Message: 3
Date: Tue, 26 Jul 2011 19:29:55 +0200
From: Colin Marquardt cmarqu42@googlemail.com
Subject: Re: [Maps-l] ptolemy-performance
To: Map integration maps-l@lists.wikimedia.org
Message-ID:
       <CA+OWQVdKjjpHGcDJGVh-3WcAhA+-m4VjD9Aq89eb7z9uTjvVPA@mail.gmail.com
...
Content-Type: text/plain; charset=ISO-8859-1
2011/7/26 Peter K?rmer osm-lists@mazdermind.de:
...
Am 26.07.2011 15:47, schrieb Kay Drangmeister:
...
Ah, good to know. Is that documented somewhere, just in case
one other guy may become MMP OSM member and needs to restart
tirex some day..? ;-)
Don't think so. Can you tell a good place to document it?
You had sent a personal mail (in German) to the involved guys about
how to deploy styles a few months ago.
https://wiki.toolserver.org/view/OpenStreetMap seems like the right
place to put it.
Cheers
 Colin

Message: 4
Date: Tue, 26 Jul 2011 20:53:33 +0200
From: Peter K?rner osm-lists@mazdermind.de
Subject: Re: [Maps-l] ptolemy-performance
To: maps-l@lists.wikimedia.org
Message-ID: 4E2F0D2D.80401@mazdermind.de
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi
Am 26.07.2011 19:29, schrieb Colin Marquardt:
...
You had sent a personal mail (in German) to the involved guys about
how to deploy styles a few months ago.
https://wiki.toolserver.org/view/OpenStreetMap seems like the right
place to put it.
I placed it there.
Peter

Message: 5
Date: Tue, 26 Jul 2011 22:00:27 +0200
From: Tim Alder tim.alder@s2002.tu-chemnitz.de
Subject: [Maps-l]  ptolemy-performance  -expire.rb
To: Map integration maps-l@lists.wikimedia.org
Message-ID: 4E2F1CDB.8000807@s2002.tu-chemnitz.de
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
FYI:
To solve the performance problems on lower zoom level, I modify
expire.rb a little bit so that we have now no fix MIN_ZOOM value.
Instead the MIN_ZOOM is changing with definable probabilities. [1]
(Expiring works on zoomlevel MIN_ZOOM to MAX_ZOOM).
Low zoom-levels cost us a lot and changes are often not visible.
So edits on OSM leads to expirings on higher zoom-level with a higher
probability than on low levels. Areas with high mapping activities will
re-render more often than death areas.
Z16-18 will expire each time. z10 will only expire with a probability of
1.5%. We can adjust this values each time.
I hope this help us if we restart the updating of the db, where this
script works.
[1]
  random= rand()
  prob=[0,0,0,0,0,0,0,0,0,0.00625,0.0125,0.025,0.05,0.1,0.2,0.4,1,1,1]
#probability for each zoomlevel to expire
  z = 18
  while z > 0
    if random<prob[z]
      mini=z
    end
    z -= 1
  end
  MIN_ZOOM=mini

Message: 6
Date: Wed, 27 Jul 2011 02:52:39 +0200
From: Christoph Holtermann c.holtermann@gmx.de
Subject: [Maps-l] Multiple Marker Layers ?
To: maps-l@lists.wikimedia.org
Message-ID: 4E2F6157.2040309@gmx.de
Content-Type: text/plain; charset=ISO-8859-15
Hello !
Is it possible to have multiple Layers of Markers in the wikimedia Maps
extension
using openstreetmap ?
I would like to have a map where the user can enable two different layers
of Markers.
(working group locations and hospital locations).
regards,
C. Holtermann

Message: 7
Date: Wed, 27 Jul 2011 08:03:38 +0200
From: Kay Drangmeister kay@drangmeister.net
Subject: Re: [Maps-l] Tile expiry is working again?
To: maps-l@lists.wikimedia.org
Message-ID: 4E2FAA3A.5080007@drangmeister.net
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Am 26.07.2011 03:48, schrieb Kai Krueger:
...
May I also suggest to not do a minutely update. If the tiles are only
expired once every 8 month or so
I really would like to see changes instantly (as most mappers do),
especially when developing or modifying styles.
I think the long tile expiry holds only for the low zoom tiles,
high zoom tiles are expired instantly, if I'm not mistaken.
Kind regards,
Kay

Message: 8
Date: Wed, 27 Jul 2011 08:18:35 +0200
From: Kay Drangmeister kay@drangmeister.net
Subject: [Maps-l] Current ptolemy performance measures
To: Map integration maps-l@lists.wikimedia.org
Message-ID: 4E2FADBB.7070809@drangmeister.net
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Hi,
there have been quite some performance tuning measures on ptolemy:
(1) number of render processes has been reduced from 8/6 to 4
(2) Kolossos modified expire.rb to render low zoom tiles with low
probability
(3) indexes have been added to the DB for geometry,hstore and osm-id
(4) clustering
Is there a good way that we can monitor the results? Especially
(1) should be carefully tracked. I can see no significant changes
in IO throughput
http://munin.toolserver.org/OSM/ptolemy/iostat.html or IO
http://munin.toolserver.org/OSM/ptolemy/io_bytes_sd.html
and not even in postgres connections
http://munin.toolserver.org/OSM/ptolemy/postgres_connections_osm_mapnik.html
The load and CPU usage has been decreased a bit. My guess would
be that more processes would result in a better CPU utilization
(and thus faster overall rendering).
To monitor this we need two figures: (a) average tile rendering
time (per process) and (b) tiles rendered per second (by all
processes). Can we set up munin to track it?
BTW:
http://munin.toolserver.org/OSM/ptolemy/tirex_status_queued_requests.htmlhas
not been updated for 13 h now,
how can that happen?
And another question: earlier, two slots have been reserved for
prio 1 queue requests (i.e. missing tiles). Is there a reserve
available currently? Otherwise one would have to wait in that
case.
Kind regards,
Kay

Message: 9
Date: Wed, 27 Jul 2011 10:28:56 +0200
From: Kay Drangmeister kay@drangmeister.net
Subject: [Maps-l] What happens with tiles that timeout?
To: Map integration maps-l@lists.wikimedia.org
Message-ID: 4E2FCC48.6080909@drangmeister.net
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Hi.
Looking at the render queue on ptolemy, I often observe metatiles for
powermap Z9 or Z10 timing out. Powermap is not the only one, bw-mapnik
and bw-noicons is another candidate, too, for other Zoom levels.
Then what happens, the metatile is not rendered, and when the browser
user zooms in and out, the same long rendering resulting in a timeout
happens again. (To me it seems that much of Z9/Z10 for the powermap
just cannot be rendered in germany, but admittedly that's a guess.)
So this is bad in two ways: first, the queue is congested with long
running tasks, second, this happens over and over again. While the
first thing cannot easily be changed, the second, worse, point could
be avoided:
Why not just put a (mostly transparent) metatile saying "timeout" as
the rendering result? So for the style creator it would easily be
trackable what zoom level (or detail) is problematic, and for the
user it would be possible to just issue a manual "/dirty", if the
user thinks it may be worthwhile to try rendering again.
Kind regards,
Kay

Message: 10
Date: Wed, 27 Jul 2011 14:44:05 +0200
From: Peter K?rner osm-lists@mazdermind.de
Subject: Re: [Maps-l] Current ptolemy performance measures
To: maps-l@lists.wikimedia.org
Message-ID: 4E300815.8060205@mazdermind.de
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi
Am 27.07.2011 08:18, schrieb Kay Drangmeister:
...
there have been quite some performance tuning measures on ptolemy:
(1) number of render processes has been reduced from 8/6 to 4
(2) Kolossos modified expire.rb to render low zoom tiles with low
probability
(3) indexes have been added to the DB for geometry,hstore and osm-id
(4) clustering
Is there a good way that we can monitor the results? Especially
(1) should be carefully tracked. I can see no significant changes
in IO throughput
This decision has been made to try if offloading the database would
result in less render timeouts.
...
http://munin.toolserver.org/OSM/ptolemy/iostat.html or IO
http://munin.toolserver.org/OSM/ptolemy/io_bytes_sd.html
and not even in postgres connections
http://munin.toolserver.org/OSM/ptolemy/postgres_connections_osm_mapnik.html
...
The load and CPU usage has been decreased a bit. My guess would
be that more processes would result in a better CPU utilization
(and thus faster overall rendering).
To monitor this we need two figures: (a) average tile rendering
time (per process) and (b) tiles rendered per second (by all
processes). Can we set up munin to track it?
I don't think tirex allows capturing the tile throughput on a
per-process base, I guess it would need to be modified to allow that.
...
BTW:
http://munin.toolserver.org/OSM/ptolemy/tirex_status_queued_requests.htmlhas
...
not been updated for 13 h now,
how can that happen?
The whole tirex block has disappeared from the statistics. Munin is not
listing the plugins anymore:
osm@ptolemy:~$ telnet localhost 4949
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at ptolemy.esi.toolserver.org
list
apache_accesses apache_processes apache_volume cpu df if_e1000g0
io_busy_sd io_bytes_sd io_ops_sd iostat load mod_tile_fresh
mod_tile_response mod_tile_zoom netstat ntp_kernel_err
ntp_kernel_pll_freq ntp_kernel_pll_off ntp_offset ntp_states
postfix_mailqueue postfix_mailstats postfix_mailvolume postgres_bgwriter
postgres_cache_osm_mapnik postgres_checkpoints postgres_connections_db
postgres_connections_osm_mapnik postgres_locks_osm_mapnik
postgres_querylength_osm_mapnik postgres_scans_osm_mapnik
postgres_size_osm_mapnik postgres_transactions_osm_mapnik
postgres_tuples_osm_mapnik postgres_users postgres_xlog processes
replication_delay2 uptime users
This seems like a munin misconfiguration. Sometimes only munin-node
needs to be restarted.
...
And another question: earlier, two slots have been reserved for
prio 1 queue requests (i.e. missing tiles). Is there a reserve
available currently? Otherwise one would have to wait in that
case.
I just reduced the max. number of render processed by two. The
configuration now looks like this:
osm@ptolemy:~$ less tirex/etc/tirex/tirex.conf
#  Buckets for different priorities.
bucket name=missing       minprio=1  maxproc=6 maxload=20
bucket name=dirty         minprio=2  maxproc=4 maxload=8
bucket name=bulk          minprio=10 maxproc=3 maxload=6
bucket name=background    minprio=20 maxproc=3 maxload=4
Peter

Message: 11
Date: Wed, 27 Jul 2011 21:08:48 +0200
From: Tim Alder tim.alder@s2002.tu-chemnitz.de
Subject: Re: [Maps-l] Multiple Marker Layers ?
To: Map integration maps-l@lists.wikimedia.org
Message-ID: 4E306240.3050803@s2002.tu-chemnitz.de
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
I'm not sure if I understand your question correct.
If you mean:
http://www.mediawiki.org/wiki/Maps_extension
you should ask the creator Jeroen De Dauw directly.
If you only want to create a map you should check the possibilities of
openlayers.org that we use for a lot of maps with multiple layers.
Greetings Kolossos
Am 27.07.2011 02:52, schrieb Christoph Holtermann:
...
have multiple Layers of Markers in the wikimedia Maps extension
using openstreetmap ?
I would like to have a map where the user can enable two different layers
of Markers.
...
(working group locations and hospital locations).

Message: 12
Date: Wed, 27 Jul 2011 21:36:46 +0200
From: Jeroen De Dauw jeroendedauw@gmail.com
Subject: Re: [Maps-l] Multiple Marker Layers ?
To: Map integration maps-l@lists.wikimedia.org
Message-ID:
       <CAMhmagCnEdi4Mi2AUY9FRCk-OkKwADJHeGc+9miNF1grduiY_A@mail.gmail.com
...
Content-Type: text/plain; charset="iso-8859-1"
Hey,
...
you should ask the creator Jeroen De  Dauw directly.
Don't panic, I'm on this list :)
...
have multiple Layers of Markers in the wikimedia Maps extension
using openstreetmap?
Like Tim pointed out, OpenLayers does support this feature, however, Maps
does not. It'd would not be to hard to implement, but requires coming up
with some way of defining layers in wikitext first.
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--

Re: [Maps-l] Maps-l Digest, Vol 27, Issue 3