On 27 July 2011 21:37, <maps-l-request@lists.wikimedia.org> wrote:

Send Maps-l mailing list submissions to
maps-l@lists.wikimedia.org

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.wikimedia.org/mailman/listinfo/maps-l
or, via email, send a message with subject or body 'help' to
maps-l-request@lists.wikimedia.org

You can reach the person managing the list at
maps-l-owner@lists.wikimedia.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Maps-l digest..."

Today's Topics:

1. Re: Discussion: is bw-mapnik useful? (Peter K?rner)
2. Re: ptolemy-performance (Peter K?rner)
3. Re: ptolemy-performance (Colin Marquardt)
4. Re: ptolemy-performance (Peter K?rner)
5. ptolemy-performance -expire.rb (Tim Alder)
6. Multiple Marker Layers ? (Christoph Holtermann)
7. Re: Tile expiry is working again? (Kay Drangmeister)
8. Current ptolemy performance measures (Kay Drangmeister)
9. What happens with tiles that timeout? (Kay Drangmeister)
10. Re: Current ptolemy performance measures (Peter K?rner)
11. Re: Multiple Marker Layers ? (Tim Alder)
12. Re: Multiple Marker Layers ? (Jeroen De Dauw)

----------------------------------------------------------------------

Message: 1
Date: Tue, 26 Jul 2011 19:25:46 +0200
From: Peter K?rner <osm-lists@mazdermind.de>
Subject: Re: [Maps-l] Discussion: is bw-mapnik useful?
To: maps-l@lists.wikimedia.org
Message-ID: <4E2EF89A.5020703@mazdermind.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

I'm currently talking to my boss about open-sourcing the
background-style used at

http://karriere.thyssenkrupp.com/jobmap/

It has been designed to be used as background-layer for overlays in the
range between z0 and z14 with only very basic information.

It could be used to replace mapnik-bw, which suffers in some points like
contrast in favour of details.

Peter

Am 26.07.2011 15:05, schrieb Kay Drangmeister:
> Hi all,
>
> while we are at deploying/upgrading those styles: I did an upgrade for
> bw-mapnik and bw-noicons, they now both derive from the latest osm-mapnik
> style.
>
> However, I think that bw-mapnik, i.e. the style that's a 1:1 copy of
> osm-mapnik, may not be needed. This is because the maps I am thinking
> of can better use bw-noicons, the osm-mapnik style without many icons
> (e.g. shop icons).
>
> What's your opinion, I think we can save quite some space/render time
> on ptolemy if we remove bw-mapnik in favour of bw-noicons. To ease
> migration it may be useful to just deliver bw-noicons tiles in case
> someone requests a bw-mapnik tile.
>
> Kind regards,
> Kay
>
> _______________________________________________
> Maps-l mailing list
> Maps-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/maps-l

------------------------------

Message: 2
Date: Tue, 26 Jul 2011 19:30:01 +0200
From: Peter K?rner <osm-lists@mazdermind.de>
Subject: Re: [Maps-l] ptolemy-performance
To: maps-l@lists.wikimedia.org
Message-ID: <4E2EF999.1060708@mazdermind.de>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed

Am 26.07.2011 15:26, schrieb Kay Drangmeister:
> Am 26.07.2011, 15:16 Uhr, schrieb Peter K?rner<osm-lists@mazdermind.de>:
>
>> The indexes seem to be back up now, so I'll restart
>> tirex with the new deployed styles.
>
> We seem to have the following problem again: (nothing is rendered):
>
> https://jira.toolserver.org/browse/TS-1100

There's another thing we should talk about. Currently we have at max. 8
render threads which is a lot. I'd suggest reducing this to 4, to reduce
load on the db as well as the time required to render a single tile.

Peter

------------------------------

Message: 3
Date: Tue, 26 Jul 2011 19:29:55 +0200
From: Colin Marquardt <cmarqu42@googlemail.com>
Subject: Re: [Maps-l] ptolemy-performance
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID:
<CA+OWQVdKjjpHGcDJGVh-3WcAhA+-m4VjD9Aq89eb7z9uTjvVPA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

2011/7/26 Peter K?rmer <osm-lists@mazdermind.de>:
> Am 26.07.2011 15:47, schrieb Kay Drangmeister:
>> Ah, good to know. Is that documented somewhere, just in case
>> one other guy may become MMP OSM member and needs to restart
>> tirex some day..? ;-)
>
> Don't think so. Can you tell a good place to document it?

You had sent a personal mail (in German) to the involved guys about
how to deploy styles a few months ago.
https://wiki.toolserver.org/view/OpenStreetMap seems like the right
place to put it.

Cheers
Colin

------------------------------

Message: 4
Date: Tue, 26 Jul 2011 20:53:33 +0200
From: Peter K?rner <osm-lists@mazdermind.de>
Subject: Re: [Maps-l] ptolemy-performance
To: maps-l@lists.wikimedia.org
Message-ID: <4E2F0D2D.80401@mazdermind.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi

Am 26.07.2011 19:29, schrieb Colin Marquardt:
> You had sent a personal mail (in German) to the involved guys about
> how to deploy styles a few months ago.
> https://wiki.toolserver.org/view/OpenStreetMap seems like the right
> place to put it.

I placed it there.

Peter

------------------------------

Message: 5
Date: Tue, 26 Jul 2011 22:00:27 +0200
From: Tim Alder <tim.alder@s2002.tu-chemnitz.de>
Subject: [Maps-l] ptolemy-performance -expire.rb
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID: <4E2F1CDB.8000807@s2002.tu-chemnitz.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

FYI:

To solve the performance problems on lower zoom level, I modify
expire.rb a little bit so that we have now no fix MIN_ZOOM value.
Instead the MIN_ZOOM is changing with definable probabilities. [1]
(Expiring works on zoomlevel MIN_ZOOM to MAX_ZOOM).

Low zoom-levels cost us a lot and changes are often not visible.

So edits on OSM leads to expirings on higher zoom-level with a higher
probability than on low levels. Areas with high mapping activities will
re-render more often than death areas.

Z16-18 will expire each time. z10 will only expire with a probability of
1.5%. We can adjust this values each time.

I hope this help us if we restart the updating of the db, where this
script works.

[1]
random= rand()
prob=[0,0,0,0,0,0,0,0,0,0.00625,0.0125,0.025,0.05,0.1,0.2,0.4,1,1,1]
#probability for each zoomlevel to expire
z = 18
while z > 0
if random<prob[z]
mini=z
end
z -= 1
end
MIN_ZOOM=mini

------------------------------

Message: 6
Date: Wed, 27 Jul 2011 02:52:39 +0200
From: Christoph Holtermann <c.holtermann@gmx.de>
Subject: [Maps-l] Multiple Marker Layers ?
To: maps-l@lists.wikimedia.org
Message-ID: <4E2F6157.2040309@gmx.de>
Content-Type: text/plain; charset=ISO-8859-15

Hello !

Is it possible to have multiple Layers of Markers in the wikimedia Maps extension
using openstreetmap ?

I would like to have a map where the user can enable two different layers of Markers.
(working group locations and hospital locations).

regards,

C. Holtermann

------------------------------

Message: 7
Date: Wed, 27 Jul 2011 08:03:38 +0200
From: Kay Drangmeister <kay@drangmeister.net>
Subject: Re: [Maps-l] Tile expiry is working again?
To: maps-l@lists.wikimedia.org
Message-ID: <4E2FAA3A.5080007@drangmeister.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Am 26.07.2011 03:48, schrieb Kai Krueger:

> May I also suggest to not do a minutely update. If the tiles are only
> expired once every 8 month or so

I really would like to see changes instantly (as most mappers do),
especially when developing or modifying styles.

I think the long tile expiry holds only for the low zoom tiles,
high zoom tiles are expired instantly, if I'm not mistaken.

Kind regards,
Kay

------------------------------

Message: 8
Date: Wed, 27 Jul 2011 08:18:35 +0200
From: Kay Drangmeister <kay@drangmeister.net>
Subject: [Maps-l] Current ptolemy performance measures
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID: <4E2FADBB.7070809@drangmeister.net>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed

Hi,

there have been quite some performance tuning measures on ptolemy:

(1) number of render processes has been reduced from 8/6 to 4
(2) Kolossos modified expire.rb to render low zoom tiles with low
probability
(3) indexes have been added to the DB for geometry,hstore and osm-id
(4) clustering

Is there a good way that we can monitor the results? Especially
(1) should be carefully tracked. I can see no significant changes
in IO throughput
http://munin.toolserver.org/OSM/ptolemy/iostat.html or IO
http://munin.toolserver.org/OSM/ptolemy/io_bytes_sd.html
and not even in postgres connections
http://munin.toolserver.org/OSM/ptolemy/postgres_connections_osm_mapnik.html
The load and CPU usage has been decreased a bit. My guess would
be that more processes would result in a better CPU utilization
(and thus faster overall rendering).

To monitor this we need two figures: (a) average tile rendering
time (per process) and (b) tiles rendered per second (by all
processes). Can we set up munin to track it?

BTW:
http://munin.toolserver.org/OSM/ptolemy/tirex_status_queued_requests.html has
not been updated for 13 h now,
how can that happen?

And another question: earlier, two slots have been reserved for
prio 1 queue requests (i.e. missing tiles). Is there a reserve
available currently? Otherwise one would have to wait in that
case.

Kind regards,
Kay

------------------------------

Message: 9
Date: Wed, 27 Jul 2011 10:28:56 +0200
From: Kay Drangmeister <kay@drangmeister.net>
Subject: [Maps-l] What happens with tiles that timeout?
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID: <4E2FCC48.6080909@drangmeister.net>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed

Hi.

Looking at the render queue on ptolemy, I often observe metatiles for
powermap Z9 or Z10 timing out. Powermap is not the only one, bw-mapnik
and bw-noicons is another candidate, too, for other Zoom levels.

Then what happens, the metatile is not rendered, and when the browser
user zooms in and out, the same long rendering resulting in a timeout
happens again. (To me it seems that much of Z9/Z10 for the powermap
just cannot be rendered in germany, but admittedly that's a guess.)

So this is bad in two ways: first, the queue is congested with long
running tasks, second, this happens over and over again. While the
first thing cannot easily be changed, the second, worse, point could
be avoided:

Why not just put a (mostly transparent) metatile saying "timeout" as
the rendering result? So for the style creator it would easily be
trackable what zoom level (or detail) is problematic, and for the
user it would be possible to just issue a manual "/dirty", if the
user thinks it may be worthwhile to try rendering again.

Kind regards,
Kay

------------------------------

Message: 10
Date: Wed, 27 Jul 2011 14:44:05 +0200
From: Peter K?rner <osm-lists@mazdermind.de>
Subject: Re: [Maps-l] Current ptolemy performance measures
To: maps-l@lists.wikimedia.org
Message-ID: <4E300815.8060205@mazdermind.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi

Am 27.07.2011 08:18, schrieb Kay Drangmeister:
> there have been quite some performance tuning measures on ptolemy:
>
> (1) number of render processes has been reduced from 8/6 to 4
> (2) Kolossos modified expire.rb to render low zoom tiles with low
> probability
> (3) indexes have been added to the DB for geometry,hstore and osm-id
> (4) clustering
>
> Is there a good way that we can monitor the results? Especially
> (1) should be carefully tracked. I can see no significant changes
> in IO throughput
This decision has been made to try if offloading the database would
result in less render timeouts.
> http://munin.toolserver.org/OSM/ptolemy/iostat.html or IO
> http://munin.toolserver.org/OSM/ptolemy/io_bytes_sd.html
> and not even in postgres connections
> http://munin.toolserver.org/OSM/ptolemy/postgres_connections_osm_mapnik.html
> The load and CPU usage has been decreased a bit. My guess would
> be that more processes would result in a better CPU utilization
> (and thus faster overall rendering).
>
> To monitor this we need two figures: (a) average tile rendering
> time (per process) and (b) tiles rendered per second (by all
> processes). Can we set up munin to track it?
I don't think tirex allows capturing the tile throughput on a
per-process base, I guess it would need to be modified to allow that.

> BTW:
> http://munin.toolserver.org/OSM/ptolemy/tirex_status_queued_requests.html has
> not been updated for 13 h now,
> how can that happen?
The whole tirex block has disappeared from the statistics. Munin is not
listing the plugins anymore:

osm@ptolemy:~$ telnet localhost 4949
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at ptolemy.esi.toolserver.org
list
apache_accesses apache_processes apache_volume cpu df if_e1000g0
io_busy_sd io_bytes_sd io_ops_sd iostat load mod_tile_fresh
mod_tile_response mod_tile_zoom netstat ntp_kernel_err
ntp_kernel_pll_freq ntp_kernel_pll_off ntp_offset ntp_states
postfix_mailqueue postfix_mailstats postfix_mailvolume postgres_bgwriter
postgres_cache_osm_mapnik postgres_checkpoints postgres_connections_db
postgres_connections_osm_mapnik postgres_locks_osm_mapnik
postgres_querylength_osm_mapnik postgres_scans_osm_mapnik
postgres_size_osm_mapnik postgres_transactions_osm_mapnik
postgres_tuples_osm_mapnik postgres_users postgres_xlog processes
replication_delay2 uptime users

This seems like a munin misconfiguration. Sometimes only munin-node
needs to be restarted.

> And another question: earlier, two slots have been reserved for
> prio 1 queue requests (i.e. missing tiles). Is there a reserve
> available currently? Otherwise one would have to wait in that
> case.

I just reduced the max. number of render processed by two. The
configuration now looks like this:

osm@ptolemy:~$ less tirex/etc/tirex/tirex.conf
# Buckets for different priorities.
bucket name=missing minprio=1 maxproc=6 maxload=20
bucket name=dirty minprio=2 maxproc=4 maxload=8
bucket name=bulk minprio=10 maxproc=3 maxload=6
bucket name=background minprio=20 maxproc=3 maxload=4

Peter

------------------------------

Message: 11
Date: Wed, 27 Jul 2011 21:08:48 +0200
From: Tim Alder <tim.alder@s2002.tu-chemnitz.de>
Subject: Re: [Maps-l] Multiple Marker Layers ?
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID: <4E306240.3050803@s2002.tu-chemnitz.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

I'm not sure if I understand your question correct.

If you mean:
http://www.mediawiki.org/wiki/Maps_extension
you should ask the creator Jeroen De Dauw directly.

If you only want to create a map you should check the possibilities of
openlayers.org that we use for a lot of maps with multiple layers.

Greetings Kolossos

Am 27.07.2011 02:52, schrieb Christoph Holtermann:
> have multiple Layers of Markers in the wikimedia Maps extension
> using openstreetmap ?
>
> I would like to have a map where the user can enable two different layers of Markers.
> (working group locations and hospital locations).

------------------------------

Message: 12
Date: Wed, 27 Jul 2011 21:36:46 +0200
From: Jeroen De Dauw <jeroendedauw@gmail.com>
Subject: Re: [Maps-l] Multiple Marker Layers ?
To: Map integration <maps-l@lists.wikimedia.org>
Message-ID:
<CAMhmagCnEdi4Mi2AUY9FRCk-OkKwADJHeGc+9miNF1grduiY_A@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hey,

> you should ask the creator Jeroen De Dauw directly.

Don't panic, I'm on this list :)

> have multiple Layers of Markers in the wikimedia Maps extension
> using openstreetmap?

Like Tim pointed out, OpenLayers does support this feature, however, Maps
does not. It'd would not be to hard to implement, but requires coming up
with some way of defining layers in wikitext first.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/maps-l/attachments/20110727/a654a250/attachment.htm

------------------------------

_______________________________________________
Maps-l mailing list
Maps-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l

End of Maps-l Digest, Vol 27, Issue 3
*************************************

--
羅禹國 - Hugo LOPEZ,
Tw. tel: 09-8343-9890
Institute of Technology & Innovation Management, Master 2
NCHU, Taizhong, Taiwan.
250 KuoKuang Rd., Taichung 402, Taiwan, R.O.C.
國立中興大學台中市402國光路250號台灣