2009/11/25 Peter Körner <osm-lists(a)mazdermind.de>de>:
Colin Marquardt schrieb:
2009/11/25 Peter Körner
<osm-lists(a)mazdermind.de>de>:
Atm. the load on cassini is around 20 and the
diff imports are fighting
to stay up. I think it's cmarq's hillshade-generator takes the most
resources on cassini. top reports ~57.1% IO waiting so I think there's a
lot of disk going on.
This must have gotten worse as generate_tiles goes to lower zoom
levels. I can kill it and maybe run with a single thread or even some
artificial slowdown tonight. Right now it's three threads, which
wasn't a problem so far.
I don't think that this is necessary atm. I just
wanted to talk about
this upcoming issue. Can you estimate how long it will keep running with
the current parameters? Getting some days out of sync is not that big
problem (we're not expiring tiles currently anyway).
Maybe two more days, but it's really no problem to slow it down a bit.
The maximum run time of the current job is bounded anyway with cassini
being repurposed.
I tried
to do a "du -hs /mnt/user-store/osm_hillshading" but ir ran
loooong until I canceled it.
Well, I generate hillshading for the land mass of Europe (plus the UK)
right now, so there isn't such a lot of empty sea tiles.
Okay, cool.
It would
indeed be nice to generate metatiles instead of the single PNGs
though.
Yes, it would. Is there a tool to pack the existing tiles into metatiles?
I just found that there is a tool called convert_meta:
http://svn.openstreetmap.org/applications/utils/mod_tile/readme.txt
I hope it will work with the paletted PNGs I'm using.
I'm also
not rendering down to zoom 18. Pre-rendering those
hillshading tiles (up to a certain zoom level at least) in general is
not bad IMO since they are very much static (they don't contain any
changing information), so will be useful for many maps for the next
several years.
Yes I see that, too, but rendering on the fly and then caching the
results is a compromise between disk space and cpu power.
ACK. I'll try to get the necessary tool support.
It's not the absolute "max number"
I'm worried about but the decreasing
performance when working with a lot of files in a directory (and maybe
the inode usage). This could both be solved by using metatiles.
I see. Let's try convert_meta on a subset of the data tonight and see
how it goes.
Cheers
Colin