On 02/11/2010 12:52 PM, Peter Körner wrote:
River Tarnell schrieb:
> Peter Körner:
>> It's more important to think about where the diff-import should be
>> running from. I'd suggest trying it from willow but keep thinking about
>> maybe shifting it to ptolemy..
>
> Does importing one minute's diffs really take so much memory than 1GB isn't
> enough? From what I remember of osm2pgsql, it took at least several minutes
> to use that much memory.
My guess would be that for diffs, it won't need that much memory, as the
cache won't be effective during imports anyway. The cache stores the
lat/lon values of nodes in a blocked-sparse array at the id-th entry in
the table. When building ways and relations, Osm2pgsql will either query
the each node in O(1) from the cache or require a SQL query over the
network. It is heavily optimized for initial import and on diffs, the
cache will mostly not be populated anyway. Therefore I guess the memory
doesn't matter in this case, but the extra latency of the network vs
local process might. On the other hand on a local lan, that will
presumably be less than a single disk seek, so not really an issue either.
No i don't think it will. As I said I'd try it from willow but I can
imagine, that eg. expiring tiles won't work this easy when osm2pgsql
does not run on the same machine as mod_tile.
Expiring tiles is a different matter, but osm2pgsql doesn't expire the
tiles directly so shouldn't effect where it has to run. Instead it spits
out a textual list of tiles that should be considered expired, that then
needs to be handled separately. The actual expiry script will most
likely need to be run on the same machine as the (meta)tiles stored
though, as it does require to touch and stat quite a lot of files.
Especially with the 200 different language styles. But if the meta tile
directory is writable from the login servers, we can experiment with
what setup works best first before moving it over to ptolemy.
Kai
Peter
_______________________________________________
Maps-l mailing list
Maps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l