On Apr 15, 2010, at 2:02 PM, Kai Krueger wrote:
On 04/15/2010 02:38 PM, River Tarnell wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Ævar Arnfjörð Bjarmason:
Those servers need a lot of love. People other
than me have been
doing
some work on it recently.
I've been distracted by other things the last couple of weeks, but I
plan to do some working on getting renderd running properly next
week.
We may need some additional programming to make it feasible to run it
with a lot of styles (e.g. loading styles on demand instead of at
startup).
It appears as if renderd/Mapnik uses up about 5 - 6 Mb of Ram per
instance.
Note, that a minor memory leak in XML loading was fixed in the Mapnik
0.7.0 release:
http://trac.mapnik.org/ticket/473
http://trac.mapnik.org/changeset/1506
Part of the problem may be that renderd uses a new
instance
per style per rendering thread.
Yes, this certainly uses more memory, but it is an intended feature
AFAIK.
The mapnik.Map object is not (by design) intended to shared between
threads.
So if you have 250 styles and 8
rendering threads you end up needing about 250*8*5.5Mb ~ 11 Gb of Ram,
which sounds roughly like the numbers given for cassini.
Yes, I can see this being a problem with that many styles! :)
If Mapniks Map Object could be reused for all of the
rendering
threads,
that would reduce memory by a factor of 8. It would still be a lot,
but
at least get back into the feasible range. But someone more familiar
with Mapniks internals would have to comment on that.
In my experience with python-based servers (deployed with mod_wsgi),
it works fine sharing a global map object if deployed in multiprocess
mode (rather than multithreaded).
I'd be interested to hear from Jon Burgess or Artem about alternative
approaches for threaded apps in C.
Kai, could you post this question to mapnik-devel? Or, if you are not
subscribed, I can - just let me know.
Otherwise implementing a load on demand with a time based expiry for
map
styles might not be a bad idea, given that most of the 250 odd styles
will likely very seldom be used and sounds like it wouldn't be too
hard
to implement.
I think that's great idea.
Dane
Kai
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (HP-UX)
iEYEARECAAYFAkvHFuIACgkQIXd7fCuc5vJWyQCfcoW6A81hZYUAFmnNKTLXOWoH
ccUAnA59GN6GxIfUHNdlGnY9LHGykK/j
=YzWW
-----END PGP SIGNATURE-----
_______________________________________________
Maps-l mailing list
Maps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l
_______________________________________________
Maps-l mailing list
Maps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/maps-l