Hello everyone,
With the osm rendering database up-to-date again, and no selective
expiry of tiles at the moment, I would like to do a global expiry of all
tiles at some point. However, unfortunately, we are still experiencing
some performance problems, particularly with low zoom rendering.
These issues are likely to be two fold. a) The performance of the
database is still not ideal, and b) stylesheets are being less efficient
with database requests as they should be.
As an example of a combination of a) and b), one of the queries from the
hikebike map at zoom level 7 took over 18 minutes to complete for just a
single query and many queries are necessary to render a tile. This
perhaps explains why a lot of lowzoom tiles don't successfully complete.
I am still trying to see if anything can be done about a), but given
that these issues have been around since pretty much the beginning, I am
not sure how much we can really improve things here.
With respect to b), we will need the help of all style sheet authors, to
try and optimize their style sheets as much as possible to limit
unnecessary database access. The hikebike map style as an example
unfortunately seems particularly problematic, but I think other
stylesheets are somewhat suffering similar issues. For example the
hikebike tiles at zoom levels below 9 are still timing out, despite
having increasing the rendering timeout now to a non sustainable 1 hour
period. With single tiles taking over an hour to render, rendering 200+
styles is clearly not going to work, so we need to try and optimize
these style sheets.
In general, one should try and limit the number of database access as
much as possible and also try and filter out as much data as one can on
the db side. In addition there are a few tips one can perhaps follow.
At low zooms, the biggest problems are access to the planet_line and
planet_polygon tables.
- At zoom levels 7 - 8 and lower, accessing the planet_line table should
probably be avoided at all costs, as it is simply to slow. All (or
hopefully most) of the information needed at low zooms should be in the
planet_roads table. Queries on planet_roads should be much faster, as it
is only 3.6GB in size rather than the 26Gb of the planet_lines table
- There is no equivalent of the planet_roads for polygons. However, I
have now created a conditional spatial index on the planet_polygons
table for the condition of "where building is null", that semi
partitions the table. Currently nearly 80% of all polygons are building
polygons. At a total size of 18Gb for the planet_polygon, limiting the
search to non buildings can significantly help speed up performance. In
order for styles to take advantage of this index, however, they will
need to add the condition of "where building is null and ([previous
where condition]). If all style sheet authors could check if they can
add this where clause to any low-zoom access to the polygon table that
would be very helpful.
- At very low zooms, e.g. something like 0 - 4, one might not want to
access the osm data at all and instead use the shape file data like the
coast lines and the natural earth[1,2] shape files that live in
~osm/data/world_boundaries/ directory
- Potentially it might be better to combine several database layers into
a single query at low zoom levels, to reduce the load on the database.
Other optimizations to the styles would of cause also be helpful and
once these styles are improved, it is hopefully possible to do an expiry
again.
Kai
P.S. I had to disable a bunch of styles from tirex as they were causing
troubles including regular crashes of mapnik. These styles are all of
the qa/qai styles, the germany style, the powermap, shapenames,
surveillance and the bw-noicon style. Once they are fixed, they can be
reactivated.
[1]
http://en.wikipedia.org/wiki/Natural_Earth
[2]
http://www.naturalearthdata.com/