Maps-l

maps-l@lists.wikimedia.org

495 discussions

Priorities_2012 for cooperation WP-OSM
by Tim Alder 11 Jan '12

11 Jan '12

Hello together, I want to use the beginning of the new year to collect our projects and project ideas to get an overview to the priorities for 2012. So please take a look to the new page: https://wiki.toolserver.org/view/OpenStreetMap/Priorities_2012 Please add your name as supporter for projects you want really like to see. You can also add your own project ideas. Greetings Kolossos

1 0

Regular Maintenance at 7. December
by DaB. 10 Jan '12

10 Jan '12

Hello all, like already announced after the last maintenance, the next maintenance will be at Wednesday, 7 December between 19:00 and 1:00 UTC. The roots will collect what they will do at [1] until Sunday night. If you have something for us to do (like a software-update) please open a bugreport at JIRA and make sure to add the label "maintaince-window" until Sunday noon. The roots plan to finish the configuration of Apache until Wednesday too, but there will be no switch yet to give you all some time for testing (I will send a eMail with more details when the time is right). Sincerly, DaB. [1] https://wiki.toolserver.org/view/Admin:Next_maintenance -- Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

2 3

Re: [Maps-l] Tile expiry, a new attempt ru.wp
by Tim Alder 04 Jan '12

04 Jan '12

Hello, for the russian map we have the OK from an ru.wp-admin so we can start. I did try to activate the russian style for all zoomlevels but missing tiles didn't go to the "missing"-queue of tirex. So I reset the change. What's wrong? Greetings Tim Am 03.01.2012 13:39, schrieb Kai Krueger: > On 01/02/2012 01:57 PM, Tim Alder wrote: >> >> People from ru.wp ask me to support russian but I did affraid that it >> will be too much load for us. But now is russian map so popular that >> it is perhaps usefull, we should use pre-rendering for it. > > We should probably test it to see how it goes. > Who would be the people on the russian wikipedia to talk to, to see what > they need? Also, should any changes take effect, who can tel us if > everything is working out and if the changes are positively received? > > Kai > >> >> |Kay: Please excuse my ignorance, but what is "default"? Is it the same as >> |"osm"? >> Yes. >> >> Greetings Kolossos >> >> >> Zitat von Kai Krueger<kakrueger(a)gmail.com>: >> >>> Hello everyone, >>> >>> Over the past month, the server has been busy trying to re-render all of >>> the tiles to try to ensure more up-to date maps. This process is more or >>> less complete now and all of the map tiles should then be at least from >>> the 1st of December 2011. >>> >>> As one month old tiles is still not overly satisfying, I am now trying >>> to re-activate diff based tile expiry again. >>> >>> The full expiry of all styles remains infeasible for the moment and so I >>> am trying to introduce expiry slowly to see how far it can be pushed. >>> >>> As a start, I have activated tile expiry only for the highest zoom >>> levels (z15 - z18) and then only for the styles that currently are >>> embedded in one of the wikipedias. >>> >>> As far as I can tell these are the following styles: >>> >>> osm-no-labels osm-labels-de osm-labels-ca osm-labels-bg osm-labels-da >>> osm-labels-el osm-labels-es osm-labels-fa osm-labels-fr osm-labels-it >>> osm-labels-lv osm-labels-nl osm-labels-no osm-labels-pl osm-labels-ru >>> germany default hikebike >>> >>> If anyone knows of a language that I have forgotten, please let me know >>> and I will include it as well. >>> >>> So far, it looks like the server can cope with this, although I have >>> only activated it earlier today. Should it remain this way, I will try >>> and expand the zoom levels. >>> >>> As it is critical to reduce the number of low zoom tiles rendered, as >>> those still kill performance, lower zoom tiles will probably not be >>> expired as frequently. E.g. perhaps Z14 - Z11 on a daily basis, below >>> that even only in the background. >>> >>> We will have to see, though, how performance behaves over time. >>> >>> Happy new year, >>> >>> Kai >>> >>> _______________________________________________ >>> Maps-l mailing list >>> Maps-l(a)lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/maps-l >>> >>> >> >> >> >> >> _______________________________________________ >> Maps-l mailing list >> Maps-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/maps-l > >

1 1

Tile usage policy
by Kai Krueger 03 Jan '12

03 Jan '12

Hello everyone, I was wondering if there currently is something like a "tile usage policy". I.e. a policy about who can use tiles served from the toolserver and how many tiles they are allowed to request? Other tileservers like the one at osm.org, have had problems with "tile scrapers", usually smart phone applications that download large areas of tiles systematically for later offline use. They have had to enforce their policy either by throttling or outright banning applications and users, as a few tile scrapers can quickly overwhelm the resources of a tileserver. So far I don't think this has been much of an issue on the toolserver tileserver and there hasn't really been a need to enforce or limit access yet. I did see some behaviour that looked suspiciously like large scale tile scraping today though (systematic rendering of large number of empty hikebike tiles at Z14) and so it might be good to have a policy in place before it becomes an issue in the future. OpenStreetMap's tile usage policy can be found at http://wiki.openstreetmap.org/wiki/Tile_usage_policy for reference. Kai

2 1

Tile expiry, a new attempt
by Kai Krueger 03 Jan '12

03 Jan '12

Hello everyone, Over the past month, the server has been busy trying to re-render all of the tiles to try to ensure more up-to date maps. This process is more or less complete now and all of the map tiles should then be at least from the 1st of December 2011. As one month old tiles is still not overly satisfying, I am now trying to re-activate diff based tile expiry again. The full expiry of all styles remains infeasible for the moment and so I am trying to introduce expiry slowly to see how far it can be pushed. As a start, I have activated tile expiry only for the highest zoom levels (z15 - z18) and then only for the styles that currently are embedded in one of the wikipedias. As far as I can tell these are the following styles: osm-no-labels osm-labels-de osm-labels-ca osm-labels-bg osm-labels-da osm-labels-el osm-labels-es osm-labels-fa osm-labels-fr osm-labels-it osm-labels-lv osm-labels-nl osm-labels-no osm-labels-pl osm-labels-ru germany default hikebike If anyone knows of a language that I have forgotten, please let me know and I will include it as well. So far, it looks like the server can cope with this, although I have only activated it earlier today. Should it remain this way, I will try and expand the zoom levels. As it is critical to reduce the number of low zoom tiles rendered, as those still kill performance, lower zoom tiles will probably not be expired as frequently. E.g. perhaps Z14 - Z11 on a daily basis, below that even only in the background. We will have to see, though, how performance behaves over time. Happy new year, Kai

4 10

Postgis
by Kay Drangmeister 30 Dec '11

30 Dec '11

Hi My shell/python script that extracts the location of parking icons from the database does no longer work: /home/kayd/parkingicons/generate-parking-icons.sh The error message is: psycopg2.InternalError: transform: couldn't project point (2.79772e+06 8.45105e+06 0): failed to load NAD27-83 correction file (-38) HINT: PostGIS was unable to transform the point because either no grid shift files were found, or the point does not lie within the range for which the grid shift is defined. Refer to the ST_Transform() section of the PostGIS manual for details on how to configure PostGIS to alter this behaviour. To narrow down the problem please try this sql statement: osm_mapnik=> SELECT ST_Y(ST_Transform(ST_line_interpolate_point(way,0.5),4326)) FROM planet_line WHERE osm_id=27720543; ERROR: transform: couldn't project point (1.12026e+06 6.38855e+06 0): failed to load NAD27-83 correction file (-38) HINT: PostGIS was unable to transform the point because either no grid shift files were found, or the point does not lie within the range for which the grid shift is defined. Refer to the ST_Transform() section of the PostGIS manual for details on how to configure PostGIS to alter this behaviour. Of course this query worked before and it seems to be an installation problem of the new postgres/postgis version: http://gis.stackexchange.com/questions/13696/postgis-st-transform-failed-to… Kind regards, Kay

2 1

Optimizing style sheets for map-tile rendering
by Kai Krueger 23 Dec '11

23 Dec '11

Hello everyone, With the osm rendering database up-to-date again, and no selective expiry of tiles at the moment, I would like to do a global expiry of all tiles at some point. However, unfortunately, we are still experiencing some performance problems, particularly with low zoom rendering. These issues are likely to be two fold. a) The performance of the database is still not ideal, and b) stylesheets are being less efficient with database requests as they should be. As an example of a combination of a) and b), one of the queries from the hikebike map at zoom level 7 took over 18 minutes to complete for just a single query and many queries are necessary to render a tile. This perhaps explains why a lot of lowzoom tiles don't successfully complete. I am still trying to see if anything can be done about a), but given that these issues have been around since pretty much the beginning, I am not sure how much we can really improve things here. With respect to b), we will need the help of all style sheet authors, to try and optimize their style sheets as much as possible to limit unnecessary database access. The hikebike map style as an example unfortunately seems particularly problematic, but I think other stylesheets are somewhat suffering similar issues. For example the hikebike tiles at zoom levels below 9 are still timing out, despite having increasing the rendering timeout now to a non sustainable 1 hour period. With single tiles taking over an hour to render, rendering 200+ styles is clearly not going to work, so we need to try and optimize these style sheets. In general, one should try and limit the number of database access as much as possible and also try and filter out as much data as one can on the db side. In addition there are a few tips one can perhaps follow. At low zooms, the biggest problems are access to the planet_line and planet_polygon tables. - At zoom levels 7 - 8 and lower, accessing the planet_line table should probably be avoided at all costs, as it is simply to slow. All (or hopefully most) of the information needed at low zooms should be in the planet_roads table. Queries on planet_roads should be much faster, as it is only 3.6GB in size rather than the 26Gb of the planet_lines table - There is no equivalent of the planet_roads for polygons. However, I have now created a conditional spatial index on the planet_polygons table for the condition of "where building is null", that semi partitions the table. Currently nearly 80% of all polygons are building polygons. At a total size of 18Gb for the planet_polygon, limiting the search to non buildings can significantly help speed up performance. In order for styles to take advantage of this index, however, they will need to add the condition of "where building is null and ([previous where condition]). If all style sheet authors could check if they can add this where clause to any low-zoom access to the polygon table that would be very helpful. - At very low zooms, e.g. something like 0 - 4, one might not want to access the osm data at all and instead use the shape file data like the coast lines and the natural earth[1,2] shape files that live in ~osm/data/world_boundaries/ directory - Potentially it might be better to combine several database layers into a single query at low zoom levels, to reduce the load on the database. Other optimizations to the styles would of cause also be helpful and once these styles are improved, it is hopefully possible to do an expiry again. Kai P.S. I had to disable a bunch of styles from tirex as they were causing troubles including regular crashes of mapnik. These styles are all of the qa/qai styles, the germany style, the powermap, shapenames, surveillance and the bw-noicon style. Once they are fixed, they can be reactivated. [1] http://en.wikipedia.org/wiki/Natural_Earth [2] http://www.naturalearthdata.com/

4 7

non-rendering application at the osm-database
by Peter Wendorff 21 Dec '11

21 Dec '11

Hi. I'm working on the look-and-listen-map (toolserver project lalm), a web map portal for the Blind and Visually Impaired. That is not a rendering application like the several mapnik stylesheets running on the toolserver, but I agreed with the Admins, that it might be worth in respect of capacities of the toolserver to use the mapnik style nevertheless. Today I finally got database access and looked deeper into the schema. Most queries in my application rely on the osm_id present in all tables (and index, too), so that's not a problem. Some queries will later on require geometric queries, but here I can combine the postprocessed tables (planet_nodes, planet_line, planet_polygon...) for filtering and the non-processed ones for fetching the values - so that's okay, too. My problem lies, where mapnik does ugly tricks for rendering: I have a node-id and want to get all ways this node is part of. That would be possible, querying planet_ways, as planet_ways contains a field nodes, that is a integer-array; but as far as I know there is no performant way to query an array-field of a database on containing a particular element. Currently my code is based on a different database schema, but I have a query with the following characteristics, that I need with the osm_mapnik-scheme: IN: node_id (int) OUT: a list of way_id, where the corresponding way contains the node with id node_id. A second question would be, if errors would occur, when I combine e.g. planet_ways with planet_line in a query, and a matching row in planet_ways is pending. Usually the intermediate tables are only accessed by the import tool, so I would do unconventional things here, but am I right? If the planet_ways-row is pending, the data of this table is not in sync with the data in planet_line. regards Peter

3 3

Re-import of the osm_mapnik database on ptolemy
by Kai Krueger 20 Dec '11

20 Dec '11

Hello everyone, as you all probably know, the osm_mapnik database is not being updated anymore since a while. About two months ago the database seems to have become corrupted, which in turn has unfortunately prevented diffs from being applied. In order to get things up-to-date again and working more smoothly than before, the database needs to be freshly imported. During this re-import period the database will not be available for use. As a full import will likely take several days plus an additional couple of days for the diffs to catch up, there will unfortunately be a bit of an extended downtime for the database. During this time postgresql will likely be updated and I would also like to use it to see if the database can be optimized further to give better performance than before. To try and minimize disruption, I would like to ask who, or which tools, are currently using the osm_mapnik database? Is the downtime going to be an issue for anyone? In return, we will hopefully have an up-to-date database again that can be kept up-to-date. Kai

5 8

Scripto, free software for transcribing documents
by Lars Aronsson 14 Dec '11

14 Dec '11

Scripto is an alternative to the ProofreadPage extension used by Wikisource. It is based on Mediawiki but also on OpenLayers, the software used to zoom and pan in OpenStreetMap. The only website I have seen that uses Scripto is the U.K. War Department papers, and in many ways it is more clumsy than ProofreadPage. But there might be a few ideas that could be worth picking up. Take a look. The software is described at http://scripto.org/ As for reference installations, they mention http://wardepartmentpapers.org/transcribe.php -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

1 0

← Newer
1
...
24
25
26
27
28
29
30
...
50
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Maps-l