Hello,
the wikimedia OSM tile server on toolserver has been running for quite a while now including the integration into the German, French and Russian wikipedia as well as some other Wikipedia's through the OSM gadget.
From the recent quietness of this list and from Kolossos' talk at FOSSGIS, it appears that the technical side of things is working fairly smooth? Are there any major issues left from the perspective of tile serving and the OSM gadget?
I was also wondering if there are any thoughts, impressions or data on what the general impression of this integration has been on the general audience and if it has been received well? Did wikipedian's like having the maps easily accessible in the articles? Are people aware of this option? How did they react to that OSM might not always be complete? And did some of wikipedia's editors start editing OSM data to improve the map for "their" article?
I was also wondering if there are any plans to expand the use of OSM in wikipedia? Particularly, do more wikipedia's want to activate their language version? What would it take to integrate it into the english wikipedia?
Sorry for all the questions, but I was curious to hear how people see the current status and its future. If there are any wiki pages, talks or other material about the current status of the OSM wikipedia integration I'd be thankful for any further links as well.
Kai
On Tue, Apr 19, 2011 at 11:51 PM, Kai Krueger kakrueger@gmail.com wrote:
Sorry for all the questions, but I was curious to hear how people see the current status and its future. If there are any wiki pages, talks or other material about the current status of the OSM wikipedia integration I'd be thankful for any further links as well.
I'm curious too. Has there been any discussion of how the change in OSM license will affect integration? Are there any Wikipedia ---> OSM information-sharing scripts that may break as a result?
SJ
What I dont understand is this: if the license is not good enough for osm, why is it not good enough for wikipedia? If the license is broken, why dont the authors of the license fix it so all projects benefit.
it is all confusing for me. mike
On Wed, Apr 20, 2011 at 5:56 AM, Samuel Klein meta.sj@gmail.com wrote:
On Tue, Apr 19, 2011 at 11:51 PM, Kai Krueger kakrueger@gmail.com wrote:
Sorry for all the questions, but I was curious to hear how people see the current status and its future. If there are any wiki pages, talks or other material about the current status of the OSM wikipedia integration I'd be thankful for any further links as well.
I'm curious too. Has there been any discussion of how the change in OSM license will affect integration? Are there any Wikipedia ---> OSM information-sharing scripts that may break as a result?
SJ
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Am 20.04.2011 06:20, schrieb Mike Dupont:
What I dont understand is this: if the license is not good enough for osm, why is it not good enough for wikipedia? If the license is broken, why dont the authors of the license fix it so all projects benefit.
The "value" of a single wikimedia article is enough to have it protected by CC.
The "value" of a single node is to small.
The "value" of osm comes from the combination of all nodes and way, from the database. CC does not protect databases in most countries.
More on the Wiki: http://www.osmfoundation.org/wiki/License/We_Are_Changing_The_License#Why_ar...
on the Legal-ML ;)
and, if you cqan understand german, in this wonderful record from the FOSSGIS 2011: http://ftp5.gwdg.de/pub/misc/openstreetmap/FOSSGIS2011/FOSSGIS2011-323-de-os...
Peter
On Apr 19, 2011, at 11:56 PM, Samuel Klein meta.sj@gmail.com wrote:
On Tue, Apr 19, 2011 at 11:51 PM, Kai Krueger kakrueger@gmail.com wrote:
Sorry for all the questions, but I was curious to hear how people see the current status and its future. If there are any wiki pages, talks or other material about the current status of the OSM wikipedia integration
Hope to be a topic of discussion and hacking at Berlin hackathon. Get some gadget enabled on enwiki, spend time to improve the dewiki gadget and work on other maps tools.
(who all will be there? who will be in Haifa for hacking days + wikimania?)
I'd be thankful for any further links as well.
I'm curious too. Has there been any discussion of how the change in OSM license will affect integration?
Compatibility of licenses is over my head. Is this something Geoff would be able to look at?
Are there any Wikipedia ---> OSM information-sharing scripts that may break as a result?
OSM already can't and doesn't use anything from wikipedia since wiki coordinates are tainted with derived google maps data. (by US standards, the coordinates are just facts, non-copyrightable and okay that wikipedians collect them that way). By UK/EU law, google maps derived data is protected by database rights and osm operates under such. ODBL is attempting to address database rights while CC-BY-SA does not.
Sharing and reusing osm data on wikipedia is okay and think still will be with license change but not sure what we need to do to ensure we comply with license terms.
Cheers, Katie
SJ
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
On Wed, Apr 20, 2011 at 2:32 AM, aude aude.wiki@gmail.com wrote:
On Apr 19, 2011, at 11:56 PM, Samuel Klein meta.sj@gmail.com wrote:
Are there any Wikipedia ---> OSM information-sharing scripts that may break as a result?
OSM already can't and doesn't use anything from wikipedia since wiki coordinates are tainted with derived google maps data.
I was thinking of data other than geocoordinates. See for instance
http://lists.openstreetmap.org/pipermail/talk/2011-April/057531.html
Sharing and reusing osm data on wikipedia is okay and think still will be with license change but not sure what we need to do to ensure we comply with license terms.
Neither am I...
SJ
Well, it would be nice if maybe the toolserver map generation can also create thematic or topical maps based on OSM data.
Right now what is displayed in articles is a general-purpose street map.
Some examples of other maps: 1) subway lines (maybe with streets dimmed) for a subway network article 2) administrative boundaries 3) university and college maps (with the surrounding area dimmed) 4) topographical maps (maybe using SRTM data) for mountain articles
On Wed, Apr 20, 2011 at 11:51 AM, Kai Krueger kakrueger@gmail.com wrote:
Hello,
the wikimedia OSM tile server on toolserver has been running for quite a while now including the integration into the German, French and Russian wikipedia as well as some other Wikipedia's through the OSM gadget.
From the recent quietness of this list and from Kolossos' talk at FOSSGIS, it appears that the technical side of things is working fairly smooth? Are there any major issues left from the perspective of tile serving and the OSM gadget?
I was also wondering if there are any thoughts, impressions or data on what the general impression of this integration has been on the general audience and if it has been received well? Did wikipedian's like having the maps easily accessible in the articles? Are people aware of this option? How did they react to that OSM might not always be complete? And did some of wikipedia's editors start editing OSM data to improve the map for "their" article?
I was also wondering if there are any plans to expand the use of OSM in wikipedia? Particularly, do more wikipedia's want to activate their language version? What would it take to integrate it into the english wikipedia?
Sorry for all the questions, but I was curious to hear how people see the current status and its future. If there are any wiki pages, talks or other material about the current status of the OSM wikipedia integration I'd be thankful for any further links as well.
Kai
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hello, here some things that running absolutely not smooth in my eyes and we should work on this problems:
*The Expiring/updating process is not running for months.[1] (To have an up-to-date map is perhaps not so important for wikipedia, but I hear from some mappers, who use hikebikemap and other maps to control there changes.)
*The rendering of the >200 styles for the different languages make it difficult to handle the server.
*We have running now a productive system and experimental systems on the same server. This means we can't make in the moment big experiments! I will be also in Berlin and I hope we can talk about a solution for this situation.
*We have enough CPU power but if I look on the I/O-power [2] it looks that we are on the end of sd3-reading power. So I'm not sure about the reason, but I don't believe that we should activate gadget in en.wp in this situation. (A upgrade to SSD's would be very interesting for I/O-power. I my eyes we would need only 400 GB so it shouldn't cost too much if we could speed-up the rendering perhaps by factor 5! This would also speed-up the other tools which uses the OSM-database. A solution by brain-power would be off course better.)
*We have a performance problem for "full-styles" at medium zoom-levels so e.g. "black-and-white"-styles running in the moment at zoom-level 10 over 1000 seconds for one meta-tile. That's too long!
So in this moment it wouldn't be a good idea to make advertisement for new developer to bring more and more map-styles on the server.
With activating the map in english wikipedia I estimate that we could increase the load on server by factor two. If we make advertisement for this map the peak on the first day could be higher.
So we have a lot to do, especially in the admin area. And we should talk more about this problems on this list. --------- To the legal concerns I give Frederik an answer on legal-talk[3]. That shouldn't be our biggest problem.
Greetings Kolossos
[1] https://jira.toolserver.org/browse/TS-971 [2] http://munin.toolserver.org/OSM/ptolemy/index.html#disk [3] http://lists.openstreetmap.org/pipermail/legal-talk/2011-April/005907.html
Hello,
thank you for the information.
On 04/20/2011 01:40 PM, Tim Alder wrote:
Hello, here some things that running absolutely not smooth in my eyes and we should work on this problems:
*The Expiring/updating process is not running for months.[1] (To have an up-to-date map is perhaps not so important for wikipedia, but I hear from some mappers, who use hikebikemap and other maps to control there changes.)
If I understood Peter's mail from the beginning of March correctly, the expiry is partially working? I.e. the touch-mode is working, but not the renderd-mode? As I presume the touch-mode is used for low-zoom and renderd-mode for low zoom, the situation shouldn't be too bad.
Mappers tend to use the high zoom levels to check for details and the low-zoom tend not to have noticeable changes that often anyway. Osm.org also only does direct tile expiry on high-zoom tiles. One possibility for a band aid until the expiry is fixed correctly, would be to touch the db time stamp file. This would expire all tiles and would therefore add some extra strain on the server for a couple of days, but it would also allow to rerender tiles for which the style sheet has changed.
*The rendering of the>200 styles for the different languages make it difficult to handle the server.
Is that still the memory issue, or did the move to tirex mitigate this?
*We have running now a productive system and experimental systems on the same server. This means we can't make in the moment big experiments! I will be also in Berlin and I hope we can talk about a solution for this situation.
*We have enough CPU power but if I look on the I/O-power [2] it looks that we are on the end of sd3-reading power. So I'm not sure about the reason, but I don't believe that we should activate gadget in en.wp in this situation.
On nearly all tile servers I have seen so far, the bottleneck has always been the I/O performance. So it wouldn't be surprising that the read IOPS of the disks are maxed out. There are two aspects to I/O performance. For one the postgis access and secondly tile access. With only ca. 100 tiles/s I wouldn't think the tile access would be a significant I/O bottleneck. Postgis on the other hand will likely take all I/O performance it can get as long as there is something to render.
(A upgrade to SSD's would be very interesting for I/O-power. I my eyes we would need only 400 GB so it shouldn't cost too much if we could speed-up the rendering perhaps by factor 5! This would also speed-up the other tools which uses the OSM-database. A solution by brain-power would be off course better.)
You would presumably only want to put the postgis-DB on an SSD? The tiles are probably better off on larger but slower disks.
*We have a performance problem for "full-styles" at medium zoom-levels so e.g. "black-and-white"-styles running in the moment at zoom-level 10 over 1000 seconds for one meta-tile. That's too long!
So in this moment it wouldn't be a good idea to make advertisement for new developer to bring more and more map-styles on the server.
With activating the map in english wikipedia I estimate that we could increase the load on server by factor two. If we make advertisement for this map the peak on the first day could be higher.
Pure tile serving wouldn't likely be the issue. A reasonably powerful server like ptolemy should be able to handle greater than 1000 tiles/s. Tiles are also well cachable through squid proxies. What the effect on rendering queues and tile disk usage would be, is probably harder to predict though.
So we have a lot to do, especially in the admin area. And we should talk more about this problems on this list.
To the legal concerns I give Frederik an answer on legal-talk[3]. That shouldn't be our biggest problem.
There has been a lot of controversial debate on the licensing issue within the OSM community, but I don't think it would have significant effects for data users like wikipedia. Tile serving is well within the anticipated use cases for either license. Possibly all that would change would be a slight change in the attribution string displayed. Tiles (as produced work) can probably even stay under CC-BY-SA.
Kai
Greetings Kolossos
[1] https://jira.toolserver.org/browse/TS-971 [2] http://munin.toolserver.org/OSM/ptolemy/index.html#disk [3] http://lists.openstreetmap.org/pipermail/legal-talk/2011-April/005907.html
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Quoting Kai Krueger kakrueger@gmail.com:
Hello,
...
*The rendering of the>200 styles for the different languages make it difficult to handle the server.
Is that still the memory issue, or did the move to tirex mitigate this?
It is more the point that it is difficult to restart tirex (waiting 15 min...). So it makes no fun to install new styles and experiment. The memory usage seems acceptable.
On nearly all tile servers I have seen so far, the bottleneck has always been the I/O performance. So it wouldn't be surprising that the read IOPS of the disks are maxed out. There are two aspects to I/O performance. For one the postgis access and secondly tile access. With only ca. 100 tiles/s I wouldn't think the tile access would be a significant I/O bottleneck. Postgis on the other hand will likely take all I/O performance it can get as long as there is something to render.
Ok, if all requests from en.wp goes to nearly the same points on the earth that if also have on de.wp it should work and we could do the same trick that we use at we start at de.wp, because we know the coordinates we could pre-render the areas. That could work.
(A upgrade to SSD's would be very interesting for I/O-power. I my eyes we would need only 400 GB so it shouldn't cost too much if we could speed-up the rendering perhaps by factor 5! This would also speed-up the other tools which uses the OSM-database. A solution by brain-power would be off course better.)
You would presumably only want to put the postgis-DB on an SSD? The tiles are probably better off on larger but slower disks.
Yes, I thought about postGIS-DB.
Greetings Kolossos
How often we will update wiki mini atlas with openstreetmap ? http://meta.wikimedia.org/wiki/WikiMiniAtlas
It
On 25 April 2011 23:17, Tim Alder tim.alder@s2002.tu-chemnitz.de wrote:
Quoting Kai Krueger kakrueger@gmail.com:
Hello,
...
*The rendering of the>200 styles for the different languages make it difficult to handle the server.
Is that still the memory issue, or did the move to tirex mitigate this?
It is more the point that it is difficult to restart tirex (waiting 15 min...). So it makes no fun to install new styles and experiment. The memory usage seems acceptable.
On nearly all tile servers I have seen so far, the bottleneck has always been the I/O performance. So it wouldn't be surprising that the read IOPS of the disks are maxed out. There are two aspects to I/O performance. For one the postgis access and secondly tile access. With only ca. 100 tiles/s I wouldn't think the tile access would be a significant I/O bottleneck. Postgis on the other hand will likely take all I/O performance it can get as long as there is something to render.
Ok, if all requests from en.wp goes to nearly the same points on the earth that if also have on de.wp it should work and we could do the same trick that we use at we start at de.wp, because we know the coordinates we could pre-render the areas. That could work.
(A upgrade to SSD's would be very interesting for I/O-power. I my eyes we would need only 400 GB so it shouldn't cost too much if we could speed-up the rendering perhaps by factor 5! This would also speed-up the other tools which uses the OSM-database. A solution by brain-power would be off course better.)
You would presumably only want to put the postgis-DB on an SSD? The tiles are probably better off on larger but slower disks.
Yes, I thought about postGIS-DB.
Greetings Kolossos
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hey Guys,
How often we will update wiki mini atlas with openstreetmap ? http://meta.wikimedia.org/wiki/WikiMiniAtlas
Well, Tim asked me to comment on this thread, so here we go. In the last few months I spent a great deal of time rewriting bot front and back end of the WikiMiniAtlas. The new front end code is now JQuery based and more compact. The backend code is more flexible and allowes me to now support clickable map labels in 70 languages. Other new features include
* Short article synopsis when Ctrl+hovering over a label * Highlighting all coordinates of the current article with click navigation (try this on list articles with 100s of coordinates)
My development version supports dynamic resizing of the map window. This would make it fairly straight forward now to support new tile sizes (WMA tiles are 128x128, OSM is 256x256). The other difference compared to OSM is the mapprojection. My label backend used to be designed for one particular mapprojection (lat-long). The added flexibility in the new backend would make it possible to add support for Mercator-like projections as OSM or GoogleMaps have.
To make the story short: Supporting OSM tiles in the WMA jumped to the top of my priority list. Due to IRL commitments I'm fairly tied up with work until June. But this project is exciting and fun enough for me to sneak in a few hours of coding here and there.
Daniel [[User:Dschwen]]
Thanks Daniel for the update.
Will there be a provision to highlight osm relation on wikipedia article ?
http://wiki.openstreetmap.org/wiki/WikiProject_India/National_Highway_Networ...
On 30 April 2011 09:28, Daniel Schwen daniel@schwen.de wrote:
Hey Guys,
How often we will update wiki mini atlas with openstreetmap ? http://meta.wikimedia.org/wiki/WikiMiniAtlas
Well, Tim asked me to comment on this thread, so here we go. In the last few months I spent a great deal of time rewriting bot front and back end of the WikiMiniAtlas. The new front end code is now JQuery based and more compact. The backend code is more flexible and allowes me to now support clickable map labels in 70 languages. Other new features include
- Short article synopsis when Ctrl+hovering over a label
- Highlighting all coordinates of the current article with click
navigation (try this on list articles with 100s of coordinates)
My development version supports dynamic resizing of the map window. This would make it fairly straight forward now to support new tile sizes (WMA tiles are 128x128, OSM is 256x256). The other difference compared to OSM is the mapprojection. My label backend used to be designed for one particular mapprojection (lat-long). The added flexibility in the new backend would make it possible to add support for Mercator-like projections as OSM or GoogleMaps have.
To make the story short: Supporting OSM tiles in the WMA jumped to the top of my priority list. Due to IRL commitments I'm fairly tied up with work until June. But this project is exciting and fun enough for me to sneak in a few hours of coding here and there.
Daniel [[User:Dschwen]]
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hello, sorry for late answer. The highlighting of osm objects is my next plan for the OpenLayers-map. See: http://commons.wikimedia.org/w/index.php?title=File%3AFossgis2011-WP-GEO.pdf... With OpenLayers it should be no problem to support such vector overlays, in WikiMiniAtlas it could be much more difficult.
We need in each case be careful that the vector-objects are not to complex.
But I don't want to use the OSM-IDs because they seem me not stable enough. My actual plan is to work with the Wikipedia-tags inside OSM. Your example with such a list of objects in one article is (if this list is in wikipedia) a new problem to solve but we could use as Tag something like "wikipedia=lang:articlename#objectname"
I would be happy to find some help for this project, so if somebody interested, please contact me.
Greetings Kolossos
Am 03.05.2011 05:25, schrieb Naveen Francis:
Thanks Daniel for the update.
Will there be a provision to highlight osm relation on wikipedia article ?
http://wiki.openstreetmap.org/wiki/WikiProject_India/National_Highway_Networ...