During the developers meeting at 21C3, we saw a pretty impressive demonstartion of a map-generation software; alas, the underlying data was (partially) contradicting itself, and the software consists of parts in at least three programming language, all in alpha/beta stages, AFAIK.
When I saw someone upload dozens of maps on commons the other day, all identical except for a different county marked in red, I though there should be something simpler. Along the ideas that I published on meta some long time ago, I wrote a PHP script for a simple map generation tool. Source: CVS HEAD, "phase3/tools/geo.php", hereby GPL.
*Disclaimer:* This is just proof-of-concept; it would need significant work to become useful.
This little script reads data from indexed sets of texts (currently hardcoded as an array key=>text, but would be title=>wikipage in a "real" implementation) and generates SVG from it. In the included example (just run it from command line) it generates a simplified ;-] map of Germany, showing east and west Germany (no political implications here, just testing!) as well as the Danube.
The drwaing is based on poly-lines made from coordinate pairs in the text entries. The poly-lines are then appended to each other as needed, eventually forming polygons. This will save a great deal of work and keep the data consistent: a border line between two states, for example, is only defined once, then "included" into the polygons of both states. To take this even further, is a river partially forms the border, the very same polyline can be used as part of the river.
Currently, it draws each polygon separately, so in my demo, "Germany" is still divided into two states :-( However, I plan on a "merging" method, which will fuse two polygons into one by removing common borders, so East and West Germany could be drawn individually, with borders, or as the entity "Germany", with one common border around it.
Imagine this for the U.S.: If we had the border data for all counties, and want to draw a world map with the U.S. as a single entity, the county polygons would be fused into state polygons, which then would be fused into a few country polygons (continental, Hawaii, Alaska).
In my implementation, all data sets can carry meta information as well. The Danube in the example is painted in blue because it carries a "type:river" marker. Different object groups can be defined for different purposes (political, geographical, etc.) Also, objects can be labeled in all languages in-place. I currently don't show the labels, though.
In a "real" implementation, all data would be stored as wiki pages. A query to render some complicated view might take some time, then. But, as long as we don't need real-time refreshment (cache as cache can!), I would deem such a soultion prefferable to uploading (and maintaining) a zillion map images...
Magnus
Magnus Manske wrote:
Imagine this for the U.S.: If we had the border data for all counties, and want to draw a world map with the U.S. as a single entity, the county polygons would be fused into state polygons, which then would be fused into a few country polygons (continental, Hawaii, Alaska).
In my implementation, all data sets can carry meta information as well. The Danube in the example is painted in blue because it carries a "type:river" marker. Different object groups can be defined for different purposes (political, geographical, etc.) Also, objects can be labeled in all languages in-place. I currently don't show the labels, though.
Isn't this reimplementing some sort of a GIS data format and visualization system? I think there was at least one person on this list who claimed to have significant knowledge of the GIS field, and I know there are some open-source systems already available.
Stuff tends to get pretty complex as you go along: dealing with projections (since the earth isn't flat), 3d data, increasing/decreasing resolution as you zoom in/out, and even automatically placing labels.
-Mark
Delirium schrieb:
Isn't this reimplementing some sort of a GIS data format and visualization system? I think there was at least one person on this list who claimed to have significant knowledge of the GIS field, and I know there are some open-source systems already available.
Well, if there is a central, open-source (software and data), editable-by-anyone solution that we can just plug into MediaWiki, great, let's do it. Otherwise I say: GIS? So what? ;-)
Seroiusly: I *am* aware that there are many map-rendering programs around, some of them open source. But, the fact that noone has come up with a working solution for MediaWiki tells me that there's some kind of problem.
Stuff tends to get pretty complex as you go along: dealing with projections (since the earth isn't flat), 3d data, increasing/decreasing resolution as you zoom in/out, and even automatically placing labels.
Well, internally I want to store "real-world" coordinates (lalt/long). All coordinates are passed through a single function to convert them into display coordinates. For the different 2D mapping systems, that should not be a problem, as it can be tweaked at a single point in the source. 3D is a different beast, of course. Personally, I'd go for a 2D solution initially. If there is pressing need for rendering a map on a 3D globe, then displaying a 2D image of it in the browser, I am certain it will not exceed the joined programming abilities of the people on this list :-)
As for resolution/zoom, all objects can be tagged with metadata. I imagine rivers with "magnitude" tags 1-5 or something. Detail level could depend on actual resolution, default settings ("global","country","state",etc.), parameter ("show_magnitude=1-3"), or any combination of the above.
Magnus
Magnus Manske wrote:
Seroiusly: I *am* aware that there are many map-rendering programs around, some of them open source. But, the fact that noone has come up with a working solution for MediaWiki tells me that there's some kind of problem.
I think part of the problem is an impedence mismatch of sorts: Most of the GIS software is designed by and for geographers who want to run large statistical queries on it, while we by and large just want pretty pictures, and want them as fast and simply as possible. Still, it'd be a shame if none of the considerable work that's been done is reusable...
Well, internally I want to store "real-world" coordinates (lalt/long). All coordinates are passed through a single function to convert them into display coordinates. For the different 2D mapping systems, that should not be a problem, as it can be tweaked at a single point in the source. 3D is a different beast, of course. Personally, I'd go for a 2D solution initially. If there is pressing need for rendering a map on a 3D globe, then displaying a 2D image of it in the browser, I am certain it will not exceed the joined programming abilities of the people on this list :-)
As for resolution/zoom, all objects can be tagged with metadata. I imagine rivers with "magnitude" tags 1-5 or something. Detail level could depend on actual resolution, default settings ("global","country","state",etc.), parameter ("show_magnitude=1-3"), or any combination of the above.
That sounds reasonable. This is probably part of why GIS systems are so complex: They want, for example, arbitrary and automatic resolution scaling based on rather complex feature-summarization algorithms, so you store the most detailed data, and it automatically abstracts it when you "zoom out" as necessary. But I imagine 5 discrete levels will suffice for 99% of our purposes. Part of the problem remaining then is that we'd have to regenerate all this data ourselves, or at least tag it: There are huge data repositories of GIS data already freely available (especially for the US), but they wouldn't have this sort of information.
Anyway, it does seem like a good way to go forward, barring anyone with significant GIS experience jump-starting integration of one of those systems into MediaWiki in the near future (their documentation is too horrid for me to have succeeded in my last attempt). I do have some vague familiarity with the existing datasets, so when I get some free time (probably in about 2 weeks... conference submission deadlines coming up) I'll see if I can write some scripts to extract some of the sorts of data that'd be useful to us from the commonly-available data sets, so we don't have to manually input every bit of data that's already out there.
-Mark
On Wed, 12 Jan 2005, Magnus Manske wrote:
Well, if there is a central, open-source (software and data), editable-by-anyone solution that we can just plug into MediaWiki, great, let's do it. Otherwise I say: GIS? So what? ;-)
Some experimental work was done by Taw:
http://meta.wikipedia.org/Maps
Alfio
Alfio Puglisi schrieb:
On Wed, 12 Jan 2005, Magnus Manske wrote:
Well, if there is a central, open-source (software and data), editable-by-anyone solution that we can just plug into MediaWiki, great, let's do it. Otherwise I say: GIS? So what? ;-)
Some experimental work was done by Taw:
Yes, that's the one that was shown and discussed on the developer meeting at 21C3. The one consisting of three different parts in different programming languages. And the underlying data has to be fixed manually anyway, as it contains numerous errors.
Again, I am not trying to push my 'ware here. I am just proposing to have a (rather simple) PHP script and a wikimedia project with lots of volunteers to do the work (in this case, data collecting and -structuring, not writing articles)
If you (the list) think this is worth exploring, I could try and get some more detailed data for a *real* demonstration. If you think it is a waste of time, I won't. Either way, tell me! :-)
Magnus
-----BEGIN PGP SIGNED MESSAGE-----
Moin,
On Wednesday 12 January 2005 17:10, Magnus Manske wrote:
During the developers meeting at 21C3, we saw a pretty impressive demonstartion of a map-generation software; alas, the underlying data was (partially) contradicting itself, and the software consists of parts in at least three programming language, all in alpha/beta stages, AFAIK.
When I saw someone upload dozens of maps on commons the other day, all identical except for a different county marked in red, I though there should be something simpler. Along the ideas that I published on meta some long time ago, I wrote a PHP script for a simple map generation tool. Source: CVS HEAD, "phase3/tools/geo.php", hereby GPL.
*Disclaimer:* This is just proof-of-concept; it would need significant work to become useful.
I sense a deja vu here :-)
(I actually toyed with the idea of SVG generation for graphs, but postponed that to later once the ideas are more stabilized :)
However, I like your idea - even though I am not in map-making myself.
Best wishes,
Tels
- -- Signed on Wed Jan 12 17:42:43 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email.
"A Thaum is the basic unit of magical strength. It has been universally established as the amount of magic needed to create one small white pigeon or three normal-sized billiard balls." -- Terry Pratchett
Hi,
how often do maps change? Seeing that there is already a lot of work being done by the servers, I wouldn't want them to do things where you don't need to collaborate (thus not needing online rendering capabilities).
Ciao, Dscho
Johannes Schindelin schrieb:
Hi,
how often do maps change? Seeing that there is already a lot of work being done by the servers, I wouldn't want them to do things where you don't need to collaborate (thus not needing online rendering capabilities).
I imagine a separate project (wikimaps.org?) on a server of its own. Especially in the beginning, many changes will be made to the map data, so I'd suggest re-rendereing maps once a day or so at an off-peak time.
Later, when these maps are actually used by wikimedia and other projects, I'd use a FIFO order to process map requests, and forcible keep the load beneath a certain threshold. Maybe a request could generate an initial image under that name, saying "This map is being rendered. please reload your browser page in 5 minutes to see the map." or something. This would be shown on the requesting page until the map is rendered.
However, since I have no idea how long rendering would take, it might just work in real-time.
Magnus
wikitech-l@lists.wikimedia.org