After a failure to install API database on ptolemy in reasonable amount of time, I have come to conclusion that we might not need full API database (ca. ~1 TB) for now, not at least for production Wikimedia use.
What Wikimedia might need is the Mapnik database (ca. 70GB for rendering) - this is what we have now on Cassini (although outdated - but see neighbouring thread).
Full API database - once *finally* imported - would have been probably more useful for tool developers, hacking properties and some other features of geodata (maybe even history).
However, only Ptolemy (currently WMF production server) has disk space available for the whole API database; I do not see - but I might be wrong - currently use for this database (except for genering mapnik DB out of it) - for the WMF production environment.
So we might have add disk space to Cassini to have space for the full API database, and start importing it there. But... Maybe we could just switch servers? (i.e. current Cassini reconfigure as WMF-production Ptolemy and current Ptolemy reinstall as a toolserver database).
Sure, it will require stopping tools for a while and do some IP (or maybe even rack-juggling) but maybe it's the solution.
We might alternatively give up on full database API altogether and just run Mapnik DB on the Ptolemy as we have.
What do you think?
Marcin Cieslak schrieb:
After a failure to install API database on ptolemy in reasonable amount of time, I have come to conclusion that we might not need full API database (ca. ~1 TB) for now, not at least for production Wikimedia use.
May I ask you where you got this dump from? It's not an official one, is it?
What Wikimedia might need is the Mapnik database (ca. 70GB for rendering)
- this is what we have now on Cassini (although outdated - but see neighbouring
thread).
Full API database - once *finally* imported - would have been probably more useful for tool developers, hacking properties and some other features of geodata (maybe even history).
Yes, it is! It may be the only way to get some historic information needed, to eg. show a changeset in some kind of cool diff-view or to develop smart changeset-revert tools (see [1] and [2] for this problem, the OSM-API does not and will in the near future not provide this kind of information).
However, only Ptolemy (currently WMF production server) has disk space available for the whole API database; I do not see - but I might be wrong
- currently use for this database (except for genering mapnik DB out of it)
- for the WMF production environment.
I do not see either. If Ptolemy won't be accessible by cassini and it's only task will be serving the mapnik render with gis-data, then I don't think we'll that much HD-Space on it. I'm unsure about the remaining system configuration (e.g. ptolemy has 8GB more RAM).
So we might have add disk space to Cassini to have space for the full API database, and start importing it there. But... Maybe we could just switch servers? (i.e. current Cassini reconfigure as WMF-production Ptolemy and current Ptolemy reinstall as a toolserver database).
Yes, please do this!
Sure, it will require stopping tools for a while and do some IP (or maybe even rack-juggling) but maybe it's the solution.
I don't think there are too many *live* tools on cassini (although there may be some in development stage).
We might alternatively give up on full database API altogether and just run Mapnik DB on the Ptolemy as we have.
The simple question is: do we want tools that can play with the history of osm-objects or not. If we want, we'll have to supply the necessary data.
Peter
Zitat von Peter Körner osm-lists@mazdermind.de:
Marcin Cieslak schrieb:
However, only Ptolemy (currently WMF production server) has disk space available for the whole API database; I do not see - but I might be wrong
- currently use for this database (except for genering mapnik DB out of it)
- for the WMF production environment.
I do not see either. If Ptolemy won't be accessible by cassini and it's only task will be serving the mapnik render with gis-data, then I don't think we'll that much HD-Space on it. I'm unsure about the remaining system configuration (e.g. ptolemy has 8GB more RAM).
For the system configuration take a look to: http://meta.wikimedia.org/wiki/OpenStreetMap#Servers It's not only a question of disk space, it's IMO more a question of disk speed and 16 HDs are faster than 4HDs.
So far I know are this servers also on different rigs and it's a question of foundation and verein. So the switch seem complicate. For tools are now also alternative servers from Fossgis online, so we should concentrate to the main target to bring the maps fast into Wikipedia. So it is perhaps simpler to replace the 4 HDs be bigger ones, add ones or do something else. I hope our hardware experts like Mark Bergsma have an idea.
Greetings Kolossos
For the system configuration take a look to: http://meta.wikimedia.org/wiki/OpenStreetMap#Servers
Yes, that's where I looked at - but I'm no hardware man, so..
For tools are now also alternative servers from Fossgis online,
I know that there are somewhere servers to play with (osm also has dev servers), but there is no known procedure to get an account on them. Furthermore there already is a bunch of cool guys having a toolserver account and it's not likely that they're going to apply for another dev-server, just to play a little with the osm data.
so we should concentrate to the main target to bring the maps fast into Wikipedia.
Yes, we should. But we should also not ignore the possible uses of cassini.
So it is perhaps simpler to replace the 4 HDs be bigger ones, add ones or do something else. I hope our hardware experts like Mark Bergsma have an idea.
First we should decide if we need sth. of the parts of ptolemy and if not, Marcin can continue his work. Next we can decide what our plans are with cassini.
Peter
Hi All
WMDE is currently planning the budget for 2010. I'll see if I can get some money for a JBOD and/or larger disks. I think we'lkl need more space at some point anyway. So, I suggest to delay this discussion for two weeks - then we know what we can or can't buy in 2010. If we can just add a JBOD to Cassini, I'd suggest to do that. If not, switching the servers might be an option.
-- daniel
Peter Körner schrieb:
For the system configuration take a look to: http://meta.wikimedia.org/wiki/OpenStreetMap#Servers
Yes, that's where I looked at - but I'm no hardware man, so..
For tools are now also alternative servers from Fossgis online,
I know that there are somewhere servers to play with (osm also has dev servers), but there is no known procedure to get an account on them. Furthermore there already is a bunch of cool guys having a toolserver account and it's not likely that they're going to apply for another dev-server, just to play a little with the osm data.
so we should concentrate to the main target to bring the maps fast into Wikipedia.
Yes, we should. But we should also not ignore the possible uses of cassini.
So it is perhaps simpler to replace the 4 HDs be bigger ones, add ones or do something else. I hope our hardware experts like Mark Bergsma have an idea.
First we should decide if we need sth. of the parts of ptolemy and if not, Marcin can continue his work. Next we can decide what our plans are with cassini.
Peter
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Zitat von Peter Körner osm-lists@mazdermind.de:
I know that there are somewhere servers to play with (osm also has dev servers), but there is no known procedure to get an account on them. Furthermore there already is a bunch of cool guys having a toolserver account and it's not likely that they're going to apply for another dev-server, just to play a little with the osm data.
Here you can found the procedure: http://wiki.openstreetmap.org/wiki/FOSSGIS/Server/Development-Server (Everything looks a little bit like on our site. ;-)) But for rest I give you absolutly right. There is no server like toolserver. :-)
Greetings Kolossos