Am Tuesday 09 June 2009 07:40:41 schrieb Daniel Kinzler:
Reading back a bit, it seems that Sven Anders was designated to be the admin for those boxes. But I expect it would be good if there was more than one OSM person with access to them. All people with Wikimedia shell/root access will be able to access the OSM production boxes too, but they probably won't except for emergencies.
Sven, Please contact Mark and Brion about getting access.
I have just mailed Mark and Brion about getting access.
When Jochen ask me, to do this admin job, he said something about: "testing system, to see what OSM/German Wikipedia intergration would go like". Now its: "Production System and International Project".
I'm a litte afraid about the job maintaining two productive machines for the Wikipedia. So I would be very happy if there could be a team of 2-3 people doing this together.
I'm sorry to tell you that I will not be on my computer in the next 3 weeks (okay perhabs I will check mails, but not having time to set up a Postgis System).
Sven
2009/6/14 Sven Anders sven@anders-hamburg.de
When Jochen ask me, to do this admin job, he said something about: "testing system, to see what OSM/German Wikipedia intergration would go like". Now its: "Production System and International Project".
We'll start with testing and then we'll learn on our way. You'll soon find out that a "Wikipedia production system" is not so much different from a test box...
I'm a litte afraid about the job maintaining two productive machines for the Wikipedia. So I would be very happy if there could be a team of 2-3 people doing this together.
I'd be ready to start with squid setup and similar things.
I'm sorry to tell you that I will not be on my computer in the next 3 weeks (okay perhabs I will check mails, but not having time to set up a Postgis System).
I can set up PostGIS, but I'm not the one to do the fine tuning.
Who can grant me access to these servers, Avar?
Regards,
Jens
On Tue, Jun 16, 2009 at 7:49 AM, Jens Frankjens.l.frank@googlemail.com wrote:
2009/6/14 Sven Anders sven@anders-hamburg.de
When Jochen ask me, to do this admin job, he said something about: "testing system, to see what OSM/German Wikipedia intergration would go like". Now its: "Production System and International Project".
We'll start with testing and then we'll learn on our way. You'll soon find out that a "Wikipedia production system" is not so much different from a test box...
I'm a litte afraid about the job maintaining two productive machines for the Wikipedia. So I would be very happy if there could be a team of 2-3 people doing this together.
I'd be ready to start with squid setup and similar things.
I'm sorry to tell you that I will not be on my computer in the next 3 weeks (okay perhabs I will check mails, but not having time to set up a Postgis System).
I can set up PostGIS, but I'm not the one to do the fine tuning.
Who can grant me access to these servers, Avar?
Nope, I don't have access (yet?), and anyway I was just going to have access to the osm-toolserver (not the tile & db one) afaik.
Nope, I don't have access (yet?), and anyway I was just going to have access to the osm-toolserver (not the tile & db one) afaik.
As a WMF root, Jens should have access per default. The production boxes should have the usual wmf root key installed. Ask Mark for detaisl, he's the master of the squid cluster.
-- daniel
2009/6/16 Daniel Kinzler daniel.kinzler@wikimedia.de
Nope, I don't have access (yet?), and anyway I was just going to have access to the osm-toolserver (not the tile & db one) afaik.
As a WMF root, Jens should have access per default. The production boxes should have the usual wmf root key installed. Ask Mark for detaisl, he's the master of the squid cluster.
Ptolemy is online and serving tiles as reverse proxy (currently mirroring the OSMF mapnik tiles). Server statistics are still missing, so that we can watch cache hit ratio and other performance indicators.
I'll have a look at Ortelius tomorrow.
Regards,
jens
On Wed, Jun 17, 2009 at 9:28 PM, Jens Frankjens.l.frank@googlemail.com wrote:
2009/6/16 Daniel Kinzler daniel.kinzler@wikimedia.de
Nope, I don't have access (yet?), and anyway I was just going to have access to the osm-toolserver (not the tile & db one) afaik.
As a WMF root, Jens should have access per default. The production boxes should have the usual wmf root key installed. Ask Mark for detaisl, he's the master of the squid cluster.
Ptolemy is online and serving tiles as reverse proxy (currently mirroring the OSMF mapnik tiles). Server statistics are still missing, so that we can watch cache hit ratio and other performance indicators.
To update everyone here are some links Jens provided:
http://ortelius.esams.wikimedia.org/rrd/1day.cgi http://munin.openstreetmap.org/openstreetmap/tile.openstreetmap.html#Network
With regards to cassini (the toolserver) it now has a free 345 GB /sql partition. We could wait for the main osm-db server to be set up and set up replication there, or set up cassini as a stand-alone server which could be moved to replication later.
When can we expect the main osm-db to be online? And did anyone who's more familiar with setting up osm replication than me volunteer to set it up already?
With regards to cassini (the toolserver) it now has a free 345 GB /sql partition. We could wait for the main osm-db server to be set up and set up replication there, or set up cassini as a stand-alone server which could be moved to replication later.
When can we expect the main osm-db to be online? And did anyone who's more familiar with setting up osm replication than me volunteer to set it up already?
Originally, no replication was planned. Would the space even be sufficient?
-- daniel
On Wed, Jun 24, 2009 at 12:22 PM, Daniel Kinzlerdaniel.kinzler@wikimedia.de wrote:
With regards to cassini (the toolserver) it now has a free 345 GB /sql partition. We could wait for the main osm-db server to be set up and set up replication there, or set up cassini as a stand-alone server which could be moved to replication later.
When can we expect the main osm-db to be online? And did anyone who's more familiar with setting up osm replication than me volunteer to set it up already?
Originally, no replication was planned. Would the space even be sufficient?
Maybe:) This is the current space the DB is taking @ OSM: http://pastebin.com/f39a15e34
Maybe a Planet.osm import is smaller than that though. I could check and see if it fails.
I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensive queries.
Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you.
I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensive queries.
Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you.
There will definitly be no direct access to the main db. from what i gathered at the dev meeting, the idea was to let people import the regions they were interested in on the map TS. No replication, no full data, nothing.
But if replication isn't an issue space-wise, why not? It's sure cooler to have the full planet data to play with, in real time.
-- daniel
On Wed, Jun 24, 2009 at 12:33 PM, Daniel Kinzlerdaniel.kinzler@wikimedia.de wrote:
I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensive queries.
Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you.
There will definitly be no direct access to the main db. from what i gathered at the dev meeting, the idea was to let people import the regions they were interested in on the map TS. No replication, no full data, nothing.
But if replication isn't an issue space-wise, why not? It's sure cooler to have the full planet data to play with, in real time.
I'm imagining things most-likely.
I was under the impression that a normal user on the osm-ts would have access to both the Wikimedia and OSM databases in raw form somehow.
But nevermind:)
I'm imagining things most-likely.
I was under the impression that a normal user on the osm-ts would have access to both the Wikimedia and OSM databases in raw form somehow.
The'll have access to a *copy* of the WIkimedia database - and if you setup replication, also to a copy of the OSM database, although that wasn't in the original plan. Or maybe it was and I got it wrong, doesn't really matter :)
-- daniel
On Wed, Jun 24, 2009 at 12:46 PM, Daniel Kinzlerdaniel.kinzler@wikimedia.de wrote:
I'm imagining things most-likely.
I was under the impression that a normal user on the osm-ts would have access to both the Wikimedia and OSM databases in raw form somehow.
The'll have access to a *copy* of the WIkimedia database - and if you setup replication, also to a copy of the OSM database, although that wasn't in the original plan. Or maybe it was and I got it wrong, doesn't really matter :)
We'll see what works:)
On Wed, Jun 24, 2009 at 8:46 AM, Daniel Kinzler <daniel.kinzler@wikimedia.de
wrote:
I'm imagining things most-likely.
I was under the impression that a normal user on the osm-ts would have access to both the Wikimedia and OSM databases in raw form somehow.
The'll have access to a *copy* of the WIkimedia database - and if you setup replication, also to a copy of the OSM database, although that wasn't in the original plan. Or maybe it was and I got it wrong, doesn't really matter :)
-- daniel
I suggest we set up replication from the production server to the toolserver, and having raw access to the database on the toolserver would be best. I'm not sure the status of the production database server, but have e-mailed Brion and Mark to find out and see how I can help out with it.
I think the XAPI should go on the production server. The XAPI is read-only, while we may want to experiment on the toolserver with adding additional map features and tags to the database, and other things that require write access.
-Aude
--
Daniel Kinzler Software Developer Wikimedia Deutschland
Phone +49 30 219 158 260
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us achieve that! http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
What would a mirror of the database be used for? A mirror of the API, which is what XAPI provides would perhaps be more useful.
Is there a list of proposed applications? The only one I know of is query-to-map.
80n
On Wed, Jun 24, 2009 at 1:28 PM, Ævar Arnfjörð Bjarmason avarab@gmail.comwrote:
On Wed, Jun 24, 2009 at 12:22 PM, Daniel Kinzlerdaniel.kinzler@wikimedia.de wrote:
With regards to cassini (the toolserver) it now has a free 345 GB /sql partition. We could wait for the main osm-db server to be set up and set up replication there, or set up cassini as a stand-alone server which could be moved to replication later.
When can we expect the main osm-db to be online? And did anyone who's more familiar with setting up osm replication than me volunteer to set it up already?
Originally, no replication was planned. Would the space even be
sufficient?
Maybe:) This is the current space the DB is taking @ OSM: http://pastebin.com/f39a15e34
Maybe a Planet.osm import is smaller than that though. I could check and see if it fails.
I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensive queries.
Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you.
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
From what i gathered from our conversation earlier, XAPI's index is too large
for Cassini. It would have to go on the main server. I propose to talk to Sven and Jens about it. Or what do you mean by "a mirror of the API"?
-- daniel
80n schrieb:
What would a mirror of the database be used for? A mirror of the API, which is what XAPI provides would perhaps be more useful.
Is there a list of proposed applications? The only one I know of is query-to-map.
80n
On Wed, Jun 24, 2009 at 1:28 PM, Ævar Arnfjörð Bjarmason <avarab@gmail.com mailto:avarab@gmail.com> wrote:
On Wed, Jun 24, 2009 at 12:22 PM, Daniel Kinzler<daniel.kinzler@wikimedia.de <mailto:daniel.kinzler@wikimedia.de>> wrote: >> With regards to cassini (the toolserver) it now has a free 345 GB /sql >> partition. We could wait for the main osm-db server to be set up and >> set up replication there, or set up cassini as a stand-alone server >> which could be moved to replication later. >> >> When can we expect the main osm-db to be online? And did anyone who's >> more familiar with setting up osm replication than me volunteer to set >> it up already? > > Originally, no replication was planned. Would the space even be sufficient? Maybe:) This is the current space the DB is taking @ OSM: http://pastebin.com/f39a15e34 Maybe a Planet.osm import is smaller than that though. I could check and see if it fails. I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensive queries. Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you. _______________________________________________ Maps-l mailing list Maps-l@lists.wikimedia.org <mailto:Maps-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
On Wed, Jun 24, 2009 at 1:37 PM, Daniel Kinzler <daniel.kinzler@wikimedia.de
wrote:
From what i gathered from our conversation earlier, XAPI's index is too large for Cassini. It would have to go on the main server. I propose to talk to Sven and Jens about it. Or what do you mean by "a mirror of the API"?
XAPI is a replication of the main OSM database that is maintained to be current within about 30 minutes of the real OSM database. It exposes the data through a similar API to the real OSM database. It requires about 300Gb of disk.
There was a point made earlier that there wouldn't be space for XAPI because that would leave no room for the replicated database. Now it seems questionable whether there's a real need for an additional replicated database on osm-ts. What role would it play, what applications would use it?
80n
-- daniel
80n schrieb:
What would a mirror of the database be used for? A mirror of the API, which is what XAPI provides would perhaps be more useful.
Is there a list of proposed applications? The only one I know of is query-to-map.
80n
On Wed, Jun 24, 2009 at 1:28 PM, Ævar Arnfjörð Bjarmason <avarab@gmail.com mailto:avarab@gmail.com> wrote:
On Wed, Jun 24, 2009 at 12:22 PM, Daniel Kinzler<daniel.kinzler@wikimedia.de <mailto:daniel.kinzler@wikimedia.de>> wrote: >> With regards to cassini (the toolserver) it now has a free 345 GB /sql >> partition. We could wait for the main osm-db server to be set upand
>> set up replication there, or set up cassini as a stand-aloneserver
>> which could be moved to replication later. >> >> When can we expect the main osm-db to be online? And did anyonewho's
>> more familiar with setting up osm replication than me volunteer to set >> it up already? > > Originally, no replication was planned. Would the space even be sufficient? Maybe:) This is the current space the DB is taking @ OSM: http://pastebin.com/f39a15e34 Maybe a Planet.osm import is smaller than that though. I could check and see if it fails. I was under the impression that replication was planned so that users on the osm-ts would have access to the DB without hammering the main DB (which would be there for production tools) with expensivequeries.
Or so I gathered from our talk on IRC on the toolserver channel, but may have misunderstood you. _______________________________________________ Maps-l mailing list Maps-l@lists.wikimedia.org <mailto:Maps-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
--
Daniel Kinzler Software Developer Wikimedia Deutschland
Phone +49 30 219 158 260
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us achieve that! http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l