Earlier this spring, there was some talk, perhaps within the newly founded Wikimedia Norway chapter, to set up a common server in Norway for analysing database dumps and running bot scripts, i.e. a toolserver with everything except the database replication.
Did this happen? Are there a bunch of such servers around the world today? Is there a list of such servers? Should we encourage people to do this, or should we try to concentrate work (and pool resources) to the (real / German) toolserver?
Concentrating to fewer servers has the advantage that people can get a single account there, and join each project that is already there. Distributing over multiple, independent servers has the advantage of offloading the central server. It might also seem like an attractive project for a new Wikimedia chapter, since a server of their own looks like an achievement in itself. But it is also a waste of resources to maintain separate servers, if these resources could instead be pooled.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Lars Aronsson:
Distributing over multiple, independent servers has the advantage of offloading the central server.
really? how does it create less load to have one TS in .nl and one in .no compared to two identical servers in .nl?
- river.
River Tarnell wrote:
really? how does it create less load to have one TS in .nl and one in .no compared to two identical servers in .nl?
If you don't have the server in .no, the Norwegian users would easily move to the central server (as easily as you can create accounts for them, which has historically not been so fast), but the funding for the Norwegian server (which could be corporate or university sponsored) most probably would not.
I would prefer to pool our efforts and resources into making the existing toolserver more useful and more powerful. But then, as a toolserver admin an employee of WMDE, I'm biased :)
Anyway, the most efficient and, at least for us, easiest way to contribute would be a donation to wmde appropriated to be spent on the toolserver project. In the past there has been some confusion about how to best do this, we are working on sorting this out now. If anyone is interested in donating money or hardware or some other assets, please contact me.
If you feel like setting up your own, we are happy to cooperate. But be warned that it not only takes money, but it also takes a lot of time and effort to manage something like this (thanks river!).
-- daniel
Lars Aronsson schrieb:
Earlier this spring, there was some talk, perhaps within the newly founded Wikimedia Norway chapter, to set up a common server in Norway for analysing database dumps and running bot scripts, i.e. a toolserver with everything except the database replication.
Did this happen? Are there a bunch of such servers around the world today? Is there a list of such servers? Should we encourage people to do this, or should we try to concentrate work (and pool resources) to the (real / German) toolserver?
Concentrating to fewer servers has the advantage that people can get a single account there, and join each project that is already there. Distributing over multiple, independent servers has the advantage of offloading the central server. It might also seem like an attractive project for a new Wikimedia chapter, since a server of their own looks like an achievement in itself. But it is also a waste of resources to maintain separate servers, if these resources could instead be pooled.
A donation to a chapter within one jurisdiction is not something that is easily moved to another chapter in another jurisdiction.
In this specific case there were a need for a solution and, well, little response from other. One admin from the nowp community managed to get a deal both on a server and hosting and got it up and running.
Sometimes it is simply more efficient to get a working solution, then to wait for something to happen.
John
Daniel Kinzler wrote:
I would prefer to pool our efforts and resources into making the existing toolserver more useful and more powerful. But then, as a toolserver admin an employee of WMDE, I'm biased :)
Anyway, the most efficient and, at least for us, easiest way to contribute would be a donation to wmde appropriated to be spent on the toolserver project. In the past there has been some confusion about how to best do this, we are working on sorting this out now. If anyone is interested in donating money or hardware or some other assets, please contact me.
If you feel like setting up your own, we are happy to cooperate. But be warned that it not only takes money, but it also takes a lot of time and effort to manage something like this (thanks river!).
-- daniel
Lars Aronsson schrieb:
Earlier this spring, there was some talk, perhaps within the newly founded Wikimedia Norway chapter, to set up a common server in Norway for analysing database dumps and running bot scripts, i.e. a toolserver with everything except the database replication.
Did this happen? Are there a bunch of such servers around the world today? Is there a list of such servers? Should we encourage people to do this, or should we try to concentrate work (and pool resources) to the (real / German) toolserver?
Concentrating to fewer servers has the advantage that people can get a single account there, and join each project that is already there. Distributing over multiple, independent servers has the advantage of offloading the central server. It might also seem like an attractive project for a new Wikimedia chapter, since a server of their own looks like an achievement in itself. But it is also a waste of resources to maintain separate servers, if these resources could instead be pooled.
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
John at Darkstar wrote:
In this specific case there were a need for a solution and, well, little response from other. One admin from the nowp community managed to get a deal both on a server and hosting and got it up and running.
Aha, so there is a Norwegian toolserver? To what extent is it sponsored by the chapter? Can Swedish users get accounts?
On Wed, Jul 29, 2009 at 12:07 PM, Lars Aronssonlars@aronsson.se wrote:
Aha, so there is a Norwegian toolserver? To what extent is it sponsored by the chapter? Can Swedish users get accounts?
As far as I know, it's this: http://toolserver.no/ and anyone who requests an account can get one (not just Norwegians or Skandinavians).
Yes, its the one. I'm not sure if its imposed any restrictions on user accounts, but I don't think so.
The server is sponsored by a company selling PC's and related equipment, and it is located in a facility for the municipality and city Tønsberg in Norway. The admin (Laaknor) who made the deal works there.
You may contact the user himself at http://no.wikipedia.org/wiki/User:Laaknor http://meta.wikimedia.org/wiki/User:Laaknor
John Erling Blad Wikimedia Norway
Casey Brown wrote:
On Wed, Jul 29, 2009 at 12:07 PM, Lars Aronssonlars@aronsson.se wrote:
Aha, so there is a Norwegian toolserver? To what extent is it sponsored by the chapter? Can Swedish users get accounts?
As far as I know, it's this: http://toolserver.no/ and anyone who requests an account can get one (not just Norwegians or Skandinavians).
Casey Brown a écrit :
On Wed, Jul 29, 2009 at 12:07 PM, Lars Aronssonlars@aronsson.se wrote:
Aha, so there is a Norwegian toolserver? To what extent is it sponsored by the chapter? Can Swedish users get accounts?
As far as I know, it's this: http://toolserver.no/ and anyone who requests an account can get one (not just Norwegians or Skandinavians).
(a bit late, but..) Since it's the case, people only using tools that does not need DB replication could go there to alleviate some the toolserver's charge ? (I sometimes run a few queries, but I'm thinking about creating an account there myself)
DarkoNeko
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Wolfgang ten Weges:
Since it's the case, people only using tools that does not need DB replication could go there to alleviate some the toolserver's charge ?
i don't think we're short of resources at the moment.
- river.
Especially with those three Sun servers you plan to buy. Fahad Sadah
2009/8/2 River Tarnell river@loreley.flyingparchment.org.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Wolfgang ten Weges:
Since it's the case, people only using tools that does not need DB replication could go there to alleviate some the toolserver's charge ?
i don't think we're short of resources at the moment.
- river.-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAkp09swACgkQIXd7fCuc5vI/ZgCfUGq71VY0H3II9V2BdlggU4dx oEsAniTJVZkNvMMF2mlJ3I6zAfz/cnLQ =vEF9 -----END PGP SIGNATURE-----
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
John at Darkstar vacuum@jeb.no wrote:
A donation to a chapter within one jurisdiction is not something that is easily moved to another chapter in another jurisdiction. [...]
Why? If a chapter can pay a hosting company for their servi- ces with a donation, they should be able to pay WMDE for their services as well. There is no need to "move a donation to another jurisdiction".
In the end, I do not think that hardware ressources are an issue with the (current) toolserver. With the backup data- base servers on the horizon, replication should become more than good enough. What needs to be done, and River has pointed repeatedly in that direction, is to work on the tools themselves: Make them "stable", make them user-friend- ly, identify stuff that should better be integrated into Me- diaWiki (interwiki bots, Templatetiger, Geohack, etc.), etc.
Tim
On Wed, Aug 5, 2009 at 7:09 AM, Tim Landscheidt tim@tim-landscheidt.dewrote:
John at Darkstar vacuum@jeb.no wrote:
A donation to a chapter within one jurisdiction is not something that is easily moved to another chapter in another jurisdiction. [...]
Why? If a chapter can pay a hosting company for their servi- ces with a donation, they should be able to pay WMDE for their services as well. There is no need to "move a donation to another jurisdiction".
Well, this ignores the political and logistical sensitivities involved in paying WMDE from outside of DE.
While nothing prevents a chapter from opting to pay WMDE for this service, I'm sure some members/contributors would be annoyed their monies are going "out of country" when there are perfectly good hosting services "in our back yard". I'm not sure on the legalities of Charities in all jurisdictions, but this could also restrict/prevent a chapter registered as a charity from sending money out of country.
In the end, I do not think that hardware ressources are an
issue with the (current) toolserver. With the backup data- base servers on the horizon, replication should become more than good enough. What needs to be done, and River has pointed repeatedly in that direction, is to work on the tools themselves: Make them "stable", make them user-friend- ly, identify stuff that should better be integrated into Me- diaWiki (interwiki bots, Templatetiger, Geohack, etc.), etc.
Some projects have "bounties" or pay people to improve/fix parts of tools. This might be something that chapters could do more easily. Just an idea on how to get money from chapters who want to contribute to things -- and having a web page somewhere that says "This tool sponsored in part by the ZZ Chapter" lets members see where money is going. Just a thought, anyways.
Gerald.
Gerald A geraldablists@gmail.com wrote:
A donation to a chapter within one jurisdiction is not something that is easily moved to another chapter in another jurisdiction. [...]
Why? If a chapter can pay a hosting company for their servi- ces with a donation, they should be able to pay WMDE for their services as well. There is no need to "move a donation to another jurisdiction".
Well, this ignores the political and logistical sensitivities involved in paying WMDE from outside of DE.
While nothing prevents a chapter from opting to pay WMDE for this service, I'm sure some members/contributors would be annoyed their monies are going "out of country" when there are perfectly good hosting services "in our back yard". I'm not sure on the legalities of Charities in all jurisdictions, but this could also restrict/prevent a chapter registered as a charity from sending money out of country. [...]
I'm not aware of any such restriction, and in the 27 coun- tries south of Norway they would probably be illegal anyhow. But maybe I'm just biased as a German member of a German Verein that pays for hosting in not-so-German Amsterdam - as long as the charitable goals are achieved, I don't see a problem.
Tim
On Wed, Aug 5, 2009 at 7:09 AM, Tim Landscheidttim@tim-landscheidt.de wrote:
Why? If a chapter can pay a hosting company for their servi- ces with a donation, they should be able to pay WMDE for their services as well. There is no need to "move a donation to another jurisdiction".
My understanding (IANAL anywhere, especially not Europe) is that they can let WM-DE host and manage the servers, but they might not be able to just give them to WM-DE. Charities can't necessarily give away assets to non-charities, otherwise there would be an obvious loophole in the whole "not for profit" idea (accept donation, give away to for-profit corporation you own, pocket as profit).
So we'd have a lot of servers owned by a lot of different foundations in different countries. If you need new hardware for this server you have to get the money from this person, for this server you need this person. You get a dozen different support contracts and warranties instead of one per vendor/manufacturer. I don't deal with the hardware at all, but it looks like enough of a pain for the admins to just have to deal with servers owned by one or two groups.
So it would be nice if one organization could own all the servers and manage all the support contracts. Like Wikimedia Deutschland. But apparently that's not easy to do, either legally or politically.
On Wed, Aug 5, 2009 at 7:09 AM, Tim Landscheidttim@tim-landscheidt.de wrote:
Why? If a chapter can pay a hosting company for their servi- ces with a donation, they should be able to pay WMDE for their services as well. There is no need to "move a donation to another jurisdiction".
As soon as there is cash flow from one chapter to another, you are affectively moving donations from a jurisdiction to another.
On Wed, Aug 5, 2009 at 19:04, Aryeh GregorSimetrical+wikilist@gmail.com wrote:
My understanding (IANAL anywhere, especially not Europe) is that they can let WM-DE host and manage the servers, but they might not be able to just give them to WM-DE. Charities can't necessarily give away assets to non-charities, otherwise there would be an obvious loophole in the whole "not for profit" idea (accept donation, give away to for-profit corporation you own, pocket as profit).
WM DE is a non-profit, so the problem is not there. Or I didn't understand your statement. The question has a lot to do with what we do.
As you point out, hardware is an asset, and it is harder to file as "mission oriented" to start with, wherever it is located.
On top of that, some countries have restrictions inherent to their non-profit legal frame. Regardless of the donor's point of view, which, with good PR, can always be turned around to show that supporting the German tolserver is within the mission, the money simply cannot go to "another organisation", because it implies legal ties or afiliations that are hard to translate into "mission oriented" projects. Moving cash around in the non-profit world is really complicated.
As far as i know, for example, most of the donations that Wikimedia PL receives come with strings attached, and Wikimedia PL can only spend that money _within Poland_. None of their doing, it's the law, you can't do anything about it. It goes with their non-profit status. In France, for example, it would take a lot of explaining to the tax office to say why we're "funding" Wikimedia Deutschland, but it would all depend on the amount (and the percentage of the chapter's financial means it represents).
My take is that as chapters grow, it will be easier to fund the German toolserver as the amounts needed will be smaller for each chapter (relatively) and can be more easily explained to the national tax offices.
So we'd have a lot of servers owned by a lot of different foundations in different countries. If you need new hardware for this server you have to get the money from this person, for this server you need this person. You get a dozen different support contracts and warranties instead of one per vendor/manufacturer. I don't deal with the hardware at all, but it looks like enough of a pain for the admins to just have to deal with servers owned by one or two groups.
So it would be nice if one organization could own all the servers and manage all the support contracts. Like Wikimedia Deutschland. But apparently that's not easy to do, either legally or politically.
Spot on, this is the headache we're facing. Too many cooks spoil the broth. Efficiency is reduced if you have to go to X for one machine, and Z for the next one.
Delphine (who knows nothing about toolservers, but a little more about chapters and chapters' finances ;-))
receives come with strings attached, and Wikimedia PL can only spend that money _within Poland_. None of their doing, it's the law, you can't do anything about it. It goes with their non-profit status. In
I find this hard to believe. By the same logic it would be impossible for a polish non-profit to perform humanitarian aid in development countries.
On Thu, Aug 6, 2009 at 00:33, Daniel Schwenlists@schwen.de wrote:
receives come with strings attached, and Wikimedia PL can only spend that money _within Poland_. None of their doing, it's the law, you can't do anything about it. It goes with their non-profit status. In
I find this hard to believe. By the same logic it would be impossible for a polish non-profit to perform humanitarian aid in development countries.
Not all donations come within this frame. I hope I'm not saying anything wrong here, as this is the way I have understood it, but there is a special way a charity can get donations in Poland, and that is through people's income tax. To make a long story short, at the end of the fiscal year as a tax payer in poland, you have the possibility to decide that 1% (or some other %?) of your income tax goes to a designated charity (approved by the tax office). Donations received through this mean are to be spent in Poland.
Charities can also fundraise in various other ways and the donations coming through those other channels do not come with those strings attached. Not all charities are "designated charities", although they can still have tax-deductible status. Which should answer your disbelief... charities in Poland _can_ spend their money elsewhere than in Poland, if it does not come through this 1%.
Delphine
2009/7/28 Lars Aronsson lars@aronsson.se:
Did this happen? Are there a bunch of such servers around the world today? Is there a list of such servers? Should we encourage people to do this, or should we try to concentrate work (and pool resources) to the (real / German) toolserver?
There are to main reasons why sometimes it's easier to create toolserver on your own then ''donating'' German toolserver:
* it's not easy to transfer money to other country from chapter (if it has charity status eg.) * not everyone accepts German rules on their toolserver :P
AJF/WarX
ps. WM-PL has 2 machines which can be called ''toolserver''
There are to main reasons why sometimes it's easier to create toolserver on your own then ''donating'' German toolserver:
- it's not easy to transfer money to other country from chapter (if it
has charity status eg.)
- not everyone accepts German rules on their toolserver :P
Right - and I think the fist reason sucks and the second is a good reason. Ironically, the second problem is probably easier to overcome.
if, when and how charities can transfer/donate to organizations in another country seems to be an issue for the lawyers. *sigh*
-- daniel
2009/7/28 Daniel Kinzler daniel@brightbyte.de:
Right - and I think the fist reason sucks and the second is a good reason. Ironically, the second problem is probably easier to overcome.
if, when and how charities can transfer/donate to organizations in another country seems to be an issue for the lawyers. *sigh*
Full agree ;)
Transfering money can create larger problem than law (which can be solved using lots of well-payed lawyers) - PR.
It can be used by journalists to attack chapter because ''they are sending our money given in *good faith* to Germany (or any other country)!'' and derivations of this topic.
Probably it would be worth to analyze creating specialized toolserver clusters in different countries. For example:
* German cluster has replicated data from WMF, so it can be used for tools that need that * AFAIK Gc is near European proxies, so it's good for running bots on wikis * historically Polish *toolserver* is used as place for keeping photos not uploaded to Commons by some of our photographers (safe storage and collaboration in choosing photos for uploading)
* If for Norwegians it's easier to buy for themselves machine with many-many processors, tons of RAM, etc. why not to create facility for dump analisys in Norway?
AJF/WarX
On 28/07/2009, at 10:55 PM, Artur Fijałkowski wrote:
- AFAIK Gc is near European proxies, so it's good for running bots
on wikis
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
-- Andrew Garrett agarrett@wikimedia.org http://werdn.us/
2009/7/29 Andrew Garrett agarrett@wikimedia.org:
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
So it seems it's not any faster to run bots form German Toolserver than from our own, local one, as long as you don't use the database - am I right? Many people use TS to run their bots on it - including myself. Wouldn't some local TSs help balance the load?
2009/7/29 Adam Dodek Michalik dodecki@gmail.com
2009/7/29 Andrew Garrett agarrett@wikimedia.org:
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
So it seems it's not any faster to run bots form German Toolserver than from our own, local one, as long as you don't use the database - am I right? Many people use TS to run their bots on it - including myself. Wouldn't some local TSs help balance the load?
Maybe. The editing load would still be in Tampa. Mighta be that multiple TSes would still be faster as there is more RAM for the pywikibot stuff.
Marco
Adam Dodek Michalik schrieb:
2009/7/29 Andrew Garrett agarrett@wikimedia.org:
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
So it seems it's not any faster to run bots form German Toolserver than from our own, local one, as long as you don't use the database - am I right? Many people use TS to run their bots on it - including myself. Wouldn't some local TSs help balance the load?
More servers to run bots on would be a good thing. From a technical point of view, it would be better to have all of them in the same cluster, so user accounts and secondary services (svn, jira, etc) can be shared and administrative overhead is minimized.
The only reasons *not* to do that are political/financial. IF those could be resolved, that would be great. If not, then yes, having another place to run bots is still good.
-- daniel
There are several other reasons.
The uptime of the cluster in Amsterdam isn't that good, an other site would probably not be much better, but it could be alive while the TS' in Amsterdam are dead.
Also, there are probably legal reasons for using a local server for some type of material, especial regarding privacy but probably also due to copyright. If something is within the laws about privacy in Norway it is necessary to know who gets access to the data, which we do not know in Amsterdam. Another example is a server for a chapter-only website, with read only access after logon for members.
And at last, sometime it seems like the wmde-admins simply ain't responsive. Why that is so I don't know, but it is nearly impossible to run a project when you can't get in touch with the server admins.
John
Daniel Kinzler wrote:
Adam Dodek Michalik schrieb:
2009/7/29 Andrew Garrett agarrett@wikimedia.org:
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
So it seems it's not any faster to run bots form German Toolserver than from our own, local one, as long as you don't use the database - am I right? Many people use TS to run their bots on it - including myself. Wouldn't some local TSs help balance the load?
More servers to run bots on would be a good thing. From a technical point of view, it would be better to have all of them in the same cluster, so user accounts and secondary services (svn, jira, etc) can be shared and administrative overhead is minimized.
The only reasons *not* to do that are political/financial. IF those could be resolved, that would be great. If not, then yes, having another place to run bots is still good.
-- daniel
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Yea, there are good reasons to run independant servers. There are also good reaons to keep everything in one place.
The uptime of the cluster in Amsterdam isn't that good, an other site would probably not be much better, but it could be alive while the TS' in Amsterdam are dead.
While some parts of the system may be down, we didn't have a long downtime of everything for quite a while now. What does happen is that db replication breaks, somtimes so badly that we need to re-import. That takes long. This will hopefull improve seen, stay tuned for a blog post later today :)
Also, there are probably legal reasons for using a local server for some type of material, especial regarding privacy but probably also due to copyright. If something is within the laws about privacy in Norway it is necessary to know who gets access to the data, which we do not know in Amsterdam.
Yes, if you keep private info there, this is an issue. We try not to.
Another example is a server for a chapter-only website, with read only access after logon for members.
Absolutely. I wouldn't put anything like that on a box with user access. WMDE's chapter wiki and internal tools are not hosted on the toolserver bot at a commercial hoster in Germany.
And at last, sometime it seems like the wmde-admins simply ain't responsive. Why that is so I don't know, but it is nearly impossible to run a project when you can't get in touch with the server admins.
Well, it's a volunteer project, so there's no guarantee that anyone does anything. But... have you tried lately? In addition to me (i suck as an admin), River and DaB, we have simetrical and werdna now, and avar for OSM-Related stuff. I think the situation has become much better compared to, say, a year ago.
Anyway - I can see that if you have a sponsor to give the hardware, a place to put them and the people to maintain them, it's faster and easier to just set them up and run them. And I'm happy if it works.
I would suggest though that we should set up some means of exchanging info between the different projects of that kind - to my knowledge, there'S the boxes in poland, now in norway, and wmde's toolserver. Do you know of more? What would be a good way to communicate? Yet another mailing list?
-- daniel
Daniel Kinzler wrote:
Adam Dodek Michalik schrieb:
2009/7/29 Andrew Garrett agarrett@wikimedia.org:
The German toolserver is in the same rack as the European proxies, but it's not better because edits and API requests have to go to Tampa.
So it seems it's not any faster to run bots form German Toolserver than from our own, local one, as long as you don't use the database - am I right? Many people use TS to run their bots on it - including myself. Wouldn't some local TSs help balance the load?
More servers to run bots on would be a good thing. From a technical point of view, it would be better to have all of them in the same cluster, so user accounts and secondary services (svn, jira, etc) can be shared and administrative overhead is minimized.
The only reasons *not* to do that are political/financial. IF those could be resolved, that would be great. If not, then yes, having another place to run bots is still good.
-- daniel
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Perhaps there could be some kind of central management of some kind.
One thing is distribution of the open databases. It should not be necessary to set up replication for each and every toolserver.
Some kind of central authentication so access can be federated perhaps, yet authorization should be enforced locally.
But then perhaps we could use the same facility if the internal access rules on the various servers could be more stringent, that is a Norwegian server is for Norwegian use only - even if its located in the cluster. More like a webhotel with server hosting for chapters. Now, make it even more general and say virtual servers for the chapters own use and make them off-limit for other bot operators.
Still, note that such a server, virtual or not, has to be under complete control of the individual chapter. Even sharing backup tapes could be troublesome.
I would suggest though that we should set up some means of exchanging info between the different projects of that kind - to my knowledge, there'S the boxes in poland, now in norway, and wmde's toolserver. Do you know of more? What would be a good way to communicate? Yet another mailing list?
I don't think another mailing list is necessary. If a toolserver-box is used for something else that need higher confidentiality then it should go on a local chapter list or something similar.
-- daniel
John
It should not be necessary to set up replication _manually_ for each and every toolserver.
More or less... ;)
John (my typos are mine!)
John at Darkstar wrote:
Perhaps there could be some kind of central management of some kind.
One thing is distribution of the open databases. It should not be necessary to set up replication for each and every toolserver.
Some kind of central authentication so access can be federated perhaps, yet authorization should be enforced locally.
But then perhaps we could use the same facility if the internal access rules on the various servers could be more stringent, that is a Norwegian server is for Norwegian use only - even if its located in the cluster. More like a webhotel with server hosting for chapters. Now, make it even more general and say virtual servers for the chapters own use and make them off-limit for other bot operators.
Still, note that such a server, virtual or not, has to be under complete control of the individual chapter. Even sharing backup tapes could be troublesome.
I would suggest though that we should set up some means of exchanging info between the different projects of that kind - to my knowledge, there'S the boxes in poland, now in norway, and wmde's toolserver. Do you know of more? What would be a good way to communicate? Yet another mailing list?
I don't think another mailing list is necessary. If a toolserver-box is used for something else that need higher confidentiality then it should go on a local chapter list or something similar.
-- daniel
John
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Kerberos authentication. Easier than pubkeys, and more secure. Fahad Sadah
2009/7/29 John at Darkstar vacuum@jeb.no
Perhaps there could be some kind of central management of some kind.
One thing is distribution of the open databases. It should not be necessary to set up replication for each and every toolserver.
Some kind of central authentication so access can be federated perhaps, yet authorization should be enforced locally.
But then perhaps we could use the same facility if the internal access rules on the various servers could be more stringent, that is a Norwegian server is for Norwegian use only - even if its located in the cluster. More like a webhotel with server hosting for chapters. Now, make it even more general and say virtual servers for the chapters own use and make them off-limit for other bot operators.
Still, note that such a server, virtual or not, has to be under complete control of the individual chapter. Even sharing backup tapes could be troublesome.
I would suggest though that we should set up some means of exchanging
info
between the different projects of that kind - to my knowledge, there'S
the boxes
in poland, now in norway, and wmde's toolserver. Do you know of more?
What would
be a good way to communicate? Yet another mailing list?
I don't think another mailing list is necessary. If a toolserver-box is used for something else that need higher confidentiality then it should go on a local chapter list or something similar.
-- daniel
John
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolserver-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Fahad Sadah:
Kerberos authentication. Easier than pubkeys, and more secure.
i don't know about that.
- - critical vulnerabilities are frequently discovered in the MIT Kerberos software, while SSH has had very few serious security issues, and none recently.
- - both use a secure protocol with crypto which is currently unbreakable. there is a possible issue when using SSH with CBC mode ciphers, but this is fixed by using CTR mode ciphers instead.
- - Kerberos only works with password authentication, meaning anyone can log into any account if they know the password; for example, because someone accidentally typed their password into IRC, or wrote it down somewhere. strong password policy requires restrictions on password contents (length, character types, etc) that encourage users to write them down (especially when you have a lot of non-technical users, like us)
- - conversely, it is very difficult to accidentally paste a private key somewhere, and it's impossible to guess. even if it was leaked, the user would also have to leak the passphrase.
- - once you've generated a public key, it is no harder to use than a password. in fact, since you never have to change it, or remember it, or use a different one for each site, it's likely to be a lot easier.
while Kerberos is not insecure /per se/, i've considered deploying it in the past, and it seems like it would reduce the security of the Toolserver compared to public key authentication. (still, if you think password authentication is more secure than public key, i'd be interested to hear why.)
- river.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
River Tarnell:
in fact, since you never have to change it, or remember it, or use a different one for each site, it's likely to be a lot easier.
one other problem with password authentication: many users do use the same one for different sites, even if it's a bad idea. if the Toolserver was compromised, and we used Kerberos auth, an attacker could easily install a trojaned ssh server or client which recorded every user's password, and then use those passwords to attack other sites. (this is how Apache.org was compromised, except it was SF with the trojaned SSH instead of the Toolserver.)
conversely, with public keys, an evil server cannot do anything to compromise the security of the private key, since it's never sent to the server.
- river.
- critical vulnerabilities are frequently discovered in the MIT Kerberos
software, while SSH has had very few serious security issues, and none recently.
I didn't know that, tbh. I've used krb5 somewhere else for a while now, and no break-ins.
- - Kerberos only works with password authentication, meaning anyone can log
into any account if they know the password; for example, because someone accidentally typed their password into IRC, or wrote it down somewhere. strong password policy requires restrictions on password contents (length, character types, etc) that encourage users to write them down (especially when you have a lot of non-technical users, like us)
This is the main problem. However "a lot of non-technical users, like us" is untrue.
- conversely, it is very difficult to accidentally paste a private key
somewhere, and it's impossible to guess. even if it was leaked, the user would also have to leak the passphrase.
I doubt many people here use passphrases
Kerberos was just an example, btw. I was just suggesting the idea of using a centralized auth system.
Fahad Sadah
toolserver-l@lists.wikimedia.org