This is most helpful.  I can work on getting this ready for next week's Signpost.

-Aude

On Mon, Jun 15, 2009 at 5:59 AM, Daniel Kinzler <daniel@brightbyte.de> wrote:
Aude schrieb:
> I'm the "tech beat" editor for the English Wikipedia's Signpost
> newspaper.  In the weekly tech reports, we can mention things going on
> with the toolserver when I'm aware of them.  Being aware of things
> generally means reading the mailing list and blogs.   (I have added the
> toolserver blog to http://open.wikiblogplanet.com/)
>
> The Signpost also runs special stories on occassion for tech topics.  I
> would be happy to run something about the toolserver.  If you would
> answer some questions, I could turn the answers into a signpost
> article.  The article could take an "interview" format like the
> WikiProject reports.  (e.g.
> http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2009-05-25/WikiProject_report)
> or I can turn it into another format (especially if others give input).

Great Idea! I'll try to answer the questions below, maybe River can add more
info. Or anyone else, for that matter, I'm sure I'll forget something important.

I'm not quite sure though if the toolserver blog should be on the planet...
raising awareness is definitly good. On the other hand, status reports like
"replication failed, will fix tomorrow" is not so helpful in the aggregated
feed. But I guess we could change it to a filtered stream when people complain.

> Anyway, here are some questions:
>
> * What is the toolserver? (e.g., it's not just one server but several...)

The Toolserver, or more accurately, the Toolserver cluster, is a project by
Wikimedia Germany that allows interested programmers to work with a live copy of
the wiki databases.

> * What is the purpose of the toolserver? / what we do?

The Toolserver is used mainly to provide specialized web based tools to the wiki
communities, but also to run bots and similar services. The Toolserver is also
open to researchers that want to study wikimedia's projects.

> * How does the toolserver operate? (e.g. it has replicated copies of the
> wiki databases, minus the full text, to allow people to develop tools.
> The servers are located in Amsterdam)

The Toolserver cluster, is located in Amsterdam, has several types of machines.
Maily, there are three:

* the database servers, which contain a live copy of all the wiki databases -
however without the full article text. Article text is stored in compressed
blocks on a special cluster. There is a copy of this in Amsterdam, but it is not
accessible directly to toolserver users. The main reason for this is that these
compressed blocks may contain private data.

* then there is the web server, where users can provide web based tools. This is
what the wiki communities usually see.

* and there are the login servers, machines where programmers can run bots and
other "offline" scripts, or run commands directly.

> * Who runs it?  / who are we? / how is it funded?

The Toolserver is run by Wikimedia Germany, it is funded entirely from
donations. The Wikimedia Foundation helps with hosting the machines in Amsterdam.

> * What is your role with the toolserver?

I am an employee of Wikimedia Germany, and I'm responsible for planning the
toolserver budget, among other things. I also have root access, and can approve
new accounts, but I'm a lousy system administrator. So mainly my role is
planning and organizing.

I am of course also a toolserver user and provide several tools, the most well
known of wich are probably CatScan and CheckUsage.

> * How did the toolserver get started?

To be honest, I don't know the details of the very beginning. I only joined when
it was already up for a couple of months. But I believe that there was some
frustration by people who had the skill to develop helpful tools for the wiki
communities, but did not get access to the main server cluster, for security
reasons. So the idea arose to create a place for such people to work with the data.

> * How do Wikipedian's get accounts on the toolserver?  What are the
> requirements and approval process?

Wikipedians (or anyone else, actually) can request an account by simply stating
what you want to do there. The requests are reviewed by DaB (and sometimes by
me), and if the project falls into the Toolserver's scope (which is: supporting
Wikimedia and, more recently, OpenStreetMap projects), seems technically
feasible and complies to our terms of use, the request is granted.

Until now, account requests have been handled at
<http://meta.wikimedia.org/wiki/Toolserver/New_accounts>. However, we are
currently reworking the process, to make it more swift and more transparent.

The terms of use for the toolserver are defined by
<https://wiki.toolserver.org/view/Rules>, the conditions for account approval
are at <https://wiki.toolserver.org/view/Account_approval_policy/en>.

> * If one gets an account, what are the responsibilities?  I know
> accounts had to be reconfirmed, are there resource limits such as disk
> space limits for each user? or what?

It's the user's responsibility to comply to the rules, that is, to do no evil.
There are no hard limits on resources, users are asked to be considerate. Heavy
tasks that slow down the system may be killed by an administrator. Generally, we
try to address problems on a case by case basis, directly with the user.

> * Who is the person to contact when things break?

That depends on what is broken. We have over a hundred active users, many of
which have several tools in diverse states of completeness or repair. Generally,
when something is broken, contact the author of that something - Toolserver
administrators usually can not fix it.

The matter is different when the Toolserver  as such has a problem - for
example, when one of the servers is down. In that case, write an email to
ts-admins@toolserver.org. You can also find us on IRC, in the channel
#wikimedia-toolserver.

> * We already are aware of the map toolserver project, but along with
> that, some other new hardware was recently ordered?  What's the purpose
> of the new hardware?

The latest hardware order included five servers, three of them for the
Toolserver cluster:

* we are replacing our oldest database server, which has been constantly
overloaded recently.

* we are putting user data (home directories) onto a new machine and move other
things around a bit. In the end, there will be a second login server, for
running bots and the like.

* and last but not least: the map toolserver, a place for the OpenStreetMap
community to develop exciting new projects.

In addition, two servers are provided for integrating "live" interactive maps
ion wikipedia articles, based on OpenStreetMap data.

> * What's coming in the pipeline (in the future)?

We are mainly working on the reliability of the toolserver - for instance, we
hope to have two copies of every wiki database in the future, so tools using
that data can keep running even when one system fails.

Generally, we are trying to make the Toolserver cluster a mature part of the
Wikimedia landscape.

> * Anything else people should know about the toolserver?

We should all say thanks to River, our senior system administrator, for keeping
the Toolserver running!

-- daniel

_______________________________________________
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l