Hi all
I'm happy to let you know that new hardware has been ordered by Wikimedia
Deutschland and will arrive probably in about two weeks. We will get two new
systems:
* A more powerful web server, to replace hemlock: Sun Fire X4150, 2x Quad-Core
Xeon, 8GB RAM, 2x73GB SAS HDD. The current web server only has two cores.
* Another database server, to be used for S1 (english wikipedia), so S1 and S3
no longer have to share a server: Sun Fire X4250, 2x Quad-Core Xeon, 32GB RAM,
16x146GB SAS RAID.
This should improve performance and give us some head space for growth. Once the
new servers arrive, S3 will be re-imported too, so we will have live data again.
Any ideas for names? To stay with the nightshade theme, how about Jurubeba and
Erubia? Or perhaps we go the "witches' weed" way, with Datura and Mandrake?
Henbane is taken, i think. Amanita sounds nice, too :)
A third server has been ordered, which will also be installed in Amsterdam, but
will not be part of the toolserver cluster. It's a storage server (X4540, 24TB
RAID) that will keep a live backup of all media files.
Cheers,
Daniel
hi all
nightshade ran out of ram. I don't know why, because i couldn't even log in via
serial to have a look. so i power-cycled it, should be back up shortly.
if you edited your scripts lately, please check if you made a fork bomb. or did
something that would consume all the ram.
cheers,
daniel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, Jan 28, 2009 at 12:53 AM, Platonides wrote:
> Marco Schuster wrote:
>> Hi all,
>>
>> I want to crawl around 800.000 flagged revisions from the German
>> Wikipedia, in order to make a dump containing only flagged revisions.
>> For this, I obviously need to spider Wikipedia.
>> What are the limits (rate!) here, what UA should I use and what
>> caveats do I have to take care of?
>>
>> Thanks,
>> Marco
>>
>> PS: I already have a revisions list, created with the Toolserver. I
>> used the following query: "select fp_stable,fp_page_id from
>> flaggedpages where fp_reviewed=1;". Is it correct this one gives me a
>> list of all articles with flagged revs, fp_stable being the revid of
>> the most current flagged rev for this article?
>
> Fetch them from the toolserver (there's a tool by duesentrieb for that).
> It will catch almost all of them from the toolserver cluster, and make a
> request to wikipedia only if needed.
I highly doubt this is "legal" use for the toolserver, and I pretty
much guess that 800k revisions to fetch would be a huge resource load.
Thanks, Marco
PS: CC-ing toolserver list.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Use GnuPG with Firefox : http://getfiregpg.org (Version: 0.7.2)
iD8DBQFJf6AjW6S2GapJUuQRAvBuAJ46G0qhk+e2axFddbHFMUqzScH4PgCeIMBL
L9WWNeZaA/6vHyzSoKrGN54=
=p/R+
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
i've made a couple of changes to how email works on the toolserver, mainly that
each account now has an associated email address in LDAP. i haven't added such
an address for existing accounts, but if you want to, you can set your email
address by running 'setmail'. your account's email address takes precedence
over $HOME/.forward for email to <username>@toolserver.org.
if you don't want to do this, you can continue to use the old .forward-style
mail forwarding. however, if you don't set an email address in LDAP, you won't
be able to make use of future services which might use this, such as
automatically resetting your SSH key or LDAP password.
to emphasise: you do not *need* to do anything because of this change if you
don't want to. if you do nothing, your account will continue to work as
before.
- river.
-----BEGIN PGP SIGNATURE-----
iD8DBQFJf+DSIXd7fCuc5vIRAjpoAJ4t1yOykKMVvCat/SRrR0aQsEocCQCeKGSF
KQ5VC9lAjWs+AsQfk4d//1g=
=BFK3
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
the email address to contact Toolserver administrators has changed. you should
now use ts-admins(a)toolserver.org, _not_ ts-admins(a)wikimedia.org. the latter
address will continue to work for now, but might stop working in the future.
- river.
-----BEGIN PGP SIGNATURE-----
iD8DBQFJfSx+IXd7fCuc5vIRAkE2AJ0WWxYy7OliRvDHSl3jJvvO/fQbUACguZ+m
fjvY/qB3VEv7Mj6fq6iX5bQ=
=Y4XC
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
mail to @toolserver.org addresses (and mail directly to any machine, e.g.
@nightshade.toolserver.org) is now being filtered for spam. the filtering is
configured fairly laxly; it will only reject mail with a SpamAssassin score
over 120, which will almost certainly not match legitimate mail. mail with a
score over 50 will be greylisted. other mail will have several headers added
(which begin with X-TS-Spam-), so you can do your own filtering.
we will monitor the effectiveness of the filtering over the next few days and
might adjust these parameters as needed.
- river.
-----BEGIN PGP SIGNATURE-----
iD8DBQFJfPvkIXd7fCuc5vIRAlr6AJ0ehjzk1S3a4maEgukBk0VD/+5jLQCgjihp
d0FbdBCZh/RoYf9hK8fgSDI=
=dUzL
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hi,
we are currently racking amaranth.toolserver.org, which will be the first US
toolserver, hosted in Tampa. this system is a Sun X4150 with 32GB RAM and an
external disk array with 12 146GB 15'000 rpm disks.
initially, this will be used for a US mirror of cache.stable.toolserver.org,
which will improve things using the cache (currently only WikiMiniAtlas) for US
users. we might also we it to host the various webapps (like JIRA), taking the
load off the existing machines in Amsterdam.
if anyone has any other suggestions as to what this machine could be used for,
let us know...
- river.
-----BEGIN PGP SIGNATURE-----
iD8DBQFJeNHsIXd7fCuc5vIRAt4tAJ4vtgwHJ91R3KiTXGxo2CPMItWVVwCcC7xW
K6+556Xgs9Le6Nyghczdni4=
=1X90
-----END PGP SIGNATURE-----
Hello All
I'm happy to announce the MediaWiki Developer Meet-Up will happen April 3.-5. in
Berlin, at the c-base. The event is for everyone who works on MediaWiki, writes
extensions, builds bots, writes scripts for the toolserver, or is otherwise
interested in the technical aspects of Wikimedia. We are happy that we can now
have the meet-up after our plans for 25C3 and FOSDEM failed. If you want to come
to the Developer Meetup, please sign up at
<http://www.mediawiki.org/wiki/Project:Developer_meet-up_2009>.
The event will take place in parallel to the Wikimedia Foundation's board
meeting and chapter meeting, so there will be a lot of Wikimedians in Berlin at
the time. We plan to have a party to bring everyone together and give an
opportunity for developers, board members and chapter people to mingle.
The meet-up will be a loose BacCamp-like event so topics and schedule are
largely up to you. The goal is to get to know new aspects of MediaWiki and
Wikimedia and to develop ideas on how we can make things even better. And of
course to have a lot of fun with wiki hackers from around the world!
-- daniel
MZMcBride liked my suggestion, apparently, and implemented it:
We have an approval voting poll available there for everyone. For each name that you like,
simply sign under that name with an indent (:~~~~).
Hopefully, this will encourage both more responses in a more easily tracked manner and
remind folks that the toolserver wiki is still up and functioning.
In an unrelated note... if folks don't have access to perform SQL queries, should they actually
be in the query-service group? I think the current membership contains folks who can't
actually perform them.
~ Kylu
__________________________________________________________________
Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now at
http://ca.toolbar.yahoo.com.