Toolserver-l January 2009

toolserver-l@lists.wikimedia.org

39 participants
22 discussions

New hardware ordered
by Daniel Kinzler 29 Jun '11

29 Jun '11

Hi all I'm happy to let you know that new hardware has been ordered by Wikimedia Deutschland and will arrive probably in about two weeks. We will get two new systems: * A more powerful web server, to replace hemlock: Sun Fire X4150, 2x Quad-Core Xeon, 8GB RAM, 2x73GB SAS HDD. The current web server only has two cores. * Another database server, to be used for S1 (english wikipedia), so S1 and S3 no longer have to share a server: Sun Fire X4250, 2x Quad-Core Xeon, 32GB RAM, 16x146GB SAS RAID. This should improve performance and give us some head space for growth. Once the new servers arrive, S3 will be re-imported too, so we will have live data again. Any ideas for names? To stay with the nightshade theme, how about Jurubeba and Erubia? Or perhaps we go the "witches' weed" way, with Datura and Mandrake? Henbane is taken, i think. Amanita sounds nice, too :) A third server has been ordered, which will also be installed in Amsterdam, but will not be part of the toolserver cluster. It's a storage server (X4540, 24TB RAID) that will keep a live backup of all media files. Cheers, Daniel

11 17

Cron
by Nikola Smolenski 15 Mar '09

15 Mar '09

I have noticed that I have no crontab on login.toolserver.org but that my crontab on toolserver.org continues to work (and affects files that I see on login.toolserver.org). Is there something I should do?

4 5

rebooted nightshade
by Daniel Kinzler 04 Feb '09

04 Feb '09

hi all nightshade ran out of ram. I don't know why, because i couldn't even log in via serial to have a look. so i power-cycled it, should be back up shortly. if you edited your scripts lately, please check if you made a fork bomb. or did something that would consume all the ram. cheers, daniel

13 16

Re: [Toolserver-l] [Wikitech-l] Crawling deWP
by Marco Schuster 28 Jan '09

28 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, Jan 28, 2009 at 12:53 AM, Platonides wrote: > Marco Schuster wrote: >> Hi all, >> >> I want to crawl around 800.000 flagged revisions from the German >> Wikipedia, in order to make a dump containing only flagged revisions. >> For this, I obviously need to spider Wikipedia. >> What are the limits (rate!) here, what UA should I use and what >> caveats do I have to take care of? >> >> Thanks, >> Marco >> >> PS: I already have a revisions list, created with the Toolserver. I >> used the following query: "select fp_stable,fp_page_id from >> flaggedpages where fp_reviewed=1;". Is it correct this one gives me a >> list of all articles with flagged revs, fp_stable being the revid of >> the most current flagged rev for this article? > > Fetch them from the toolserver (there's a tool by duesentrieb for that). > It will catch almost all of them from the toolserver cluster, and make a > request to wikipedia only if needed. I highly doubt this is "legal" use for the toolserver, and I pretty much guess that 800k revisions to fetch would be a huge resource load. Thanks, Marco PS: CC-ing toolserver list. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Use GnuPG with Firefox : http://getfiregpg.org (Version: 0.7.2) iD8DBQFJf6AjW6S2GapJUuQRAvBuAJ46G0qhk+e2axFddbHFMUqzScH4PgCeIMBL L9WWNeZaA/6vHyzSoKrGN54= =p/R+ -----END PGP SIGNATURE-----

3 4

new mail system
by River Tarnell 28 Jan '09

28 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, i've made a couple of changes to how email works on the toolserver, mainly that each account now has an associated email address in LDAP. i haven't added such an address for existing accounts, but if you want to, you can set your email address by running 'setmail'. your account's email address takes precedence over $HOME/.forward for email to <username>@toolserver.org. if you don't want to do this, you can continue to use the old .forward-style mail forwarding. however, if you don't set an email address in LDAP, you won't be able to make use of future services which might use this, such as automatically resetting your SSH key or LDAP password. to emphasise: you do not *need* to do anything because of this change if you don't want to. if you do nothing, your account will continue to work as before. - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJf+DSIXd7fCuc5vIRAjpoAJ4t1yOykKMVvCat/SRrR0aQsEocCQCeKGSF KQ5VC9lAjWs+AsQfk4d//1g= =BFK3 -----END PGP SIGNATURE-----

1 0

ts-admins email address is changing
by River Tarnell 26 Jan '09

26 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, the email address to contact Toolserver administrators has changed. you should now use ts-admins(a)toolserver.org, _not_ ts-admins(a)wikimedia.org. the latter address will continue to work for now, but might stop working in the future. - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJfSx+IXd7fCuc5vIRAkE2AJ0WWxYy7OliRvDHSl3jJvvO/fQbUACguZ+m fjvY/qB3VEv7Mj6fq6iX5bQ= =Y4XC -----END PGP SIGNATURE-----

1 0

spam filtering for @toolserver.org addresses
by River Tarnell 26 Jan '09

26 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, mail to @toolserver.org addresses (and mail directly to any machine, e.g. @nightshade.toolserver.org) is now being filtered for spam. the filtering is configured fairly laxly; it will only reject mail with a SpamAssassin score over 120, which will almost certainly not match legitimate mail. mail with a score over 50 will be greylisted. other mail will have several headers added (which begin with X-TS-Spam-), so you can do your own filtering. we will monitor the effectiveness of the filtering over the next few days and might adjust these parameters as needed. - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJfPvkIXd7fCuc5vIRAlr6AJ0ehjzk1S3a4maEgukBk0VD/+5jLQCgjihp d0FbdBCZh/RoYf9hK8fgSDI= =dUzL -----END PGP SIGNATURE-----

1 0

new US toolserver
by River Tarnell 23 Jan '09

23 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, we are currently racking amaranth.toolserver.org, which will be the first US toolserver, hosted in Tampa. this system is a Sun X4150 with 32GB RAM and an external disk array with 12 146GB 15'000 rpm disks. initially, this will be used for a US mirror of cache.stable.toolserver.org, which will improve things using the cache (currently only WikiMiniAtlas) for US users. we might also we it to host the various webapps (like JIRA), taking the load off the existing machines in Amsterdam. if anyone has any other suggestions as to what this machine could be used for, let us know... - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJeNHsIXd7fCuc5vIRAt4tAJ4vtgwHJ91R3KiTXGxo2CPMItWVVwCcC7xW K6+556Xgs9Le6Nyghczdni4= =1X90 -----END PGP SIGNATURE-----

3 3

MediaWiki developer meet-up in Berlin, April 3-5
by Daniel Kinzler 19 Jan '09

19 Jan '09

Hello All I'm happy to announce the MediaWiki Developer Meet-Up will happen April 3.-5. in Berlin, at the c-base. The event is for everyone who works on MediaWiki, writes extensions, builds bots, writes scripts for the toolserver, or is otherwise interested in the technical aspects of Wikimedia. We are happy that we can now have the meet-up after our plans for 25C3 and FOSDEM failed. If you want to come to the Developer Meetup, please sign up at <http://www.mediawiki.org/wiki/Project:Developer_meet-up_2009>. The event will take place in parallel to the Wikimedia Foundation's board meeting and chapter meeting, so there will be a lot of Wikimedians in Berlin at the time. We plan to have a party to bring everyone together and give an opportunity for developers, board members and chapter people to mingle. The meet-up will be a loose BacCamp-like event so topics and schedule are largely up to you. The goal is to get to know new aspects of MediaWiki and Wikimedia and to develop ideas on how we can make things even better. And of course to have a lot of fun with wiki hackers from around the world! -- daniel

1 0

Server naming poll
by Lilewyn 18 Jan '09

18 Jan '09

MZMcBride liked my suggestion, apparently, and implemented it: We have an approval voting poll available there for everyone. For each name that you like, simply sign under that name with an indent (:~~~~). Hopefully, this will encourage both more responses in a more easily tracked manner and remind folks that the toolserver wiki is still up and functioning. In an unrelated note... if folks don't have access to perform SQL queries, should they actually be in the query-service group? I think the current membership contains folks who can't actually perform them. ~ Kylu __________________________________________________________________ Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now at http://ca.toolbar.yahoo.com.

4 3

New hardware ordered
by Purodha 18 Jan '09

18 Jan '09

Title: Poisonous plant names. Personal Authors: Wagstaff, D. J., Wiersema, J. H. Author Affiliation: FDA, Center for Food Safety and Applied Nutrition, Washington, DC 20204, USA. Editors: Garland, T., Barr, A. C. Document Title: Toxic plants and other natural toxicants Abstract: This chapter describes a checklist of poisonous plant names, which currently includes 4672 species in 1646 genera and 255 families. There are 11 614 common names. This list of plants reported to be toxic represents approximately 1% of the known vascular plants of the world. The checklist is organized in a DataPerfect database. Publisher: CAB International _________________________________________________________________ Greetings, Purodha

1 0

Re: [Toolserver-l] New hardware ordered
by Lilewyn 18 Jan '09

18 Jan '09

We've got this nice clean (relatively unused) wiki sitting around. Why not put suggestions on there and use some approval voting to see which names have the most support? ~ Kylu __________________________________________________________________ Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now at http://ca.toolbar.yahoo.com.

2 1

Re: [Toolserver-l] New hardware ordered
by Larry Pieniazek 18 Jan '09

18 Jan '09

Xanthium works for me too... should really confuse people nicely. (I'd say it tsanthium but I could be wrong...) Larry Pieniazek Hobby mail: Lar at Miltontrainworks dot com

2 1

New hardware ordered
by Lilewyn 18 Jan '09

18 Jan '09

When we named nightshade, the theme was "poisonous plants". Current servers (from https://wiki.toolserver.org/view/Servers ) * Hemlock * Nightshade * Willow * Zedler (Not a poisonous plant, but names after Johann Heinrich Zedler, who was an 18th century German encyclopedist. This is in keeping with the WMF naming scheme.) * Yarrow * Vandale (As above, but Van Dale was Dutch.) * Clematis (Poisonous plant, not an encyclopedist) Short, usable, and thematic (poisonous plant) names might be: * Azalea * Daphne * Digitalis * Foxglove * Hellebore * Holly * Lilly * Oleander * Privet * Wolfsbane * Yew "Daphne", "Holly" and "Lilly" have the distinction of also being female names, which would give us another opportunity to confuse those trying to understand our naming schema. I'm rather fond of Azalea personally, though I think someone reminded me recently that "nightshade" was my suggestion, so I'll sit this one out. ~ Kylu __________________________________________________________________ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/

13 16

Re: [Toolserver-l] Toolserver-l Digest, Vol 39, Issue 13
by Betacommand 17 Jan '09

17 Jan '09

My suggestion for the new server name is wolfsbane also Betacommand

1 0

commons-s1 replication lag
by Eusebius 15 Jan '09

15 Jan '09

Hi there, When looking at the replication lag, I see that s3 is not replicating and I understand the impact of it on the relationship between the tools and the projects hosted on s3. But I also see that commons-s1 is not replicating, and I don't understand anymore: if it is a replica of the Commons DB, tools working on Commons should be affected by it, right? But it seems not to be the case. What didn't I understand? Regards, Eusebius

2 1

potential abuse by 84.114.164.84
by Daniel Schwen 12 Jan '09

12 Jan '09

While looking at the access statistics of the Query-to-map tool I discovered that a large portion of the hits are generated by one single austrian DSL IP (90000 hits yesterday (9% of all hits that day), 55000 hits so far today, ). Seemingly random queries to lots of tools. Example: 84.114.164.84 - - [11/Jan/2009:14:03:43 +0000] "HEAD /%7Ekolossos/wp-world/umkreis.php?la=pt&lon=16.285&lat=48.134444&rang=50&map=1 HTTP/1.1" 200 0 "-" "Jakarta Commons-HttpClient/3.1" 84.114.164.84 - - [11/Jan/2009:14:03:49 +0000] "HEAD /~para/earth.php?latdegdec=48.134444&londegdec=16.285&scale=300000 HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1" 84.114.164.84 - - [11/Jan/2009:14:03:49 +0000] "HEAD /~kolossos/wp-world/umkreis.php?la=nl&lon=16.285&lat=48.134444&rang=50&map=1 HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1" 84.114.164.84 - - [11/Jan/2009:14:03:50 +0000] "HEAD /~kolossos/wp-world/umkreis.php?la=pt&lon=16.285&lat=48.134444&rang=50&map=1 HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1" What is going on here? Proxy? Daniel P.S.: there is a webserver on that IP. Looks like a broken Mediawiki installation, a broken forum installation. No further info though.

6 5

Toolserver status for Signpost
by Aude 10 Jan '09

10 Jan '09

I am preparing the Technology Report for this week's edition of the Signpost on enwiki. (see http://en.wikipedia.org/wiki/User:Aude/Technology_report) I have some idea about how toolserver works, aware that there are issues with replication and that Yarrow (one of the servers) was down. What is the status now? It sounds like things are not 100% back to normal. How long (estimated) will it take to be back to normal? There are a number of tools linked from the featured article candidates pages. Are these all working 100%? What other tools are affected? http://en.wikipedia.org/wiki/Wikipedia:Featured_article_candidates/Typhoon_… Any information for the Signpost would be most appreciated. Cheers, -Aude

3 7

s3 replication is halted
by River Tarnell 06 Jan '09

06 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, as Wikimedia deleted the MySQL binlogs required for replicating s3 on the Toolserver, replication is now halted. a full dump/import is required to fix replication. because disk space on yarrow is limited at the moment, and we are about to receive new servers, i might wait until this before reimporting. (however, if that's likely to take a long time, i might do it sooner.) - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJYWeSIXd7fCuc5vIRAo03AJ9ouEHS98oTwKPugzcUsc8Lp+LmSgCffFmd gK3xul+k9UinKPPHH5Q3PWg= =pnlB -----END PGP SIGNATURE-----

9 22

1200 MB every night
by emijrp 04 Jan '09

04 Jan '09

Hi; I run a script that downloads 1200 MB every night (from http://dammit.lt/wikistats/) to generate some stats (http://es.wikipedia.org/wiki/Wikipedia:Ranking_de_visitas). Any problem with that bandwidth usage? 30GB by month. Thanks, regards

5 5

maintenance 28th to 31st Dec: all servers
by River Tarnell 04 Jan '09

04 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 at some point between these two dates (exact date isn't yet known), Wikimedia will performance maintenance at the knams hosting facility, where the Toolserver is hosted. this will involve downtime for all servers as we will be moving to a new rack. i will combine this maintenance with the next Toolserver quarterly maintenance, which was previously scheduled for January 5th. the total outage period will be several hours. - river. -----BEGIN PGP SIGNATURE----- iD8DBQFJS08dIXd7fCuc5vIRAiVUAKCEKhl4a/K1J0gyoRHoZbocgtfGqgCeP4L5 P5rCTBWgi6D0nU4m7aWglE8= =dsYJ -----END PGP SIGNATURE-----

4 12

IPv6 on stable - thank you
by Marcin Cieslak 01 Jan '09

01 Jan '09

Hi, Big thank you for providing IPv6 for the stable toolserver. I think I was using it unknowingly since around Dec 14th, excellent idea - so thanks again. --Marcin

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l January 2009