-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Due to the current high replag on thyme, I've switched sql-s1 to
rosemary. If your tools require a user database and you haven't updated
them to use sql-s1-user instead, you will find your databases are
missing. Once the replag is down to a reasonable level I'll switch it
back.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk13HAsACgkQIXd7fCuc5vIvdQCffgfjFv9OKmvpoWNY/0+LiTqy
IeUAniODzwDbdQG7jcf3Jp/3awc40gLR
=MBKi
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
We're currently applying schema changes from WMF for a MediaWiki
upgrade. To reduce replication lag, we will first apply them to an
inactive server for each cluster, then switch users to that server and
apply them on the other one. The first part has started now, and we
will be switching in an hour or two.
During the second part, user databases will be unavailable until we
finish the changes and switch back. This may take several hours to
finish.
The exception is s2/s5, which only has one server; these clusters will
be lagged during the change.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk11bNYACgkQIXd7fCuc5vIIowCgqKiJvPxtr+B19TRDrbJTy/70
fHQAn3CW69mS7tQPWrRxdMTKYnuFM5St
=DpI/
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I've added yarrow as a login server and SGE execution host for all.q and
longrun. All SGE jobs are now eligible to run there; no action is
required from users.
This server is for SGE only, tools must not be started there manually.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk11fv4ACgkQIXd7fCuc5vK24wCgjGrOQSwxLdndb3FeVbN/Dr4l
FbQAn1GMC00MN8vDklXQqFBgDVv1mS7J
=yQue
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I'm currently copying user-store to a new disk array; this should be
finished in a couple of days. I hope the new array will be faster, as
well as much larger. There will be a separate maintenance notice for
the actual switch later.
Once the move is done, in additional to having more storage now (about
14TB), it will be much easier to add additional storage in the future.
However, I don't think having a single 14+ TB filesystem is a good idea
(the initial volume will be 5TB), so I'm considering new ways to split
up the storage.
The most obvious is to have one volume for miscellaneous files, like the
current user-store, and then one volume per large project: stats, dumps,
etc. People (or MMTs) who wanted additional storage for their project
could request a new volume of a particular size. As well as making it
easier for us to account for space, this would mean you were guaranteed
to get the amount of storage you asked for, unlike the current situation
where the filesystem can (and does) fill up.
Does this seem like something that would be useful to users? Otherwise,
does anyone have another suggestions for how to allocate the space?
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk1wAlwACgkQIXd7fCuc5vKrKACfXttPL1HS1EzLxykm0t5/dggn
BzkAnj3KiTWGs8QR55sb0/Jv5XQY0uie
=nN4u
-----END PGP SIGNATURE-----
> River Tarnell:
> Well, no one commented on this, so I propose the following:
>
> * Dumps will go in /store/dumps/<wiki>/
> * Page view stats will go in /store/stats/
> * Everything else will go in /store/misc/
Thanks a lot river. Yes, I think dividing in such a way is wise.
Also, the current user-store is kinda messy, with lots of dumps here
and there, and some files we don't really know what they are for.
Maybe it is the right time to clean up a bit.
If I may, I'd strongly urge anyone that have files there to create
explicit directories for their project and indicate what these files
are for in the INDEX (as indicated in README).
I would also like all the dump files to be properly, hierarchically
stored like /store/dumps/<wiki>/dump_files.
That would make things easier (and cleaner).
Thanks again,
Darkdadaah
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Tonight I will implement the mail forwarding changes described here:
<http://lists.wikimedia.org/pipermail/toolserver-announce/2011-February/0004…>.
There should be no interruption in service, except that mail to
user(a)toolserver.org might be delayed during the maintenance.
Start time: Monday, 7th March 0000h UTC
http://time.tcx.org.uk/utc/2011-03-07/00:00
End time: Monday, 7th Match 0100h UTC
http://time.tcx.org.uk/utc/2011-03-07/01:00
Sorry for the short notice.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk1z3dgACgkQIXd7fCuc5vJtSwCgiZO5Mu3pMIOJrhcuYUekOHvw
awQAni555uAR0bEORIUV869VqSlS1hT/
=kRiq
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
During the maintenance on December 6th, 2010 I switched the Toolserver
SSH server from Sun SSH to OpenSSH. A difference in how OpenSSH uses
PAM to authenticate users meant that after the change, users were able
to log via SSH using their LDAP password, without using an SSH key.
This error has now been fixed.
If you have no LDAP password set, or if you have a strong password[0],
then this should not have affected you. However, if you had a weak or
easily guessable password set, or if your LDAP password could have been
compromised (e.g. if you wrote it down in plain text somewhere) then
it's possible someone could have used it to gain access to your account.
In that case, I suggest you immediately change your password (via
'passwd'), then review your home directory to ensure no unauthorised
changes have been made (e.g. new SSH keys added, or shell rc files
changed). If you have sensitive data such as SSH or PGP keys on the
Toolserver, you may wish to revoke them and issue new ones. (However,
storing that kind of data on the Toolserver is probably a bad idea in
any case.)
I'm very sorry for the inconvenience this issue might cause to users,
and I will be reviewing our authentication configuration to reduce the
chance of something like this happening in the future.
- river.
[0] Which is somewhat enforced by the LDAP password policy, but it's
still possible to set a weak password if you try hard enough.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk1zvOgACgkQIXd7fCuc5vJpowCeMoLig31BAHnStWakKgeU/ZOr
pCYAoKMEF/6+yzzKGQNVYxXqJuhM2f63
=ykB1
-----END PGP SIGNATURE-----
Hello,
I'm from the French chapter and we need sometimes a lot of CPU power
and/or a lot of memory for some projects. For now it happened two times:
* the partnership with the BnF where we need CPU and disk space (for image
and DjVu processing, we rent a server dedicated to that);
* the treatment of the videos of our GLAM meeting (split the videos and
OGV conversion, done on a personal computer with a lot of time).
So is it possible to use the toolserver in these cases? And/or ask for a
particular use when important ressources are required? I didn't seen
anything related to the ressources in the rules.
Thanks,
~ Seb35 [^_^]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Later tonight (around 12AM UTC[0]) I will restart the NFS server on
hemlock, which serves user-store. This will cause an interruption to
service that should last for under 1 minute.
- river.
[0] http://time.tcx.org.uk/utc/2011-03-05/00:00
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk1xNqIACgkQIXd7fCuc5vL22QCeM2IXEqx0x3yRrcbXIbjVbCsW
GR0An1hYPDoUq58rs1G9fSxgTrT4P1h1
=4BKf
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
On the morning (UTC) of March 1 we will install some new hardware at the
Toolserver. Services will be affected as follows:
Service | Expected impact
----------------------------+--------------------------------
Databases: s3, s4, s6, s7 | User databases unavailable for
| < 10 minutes.
user-store filesystem | Unavailable for < 10 minutes.
Start time: Tuesday, 1st March, 1PM UTC (estimated)
http://time.tcx.org.uk/utc/2011/3/1/13/0
End time: Tuesday, 1st March, 4PM UTC (estimated)
http://time.tcx.org.uk/utc/2011/3/1/16/0
Details:
We will install new storage for databases and the user-store filesystem.
This requires installing HBAs in three servers: hemlock, cassia, and
hyacinth, which requires downtime. We will try to stagger the
installation of the two database servers so that those clusters
remaining accessible during the maintenance, but user databases will be
unavailable while hyacinth is down.
Before the maintenance on hemlock, we will forcibly unmount the
user-store filesystem. Any program accessing it at the time will
receive an I/O error.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (NetBSD)
iEYEARECAAYFAk1ro3AACgkQIXd7fCuc5vLhsQCfbKKphnxKxXCM3VNCJC+YOsz+
E5sAn3EBKJjF1PeU4WZCT5W9KFNuqGqM
=TGse
-----END PGP SIGNATURE-----