Hi all
I'm happy to let you know that new hardware has been ordered by Wikimedia
Deutschland and will arrive probably in about two weeks. We will get two new
systems:
* A more powerful web server, to replace hemlock: Sun Fire X4150, 2x Quad-Core
Xeon, 8GB RAM, 2x73GB SAS HDD. The current web server only has two cores.
* Another database server, to be used for S1 (english wikipedia), so S1 and S3
no longer have to share a server: Sun Fire X4250, 2x Quad-Core Xeon, 32GB RAM,
16x146GB SAS RAID.
This should improve performance and give us some head space for growth. Once the
new servers arrive, S3 will be re-imported too, so we will have live data again.
Any ideas for names? To stay with the nightshade theme, how about Jurubeba and
Erubia? Or perhaps we go the "witches' weed" way, with Datura and Mandrake?
Henbane is taken, i think. Amanita sounds nice, too :)
A third server has been ordered, which will also be installed in Amsterdam, but
will not be part of the toolserver cluster. It's a storage server (X4540, 24TB
RAID) that will keep a live backup of all media files.
Cheers,
Daniel
Hello all,
I was noticed today by a steward that one tool (I will message the author
separately) does not respect the rev_deleted-field and displays revisions,
which are already deleted, to the world.
Our rules forbids displaying of data which is not visible on the Wikimedia-
wikis any more. I know that the usage of the rev_delete-field by mediaiwiki is
quite new (the mediawiki-devs didn't update their documentation until now
AFAIS), so please look at your tools. If you use the revision-table and do
showing single edits, then add a "AND rev_delete=0" to the where-clause where
needed (it's of corse ok to ignore the field if you just display abstract data
like "How many edits have a wiki").
Please always remember: Just because you can see the data in the database,
that is no allowance to show it to the world (if in doubt: ask a root).
Sincerly,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hi all,
just a quick status update: The dump is currently running at 2req/s
and ignores all pages which have is_redirect set; also I changed the
storage method: the new files are appended to
/mnt/user-store/dewiki_static/articles.tar, as I noticed I was filling
up the inodes of the file system; doing the storage inside a tarball
will prevent this and I don't have to waste time downloading the tons
of files to my PC, only one huge tarball when its done.
I also managed to get a totally stripped down version of the Vector
skin file loading an article via JSON (I won't release it now though,
it's a damn hack - nothing except loading works, as I have removed
every JS file... should be pretty till Sunday).
Current dump position is at 92927, stripping out the redirects 53171
articles have really been downloaded, resulting in 770MB of
uncompressed tar (I expect gzip or bz2 compression to save lots of
space though).
For the redirects: how do I get the redirect target page (maybe even
the #section)?
Marco
PS: Are there any *fundamental* differences between the Vector skin
files of different languages except the localisation? Could this maybe
be converted to Javascript, maybe $("#footer-info-lastmod").html("page
was last changed at foobar")?
--
VMSoft GbR
Nabburger Str. 15
81737 München
Geschäftsführer: Marco Schuster, Volker Hemmert
http://vmsoft-gbr.de
Hi all,
I have made a list of all the 1.9M articles in NS0 (including
redirects / short pages) using the Toolserver; now I have the list I'm
going to download every single of 'em (after the trial period tonight,
I want to see how this works out. I'd like to begin with downloading
the whole thing in 3 or 4 days, if noone objects) and then publish a
static dump of it. Data collection will be on the Toolserver
(/mnt/user-store/dewiki-static/articles/); the request rate will be 1
article per second and I'll download the new files once or twice a day
to my home PC, so there should be no problem with the TS or Wikimedia
server load.
When this is finished in ~ 21-22 days, I'm going to compress them and
upload them to my private server (well, if Wikimedia has an archive
server, that 'd be better) as a tgz file so others can play with it.
Furthermore, though I have no idea if I'll succeed, I plan on hacking
a static Vector skin file which will load the articles using jQuery's
excellent .load() feature, so that everyone with JS can enjoy a truly
offline Wikipedia.
Marco
PS: When trying to invoke /w/index.php?action=render with an invalid
oldid, the server returns HTTP/1.1 200 OK and an error message, but
shouldn't this be a 404 or 500?
--
VMSoft GbR
Nabburger Str. 15
81737 München
Geschäftsführer: Marco Schuster, Volker Hemmert
http://vmsoft-gbr.de
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Perl 5.10 has now been removed from the Solaris login servers.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)
iEYEARECAAYFAkycQ3YACgkQIXd7fCuc5vIoxQCgo5+uWmAWXMZEKNsKxOSxNl6Q
wNsAn1ugrNCzBtmBhckvfjTpFLExnu9Q
=9o8F
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Python 3.1 is now available on the Solaris login servers as /usr/bin/python3
(or /usr/bin/python3.1 to require a specific version). The only module
installed so far is "oursql", to provide MySQL connectivity. (MySQLdb is not
installed, since it does not work with Python 3.)
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)
iEYEARECAAYFAkycQpYACgkQIXd7fCuc5vIRrQCfQIuLAfVLAcPDlMDHGW7lZTNm
mWYAoLr89cgB3VMW1X3zAEtTgTGrYsUA
=zjqB
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
This is a maintenance notice for today, Tuesday the 21st.
Service | Expected impact
- ----------------------------------+-------------------------------------
ptolemy: OpenStreetMap | Complete outage for under 10 minutes
PostgreSQL (sql-mapnik), |
mod_tile, Tirex, |
http://toolserver.org/tiles/ |
All other services | No impact
Start time: Wednesday, 21st September, 12AM (0000h) UTC
End time: Wednesday, 21st September, 1AM (0100h) UTC
Details:
We will upgrade the OS on ptolemy, the OSM database and tile server, to Solaris
10 Update 9. This will require downtime for a reboot after the upgrade is
completed.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)
iEYEARECAAYFAkyYrFYACgkQIXd7fCuc5vLObwCcClnsS0iHGXc5owmXlHNNJuBS
xdAAn2sd2HRT6OnRqIzKHh7s/EQ2GCIG
=J9wf
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
nightshade was rebooted at 5:05 AM on Tuesday (UTC) for a kernel update.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)
iEYEARECAAYFAkyYPR4ACgkQIXd7fCuc5vLpAQCeNEff17OzRYlfvIXhunm+3+xU
QnAAnjByPEDEijD1Pm7waivrNUdhX5QI
=N3th
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
Mono on the Solaris login servers has been upgraded from 2.4.2.2 to 2.6.7.
- river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (FreeBSD)
iEYEARECAAYFAkyX/5MACgkQIXd7fCuc5vJ7fACfVJDz3q/ZhLtHmfbaqmDjbXMZ
kGAAnjqs6m699q31R0LmDTvQcgpFxLwc
=GOnY
-----END PGP SIGNATURE-----