I have recently started with an advanced MySQL book, and though I'm far
from through with it (still missing replication, among others), it got
me thinking on how to improve our DB server setup. I have some ideas
that I would like to share with you. Note that I didn't run any tests to
prove they're wise ;-)
It might be of use to add a single harddrive to each DB slaves, and use
it exclusively for temporary tables. That way, I/O won't have to wait
for other things, and many queries that don't use temporary tables will
run faster. Some (used) IDE drive might do.
From what I can see on meta, at least bacon has a RAID that can be
fitted with a battery. This will not only improve data safety, it will
also speed up writes, as a disk write can already be handeled as written
even if it only in the RAID write cache.
After a quick look at the indices (indexes?) on the MySQL tables, these
seem fine, although there are quite a lot of them. Once the new database
schema is in place, we might want to look into index usage again. If we
could rewrite some queries, we might be able to get rid of one or the
other index, which will not only save disk space, but RAM as well,
reducing the chances of Linux swapping indexes to the disk.
Also, we currently leave the choice which index to use entirely up to
MySQL, as far as I can tell. Enforcing the use of one index, depending
on the query of course, might prevent strange side effects if MySQL
choses to use the wrong index which would result in a slow query.
I take it that we have a slow query log. Do we also have a "slow page"
log from within MediaWiki? That might give us more insight into what
pages really s**k performance-wise.
There's more, but I'll stop for now. If I'm talking BS here, just tell
me ;-)
Magnus
this is a one-off observation, so it's too little to file a bug report
over, but maybe someone can corroborate this:
If I overwrite an existing image with a new version and a thumbnailed
version of that image is part of some article, then the thumbnail will
not change to the new version until and unless I change the thumbnail
size. Also, I found I had to upload the new version of the image twice
for the actual main image to change.
-- ropers [[en:User:Ropers]]
www.ropersonline.com
Hi,
Someone on IRC, I forget who, suggested that we should end this ability
to upload new revisions of images. I agree.
It's something vandals can use to alter articles without affecting the
watchlist; it also means that when looking at an old version of a page,
you don't necessarily see what content was there at the time.
In addition, the image usage (with reversions, etc) is quite tricky, and
there seem to be some caching problems that can lead to people seeing an
old version.
I suggest that we simply change the system so that, if you want to
upload an image, you must give it a unique filename. You can then alter
articles as necessary to link to the new version.
--
Allan Crossman - a.crossman(a)blueyonder.co.uk
The Moon is Waxing Crescent (41% of Full)
I've just heard that ariel, our master DB server, recently ran out of
disk space. Writes to the binlogs stopped but it kept committing to its
local disk, thereby putting all the slaves irreversibly out of sync.
This is a nasty failure mode that we previously weren't aware was possible.
To resync the slaves, we will have to put the site into read-only mode
for maybe 6-12 hours. I'm hoping this will be done as soon as possible,
because in the meantime, we will have poor performance. The slaves used
to share the read load. Also, if ariel dies, we'll be down to periodic
backups, meaning data loss and days of downtime.
Search will be disabled until the slaves are resynced, and other
emergency optimisation measures such as watchlist caching might also
need to be used.
-- Tim Starling
Dear developers,
I do not know what policy is used for those who are allowed to actually
commit changes to the CVS, but the current CVS is not working -
neither a fresh install (empty database) nor an php
/maintenance/update.php helps -- A DATABASE ERROR OCCURED create table
user_rights .....
Please allow me to propose, that developers who change the database
structure always submit also a correct module for
/maintenance/updaters.inc and /maintenance/tables.sql, so that other
developers can resume their work on their projects without interruption.
Thank you.
Somebody has left the CVS 1.4 in an unstable form, installing from CVS
1.4 currently fails with the following error message during install.php:
Creating tables...A database error has occurred Query: CREATE TABLE
user_rights ( ur_user int(5) unsigned NOT NULL, ur_rights tinyblob NOT
NULL default '', UNIQUE KEY user_id (user_id) ) Function: Error: 1072
Key column 'user_id' doesn't exist in table Backtrace: Database.php line
342 calls wfdebugdiebacktrace() Database.php line 294 calls
database::reportqueryerror() install-utils.inc line 116 calls
database::query() index.php line 493 calls dbsource()
Please can someone repair this in the CVS 1.4, thanks.
I am working on an update of really outdated LanguageCs.php (using the
up-to-date messages in cs:MediaWiki namespace) and I have found a
problem I do not understand:
In SpecialBlockip.php, there are commands like
$wgOut->addWikiText( htmlspecialchars( wfMsg( "blockiptext" ) ) );
AFAICS this means that no HTML entities may occur in the blockiptext
message (as every & gets converted to &). Is there any reason for
that? Why not drop the htmlspecialchars() out? Is there any generic
rule when it is used and when not? (I mean, some other special pages
seem to use addHTML( wfMsg( ... ) ) etc.)
Thanks,
[[cs:User:Mormegil|Mormegil]]
Hello,
The new user management things is commited in cvs HEAD.
Anyone actually using HEAD for any kind of public / production site
should STOP updating from HEAD unless it is for development purposes. My
patch is full of bugs, security holes and will certainly crash the
database :o)
For developpers:
Groups are managed through the group class (group.php), it's almost a
copy and paste of the user class so you should not be too lost.
The SpecialUserlevels.php uses HTMLForm class to generate the select /
fieldset ... I am willing to rewrite part of it so HTMLForm provide us a
good HTML abstraction layer.
I think SQL patches are up to date, hopefully modifications made this
morning by JeLuF will still be there.
I provided a dump of my 'groups' table, it's available at:
./maintenance/archives/patch-userlevels-defaultgroups.sql
The whitelist thing is partially re implemented with two new globals in
default settings:
$wgAnonGroupId = 1;
$wgLoggedInGroupId = 2;
They define the groupId to be used for user rights. It's not complete.
Some logs are generated through the LogPage object and available at
Special:Log :o)
Bug tracker:
http://bugzilla.wikipedia.org/show_bug.cgi?id=767
Enjoy.
--
Ashar Voultoiz - WP++++
http://en.wikipedia.org/wiki/User:Hashar
Hello,
I have been reading the "recent changes" and "new
articles" RSS-feeds through My Yahoo. Now I've written
a small Python program to fetch and parse these feeds,
but it fails most of the time with a message about
"intermittent server problem" and warning that my
user-agent may be blocked. Since the feeds are
available through my browser, I conclude that I have
been blocked. (My program sends the user-agent,
'WikiWalker' and provides my email address).
I have also published RSS feeds, and I know that RSS
readers and aggregation sites are among the most
aggressive 'attackers' - frequently ignoring
robots.txt, cache-busting, and requesting pages at
outrageous speeds. But my WikiWalker is really benign,
so how can I get un-blocked?
TIA,
Ken