[QA] beta cluster is too busy

[James Forrester]

On 22 July 2014 09:24, Antoine Musso <hashar+wmf at free.fr> wrote:

> Hello,
>
> Yesterday a patch landed in MediaWiki core that causes the database
> update script to update a bunch of tables and do a lot of queries on the
> database.
>
> The slave database is thus lagged out by 4+ hours as I write this and
> that causes a bunch of errors when browsing the beta sites.
>
> There is a few more details on:
>
>   populateBacklinkNamespace script causing massive slave lag on beta
>   https://bugzilla.wikimedia.org/show_bug.cgi?id=68349
>
> Bryan Davis pointed me to a quite useful link to check the database lag:
>
>  http://en.wikipedia.beta.wmflabs.org/w/index.php?maxlag=-1
>
>
> Jenkins is still processing the upgrade of simplewiki which is the
> largest wiki on beta cluster.  The job output is:
>
>
> https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/label=deployment-bastion-eqiad,wikidb=simplewiki/2742/console
>
> Whenever that is completed, the database slave will have to catch up and
> we will eventually resume normal operations.
>

​As an update, the database upgrade finally finished about 20 minutes ago,
and slave lag is slowly adjusting back to normal. Unfortunately the lag is
currently nearly four hours long, so this will take a while.

J.
-- 
James D. Forrester
Product Manager, Editing
Wikimedia Foundation, Inc.

jforrester at wikimedia.org | @jdforrester
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140722/7b84ce40/attachment.html>