[Labs-l] Labs restart & mysql connections dropping?

[Nuria Ruiz]

>Which recent restart?  The GHOST restarts in late January, or the labsdb
restarts last week? Something else?
The labs restart last week.


>Is there any pattern? Eg, time of day, specific queries, etc
We do not have enough usage sin labs such we would see daily patterns or
such but several queries were getting errors like  "MySQL server has gone
away." Not all. Please see below.

>What lead you to look at the problem? Is it definite that connections
started dropping at a known point in time?
Users complaining and us looking at logs.


Some queries an errors:

OperationalError: (OperationalError) (2013, 'Lost connection to MySQL
server during query') 'SELECT anon_1.user_id AS anon_1_user_id \nFROM
(SELECT anon_2.user_id AS user_id, anon_2.count AS count \nFROM (SELECT
revision_userindex.rev_user AS user_id, count(*) AS count \nFROM
revision_userindex \nWHERE revision_userindex.rev_timestamp BETWEEN %s AND
%s AND revision_userindex.rev_user IN (SELECT logging.log_user \nFROM
logging \nWHERE logging.log_type = %s AND logging.log_action = %s AND
logging.log_timestamp BETWEEN %s AND %s) GROUP BY
revision_userindex.rev_user UNION ALL SELECT archive_userindex.ar_user AS
user_id, count(*) AS count \nFROM archive_userindex \nWHERE
archive_userindex.ar_timestamp BETWEEN %s AND %s AND
archive_userindex.ar_user IN (SELECT logging.log_user \nFROM logging
\nWHERE logging.log_type = %s AND logging.log_action = %s AND
logging.log_timestamp BETWEEN %s AND %s) GROUP BY
archive_userindex.ar_user) AS anon_2) AS anon_1 \nWHERE anon_1.user_id NOT
IN (SELECT user_groups.ug_user \nFROM user_groups \nWHERE
user_groups.ug_group = %s) GROUP BY anon_1.user_id \nHAVING
SUM(anon_1.count) >= %s' ('20150115000000', '20150214000000', 'newusers',
'create', '20150115000000', '20150214000000', '20150115000000',
'20150214000000', 'newusers', 'create', '20150115000000', '20150214000000',
'bot', 5)



OperationalError: (OperationalError) (2006, 'MySQL server has gone away')
'SELECT user.user_id AS user_user_id, user.user_name AS user_user_name,
user.user_real_name AS user_user_real_name, u
ser.user_password AS user_user_password, user.user_newpassword AS
user_user_newpassword, user.user_newpass_time AS user_user_newpass_time,
user.user_email AS user_user_email, user.user_touched AS user_user_touched,
user.user_token AS user_user_token, user.user_email_authenticated AS
user_user_email_authenticated, user.user_email_token AS
user_user_email_token, user.user_email_tok
en_expires AS user_user_email_token_expires, user.user_registration AS
user_user_registration, user.user_editcount AS user_user_editcount \nFROM
user \nWHERE user.user_name IN (%s, %s)' ('SOME', 'SOME')






On Sun, Feb 15, 2015 at 7:25 AM, Russell Blau <russblau at imapmail.org> wrote:

> Sean Pringle <springle <at> wikimedia.org> writes:
>
> > Which recent restart?  The GHOST restarts in late January, or the labsdb
> restarts last week? Something else?
> >
> >
> > What error codes and messages are you seeing client side?
> >
> > Is there any pattern? Eg, time of day, specific queries, etc
> > What lead you to look at the problem? Is it definite that connections
> started dropping at a known point in time?
> >
> I can give you some answers for a different project. Tool Labs project
> 'dplbot' has a database job that runs twice a day. Starting after 18:00
> UTC on 8 Feb 2015, a particular PHP script that involves a lot of writes
> to a project database (p50380g50692__DPL_p) on enwiki.labsdb started to
> fail consistently with the error message "MySQL server has gone away." The
> script had not changed, and this error had not occurred before this time,
> but it now occurs every time we attempt to run this particular script.
>
> Russ
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20150215/18ca7eac/attachment.html>