On Wed, Mar 30, 2016 at 10:24 AM, Nuria Ruiz <nuria@wikimedia.org> wrote:
> Jaime:
>
> Seems that there are still problems.
> Please see: https://phabricator.wikimedia.org/T131236
>
Thanks Nuria - actually I filed that bug on Jaime's request (i.e. he
already knows about it), and the data discrepancies described there
concern other tables and have mostly been observed two weeks ago
already.

> Thanks,
>
> Nuria
>
> On Wed, Mar 30, 2016 at 1:54 AM, Jaime Crespo <jcrespo@wikimedia.org> wrote:
>>
>> You can call it restart, or you can call it (un)controlled crash. The
>> data issues where affecting the whole server, and I had to drop the
>> previously mentioned table (which mysql refused to do while running).
>>
>> Things seem back to normal now, but of course tables will take some
>> time to get refilled, specially Edit_13457736.
>>
>> On Wed, Mar 30, 2016 at 10:27 AM, Tilman Bayer <tbayer@wikimedia.org>
>> wrote:
>> > On Tue, Mar 29, 2016 at 3:44 AM, Jaime Crespo <jcrespo@wikimedia.org>
>> > wrote:
>> >> There seems to be issues with the Edit_13457736 table on the db log,
>> >> on the db1047 host.
>> >>
>> >> db1047 may be known better by some of you as you as
>> >> "s1-analytics-slave" or "analytics-slave". You can use dbstore1002 for
>> >> the time being ("analytics-store" or "sX-analytics-slave", where X is
>> >> 2-7) to read from that table. Other "schemas" are not affected.
>> >>
>> >> I am fixing those right now, but it will take some time, as I may have
>> >> to handle (again) the recently purged rows on that table. Worst case
>> >> scenario, I may have to do a quick reboot of that machine.
>> >>
>> > Did that happen? I just had a query on that server abort with "ERROR
>> > 1053 (08S01) at line 5: Server shutdown in progress". It had been
>> > running since Sunday :(
>> >
>> >> I will update the status soon to communicate next steps.
>> >>
>> >> --
>> >> Jaime Crespo
>> >> <http://wikimedia.org>
>> >>
>> >> _______________________________________________
>> >> Analytics mailing list
>> >> Analytics@lists.wikimedia.org
>> >> https://lists.wikimedia.org/mailman/listinfo/analytics
>> >
>> >
>> >
>> > --
>> > Tilman Bayer
>> > Senior Analyst
>> > Wikimedia Foundation
>> > IRC (Freenode): HaeB
>> >
>> > _______________________________________________
>> > Analytics mailing list
>> > Analytics@lists.wikimedia.org
>> > https://lists.wikimedia.org/mailman/listinfo/analytics
>>
>>
>>
>> --
>> Jaime Crespo
>> <http://wikimedia.org>
>>
>> _______________________________________________
>> Analytics mailing list
>> Analytics@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
>
> _______________________________________________
> Analytics mailing list
> Analytics@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>



--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics