[Labs-l] wiki replication stopped since September

[Gerard Meijssen]

Hoi,
Well actually, if the product is causing production problems and, it is.
Something needs to be done. The question is not if the people are amiable
(they are) or if they are well intentioned (they are) but if the service
that is provided is good enough (it is not).

The question is therefore why does it take so long of suffering for so many
people.  Why is this not noticed and why is this not escalated to the
people ultimately responsible. It things are of a professional quality,
escalation happened. I am therefore really interested in the plans that are
in the process of implementation that will remedy this situation. If there
are no such plans, it is relevant for stakeholders to address the existing
situation. Maybe they should consider this anyway.,
Thanks,
      GerardM

On 26 November 2014 at 17:14, Dan Andreescu <dandreescu at wikimedia.org>
wrote:

> We (analytics team) run wikimetrics on top of the labs data.  For the last
> couple of months, we've seen missing data (data in prod that's not in labs)
> and a big slowdown on some queries that used to be fast.  I was told about
> the cold indices theory but if it takes this long to warm them up maybe we
> should turn up the thermostat.  Or... however that works in db world :)
>
> Note: we absolutely adore our DBA and labs support folks, we think there
> may just be too much work that needs to get done and these problems are
> definitely affecting our products.  Right now, we have to tell people to
> basically ignore the data for the last couple of months until this get
> sorted out.
>
> On Wed, Nov 26, 2014 at 9:57 AM, Maximilian Doerr <
> maximilian.doerr at gmail.com> wrote:
>
>> Could this in anyway, no matter how remote, be a link to the slow query
>> times enwiki has been spitting out?  Anomie suggests they are cold files in
>> the database, but I don’t understand how that should be happening with
>> indexes in that area.
>>
>> Cyberpower678
>> English Wikipedia Account Creation Team
>> Mailing List Moderator
>>
>>
>>
>> On Nov 26, 2014, at 09:54, Magnus Manske <magnusmanske at googlemail.com>
>> wrote:
>>
>> I already submitted
>>
>> https://phabricator.wikimedia.org/T75980
>>
>> and
>>
>> https://phabricator.wikimedia.org/T75511
>>
>> On Wed Nov 26 2014 at 13:44:39 John <phoenixoverride at gmail.com> wrote:
>>
>>> Actually it looks like things in the databases are completely screwy.
>>> idwiki.labsdb shows zero tables, but enwiki.labsdb "use idwiki_p" shows
>>> what it should. Looks like there may be a bigger issue
>>>
>>> On Wed, Nov 26, 2014 at 7:58 AM, Krinkle <krinklemail at gmail.com> wrote:
>>>
>>>> Seems to be at least 62 days behind on commonswiki
>>>>
>>>> tin.eqiad.wmnet$ sql commonswiki
>>>> rc_id: 142360816
>>>> rc_timestamp: 2014-11-26 12:55:31
>>>>
>>>> tools-login$ sql commonswiki
>>>> rc_id: 136903866
>>>> rc_timestamp: 2014-09-24 12:51:42
>>>>
>>>> See also https://tools.wmflabs.org/unpatrollededitstats/commonswiki/
>>>>
>>>> — Krinkle
>>>>
>>>> On 23 Nov 2014, at 12:56, John <phoenixoverride at gmail.com> wrote:
>>>>
>>>> enwiki's replication lag is at 114256 and growing.
>>>> _______________________________________________
>>>> Labs-l mailing list
>>>> Labs-l at lists.wikimedia.org
>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Labs-l mailing list
>>>> Labs-l at lists.wikimedia.org
>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>>>
>>>>
>>> _______________________________________________
>>> Labs-l mailing list
>>> Labs-l at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>>
>> _______________________________________________
>> Labs-l mailing list
>> Labs-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>>
>>
>> _______________________________________________
>> Labs-l mailing list
>> Labs-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>>
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20141126/3f311f09/attachment-0001.html>