[Engineering] [Analytics] Eventlogging Mysql consumers downtime

[Amir Aharoni]

Is the backfilling done? I see some data, but I can't be sure whether it's
complete.

2016-01-25 20:34 GMT+02:00 Nuria Ruiz <nuria at wikimedia.org>:

> We are working at this time on backfilling data for the scheduled
> maintenance, we shall report when we think backlogging has finished.
>
> Thanks,
>
> Nuria
>
> On Mon, Jan 25, 2016 at 5:34 AM, Oliver Keyes <okeyes at wikimedia.org>
> wrote:
>
>> Awesome; thanks :). Anything on the switchon time? (I appreciate it's
>> only barely Monday ;p)
>>
>> On 21 January 2016 at 11:36, Andrew Otto <otto at wikimedia.org> wrote:
>> > FYI, This has started.  I have turned off the MySQL processes.  No new
>> data
>> > will be inserted into MySQL until we turn them back on.
>> >
>> > On Wed, Jan 20, 2016 at 4:43 PM, Andrew Otto <otto at wikimedia.org>
>> wrote:
>> >>
>> >> (+Jaime)
>> >>
>> >> It’s hard to tell how long it will take, but I imagine for quite a
>> while.
>> >> I guess a day max, but probably much less.  If Jaime let’s me know
>> that he’s
>> >> finished early, and I am close enough to a computer, I’ll turn the
>> >> EventLogging MySQL processes back on asap.
>> >>
>> >>
>> >>
>> >> On Wed, Jan 20, 2016 at 4:12 PM, Oliver Keyes <okeyes at wikimedia.org>
>> >> wrote:
>> >>>
>> >>> Thanks Madhu! Awesome to see the transparency and announces around
>> this
>> >>> :D.
>> >>>
>> >>> Do you know how long the reimport should take on Monday? I want to
>> >>> make sure I know when to expect we can safely launch all our scripts.
>> >>>
>> >>> On 20 January 2016 at 13:08, Madhumitha Viswanathan
>> >>> <mviswanathan at wikimedia.org> wrote:
>> >>> > Hi all,
>> >>> >
>> >>> > In order to convert tables on db1046 to the TokuDB engine - we have
>> to
>> >>> > schedule some downtime on the Eventlogging databases between
>> tomorrow,
>> >>> > Thursday Jan 21, 2016 at 16:00 UTC to Monday Jan 25, 2016 16:00 UTC.
>> >>> >
>> >>> > What this means for EL users:
>> >>> >
>> >>> > 1. Eventlogging will still receive data and it will be available in
>> >>> > Kafka.
>> >>> > The data will continue to be imported into Hadoop and files.
>> >>> > 2. The Mysql consumers of Eventlogging will be stopped, so no data
>> will
>> >>> > get
>> >>> > imported into the master(db1046 or m4-master) and by extension to
>> the
>> >>> > analytics-store.
>> >>> > 3. Querying existing data from analytics-store will still be
>> available,
>> >>> > but
>> >>> > data for the next 4 days won't be available.
>> >>> > 4. On Monday, after the maintenance window we'll restart the Mysql
>> >>> > consumers, and all the data should get reimported from Kafka.
>> >>> >
>> >>> > Analytics and Ops(DBA) will work on this together.
>> >>> >
>> >>> > Feel free to reach out to us here or on #wikimedia-analytics if you
>> >>> > have any
>> >>> > concerns/questions.
>> >>> >
>> >>> >
>> >>> > -- Madhu Viswanathan
>> >>> > Software Engineer, Analytics
>> >>> >
>> >>> > _______________________________________________
>> >>> > Engineering mailing list
>> >>> > Engineering at lists.wikimedia.org
>> >>> > https://lists.wikimedia.org/mailman/listinfo/engineering
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Oliver Keyes
>> >>> Count Logula
>> >>> Wikimedia Foundation
>> >>>
>> >>> _______________________________________________
>> >>> Engineering mailing list
>> >>> Engineering at lists.wikimedia.org
>> >>> https://lists.wikimedia.org/mailman/listinfo/engineering
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Oliver Keyes
>> Count Logula
>> Wikimedia Foundation
>>
>> _______________________________________________
>> Analytics mailing list
>> Analytics at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>
>
>
> _______________________________________________
> Engineering mailing list
> Engineering at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/engineering
>
>


-- 
Amir Elisha Aharoni‏ ። אָמִיר אֱלִישָׁע אַהֲרוֹנִי
Language Engineering‏ ። הַנְדָּסָה לְשׁוֹנִית
Wikimedia Foundation‏ ። קֶרֶן וִיקִימֶדְיָה
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/engineering/attachments/20160128/5740ac99/attachment.html>