[Labs-l] IRC logs are gone

[legoktm]

Wow. I'm really really sorry about this. I'm guessing it's my fault based
on the timing.
In order to make up for this, I'll try and contact enough users to
re-assemble the logs for the missing days. My bouncer should have a good
portion of the logs to start with. Petr, can you give me a list of the
channels that are missing and the timestamps to start and end from?
-- Legoktm
http://enwp.org/User:Legoktm



On Sat, Dec 15, 2012 at 7:12 AM, Petr Bena <benapetr at gmail.com> wrote:

> I found out exactly what happened, though I don't know who did it. The
> whole logs folder was recursively deleted 4 days ago, interesting on this
> is that the bot is only creating the log folders when the channel config is
> loaded and log folder doesn't exist. Because the bot was running and all
> channels were loaded, it was attempting to flush its own cache to disk but
> it wasn't able to do so because the path of target file wasn't valid (when
> you write to a file, it doesn't automatically create the folders). That's
> why we are missing logs for so many days - the bot simply wasn't producing
> any, backup is running every day but there was nothing to backup, and
> yesterday when I inserted new module for labs tools, the bot needed to
> reboot, and that was the time when folders were created and logging started
> to continue.
>
>
>
> On Sat, Dec 15, 2012 at 1:45 AM, Petr Bena <benapetr at gmail.com> wrote:
>
>> of course that's not the way I prefer we need to find out what is
>> happening, it may be even some my mistake or some weird bug, there is a
>> script in cron which generates the .gz files, that one is running as root
>> as far as I know, because it creates the backups too and sets some
>> permissions on them, it's cronned on bots-1 but I didn't change it for
>> weeks and it never happened, so i don't think it has something to do with
>> that...
>>
>>
>>
>> On Sat, Dec 15, 2012 at 1:38 AM, Jeremy Baron <jeremy at tuxmachine.com>wrote:
>>
>>> On Fri, Dec 14, 2012 at 7:12 PM, Jeremy Baron <jeremy at tuxmachine.com>
>>> wrote:
>>> > On Fri, Dec 14, 2012 at 6:55 PM, Petr Bena <benapetr at gmail.com> wrote:
>>> >> Hi, I don't know what happened, but someone removed recursively all
>>> data in
>>> >> ~wm-bot/logs
>>> >
>>> > Wasn't that accident already announced?
>>>
>>> erm, whooops. seems I misinterpreted
>>> http://lists.wikimedia.org/pipermail/labs-l/2012-December/000553.html
>>> and that prior deletion was narrower than you've just noticed?
>>>
>>> >> I may have a backup but because this removal was done yesterday or
>>> earlier
>>> >> and apparently happens every day, it's a problem, because even the
>>> complete
>>> >> logs archive is almost empty now.
>>> >
>>> > Every day?? elaborate?
>>>
>>> Well, I'm still not too clear on this point but if this really is
>>> happening repeatedly (cron'd??) then we need to track down what's
>>> doing it. We can't be needing to manually restore backups multiple
>>> times per week. (and we'd probably have incremental data loss between
>>> last backup and deletion too)
>>>
>>> -Jeremy
>>>
>>> _______________________________________________
>>> Labs-l mailing list
>>> Labs-l at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>>
>>
>>
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20121215/16393a10/attachment.html>