No, it's ok I don't want to blame anyone unless it's clear if they did it or not, in fact it's more important to secure bots project so that it can never happen again, rather than seeking who did it, given that it was likely a mistake. No worries, and 3 days of logs is not really a problem.<br>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 15, 2012 at 5:20 PM, legoktm <span dir="ltr"><<a href="mailto:legoktm.wikipedia@gmail.com" target="_blank">legoktm.wikipedia@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Wow. I'm really really sorry about this. I'm guessing it's my fault based on the timing.<br>In order to make up for this, I'll try and contact enough users to re-assemble the logs for the missing days. My bouncer should have a good portion of the logs to start with. Petr, can you give me a list of the channels that are missing and the timestamps to start and end from?<span class="HOEnZb"><font color="#888888"><br clear="all">
<div>-- Legoktm<br><a href="http://enwp.org/User:Legoktm" target="_blank">http://enwp.org/User:Legoktm</a><br></div></font></span><div class="HOEnZb"><div class="h5"><br>
<br><br><div class="gmail_quote">On Sat, Dec 15, 2012 at 7:12 AM, Petr Bena <span dir="ltr"><<a href="mailto:benapetr@gmail.com" target="_blank">benapetr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I found out exactly what happened, though I don't know who did it. The whole logs folder was recursively deleted 4 days ago, interesting on this is that the bot is only creating the log folders when the channel config is loaded and log folder doesn't exist. Because the bot was running and all channels were loaded, it was attempting to flush its own cache to disk but it wasn't able to do so because the path of target file wasn't valid (when you write to a file, it doesn't automatically create the folders). That's why we are missing logs for so many days - the bot simply wasn't producing any, backup is running every day but there was nothing to backup, and yesterday when I inserted new module for labs tools, the bot needed to reboot, and that was the time when folders were created and logging started to continue.<div>
<div><br>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 15, 2012 at 1:45 AM, Petr Bena <span dir="ltr"><<a href="mailto:benapetr@gmail.com" target="_blank">benapetr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
of course that's not the way I prefer we need to find out what is happening, it may be even some my mistake or some weird bug, there is a script in cron which generates the .gz files, that one is running as root as far as I know, because it creates the backups too and sets some permissions on them, it's cronned on bots-1 but I didn't change it for weeks and it never happened, so i don't think it has something to do with that...<div>
<div><br>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 15, 2012 at 1:38 AM, Jeremy Baron <span dir="ltr"><<a href="mailto:jeremy@tuxmachine.com" target="_blank">jeremy@tuxmachine.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Fri, Dec 14, 2012 at 7:12 PM, Jeremy Baron <<a href="mailto:jeremy@tuxmachine.com" target="_blank">jeremy@tuxmachine.com</a>> wrote:<br>
> On Fri, Dec 14, 2012 at 6:55 PM, Petr Bena <<a href="mailto:benapetr@gmail.com" target="_blank">benapetr@gmail.com</a>> wrote:<br>
>> Hi, I don't know what happened, but someone removed recursively all data in<br>
>> ~wm-bot/logs<br>
><br>
> Wasn't that accident already announced?<br>
<br>
</div>erm, whooops. seems I misinterpreted<br>
<a href="http://lists.wikimedia.org/pipermail/labs-l/2012-December/000553.html" target="_blank">http://lists.wikimedia.org/pipermail/labs-l/2012-December/000553.html</a><br>
and that prior deletion was narrower than you've just noticed?<br>
<div><br>
>> I may have a backup but because this removal was done yesterday or earlier<br>
>> and apparently happens every day, it's a problem, because even the complete<br>
>> logs archive is almost empty now.<br>
><br>
> Every day?? elaborate?<br>
<br>
</div>Well, I'm still not too clear on this point but if this really is<br>
happening repeatedly (cron'd??) then we need to track down what's<br>
doing it. We can't be needing to manually restore backups multiple<br>
times per week. (and we'd probably have incremental data loss between<br>
last backup and deletion too)<br>
<div><div><br>
-Jeremy<br>
<br>
_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org" target="_blank">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org" target="_blank">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br>
</div></div><br>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div>