<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I might be a 'little' guilty here....<br>
1.5T logs<br>
67G mysql_backups<br>
<br>
Adding some crons for cleanup :)<br>
<br>
Do you have a good suggestion for rotating the log files written out
by jsub? Not sure how the grid engine handles things in regards to
hup/removing files/stale file handlers etc... would rather not have
to restart the processes as they're very long running bots.<br>
<br>
- Damian<br>
<br>
<div class="moz-cite-prefix">On 09/02/2016 19:17, Chase Pettet
wrote:<br>
</div>
<blockquote
cite="mid:CANBEROxXbrF=+W2WD1xMKeju6B7Xgw9ez7X1pt=8RLk73srv2w@mail.gmail.com"
type="cite">
<div dir="ltr"><span style="font-size:13px">Greetings,</span>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">We have 8T allocated to the Tools
project and are now getting space warnings. A log/temp/cache
cleanup was coming regardless for capacity planning purposes
but this has moved it up the priority list. We will try to
truncate in use logs, and generally be reasonable about
cleanup.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">A few thoughts:</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">* Primary copies of scripts and bots
should be elsewhere. Keep these in some source control
mechanism.</div>
<div style="font-size:13px">* Please try to be mindful of logs
and space usage over time. It is not difficult to find >10G
log files at the moment.</div>
<div style="font-size:13px">* Be aware that access, error, and
log files are excluded from backups</div>
<div style="font-size:13px">* This situation is partly holistic
and hopefully we can transition to a central logging mechanism
that is better for all.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">As always, we are around to discuss
in #wikimedia-labs on irc</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Thanks,</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Chase Pettet</div>
<div style="font-size:13px">Operations Engineer</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Labs-announce mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Labs-announce@lists.wikimedia.org">Labs-announce@lists.wikimedia.org</a>
<a class="moz-txt-link-freetext" href="https://lists.wikimedia.org/mailman/listinfo/labs-announce">https://lists.wikimedia.org/mailman/listinfo/labs-announce</a>
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Labs-l mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a>
<a class="moz-txt-link-freetext" href="https://lists.wikimedia.org/mailman/listinfo/labs-l">https://lists.wikimedia.org/mailman/listinfo/labs-l</a>
</pre>
</blockquote>
<br>
</body>
</html>