[Labs-l] *COMPLETE* Planned maintenance

[addshorewiki]

-rw-rw---- 1 50584 local-50584 990G Oct 10 17:59 bz2xml2sql.out


On 11 October 2013 09:54, YiFei <zhuyifei1999 at gmail.com> wrote:

> which log anyway?
>
> 2013/10/11, Petr Bena <benapetr at gmail.com>:
> > 990 GIGABYTES in single file?
> >
> > "Holy Jesus on a stick" --Debra Morgan
> >
> > On Fri, Oct 11, 2013 at 7:21 AM, Marc-André Pelletier
> > <mpelletier at wikimedia.org> wrote:
> >> Hello all,
> >>
> >> The switch to the new NFS server hardware is now complete, with only 12
> >> hours of delay.  :-)
> >>
> >> Seriously though, the filesystem copy took so long because - despite
> >> having prepared with a copy a day early so that only a quick rsync would
> >> suffice - there ended up being a bit over 4T worth of files that were
> >> touched in that 24h period.
> >>
> >> Log files, by their nature, are constantly updated.  Normally, that
> >> shouldn't have been a major issue since the rsync would just copy them
> >> again...  except that many of those log files were *hundreds of gigs* in
> >> size.  This caused the filesystem resync to take a bit over 11 hours.
> >>
> >> While Cyberbot was the overall winner, with very nearly 2G of logs to
> >> his name, our winner for most impressive single log is yifeibot with
> >> 990G in a single log file!
> >>
> >> I would really much rather not have to turn quota on on our file system:
> >> it is very useful to be occasionally able to handle huge datasets.
> >> However, if users abuse the freedom I will have no choice but to do so
> >> in order to protect reliability and QoS.
> >>
> >> So, the delay having been that long, some things may have broken that I
> >> was no longer able to notice (I've been at this for 14h straight, now,
> >> and need sleep).  I'll be on hand tomorrow to help work out any kinks
> >> that may have slipped in.
> >>
> >> On the positive side, however, we are now on new hardware for the NFS
> >> server and it seems to be working quite fine.  Yeay!
> >>
> >> -- Marc
> >>
> >>
> >> _______________________________________________
> >> Labs-l mailing list
> >> Labs-l at lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/labs-l
> >
> > _______________________________________________
> > Labs-l mailing list
> > Labs-l at lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/labs-l
> >
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20131011/fe1791c8/attachment.html>