[Labs-l] NFS outage in progress [UPDATE]

[Maximilian Doerr]

Any ETA when I can SSH back in and restore my bots?

Cyberpower678
English Wikipedia Account Creation Team
Mailing List Moderator



> On Jun 18, 2015, at 09:48, Steinsplitter Wiki <steinsplitter-wiki at live.com> wrote:
> 
> the foundation isn't able to provide a service that works in real life.
> 
> From: maximilian.doerr at gmail.com <mailto:maximilian.doerr at gmail.com>
> Date: Thu, 18 Jun 2015 09:47:07 -0400
> To: labs-l at lists.wikimedia.org <mailto:labs-l at lists.wikimedia.org>
> Subject: Re: [Labs-l] NFS outage in progress [UPDATE]
> 
> Mist.  Ich bekomme wieder beschwerte dass meine Bots schon wieder nicht funktionieren. :/
> 
> Cyberpower678
> English Wikipedia Account Creation Team
> Mailing List Moderator
> 
> 
> 
> On Jun 18, 2015, at 09:45, Steinsplitter Wiki <steinsplitter-wiki at live.com <mailto:steinsplitter-wiki at live.com>> wrote:
> 
> >> Hat was hier los ist damit was zum tun? 
> 
> ja :(
> 
> From: maximilian.doerr at gmail.com <mailto:maximilian.doerr at gmail.com>
> Date: Thu, 18 Jun 2015 09:41:53 -0400
> To: labs-l at lists.wikimedia.org <mailto:labs-l at lists.wikimedia.org>
> Subject: Re: [Labs-l] NFS outage in progress [UPDATE]
> 
> Ich kann weder einloggen oder meine Bot Seiten aufrufen.  Hat was hier los ist damit was zum tun?  ACC funktioniert auch nicht.
> 
> Cyberpower678
> English Wikipedia Account Creation Team
> Mailing List Moderator
> 
> 
> 
> On Jun 18, 2015, at 09:40, Steinsplitter Wiki <steinsplitter-wiki at live.com <mailto:steinsplitter-wiki at live.com>> wrote:
> 
> @Martin Domdey <>: hi, ist exakt das selbe was in der vorherigen mail zu finden ist, ich erwarte mir von seiten der foundation eine genauere antwort. z.B. noch zwei stunden
> 
> um ehrlich zu sein dürfte so etwas nicht passieren, toolslab ist stark genutzt.
> 
> From: animalia at gmx.net <mailto:animalia at gmx.net>
> To: labs-l at lists.wikimedia.org <mailto:labs-l at lists.wikimedia.org>
> Date: Thu, 18 Jun 2015 15:08:15 +0200
> Subject: Re: [Labs-l] NFS outage in progress [UPDATE]
> 
> Hey Steini, soll noch mehrere Stunden dauern laut den Technikern ...
>  
> Gesendet: Donnerstag, 18. Juni 2015 um 14:55 Uhr
> Von: "Steinsplitter Wiki" <steinsplitter-wiki at live.com <mailto:steinsplitter-wiki at live.com>>
> An: "Wikimedia Labs" <labs-l at lists.wikimedia.org <mailto:labs-l at lists.wikimedia.org>>
> Betreff: Re: [Labs-l] NFS outage in progress [UPDATE]
> Toolslabs will be back soon?  When approximately?
>  
> > Date: Wed, 17 Jun 2015 23:09:11 -0400
> > From: marc at uberbox.org <mailto:marc at uberbox.org>
> > To: abogott at wikimedia.org <mailto:abogott at wikimedia.org>; labs-l at lists.wikimedia.org <mailto:labs-l at lists.wikimedia.org>
> > Subject: Re: [Labs-l] NFS outage in progress [UPDATE]
> >
> > On 15-06-17 09:29 PM, Andrew Bogott wrote:
> > > Coren is rebooting and fscking the system -- with luck it'll be up
> > > again within the hour.
> >
> > Fortune does not smile upon us; the fsck is taking a LONG time to
> > progress (over a 40T filesystem), but the repair goes apace.
> >
> > There is already indication of some lost data - though the failures are
> > likely to date from a past crash and may have gone unnoticed for some time.
> >
> > Either way, we have a fairly recent (8-9 days) backup of the entire
> > filesystem so we may be able to recover lost or corrupted files if any
> > turn up.
> >
> > Worst case scenario is a catastrophic loss of the entire filesystem
> > (which does not seem likely at this time), in which case we will restore
> > the filesystem completely from the backup.
> >
> > The repair process will take at least a few more hours, during which I
> > will be keeping watch. I will update this list as soon as I have a
> > better assessment of the damage, if any.
> >
> > -- Marc
> >
> >
> > _______________________________________________
> > Labs-l mailing list
> > Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
> > https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>_______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>
> _______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>_______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>
> 
> _______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>_______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>
> 
> _______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>_______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org <mailto:Labs-l at lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/labs-l <https://lists.wikimedia.org/mailman/listinfo/labs-l>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20150618/086c9c50/attachment-0001.html>