[Labs-l] [Labs-announce] [Maintenance] NFS Maintenance 2015-10-28 13:30 UTC

[Marc-André Pelletier]

Hello Labs,

As the very last step in recovering completely from our past NFS
failure, we are scheduling a switch back to the primary file server
(labstore1001) as part of the recovery process.  This requires a
maintenance window during which NFS service will be briefly unavailable
since physical wiring has to be moved for the procedure.  We are
planning a brief maintenance window on Wednesday, October 28 starting at
13:30 UTC to return NFS service to the primary server.  The maintenance
is tracked in phabricator at:

https://phabricator.wikimedia.org/T107038

The maintenance window will cause a (planned brief) suspension of NFS
service to labs in the interval between shutting down labstore1002 and
starting service on labstore1001, during which I/O operations to NFS
will stall (be unable to complete until service is restored).

Most labs projects have little to no dependency on NFS anymore, and will
not be affected by the switch.  Projects that still require use of NFS
will lose the ability to access files in /data/project and /home during
the outage.  This includes tools, maps, and the beta cluster projects.

We expect the actual outage will last less than 20 minutes, at some
point during the maintenance window.  Notices will be posted on this
mailing list and on the #wikimedia-labs freenode IRC channel as
maintenance begins and ends.

In case of issues during the switch, we will rollback the change and
return the secondary server (currently in use) to function before we
evaluate the situation and prepare an alternative plan.

-- Marc

_______________________________________________
Labs-announce mailing list
Labs-announce at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-announce