Fallback is: cable up the old 1GB nic (Chris has done this and set up the port), PXE install on that, move to 10gb NIC once we're back up. Gross but it gets the job done.
Set for tomorrow (Friday) 1 to 4 pm UTC, this time should be much smoother. Same caveats apply as before.
Ariel
On Wed, Mar 2, 2016 at 8:47 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
PXE boot from non-embedded nic failed spectacularly despite our best efforts. This means we'll have to schedule another window once we have someting new to try. I apologize for the extra inconvenience. All services are back exactly the way they were.
Ariel
On Wed, Mar 2, 2016 at 6:01 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
Extending this downtime window because we ran into unexpected issues with PXE boot.
On Tue, Mar 1, 2016 at 3:53 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
Dataset1001, the host which serves dumps and other datasets to the public, as well as providing access to various datasets directly on stats100x, will be unavailable tomorrow for an upgrade to jessie. While I don't expect to need nearly 3 hours for the upgrade, better safe than sorry. In the meantime all files will be accessible via ms1001.wikimedia.org via the web, and all dumps and page view files from our mirrors as well.
Thanks for your understanding.
Ariel Glenn
This upgrade has concluded successfully and all services are again operational.
Ariel
On Thu, Mar 3, 2016 at 8:15 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
Fallback is: cable up the old 1GB nic (Chris has done this and set up the port), PXE install on that, move to 10gb NIC once we're back up. Gross but it gets the job done.
Set for tomorrow (Friday) 1 to 4 pm UTC, this time should be much smoother. Same caveats apply as before.
Ariel
On Wed, Mar 2, 2016 at 8:47 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
PXE boot from non-embedded nic failed spectacularly despite our best efforts. This means we'll have to schedule another window once we have someting new to try. I apologize for the extra inconvenience. All services are back exactly the way they were.
Ariel
On Wed, Mar 2, 2016 at 6:01 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
Extending this downtime window because we ran into unexpected issues with PXE boot.
On Tue, Mar 1, 2016 at 3:53 PM, Ariel Glenn WMF ariel@wikimedia.org wrote:
Dataset1001, the host which serves dumps and other datasets to the public, as well as providing access to various datasets directly on stats100x, will be unavailable tomorrow for an upgrade to jessie. While I don't expect to need nearly 3 hours for the upgrade, better safe than sorry. In the meantime all files will be accessible via ms1001.wikimedia.org via the web, and all dumps and page view files from our mirrors as well.
Thanks for your understanding.
Ariel Glenn