Tuesday starting at around 17:00 UTC I'm going to relocate the paws and kubernetes masters to the new network region. While the VMs are copying, launches of new kubernetes jobs and creation of new PAWS notebooks will fail.
The outage should last about an hour -- less if everything goes well, somewhat more if not. Jobs that are already running when the copy begins should be unaffected.
Apologies for any inconvenience caused!
-Andrew
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
My apologies, the earlier version of this email had an incorrect subject line. This outage will be happening on Tuesday, not Monday.
-Andrew
On 4/11/19 8:30 PM, Andrew Bogott wrote:
Tuesday starting at around 17:00 UTC I'm going to relocate the paws and kubernetes masters to the new network region. While the VMs are copying, launches of new kubernetes jobs and creation of new PAWS notebooks will fail.
The outage should last about an hour -- less if everything goes well, somewhat more if not. Jobs that are already running when the copy begins should be unaffected.
Apologies for any inconvenience caused!
-Andrew
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Reminder: this is happening today, in about three hours.
-Andrew
On 4/11/19 8:30 PM, Andrew Bogott wrote:
Tuesday starting at around 17:00 UTC I'm going to relocate the paws and kubernetes masters to the new network region. While the VMs are copying, launches of new kubernetes jobs and creation of new PAWS notebooks will fail.
The outage should last about an hour -- less if everything goes well, somewhat more if not. Jobs that are already running when the copy begins should be unaffected.
Apologies for any inconvenience caused!
-Andrew
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
This work is still underway. There are some unforeseen issues but we should be back to normal shortly.
On 4/16/19 9:04 AM, Andrew Bogott wrote:
Reminder: this is happening today, in about three hours.
-Andrew
On 4/11/19 8:30 PM, Andrew Bogott wrote:
Tuesday starting at around 17:00 UTC I'm going to relocate the paws and kubernetes masters to the new network region. While the VMs are copying, launches of new kubernetes jobs and creation of new PAWS notebooks will fail.
The outage should last about an hour -- less if everything goes well, somewhat more if not. Jobs that are already running when the copy begins should be unaffected.
Apologies for any inconvenience caused!
-Andrew
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
This is done now. Paws broke in a thousand ways after the move so it lagged well behind the expected timeline, but normal function of the toolforge k8s grid and Paws should be restored.
Let us know if you run into unexpected issues.
-Andrew
On 4/16/19 1:03 PM, Andrew Bogott wrote:
This work is still underway. There are some unforeseen issues but we should be back to normal shortly.
On 4/16/19 9:04 AM, Andrew Bogott wrote:
Reminder: this is happening today, in about three hours.
-Andrew
On 4/11/19 8:30 PM, Andrew Bogott wrote:
Tuesday starting at around 17:00 UTC I'm going to relocate the paws and kubernetes masters to the new network region. While the VMs are copying, launches of new kubernetes jobs and creation of new PAWS notebooks will fail.
The outage should last about an hour -- less if everything goes well, somewhat more if not. Jobs that are already running when the copy begins should be unaffected.
Apologies for any inconvenience caused!
-Andrew
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce