We have completed all of the updates and reboots for the hypervisors and
instances in https://phabricator.wikimedia.org/T184910, but there are more
maintenance events that are less invasive to come. This is being tracked
in https://phabricator.wikimedia.org/T184910
Most of this will be handled gracefully without user impact, but not all of
it.
We will reboot the `dumps` NFS server that also provides the `maps` and
`scratch` NFS shares tomorrow (1/18/2018). Note the reason this is an
outage event is that this server is a single point of failure. Efforts to
improve this are happening in https://phabricator.wikimedia.org/T168486.
More announcements will come for maintenance that is impactful.
--
Chase Pettet
chasemp on phabricator <https://phabricator.wikimedia.org/p/chasemp/> and
IRC
Sometime soon (probably in the next day or two) we will be applying
kernel patches to all VMs and physical hosts in WMCS. This is to address
an urgent security issue[1] , so we'll be skipping the traditional 7-day
warning period -- basically as soon as proper fixes are available we'll
start patching and rebooting.
As usual, we'll do our best to re-balance Toolforge grid nodes, so
impact on Toolforge users should be minimal (worst case you may need to
manually restart interrupted tasks).
For other users: if your VPS project requires special handling or
specific notice about when a particular VM will reboot, please add a
subtask describing your need to https://phabricator.wikimedia.org/T184189 .
[1] https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)
On 2018-01-09 the wikidatawiki database will move from its current
home on the "s5" slice to a brand new "s8" slice. The
wikidatawiki.{analytics,web}.db.svc.eqiad.wmflabs and
wikidatawiki.labsdb DNS service names will be updated to point to the
new slice host by system administrators. This change should not affect
most users of the Wiki Replica servers.
Only applications that are connecting to
s5.{analytics,web}.db.svc.eqiad.wmflabs or s5.labsdb and expecting the
wikidatawiki_p database to be present will be affected. These
applications should update their configuration to connect to the new
"s8" slice instead.
This is the end point (or nearly so) of a large amount of work that
has been done by Wikimedia's fabulous DBA team of Jamie and Manuel to
improve the health of the Wikidata wiki. See
<https://phabricator.wikimedia.org/T177208> for more details.
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855