*Scheduled downtime for Hadoop - Monday Jan 15th - 10:00 until 12:00 UTC*
Hello,
We need to perform some maintenance on our primary Hadoop cluster, which will require a period of *downtime*. This work is scheduled for *Monday Jan 15th - 10:00 until 12:00 UTC* - which is a US holiday for WMF and also Wikipedia Day https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Day.
This 2 hour maintenance window has been chosen in the hope of minimising disruption for you, whilst the cluster and the various tools that depend upon it, such as Superset and JupyterLab, are largely unavailable.
The work being undertaken is a replacement of the Hadoop nameserver hosts https://phabricator.wikimedia.org/T332573 which, unfortunately, requires a full cluster restart. We will be disabling ingestion to HDFS, pausing Airflow DAGs on all instances, and stopping production data processing pipelines, prior to the work, then re-enabling them all afterwards. We are not expecting any gaps in data, once the pipelines have caught up again.
If you have any queries or concerns about this work, or the time or date is particularly in convenient for you, please don't hesitate to let us know, so that we can look to reschedule.
Kind regards, Ben