MW Train status: Incomplete rollout, blocked on two issues

List overview All Threads
Download

newer

older

[Train] 1.32.0-wmf.14 status update

Grievances about future...

Greg Grossmeier

20 Jul 2018 20 Jul '18

5:19 p.m.

Hello, The 1.32.0-wmf.13 version of MediaWiki is rollout to almost all group0 and group1 wikis, but not group2. In plain wording: It is deployed to all non-wikipedias (Commons, wiktionaries, etc) except Wikidata. It was blocked from going to all wikis yesterday due to two issues found during the week: * Fatal MWException in Babel: "Language::isValidBuiltInCode must be passed a string" - https://phabricator.wikimedia.org/T199941 * Wikidata showing wrong language for page elements - https://phabricator.wikimedia.org/T199983 Assuming these issues are resolved before Monday we hope to resume the deployment of this version Monday during European working hours. The tracking task for this deployment: https://phabricator.wikimedia.org/T191059 A handy tool to see which wikis have which version: https://tools.wmflabs.org/versions/ Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

Attachments:

signature.asc (application/pgp-signature — 833 bytes)

Show replies by date

Željko Filipin

23 Jul 23 Jul

5:22 p.m.

New subject: [Engineering] MW Train status: Incomplete rollout, blocked on two issues

Hi, This is the second ever train I am conducting and it has been very interesting. I've been told that is not unusual. I have good news and bad news. Good news is that some problems are resolved. I would like to thank to everybody that helped. Bad news is that train is still blocked. :( Blocking tasks are: - T199941 Fatal MWException in Babel: "Language::isValidBuiltInCode must be passed a string" - T199983 Wikidata showing wrong language for page elements - T200136 Does not work for change a log type drop down when the log type specified by URL / argument @ 1.32.0-wmf.13 (360f7b5) I will cut the new branch tomorrow, but I will not be able to deploy it to group 0 until blockers from last week are resolved. If you can help resolving the current problems, please do. If you know that somebody can help, please let them know. Željko On Fri, Jul 20, 2018 at 7:19 PM Greg Grossmeier <greg(a)wikimedia.org> wrote:

...

C. Scott Ananian

7:47 p.m.

New subject: [Ops] [Engineering] MW Train status: Incomplete rollout, blocked on two issues

Deployments during the week of wikimania are always tough. I'm surprised we even tried to do that this year. --scott On Mon, Jul 23, 2018, 7:23 PM Željko Filipin <zfilipin(a)wikimedia.org> wrote:

...

_______________________________________________ Ops mailing list Ops(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/ops

Greg Grossmeier

8:42 p.m.

New subject: [Ops] [Engineering] MW Train status: Incomplete rollout, blocked on two issues

...

Deployments during the week of wikimania are always tough. I'm surprised we even tried to do that this year.

There is/was sufficient coverage from both RelEng and SRE so there wasn't an issue of site outage without available people to respond. IOW, no one from RelEng nor SRE was attending Wikimania (sadly). There is the issue of not getting quick feedback and resolution on UBN! bugs from attendees at the conference, though. (This was obviously dependent on who was at the conference and only manifested itself in a couple cases.) Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

C. Scott Ananian

9:29 p.m.

New subject: [Engineering] [Ops] MW Train status: Incomplete rollout, blocked on two issues

FWIW, I think the Babel bug was waiting for several days for review from Niklas, the extension owner, who wasn't at wikimania. But the folks who ended up taking action and doing the revert we're at wikimania -- I think it's a general matter of lack of communication during wikimania, with folks distracted and assuming someone else is responsible, rather than a specific "X person is at location Y". Wikimania just disrupts normal communications and work patterns. --scott (To be clear, I'm not trying to pass blame, I'm just pointing out that the disruption is not as simple of who is where or whether "enough" people are at home.) On Mon, Jul 23, 2018, 4:42 PM Greg Grossmeier <greg(a)wikimedia.org> wrote:

...

Deployments during the week of wikimania are always tough. I'm surprised we even tried to do that this year.

Runa Bhattacharjee

24 Jul 24 Jul

8:54 a.m.

New subject: [Engineering] [Ops] MW Train status: Incomplete rollout, blocked on two issues

On Tue, Jul 24, 2018 at 2:59 AM, C. Scott Ananian <cananian(a)wikimedia.org> wrote:

...

Niklas is on vacation and has marked himself as unavailable on phabricator. He won't be back until next week. Meanwhile, the Language team did monitor the conversation and based on that expected the patches to be reverted to unblock the train. Thanks Runa -- Senior Engineering Manager, Language Wikimedia Foundation

Željko Filipin

10:18 a.m.

New subject: [Ops] [Engineering] MW Train status: Incomplete rollout, blocked on two issues

On Tue, Jul 24, 2018 at 10:54 AM Runa Bhattacharjee < rbhattacharjee(a)wikimedia.org> wrote:

...

Meanwhile, the Language team did monitor the conversation and based on that expected the patches to be reverted to unblock the train.

Hi Runa, T199941 (Fatal MWException in Babel: "Language::isValidBuiltInCode must be passed a string") is the only remaining blocker for the train. I see some activity at the task (commit merges, deployments) but the task is still open, UBN and a subtask of T191059 (1.32.0-wmf.13 deployment blockers), making it a blocker. If the issue is resolved, please resolve the phabricator task. If there is still some cleanup to do, but it's not blocking the train, please remove the task from T191059 blockers. Thanks, Željko

Željko Filipin

2:43 p.m.

New subject: [Engineering] MW Train status: Incomplete rollout, blocked on two issues

Hi, 1.32.0-wmf.13 train continues to be "interesting". Trains should not be interesting. They should be boring. I would like to thank again everybody that reported problems and worked on resolving them. There is currently one blocker: - T200269 Unable to undelete revision (Fatal error: given Title does not belong to page ID, RevisionStoreRecord) I have cut the 1.32.0-wmf.14 branch today, but it's deployment (to groups 0, 1 and 2) is blocked on 1.32.0-wmf.13. Željko

Željko Filipin

5:50 p.m.

New subject: [Engineering] MW Train status: Incomplete rollout, blocked on two issues

Prateek Saxena

25 Jul 25 Jul

2:37 a.m.

New subject: [Engineering] MW Train status: Incomplete rollout, blocked on two issues

As one of the people who made the train "interesting" I'd like to thank you for taking care of this :) On Tue, Jul 24, 2018 at 11:20 PM, Željko Filipin <zfilipin(a)wikimedia.org> wrote:

...

Good news! A few minutes ago we have deployed 1.32.0-wmf.13 to all wikis. Logs look OK to me so far. A big thank you to everybody that reported problems and helped resolve them. I want to tank every single one of you, but I'm to tired right now, so I'll leave that for tomorrow. For now, I would like to thank Tyler Cipriani and Greg Grossmeier for all the help with finally getting 1.32.0-wmf.13 everywhere. Željko _______________________________________________ Wikitech-ambassadors mailing list Wikitech-ambassadors(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors

Željko Filipin

30 Jul 30 Jul

2:26 p.m.

New subject: [Engineering] MW Train status: Incomplete rollout, blocked on two issues

This will be my final notification about 1.32.0-wmf.13. I have started working on incident report: https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train Any help in completing the report is greatly appreciated. Thanks, Željko On Tue, Jul 24, 2018 at 7:50 PM Željko Filipin <zfilipin(a)wikimedia.org> wrote:

...

2112

days inactive

2122

days old

wikitech-ambassadors@lists.wikimedia.org

Manage subscription

10 comments

5 participants

tags (0)

participants (5)

C. Scott Ananian
Greg Grossmeier
Prateek Saxena
Runa Bhattacharjee
Željko Filipin