[QA] Gerrit cleanup day - QA ramifications

[Antoine Musso]

Le 20/08/2015 01:52, Greg Grossmeier a écrit :
> Hello all,
> 
> This is a heads up and discussion starter about the upcoming "Gerrit
> Cleanup Day" (see: https://phabricator.wikimedia.org/T88531).
> 
> tl;dr: On Wed Sept. 23rd there will be a concerted effort to reduce the
> backlog of open patchsets in Gerrit (by reviewing and/or merging them,
> as appropriate). What does this mean for us, those who care about
> quality and site stability?

Hello,

That is a Wednesday, hence after the branch cut so it seems fine.

We should emphasis there is probably no hurry to have patches finished /
merged since they can well undermine the site stability.  A patch that
has been idling for a few months can probably wait a few more days.

What I am worried about are the metrics:

* Number of patchsets in queue before and after.
* Median age of patchsets in queue before and after.
* Nice-to-have: Identify repositories / teams championing code review,
and the black holes (by having a log of all actions that day; and doing
analysis after that day.)


And we all know how engineers react when they are metric driven. They
fulfil those metrics and tends to ignore effects that are not taken in
account.  So if we merge a ton of patch and end up with a week or so of
site instability, the whole event would be a huge burden and definitely
a failure.


I would get a list of bug fixed and new features implemented and add to
the metrics reverts and potential site issues to mitigate the raw
metrics.  Would need a few additional days of monitoring though and we
would not know the result until the next branch is fully deployed.


Another thing, with so many changes potentially landing, it would be
rather nice to have all teams to freeze new features introduction a few
days before the event.  That is to minimize the amount of code that is
going to land.


As usual, remember people to test merged patchset on beta cluster once
merged and set up a QA sprint the day after.

May I say that to land a patch that day they should have tests and more
than one +2 approval?

> What can we do as a group to ensure that all of the code that does get
> merged during this day doesn't significantly reduce the quality of our
> codebase? A site outage the following week would be A Bad Thing(TM).
> 
> 
> Two basic response:
> 1) do nothing out of the ordinary (iow: trust our current processes)
> 2) do something different
> 
> 
> Thoughts?  (I have some ideas, but I'd like to hear other people's
> opinions first)
> 
> Greg
> 


-- 
Antoine "hashar" Musso