On Thu, May 21, 2015 at 12:05 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
"Test Plan" is required.
Sounds like a good practice to me. Worst case scenario, type "I didn't
test
this patch at all."
I think it's not a good idea, as it trains people to do the opposite thing of what it's intended to train people for. I.e. "mandatory, but put 'whatever' there" IMHO is worse than non-mandatory but supported by common consensus. If we're setting up a new system, we shouldn't set it up so we constantly work around it.
I don't agree, or I don't understand, "Test Plan:" is fantastic and I've wanted it in commit messages for years. Thinking about how to test is essential to writing good code, and a reminder to express your thought process is invaluable, whether it's "qunit tests now pass", or "it should no longer break when you visit the special page", or "obvious fix". Expecting reviewers to be Nostradamus and infer what the engineer might have thought about testing, or whether the engineer tested at all, is dumb. If someone writes "Test Plan: whatever", they'll get -1s until they respect reviewers enough to write "Test Plan: trivial untested fix in an area that lacks tests".
In the last week my instance crashed on two checkins that were +2'd but never run; both useful improvements to MediaWiki-Vagrant roles that looked completely legit. I appreciate people writing needed code and others reviewing it promptly, but typing: "Test Plan: I didn't test this patch at all" might have saved me 20 minutes of wondering where I screwed up, re-provisioning, hunting down log files, etc. (Both are fixed BTW.)
Can/will we add a "Test Plan: " line to the default commit.message https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#idp25045696 just to get developers in the mood and encourage a standard format? Independent of Differential and separately from whether we make it mandatory. It seems developers have to do git config --global commit.template ~/path/to/better/git_commit.message , but we could put the file in a MediaWiki project like core
Regards, -- =S Page WMF Tech writer