Hi all,
Our current release note strategy is clearly not working.
Too many times do people omit it. Either because they consider the commit not important enough for release notes or because it is a pain to do due to the continuous merge conflicts. Not to mention fix-ups and small clarifications are often annoying to do. It also often results in inconsistent formatting of the notes as everybody does it differently.
For this and other reasons[1], I'd recommend we stop updating release notes through git commits in a separate file.
For more details see Proposal 2 on this page:
https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation...
It is important that we settle quickly and perfect later since this is affecting our current release cycle already.
In case people worry about the scalability of the approach, I'm willing to take this on for the near future. However I'm convinced we have enough people who care and will filter the incoming stream on the wiki page. Simply look at the git history of release notes files (I expect at least Reedy and hexmode will naturally be drawn to this).
-- Krinkle
[1] Other reasons include keeping the change atomic and easy to backport. For what its worth, Git[2], Nodejs[3] and many jQuery projects such as QUnit[4] already maintain release notes this way. [2] https://github.com/git/git/commit/v1.8.2.2 [3] https://github.com/joyent/node/commit/v0.10.5 [4] https://github.com/jquery/qunit/commit/v1.11.0
I like the idea of not having to mess with RELEASE-NOTES-#.## merge conflicts. But I'm not entirely sure about everything in the proposal.
On Thu, May 2, 2013 at 12:30 PM, Krinkle krinklemail@gmail.com wrote:
For more details see Proposal 2 on this page:
https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation...
To avoid wrong entries for bugs that were merely mentioned in the commit message but not fixed in that commit, it becomes more important that we enforce a consistent format. "bug 123" is a mention. "Bug: 123" in footer meta-data indicates the bug was fixed.
So we're changing the semantics of that "Bug" footer already? Instead of being used for searching changesets related to a bug, now it's only for searching changesets actually fixing a bug? And what about changesets that only partially fix a bug? Or bugs where changes to both core and extensions are needed to fix a bug?
We can't use the commit message subject because they aren't always the same as what you want to write in the release notes.
So are we getting rid of the recommended less-than-50-character limit on commit subjects? Or are we assuming that whoever volunteers to clean up the subject-dump will expand on it as necessary?
Mark entries with the relevant labels ("New", "Removed", "Breaking" etc.)
I worry that whoever is doing this for API changes (if it doesn't wind up being me) won't know how we determine "Breaking" for API changes. I.e. "anything that's not backwards-compatible for API users".
On May 2, 2013, at 7:41 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
I like the idea of not having to mess with RELEASE-NOTES-#.## merge conflicts. But I'm not entirely sure about everything in the proposal.
On Thu, May 2, 2013 at 12:30 PM, Krinkle krinklemail@gmail.com wrote:
For more details see Proposal 2 on this page:
https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation...
To avoid wrong entries for bugs that were merely mentioned in the commit message but not fixed in that commit, it becomes more important that we enforce a consistent format. "bug 123" is a mention. "Bug: 123" in footer meta-data indicates the bug was fixed.
So we're changing the semantics of that "Bug" footer already? Instead of being used for searching changesets related to a bug, now it's only for searching changesets actually fixing a bug? And what about changesets that only partially fix a bug? Or bugs where changes to both core and extensions are needed to fix a bug?
Fair point. We could use some other detectable way instead. However I'd prefer to keep things simple where "bug 123" is a mention and "Bug: 123" a resolution. Both are linkified by Gerrit, so that's not an issue.
We could introduce something like "Bug-Fixed: 123". But I'm not sure if more syntax is the answer here. Note we also have people using "(bug 123)" in the subject still, and even people copying BugZilla's format of "Bug 123: title".
Or we could simply omit this detection altogether and build the list from Bugzillla instead (resolved bugs marked as fixed for the appropriate milestone). Then we'd only have to make sure periodically that recently closed bugs without a milestone set get a milestone set. I believe several people perform such query already and fill in the missing milestones.
We can't use the commit message subject because they aren't always the same as what you want to write in the release notes.
So are we getting rid of the recommended less-than-50-character limit on commit subjects? Or are we assuming that whoever volunteers to clean up the subject-dump will expand on it as necessary?
No, the commit subject character limit is there for a good reason. We shouldn't change that for this.
First of all, I think a lot of our commit subjects are poorly written, even for a commit message. Having said that, a good commit subject is also a good release note (that is, if the change itself is notable for release notes). I don't think that these extensive paragraphs of text we are known for in release notes are a good habit.
We already have a 62-char limit for the commit subject. That seems to be going well. Assuming that we properly summarise changes that way already, why would one need more room in the release notes? It is the same event.
There are cases where it is important to write more elaborate notes for maintenance for extension developers and migration/upgrading for site administrators.
And yes when the release notes are processed on the wiki page, things that need further documentation can be freely written in (for example) an "Upgrade" or "To extension maintainers" section – instead of scattered through the bullet point lists.
For example:
== Changes ==
- foo: Upgrade to version 1.20
- bar: Don't silently ignore invalid values in event handler
- ..
== To extension maintainers ==
Various bar plugins are known to call the event handler with an object instead of an array containing the object. This was previously silently skipped. This failure is now explicit by throwing an invalid arguments exception.
Mark entries with the relevant labels ("New", "Removed", "Breaking" etc.)
I worry that whoever is doing this for API changes (if it doesn't wind up being me) won't know how we determine "Breaking" for API changes. I.e. "anything that's not backwards-compatible for API users".
"If you don't know what you're doing, don't do it?"
I think we can reasonably assume good faith that people won't go and mislabel things on the wiki page as "breaking" for a component they are not knowledgeable in.
Plus, it is a wiki page, so it'll be a lot easier to keep track of and fix for everyone involved.
This will make that easier since this would basically turn to doing a "post-merge" review of activities to some degree.
-- Krinkle
On Fri, May 3, 2013 at 1:02 PM, Krinkle krinklemail@gmail.com wrote:
First of all, I think a lot of our commit subjects are poorly written, even for a commit message. Having said that, a good commit subject is also a good release note (that is, if the change itself is notable for release notes). I don't think that these extensive paragraphs of text we are known for in release notes are a good habit.
In my opinion, a good commit summary and a good release note are not necessarily the same thing at all, otherwise we could just dump the git log as the release notes and be done with it. Release notes *should* go into sufficient detail to tell the reader what it is they should be noting.
We already have a 62-char limit for the commit subject. That seems to be going well. Assuming that we properly summarise changes that way already, why would one need more room in the release notes? It is the same event.
Taking a recent example,[1] please tell me how to compress the following into 62 characters:
(in the New features section)
* (bug 45535) introduced the new 'LanguageLinks' hook for manipulating the language links associated with a page before display.
(in the API section)
* BREAKING CHANGE: action=parse no longer returns all langlinks for the page with prop=langlinks by default. The new effectivelanglinks parameter will request that the LanguageLinks hook be called to determine the effective language links. * BREAKING CHANGE: list=allpages, list=langbacklinks, and prop=langlinks do not apply the new LanguageLinks hook, and thus only consider language links stored in the database.
I don't think "Add LanguageLinks hook with breaking changes to 4 API modules" is detailed enough for release notes. And before you try to cheat and split it into multiple commits, note that the new hook and what it means for how langlinks are stored in the database is what is the breaking change in those API modules; the actual changes to the API modules are just mitigating or noting it.
The summary actually used for that revision, BTW, was "(bug 45535) Hook for changing language links".
[1]: https://gerrit.wikimedia.org/r/#/c/59997/
"If you don't know what you're doing, don't do it?"
But do you know that you don't know?
On May 3, 2013, at 9:33 PM, Anomie wrote:
On Fri, May 3, 2013 at 1:02 PM, Krinkle wrote:
First of all, I think a lot of our commit subjects are poorly written, even for a commit message. Having said that, a good commit subject is also a good release note (that is, if the change itself is notable for release notes). I don't think that these extensive paragraphs of text we are known for in release notes are a good habit.
In my opinion, a good commit summary and a good release note are not necessarily the same thing at all, otherwise we could just dump the git log as the release notes and be done with it. Release notes *should* go into sufficient detail to tell the reader what it is they should be noting.
I believe that a (filtered) list of good summaries is indeed sufficient for the release notes. The projects I referenced as example already proof this fact. I don't think it is realistic to think that we need a different type of message for the release notes for each change.
There are some changes (by far not the majority) that require special attention, for example when:
* the site admin needs to make changes to their configuration prior or during upgrading * the site admin needs to update specific extensions at the same time due to breaking compatibility * an extension maintainer should make changes soon due to deprecation of a feature * an extension maintainer needs to ensure changes are made due to removal of a feature * etc.
However in such case an entry in the list of changes in the release notes that is more elaborate than the others doesn't really stand out. In such case, as I believe we have in most cases already, the text in question needs to be written in a paragraph in the Compatibility, Upgrading or similar sections.
We already have a 62-char limit for the commit subject. That seems to be going well. Assuming that we properly summarise changes that way already, why would one need more room in the release notes? It is the same event.
Taking a recent example[1], please tell me how to compress the following into 62 characters:
(in the New features section)
- (bug 45535) introduced the new 'LanguageLinks' hook for manipulating the
language links associated with a page before display.
(in the API section)
- BREAKING CHANGE: action=parse no longer returns all langlinks for the page
with prop=langlinks by default. The new effectivelanglinks parameter will request that the LanguageLinks hook be called to determine the effective language links.
- BREAKING CHANGE: list=allpages, list=langbacklinks, and prop=langlinks do not
apply the new LanguageLinks hook, and thus only consider language links stored in the database.
I don't think "Add LanguageLinks hook with breaking changes to 4 API modules" is detailed enough for release notes. And before you try to cheat and split it into multiple commits, note that the new hook and what it means for how langlinks are stored in the database is what is the breaking change in those API modules; the actual changes to the API modules are just mitigating or noting it.
The summary actually used for that revision, BTW, was "(bug 45535) Hook for changing language links".
Though this is not a typical type of change and I think you already know the answer, I'll give you my take on this one.
As commit subject (and thus release notes change log entry) I'd use:
"Add hook LanguageLinks for changing langlinks before display"
Regarding the change itself:
1) I think this hook should be renamed as it ambiguous. It could be a hook for changing langlinks when parsing/saving a page (input) or a hook to implement a custom source of langlinks when requesting them (output).
2) I'm not sure why you'd make ApiParse not call the hook by default. An option to get the raw langlinks may be useful (I'd be curious as to the use cases, but I can imagine), but doing so by default seems odd.
Regarding the release notes:
This change is a typical case where extra-awareness notes are in order. I personally wouldn't consider these breaking changes, but anyway, they are certainly important. So these are are the kind of changes for which you'd include notes in a separate section.
Which brings me to another point.
No doubt these are breaking changes, but in a way almost every change is potentially a breaking change for something for someone, somewhere.
The kind of changes that break things we previously supported should be noted in those separate sections. Thus resulting in a situation where the change log is skimmable for the curious and only containing summaries of notable changes. And anything that requires attention is clearly separate and to be read first for all eyes (breaking changes for site admins or extension maintainers and configuration changes).
-- Krinkle
On Mon, May 6, 2013 at 10:09 PM, Krinkle krinklemail@gmail.com wrote:
On May 3, 2013, at 9:33 PM, Anomie wrote:
Taking a recent example[1], please tell me how to compress the following into 62 characters:
(in the New features section)
- (bug 45535) introduced the new 'LanguageLinks' hook for manipulating the
language links associated with a page before display.
(in the API section)
- BREAKING CHANGE: action=parse no longer returns all langlinks for the page
with prop=langlinks by default. The new effectivelanglinks parameter will request that the LanguageLinks hook be called to determine the effective language links.
- BREAKING CHANGE: list=allpages, list=langbacklinks, and prop=langlinks do not
apply the new LanguageLinks hook, and thus only consider language links stored in the database.
I don't think "Add LanguageLinks hook with breaking changes to 4 API modules" is detailed enough for release notes. And before you try to cheat and split it into multiple commits, note that the new hook and what it means for how langlinks are stored in the database is what is the breaking change in those API modules; the actual changes to the API modules are just mitigating or noting it.
The summary actually used for that revision, BTW, was "(bug 45535) Hook for changing language links".
Though this is not a typical type of change and I think you already know the answer, I'll give you my take on this one.
As commit subject (and thus release notes change log entry) I'd use:
"Add hook LanguageLinks for changing langlinks before display"
Oh, so not mentioning the breaking API change at all? Definitely not good.
- I'm not sure why you'd make ApiParse not call the hook by default.
An option to get the raw langlinks may be useful (I'd be curious as to the use cases, but I can imagine), but doing so by default seems odd.
I suggested that too. The Wikidata people disagreed, and I didn't feel like arguing over it.
This change is a typical case where extra-awareness notes are in order. I personally wouldn't consider these breaking changes, but anyway, they are certainly important. So these are are the kind of changes for which you'd include notes in a separate section.
How do these "extra" notes get noted wherever you intend for them to be noted? That seems to be missing from the proposal.
And this brings me back to my concern that others will incorrectly think they know what is considered a "breaking" change in the API.
On 7 May 2013 08:52, Brad Jorsch bjorsch@wikimedia.org wrote:
Oh, so not mentioning the breaking API change at all? Definitely not good.
I think you're missing the bit of Timo's proposal where all breaking changes have to *additionally* have a "Breaking change: Foo bar baz" in the "Breaking changes" section of the release notes, like they do already.
How do these "extra" notes get noted wherever you intend for them to be noted? That seems to be missing from the proposal.
Well, they're currentlyhttps://www.mediawiki.org/wiki/MediaWiki_1.21/wmf10#Breaking_changes documentedhttps://www.mediawiki.org/wiki/MediaWiki_1.21/wmf11#Breaking_changes withinhttps://www.mediawiki.org/wiki/MediaWiki_1.21/wmf12#Breaking_changes the https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf1#Breaking_changes releasehttps://www.mediawiki.org/wiki/MediaWiki_1.22/wmf2#Breaking_changes processhttps://www.mediawiki.org/wiki/MediaWiki_1.22/wmf3#Breaking_changes. I don't think Timo's suggesting changing this part of the process at all, just dumping the attempts to have the RELEASE-NOTES-1.XX file written through commits (which is often full of fail and people too often forget) and replacing it with commit logs. It doesn't remove the obligation on people making breaking changes to note this in the top-level notes.
And this brings me back to my concern that others will incorrectly
think they know what is considered a "breaking" change in the API.
Yes, what constitutes a breaking change is in the eye of the beholder, but that's a distinct argument that we clearly have an operating definition for, because, umm, people are writing or not writing such notes right now. :-)
J.
On May 7, 2013, at 5:52 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
On Mon, May 6, 2013 at 10:09 PM, Krinkle krinklemail@gmail.com wrote:
On May 3, 2013, at 9:33 PM, Anomie wrote:
Taking a recent example[1], please tell me how to compress the following into 62 characters:
(in the New features section)
- (bug 45535) introduced the new 'LanguageLinks' hook for manipulating the
language links associated with a page before display.
(in the API section)
- BREAKING CHANGE: action=parse no longer returns all langlinks for the page
with prop=langlinks by default. The new effectivelanglinks parameter will request that the LanguageLinks hook be called to determine the effective language links.
- BREAKING CHANGE: list=allpages, list=langbacklinks, and prop=langlinks do not
apply the new LanguageLinks hook, and thus only consider language links stored in the database.
I don't think "Add LanguageLinks hook with breaking changes to 4 API modules" is detailed enough for release notes. And before you try to cheat and split it into multiple commits, note that the new hook and what it means for how langlinks are stored in the database is what is the breaking change in those API modules; the actual changes to the API modules are just mitigating or noting it.
The summary actually used for that revision, BTW, was "(bug 45535) Hook for changing language links".
Though this is not a typical type of change and I think you already know the answer, I'll give you my take on this one.
As commit subject (and thus release notes change log entry) I'd use:
"Add hook LanguageLinks for changing langlinks before display"
Oh, so not mentioning the breaking API change at all? Definitely not good.
- I'm not sure why you'd make ApiParse not call the hook by default.
An option to get the raw langlinks may be useful (I'd be curious as to the use cases, but I can imagine), but doing so by default seems odd.
I suggested that too. The Wikidata people disagreed, and I didn't feel like arguing over it.
This change is a typical case where extra-awareness notes are in order. I personally wouldn't consider these breaking changes, but anyway, they are certainly important. So these are are the kind of changes for which you'd include notes in a separate section.
How do these "extra" notes get noted wherever you intend for them to be noted? That seems to be missing from the proposal.
And this brings me back to my concern that others will incorrectly think they know what is considered a "breaking" change in the API.
Extra notes are added like usual, except now on the release notes wiki page instead of the file in version control.
Now I hear you thinking, does that mean every patch contributor now needs to know about this wiki page and wait for the change to be approved and then edit the page to add notes? No.
It is the duty of repository co-owners to make wise decisions beyond just code quality. About what changes go in what release (if at all), whether the introduced features are in the best interest of the users and that we can maintain them and are willing to support them. And to be aware of whether a change is breaking or not, and if so if whether the change should still go in the current release or a the next (e.g. don't remove/break without deprecation first, if possible).
As such the co-owners of a repository (not per se the patch contributor) will know these things and should take care (or delegate) the addition of proper release notes as you deem appropriate for the users of the component(s) you maintain.
This also means that there's no uncomfortable waiting time between submission and (if needed for this particular change) the editing of the wiki page. Because can write them right after you approve a change (or do it later).
-- Krinkle
On Tue, 07 May 2013 20:51:07 +0200, Krinkle krinklemail@gmail.com wrote:
It is the duty of repository co-owners to make wise decisions beyond just code quality. About what changes go in what release (if at all), whether the introduced features are in the best interest of the users and that we can maintain them and are willing to support them. And to be aware of whether a change is breaking or not, and if so if whether the change should still go in the current release or a the next (e.g. don't remove/break without deprecation first, if possible).
So in other words, this puts more burden on reviewers, making it harder to get changes merged, especially for new users?
Because that's what this sounds like. Changes are already rotting in gerrit for a year (see the recent watchlist thread), and this certainly will not help.
The current process for release notes is fine; we just need someone to write a custom merge driver for JGit to avoid the merge conflicts. This is a technical issue, not a policy one.
On Tue, May 7, 2013 at 2:56 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
On Tue, 07 May 2013 20:51:07 +0200, Krinkle krinklemail@gmail.com wrote:
It is the duty of repository co-owners to make wise decisions beyond just code quality. About what changes go in what release (if at all), whether the introduced features are in the best interest of the users and that we can maintain them and are willing to support them. And to be aware of whether a change is breaking or not, and if so if whether the change should still go in the current release or a the next (e.g. don't remove/break without deprecation first, if possible).
So in other words, this puts more burden on reviewers, making it harder to get changes merged, especially for new users?
Because that's what this sounds like. Changes are already rotting in gerrit for a year (see the recent watchlist thread), and this certainly will not help.
The current process for release notes is fine; we just need someone to write a custom merge driver for JGit to avoid the merge conflicts. This is a technical issue, not a policy one.
As I said many times before, this isn't really necessary since JGit now supports recursive merges, it's just experimental so I hadn't turned it on.
-Chad
On Tue, 07 May 2013 21:00:05 +0200, Chad innocentkiller@gmail.com wrote:
The current process for release notes is fine; we just need someone to write a custom merge driver for JGit to avoid the merge conflicts. This is a technical issue, not a policy one.
As I said many times before, this isn't really necessary since JGit now supports recursive merges, it's just experimental so I hadn't turned it on.
How catastrophic could turning it on be? Because if not very much, then this would greatly improve everyone's workflows. Just count the manual rebase patchsets on anything involving release notes...
On Tue, May 7, 2013 at 3:07 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
On Tue, 07 May 2013 21:00:05 +0200, Chad innocentkiller@gmail.com wrote:
The current process for release notes is fine; we just need someone to write a custom merge driver for JGit to avoid the merge conflicts. This is a technical issue, not a policy one.
As I said many times before, this isn't really necessary since JGit now supports recursive merges, it's just experimental so I hadn't turned it on.
How catastrophic could turning it on be? Because if not very much, then this would greatly improve everyone's workflows. Just count the manual rebase patchsets on anything involving release notes...
I haven't tried, so who knows. We could turn it on for a test repo (it's configured like normal merge strategies, just no UI for it) and give it a whirl.
-Chad
On May 7, 2013, at 8:56 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
On Tue, 07 May 2013 20:51:07 +0200, Krinkle krinklemail@gmail.com wrote:
It is the duty of repository co-owners to make wise decisions beyond just code quality. About what changes go in what release (if at all), whether the introduced features are in the best interest of the users and that we can maintain them and are willing to support them. And to be aware of whether a change is breaking or not, and if so if whether the change should still go in the current release or a the next (e.g. don't remove/break without deprecation first, if possible).
So in other words, this puts more burden on reviewers, making it harder to get changes merged, especially for new users?
Because that's what this sounds like. Changes are already rotting in gerrit for a year (see the recent watchlist thread), and this certainly will not help.
The current process for release notes is fine; we just need someone to write a custom merge driver for JGit to avoid the merge conflicts. This is a technical issue, not a policy one.
How does this make anything harder for new users? If anything, it makes it easier by not having to worry about which file to edit, what to put in it etc.
As for more burden on reviewers, I disagree. It might inspire them to give more care to proper commit subjects (and edit them liberally as needed) because if you leave it in bad shape, it needs to be fixed later in the notes.
And here again, it simplifies things by maintaining release notes centrally and away from inside the tight development loop. Some of the same people will be doing both, but in the right frame of mind, instead of when you don't want to.
The current process for release notes is not fine. It hasn't been fine for at least 2 or 3 releases. It is missing a lot, and what's there is (imho) poor quality (my own notes included).
Improving that doesn't require moving the process, but I think this is an opportunity to fix a mess the right way at once.
-- Krinkle
On Wed, 08 May 2013 19:24:48 +0200, Krinkle krinklemail@gmail.com wrote:
How does this make anything harder for new users? If anything, it makes it easier by not having to worry about which file to edit, what to put in it etc.
You're either not reading what I wrote or intentionally pulling my words out of context. I said "harder to get changes merged", which is also precisely what I meant - the more actions a reviewer has to perform, the lesser the chance of a changeset being merged (I can provide detailed examples if you want, but I'm sure you know it just as well as I do). And please don't try to convince me figuring out which one of the two files with release notes is hard for anyone with their mind in order.
What to put in is trivial for simple changes - you're right that the commit message will do. And for complex ones, well, I would expect that anybody making them is already knowledgeable enough to know what a good release note is; and your proposed workflow more complicated than what we have now anyway.
(On a side-note, the 1.21 release notes file should have been killed a long time ago when we branched REL1_21; I have no idea what is it still doing in master.)
As for more burden on reviewers, I disagree. It might inspire them to give more care to proper commit subjects (and edit them liberally as needed) because if you leave it in bad shape, it needs to be fixed later in the notes.
"Liberally editing" commit subject is an extremely annoying thing when the threads in your mail client get split and when the items on your dashboard change. If the message has to be fixed, please point it out or just fix it properly *once*.
And here again, it simplifies things by maintaining release notes centrally and away from inside the tight development loop. Some of the same people will be doing both, but in the right frame of mind, instead of when you don't want to.
This is a fallacy; we just need to get gerrit to merge them properly. I have already worked on prototyping a solution (https://github.com/MatmaRex/mediawikireleasenotes-driver), but, being a volunteer, I am unable to spend as much time on it as I'd like. And if switching the merging mode will help as Chad says, then the point is moot anyway.
wikitech-l@lists.wikimedia.org