Hello.
TLDR: In MW core, RELEASE-NOTES is a cause of the majority of merge conflicts. I propose to restructure it in a way that could let us do less manual conflict resolutions in Gerrit.
The problem
When working on MediaWiki core, all of us often need to update RELEASE-NOTES-XX file in case some deprecations or breaking changes were made. While the change is under review, other changes touching RELEASE-NOTES-XX are made, causing merge conflicts that need to be resolved manually. Non insignificant portion of manual conflict resolutions are needed exclusively due to RELEASE-NOTES.
This is not a surprise due to the structure of the notes file. It has several sections, with every section being a list of individual notes. Since most of us only append lines to the end of the list, it creates a contention.
The proposal
I propose to reorganize RELEASE-NOTES. We keep the sections, but within each section we start the note with the name of the class that’s been touched, if it makes sense.
For example, instead of writing
* In AbstractBlock, the getTargetAndType() and getTarget() methods are hard deprecated. Instead use getTargetName() and getTargetUserIdentity() together with getType().
We would write
* AbstractBlock: the following method were hard-deprecated: - getTarget() - use getTargetName() or getTargetUserIdentity() - getTargetAndType() - use getType() with getTargetUserIdentity() or getTargetName()
After switching to a more standardized format, we sort individual notes alphabetically by class name. Method names within messages need to be sorted alphabetically as well. This will make an insertion point of new release notes more random, hopefully reducing the contention and thus reducing the probability of a merge conflict. Additionally, I personally think this will look nicer and will be easier to read for extension developers.
Not all the release notes can follow this pattern, and it’s ok. The point is not to be perfect, the point is to try to be better.
Alternative solutions
We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Feedback
Please tell me what you think about this. - Do you think it is a real problem? - Do you think simply restructuring the notes with help? - Do you think writing a custom merge driver is a way to go? - Will you remember to follow the new structure if we agree on it even though it’s not really possible to enforce with linters since not all the notes can follow this structure? - Extension developers - do you care? Do you think alphabetically sorted RELEASE-NOTES will be any easier for you to read?
Thank you. Petr Pchelko. Staff Software Engineer. WMF Platform Engineering Team.
We could write a custom merge driver for RELEASE-NOTES which always puts
‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Didnt someone already do that? There used to be a ruby script floating around for that.
On Monday, July 19, 2021, Petr Pchelko ppchelko@wikimedia.org wrote:
Hello.
TLDR: In MW core, RELEASE-NOTES is a cause of the majority of merge conflicts. I propose to restructure it in a way that could let us do less manual conflict resolutions in Gerrit.
*The problem*
When working on MediaWiki core, all of us often need to update RELEASE-NOTES-XX file in case some deprecations or breaking changes were made. While the change is under review, other changes touching RELEASE-NOTES-XX are made, causing merge conflicts that need to be resolved manually. Non insignificant portion of manual conflict resolutions are needed exclusively due to RELEASE-NOTES.
This is not a surprise due to the structure of the notes file. It has several sections, with every section being a list of individual notes. Since most of us only append lines to the end of the list, it creates a contention.
*The proposal*
I propose to reorganize RELEASE-NOTES. We keep the sections, but within each section we start the note with the name of the class that’s been touched, if it makes sense.
For example, instead of writing
- In AbstractBlock, the getTargetAndType() and getTarget() methods are hard deprecated. Instead use getTargetName() and getTargetUserIdentity() together with getType().
We would write
- AbstractBlock: the following method were hard-deprecated:
- getTarget() - use getTargetName() or getTargetUserIdentity()
- getTargetAndType() - use getType() with getTargetUserIdentity() or
getTargetName()
After switching to a more standardized format, we sort individual notes alphabetically by class name. Method names within messages need to be sorted alphabetically as well. This will make an insertion point of new release notes more random, hopefully reducing the contention and thus reducing the probability of a merge conflict. Additionally, I personally think this will look nicer and will be easier to read for extension developers.
Not all the release notes can follow this pattern, and it’s ok. The point is not to be perfect, the point is to try to be better.
*Alternative solutions*
We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
*Feedback*
Please tell me what you think about this.
- Do you think it is a real problem?
- Do you think simply restructuring the notes with help?
- Do you think writing a custom merge driver is a way to go?
- Will you remember to follow the new structure if we agree on it even
though it’s not really possible to enforce with linters since not all the notes can follow this structure?
- Extension developers - do you care? Do you think alphabetically sorted
RELEASE-NOTES will be any easier for you to read?
Thank you. Petr Pchelko. Staff Software Engineer. WMF Platform Engineering Team.
Le 20/07/2021 à 06:06, Brian Wolff a écrit :
We could write a custom merge driver for RELEASE-NOTES which always
puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Didnt someone already do that? There used to be a ruby script floating around for that.
The tool is: https://github.com/MatmaRex/mediawikireleasenotes-driver https://github.com/MatmaRex/mediawikireleasenotes-driver created in 2013. It leads me to some material we wrote at that time when the issue got first identified:
* 2013 RFC https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation
* 2018 task (still open) https://phabricator.wikimedia.org/T200392 https://phabricator.wikimedia.org/T200392
If we go with a merge driver, it would needs to be installed on CI (zuul-merger does use cgit) and on Gerrit (which uses jGit).
Antoine "hashar" Musso
On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko ppchelko@wikimedia.org wrote:
*Alternative solutions*
We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Probably overkill for MediaWiki but I'd like to mention the way that Python developers (CPython) manages their release notes, in case it might be useful. The TL;DR is that each change has a unique valid reStructuredText file in a specific directory and then there is a tool to merge all changes when a release is made.
The full process from a contributor point of view is described in [1]. The tool used to both generate the change files and merge them into a release file is [2].
[1] https://devguide.python.org/committing/#updating-news-and-what-s-new-in-pyth... [2] https://pypi.org/project/blurb/
Le 20/07/2021 à 09:48, Riccardo Coccioli a écrit :
On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko <ppchelko@wikimedia.org mailto:ppchelko@wikimedia.org> wrote:
*Alternative solutions* We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Probably overkill for MediaWiki but I'd like to mention the way that Python developers (CPython) manages their release notes, in case it might be useful. The TL;DR is that each change has a unique valid reStructuredText file in a specific directory and then there is a tool to merge all changes when a release is made.
The full process from a contributor point of view is described in [1]. The tool used to both generate the change files and merge them into a release file is [2].
[1] https://devguide.python.org/committing/#updating-news-and-what-s-new-in-pyth... https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python [2] https://pypi.org/project/blurb/ https://pypi.org/project/blurb/
OpenStack has a similar tool: reno. The doc has an overview of the requirements: https://docs.openstack.org/reno/latest/user/design.html https://docs.openstack.org/reno/latest/user/design.html and the usage doc for quick glance: https://docs.openstack.org/reno/latest/user/usage.html https://docs.openstack.org/reno/latest/user/usage.html
There is an a 30 mins presentation https://www.youtube.com/watch?v=tEOGJ_h0Lx0 https://www.youtube.com/watch?v=tEOGJ_h0Lx0
Short of having to introduce a Python tool to the developers, maybe the PHP ecosystem has a similar tool? Or we can reach out to other high traffic projects and see how they are managing their changelog and maybe forge a common tool.
Antoine "hashar" Musso
On Tue, Jul 20, 2021 at 5:22 AM Antoine Musso hashar@free.fr wrote:
Le 20/07/2021 à 09:48, Riccardo Coccioli a écrit :
On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko ppchelko@wikimedia.org wrote:
Alternative solutions
We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’, but I’m not sure the result will justify the investment.
Probably overkill for MediaWiki but I'd like to mention the way that Python developers (CPython) manages their release notes, in case it might be useful. The TL;DR is that each change has a unique valid reStructuredText file in a specific directory and then there is a tool to merge all changes when a release is made.
The full process from a contributor point of view is described in [1]. The tool used to both generate the change files and merge them into a release file is [2].
[1] https://devguide.python.org/committing/#updating-news-and-what-s-new-in-pyth... [2] https://pypi.org/project/blurb/
OpenStack has a similar tool: reno. The doc has an overview of the requirements: https://docs.openstack.org/reno/latest/user/design.html and the usage doc for quick glance: https://docs.openstack.org/reno/latest/user/usage.html
There is an a 30 mins presentation https://www.youtube.com/watch?v=tEOGJ_h0Lx0
Short of having to introduce a Python tool to the developers, maybe the PHP ecosystem has a similar tool? Or we can reach out to other high traffic projects and see how they are managing their changelog and maybe forge a common tool.
https://github.com/Automattic/jetpack-changelogger is a similar tool to reno that is pure php and composer installable. Some of you may recognize the primary author too if you check the git history there.
Bryan
wikitech-l@lists.wikimedia.org