Hi Damian, On 4/21/21 2:32 PM, Damian Johnson wrote: I'm not aware of Pywikibot having such a policy, but I think it would be a good idea to have one. MediaWiki has a stable interface policy[1] which defines what parts are stable to build on top of and which are considered internal and then a process on how to deprecate and make changes to what's supposed to be stable. One of the things I worked on for MediaWiki's deprecation process is developing codesearch[2] which makes it pretty straightforward for developers to see what methods/functions are practically being used and see what use cases are. I think something like that would be valuable for Pywikibot as well, but code for most bots/scripts is really all over the place. Something like Toolhub[3] would help with this too. [1] https://www.mediawiki.org/wiki/Stable_interface_policy [2] https://codesearch.wmcloud.org/search/ [3] https://meta.wikimedia.org/wiki/Toolhub HTH, -- Legoktm

Hi guys, I'm with Strainu on this one. Let's take an example: Someone decided that handleArg didn't look pretty enough and renamed it in January this year. So far so good, but 4 months later the redirect was removed [1]. Why was this redirect removed? Don't you realize dozens of robots use Pywikibot as a library? For me alone this broke over 40 jobs. I appreciate time and effort being spend on improving the framework, but here you derailed. Maarten [1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/689513/3/pywikibot/pagege… On 13-11-2021 13:12, Strainu wrote: > Hi folks, > > I'd like to bring this thread back to life since in the months passed > *a lot* of deprecated code has been removed, some of which was quite > "recent". I totally understand that projects move forward, and so does > Python, I totally understand the developer's whish to use new language > features and I don't believe we should keep deprecated code for > decades, as it often happens in wikis. However, given that the vast > majority of Pywikibot users are busy volunteers and that many projects > depend on (mostly unsupervised) pywikibot code for critical > maintenance work, I believe we need some kind of pedictibility on > deprecation. > > As a user, I would like to understand: > 1. when and how can a function become deprecated > 2. when and how can a parameter become deprecated > 3. how long will I still be able to use a deprecated function or parameter > 4. what Python versions will Pywikibot support. > > For 3, I propose to maintain a compatibility for *at least* 3 years. > This roughly matches the Debian lifecycle, as the longest-maintained > non-LTS release of major Linux distributions. > > For 4, I propose to support a Python version (e.g. 3.5) untill all of > the following are true (that is, the longest period between them): > - two years after official support has ended (e.g. 7 years after launch) AND > - one year after Toolforge moved to a newer Python version. AND > - the percentage of users of a version goes under 5% > > This should allow both Toolforge and independent users ample time to > update their code without surprises, even when using LTS releases. > > What do you think? I would love to see feedback from both developers > and users on these questions and possible answers. Even if you don't > agree with these proposals, please make your own, so that we can > hopefully agree on some rules - any deadlines would be better than > none. > > Regards, > Strainu > > În vin., 30 apr. 2021 la 00:33, Kunal Mehta <legoktm(a)debian.org> a scris: >> Hi Damian, >> >> On 4/21/21 2:32 PM, Damian Johnson wrote: >>> What is pywikibot's policy regarding code deprecation? Can we remove >>> it after a set duration and, if so, what is it? >> I'm not aware of Pywikibot having such a policy, but I think it would be >> a good idea to have one. MediaWiki has a stable interface policy[1] >> which defines what parts are stable to build on top of and which are >> considered internal and then a process on how to deprecate and make >> changes to what's supposed to be stable. >> >> One of the things I worked on for MediaWiki's deprecation process is >> developing codesearch[2] which makes it pretty straightforward for >> developers to see what methods/functions are practically being used and >> see what use cases are. I think something like that would be valuable >> for Pywikibot as well, but code for most bots/scripts is really all over >> the place. Something like Toolhub[3] would help with this too. >> >> [1] https://www.mediawiki.org/wiki/Stable_interface_policy >> [2] https://codesearch.wmcloud.org/search/ >> [3] https://meta.wikimedia.org/wiki/Toolhub >> >> HTH, >> -- Legoktm >> >> _______________________________________________ >> pywikibot mailing list >> pywikibot(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/pywikibot > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org

Hi Strainu, Thanks for bringing this up. And sorry to hear that many have had negative experiences with recent removals of deprecated code (though I had nothing to do with it). The Pywikibot project is barely maintained; a small community of interested folks are pushing it forward, and most code gets merged without review. I don't think we can claim, in good conscience, that there are folks who still keep a full architectural view of all elements of this project (as is the case with MediaWiki and many other tools we use daily). The point is: if the framework is too difficult to execute, it won't happen. So, although I like your proposed framework, I find it unrealistic to happen. It would assume someone is pulling data on Python version usage, and they would be involved in code review in such a way to ensure that code incompatible with legacy Python versions is not introduced too soon. I would encourage us to come up with a framework that is easier to maintain, and mostly automatable. Maybe we should just use the "7 year" rule, make sure that we have a CI pipeline that tries to build Pywikibot for all Python version that are currently within that 7 years, and at the top of the CI pipelines' code we have a comment that educates the users who may decide to edit it when the 7 year period ends for each existing version of Python. This way, incompatible changes would just fail on CI, and the enforcement of rules is quite simple, and breaking them by accident (such as by someone unknowingly modifying the CI pipeline) is less likely. This was just an idea, and may not be the right way, but I hope it shows what I mean by simplifying matters. Similarly, maybe we should find a way to create a CI pipeline that would check for "@deprecated" methods, and cross-checks it with a reference JSON file in which each deprecated function is listed along with its deprecation date. If someone tries to remove the function before that date + 3 months, the CI would fail. If someone tries to add "@deprecated" without updating the JSON file, the CI would fail. This way, we have one single source of truth about when things were marked as deprecated (the JSON file) and an easy mechanism to track when deprecated methods are getting removed. Again, just a raw idea. On Sat, Nov 13, 2021 at 7:13 AM Strainu <strainu10(a)gmail.com> wrote:

Hi Huji, Thanks for your message, it's very thoughtful and raises many valid points. The three year limit for deprecation should be trivial to enforce using git hooks and I can certainly help put it in place. Now for the Python support. I totally support setting up checks and tests, although I don't consider them critical. If incompatible code does pass through review (it has happened before during GSOC) I feel confident the community will catch that in a few days or weeks. If we can have automation, all the better. Also, I don't have any bias against a single nimerical limit myself. The algorithm I proposed was based on the issues raised in the latest deprecation discussion ( https://phabricator.wikimedia.org/T286867 ). If all these can be handled in a single limit, then that is clearly preferable and I support it. Note that the limit might well be larger than 7 years in that case. Regards, Strainu Pe sâmbătă, 13 noiembrie 2021, Huji Lee <huji.huji(a)gmail.com> a scris: negative experiences with recent removals of deprecated code (though I had nothing to do with it). interested folks are pushing it forward, and most code gets merged without review. I don't think we can claim, in good conscience, that there are folks who still keep a full architectural view of all elements of this project (as is the case with MediaWiki and many other tools we use daily). happen. So, although I like your proposed framework, I find it unrealistic to happen. It would assume someone is pulling data on Python version usage, and they would be involved in code review in such a way to ensure that code incompatible with legacy Python versions is not introduced too soon. maintain, and mostly automatable. Maybe we should just use the "7 year" rule, make sure that we have a CI pipeline that tries to build Pywikibot for all Python version that are currently within that 7 years, and at the top of the CI pipelines' code we have a comment that educates the users who may decide to edit it when the 7 year period ends for each existing version of Python. This way, incompatible changes would just fail on CI, and the enforcement of rules is quite simple, and breaking them by accident (such as by someone unknowingly modifying the CI pipeline) is less likely. This was just an idea, and may not be the right way, but I hope it shows what I mean by simplifying matters. check for "@deprecated" methods, and cross-checks it with a reference JSON file in which each deprecated function is listed along with its deprecation date. If someone tries to remove the function before that date + 3 months, the CI would fail. If someone tries to add "@deprecated" without updating the JSON file, the CI would fail. This way, we have one single source of truth about when things were marked as deprecated (the JSON file) and an easy mechanism to track when deprecated methods are getting removed. Again, just a raw idea. parameter launch) AND be over