Hi Maarten,
thank you for you feedback. There is the same "redundancy" in some Page methods which calls the corresponding site method and I wouldn’t drop one over the other. Page methods have a higher tier than Site methods which calls api functions and finally the comms functions.
pagegenerators implements generators and filters for command line options and provides additional page generators which are outside of site methods like multiple site generators and other stuff.
But I don’t see any advantage using the same functionality from pagegenerators module over those site methods who are called by that function. The (now removed) functions had the form
def this_generator(total=None, site=None) if not site: site = Pywikibot.Site() site.this(total=total)
There is a disadvantage in that sense that site must not be determined and another disadvantage that most site parameters were not provided with the pg function.
Can you tell me your POV about advantage/disadvantage of those pg function?
There are about 50 generator functions in site.GeneratorMixin but pg had only a few of them.
I made a proposal in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would implement all site generators as pagegenerators function. Is this in your sense? Unfortunately this only works with Python 3.7+ and needs some additional work for older Pythons.
Best xqt
Am 13.02.2022 um 14:40 schrieb Maarten Dammers maarten@mdammers.nl:
Hi Xqt,
I don't agree with the removal of the generators that someone decided to deprecate [1]. We always had the principle that pagegenerators is the point of entry to get pages to work on. The pagegenerators are a (thin) layer of abstraction so that you don't have to directly use function in the site objects. With this removal you're breaking one of the very fundamental functions and principles of our framework.
Why did you remove it? Any arguments beside than the incorrect argument that it's redundant? Was the deprecation and removal discussed somewhere? Can't find it on this list.
Maarten
[1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagegen...
On 12-02-2022 13:31, info@gno.de wrote:
Hi all,
a new stable release Pywikibot 7 will be deployed at the end of this month. There are a lot important points with this release, currently implemented in master branch:
Python Plattform:
Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum plattform required is Python 3.5.3 which is preinstalled at toolforge. On the other hand Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3. Most used plattform measured by PyPy download statistic is Python 3.8 (T266984). For developers I highly propose to use Python 3.10 due to "Better error messages" (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
Framework Scripts:
The both scripts generate_family_file and generate_user_files were moved from framework root directory to pywikibot/scripts folder. The two scripts shell and version were moved from scripts folder to pywikibot/scripts. This arrangement was necessary to implement a code entry point for Pywikibot site-package. Possibly you have to change the path environment setting to use these scripts. But all of them are still available through the pwb.py wrapper script.
Linux:
Scripts hash bang was changed from python to python3. This enables to invoke scripts without preleading interpreter.
PAWS:
Pywikibot 7 is a stable release and becomes the Pywikibot framework base on PAWS.
Toolforge:
After deployment of Pywikibot 7 this stable release will be available under /shared/pywikibot/stable and the current master can be accessed under /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge and the similar script search is functional. https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
MediaWiki:
No changes was made for supporting MediaWiki releases. Pywikibot 7 supports MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum Pywikibot version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4. (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
Pywikibot site-package (PyPi):
Installing Pywikibot as a site package (pip install pywikibot==7.0) comes with many innovations. pywikibot i18n messages bundle is available. Framework scripts listed above can be used with Pywikibot 7.The pwb wrapper script is the code entry point to invoke these scripts: pwb [global options] <framework script> [script options|global options]
Pywikibot Tests:
Pywikibot CI tests were moved from Travis to GitHub action. There are two actions implemented. The first runs up tp 20 workers in parallel and does the most tests whereas the second only runs login/logout tests but there are no parallel task which would lead to failures due to not logged-in test account.
Module Changes:
Support for API:Redirects was added in site and page module. User.is_locked() and APISite.is_locked() were added to determine whether a given user or user id is locked globally. The cached output functionality from compat release was re-implemented.
families: Allow family files reside in `families` folder in base_dir by default. This is important if Pywikibot is used as a site-package. generate_family_file uses this new place to save the family file. Wikihow family file was added.
i18n module: bundles() yields message bundle names or Path objects; known_languages() gives all languages we have localizations for.
page module: isStaticRedirect() also considers transcluded __STATICREDIRECT__. put() and change_category() now has a show_diff parameter. User.is_blocked() can also detect range blocks.
proofreadpage: Wikimedia OCR engine is supported.
site module: APISite.get_globaluserinfo() method was added to retrieve globaluserinfo. New "maxlimit" property was added to determine the limits for API loads.
textlib module: textlib comes with to_latin_digits() function as counterpart of to_local_digits() and case_escape() which gives an escaped regex pattern depending on 'first-letter' case.
tools module: intersect_generators was rewritten which makes it running up to 10’000 times faster.
Scripts Changes:
Scripts descriptions can be viewed at https://doc.wikimedia.org/pywikibot/master/scripts/index.html
ConfigParserBot is provided for several scripts. Any option can be set within scripts.ini file. https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot....
add_text scripts provided -create and -createonly options. A CleanBot was added in category script which can be invoked by clean action option. It removes redundant grandchildren categories. The fixing_redirect script uses concurrent.futures to retrieve redirect targets; this decreases processing time by 90% if solving moved pages is enabled.
Deprecations:
Most of the deprecated code parts were removed to start with a new proper and maintainable code without budensome overhead. The deprecations must be solved if you have your own code based on Pywikibot before you upgrade to the new release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html to see all changes. Run your scripts and focus on FutureWarnings about deprecated code parts. Refer the logs for deprecation warnings. Scripts must be run with global -log option; if your script does not use handle_args() function you can invoke it with the pwb wrapper script: pwb.py -log <yourscript> [your options]
Thank you all, comments are welcome
xqt
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org