TL;DR:
* The #pywikibot IRC channel is moving from Freenode to Libera.Chat.
* Register an account on Libera.Chat and join us there!
There has been a lot of activity over the last 2-3 days related to
staffing changes on the Freenode IRC network [0]. The Wikimedia IRC
Group Contacts (GCs) [1] evaluated the situation and decided that
moving the Wikimedia IRC channels from Freenode to the brand new
Libera.Chat IRC network [2] would be the best course of action [3].
So, we are moving!
A new #pywikibot channel has been created on irc.libera.chat for
handling Pywikibot discussion, support, Phab/Gerrit updates, etc. I am
waiting for a bit more people to join us over on Libera before closing
down the Freenode room.
There is a new subpage on Meta-Wiki [4] for information on how to create
a new account for yourself on Libera.Chat and other related information.
There is also a tracking task [5] that you can look at to see various
activities that the community hopes to take action on to complete the
migration.
One last thing: The #wmhack Freenode channel is bridged to
#wikimedia-hackathon on Libera.Chat. The new channel name will make it
easier for the GCs to help manage spam and other issues that come up
occasionally on IRC. Don't miss the fun of our 2021 virtual hackathon
from Friday, May 21st to Sunday, May 23rd! [6]
[0]: https://www.kline.sh/
[1]: https://meta.wikimedia.org/wiki/IRC/Group_Contacts
[2]: https://libera.chat/
[3]: https://meta.wikimedia.org/w/index.php?diff=21476411
[4]: https://meta.wikimedia.org/wiki/IRC/Migrating_to_Libera_Chat
[5]: https://phabricator.wikimedia.org/T283247
[6]: https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2021
-- Legoktm
Hi folks,
a new stable release 5.0 will be deployed soon. There are some important points with this release:
- Support for MediaWiki < 1.19 has been dropped. Support for LTS 1.19 and above will be kept until fandom wikis were updated
- A lot of methods and classes which are deprecated for 5 years and more were removed. This includes all textlib functions which must be imported from textlib now and are no longer accessable directly from pywikibot module. The removal was announced previously by a FutureWarning. Please check your code, take attention to FutureWarning hints and refer the roadmap where other changed are announced.
- Deprecated script options were removed and are no longer functional.
Other changes:
- The -help option does show global options anymore. You get them via -help:global or by the pwb.py wrapper script pwb -help. By the way the pwb.py wrapper is able to ignore misspellings for script calls bot also can handle all global options even a user script does not call the handle_args() function e.g. to change the default lang code and family. Refer pwb.py (without parameters) to see the usage.
- APISite.recentchanges() does no longer accept positional arguments
- The new method Page.get_best_claim () retrieves the preferred Claim of a property referring to the given page
- BasePage.has_deleted_revisions() method was added deleted revs can be retrieved without the deletedhistory right
- Siteinfo.get() loads data via API instead from cache if expiry parameter is True
- Wikibase entities are loaded when necessary
- some circular imports are solved
refer https://github.com/wikimedia/pywikibot/blob/master/ROADMAP.rst for the full changelog after
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/630579 has been merged.
There are some outstanding bugfixes and improvements waiting for review at gerrit; see if you can help.
Best
xqt
Hi folks,
a new stable release 6.0 will be deployed soon. There are some important points with this release, currently implemented in master branch:
Breaking changes:
- Support for MediaWiki < 1.23 will been dropped after fandom wikis updated most platforms to MW 1.33. (T268979)
- comms.requests returns a requests.Response object instead of plain text. Normally this does not care because comms.requests result is passed through data.api anyway. But this changes enables to use a lot of requests functionality like exception handling, json conversion, encodings and we have access to headers response and others. Thanks to Mpaa who made a lot of commits to get the this done.
- config.db_hostname was renamed to db_hostname_format to take into account that replicas hosts got redesigned and may have different host names.
- APISite.loadimageinfo no longer returns any content; it just loads it to the instance cache.
- A lot of scripts will be moved to archive folder (T223826). They are still usable if you add 'scripts.archive' to user_script_paths variable in cuser-config.py but they aren't actively maintained. Please visit https://phabricator.wikimedia.org/T223826 and mark scripts you need. After moving to archive, just open a task to recover a script to be developed further. Currently archived scripts (featured.py and cfd.py) will be removed.
Other changes:
- Load time of Wikibase access to SiteLinks will be decreased a lot. Currently retrieving data from an ItemPage needs upto 6 minutes; after that change it might be only seconds. Using toJSON() method to a SiteLinkCollection will still need time to create the data dict. You may use a maintenance script "preload_pages.py" to optimize you bot in this special case.
Refer https://github.com/wikimedia/pywikibot/blob/master/ROADMAP.rst for the full changelog of the current release and https://github.com/wikimedia/pywikibot/blob/master/HISTORY.rst for all release changes.
A lot of thanks to all who have contributed new code or reviewed proposed changed. Also thanks for all reported bugs, improvement proposals and feature request. Thanks using Pywikibot.
Thanks for participating to "2020 Coolest Tool Award Winner": https://meta.wikimedia.org/wiki/Coolest_Tool_Award.
Thank you all
xqt
2009/4/9 Russell Blau <russblau(a)imapmail.org>
> I have created a table at
> http://www.botwiki.sno.cc/wiki/Rewrite/Porting_status to keep track of
> progress in porting files from the existing framework to the rewrite
> branch.
> So far it is quite incomplete (the first column is complete), but I intend
> to keep adding to it and encourage other developers to contribute. This
> can
> serve as a useful to-do list for those working on the project.
>
> Russ
>
Thanks for creating this list!
Should I understand this as "feel free to port any script to the new branch"
? =)
Converting scripts is not the hardest part, I could start working seriously
on this.
I am trying to split the scripts into categories, so that the scripts/
folder would
not be cluttered by 100+ files, like it is the case in the current
framework.
I'd like to have several categories, such as "image", "interwiki",
"references", to sort the scripts into subfolders. Any comments on this idea
are welcome: finding a "perfect" categorization doesn't look straightforward
:p
Also, I spent (again) some time looking at how to convert issues from the
sourceforge tracker
to another bug system, more specifically JIRA. It seems there is no
satisfying way
to convert those tickets at the moment, especially if we want to keep
information
such as the time when the ticket was opened, and last updated.
For months now we've been saying, periodically, "sometime we should leave
sourceforge".
I'm starting to think that now is the time: we have created a "rewrite"
category on sourceforge.
Wouldn't it be interesting to simply open tickets related to rewrite core &
rewrite-ported scripts on
the JIRA on the toolserver? ( https://jira.toolserver.org/browse/PYWP )
When porting scripts, some issues of course are still independant of
pywikibot implementation,
but not creating new pywikibot issues on sourceforge would be great.
--
Nicolas Dumazet — NicDumZ [ nɪk.d̪ymz ]
This is great! Thank you for all the hard work.
On Sat, Feb 12, 2022 at 7:34 AM <info(a)gno.de> wrote:
>
> Hi all,
>
> a new stable release Pywikibot 7 will be deployed at the end of this month.
> There are a lot important points with this release, currently implemented
> in
> master branch:
>
>
> Python Plattform:
> -----------------
>
> Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum
> plattform
> required is Python 3.5.3 which is preinstalled at toolforge. On the other
> hand
> Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3.
> Most
> used plattform measured by PyPy download statistic is Python 3.8
> (T266984). For
> developers I highly propose to use Python 3.10 due to "Better error
> messages"
> (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
>
>
> Framework Scripts:
> ------------------
>
> The both scripts generate_family_file and generate_user_files were moved
> from
> framework root directory to pywikibot/scripts folder. The two scripts
> shell and
> version were moved from scripts folder to pywikibot/scripts. This
> arrangement
> was necessary to implement a code entry point for Pywikibot site-package.
> Possibly you have to change the path environment setting to use these
> scripts.
> But all of them are still available through the pwb.py wrapper script.
>
>
> Linux:
> ------
>
> Scripts hash bang was changed from python to python3. This enables to
> invoke
> scripts without preleading interpreter.
>
>
> PAWS:
> -----
>
> Pywikibot 7 is a stable release and becomes the Pywikibot framework base on
> PAWS.
>
>
> Toolforge:
> ----------
>
> After deployment of Pywikibot 7 this stable release will be available under
> /shared/pywikibot/stable and the current master can be accessed under
> /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge
> and the similar script search is functional.
> https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
>
>
> MediaWiki:
> ----------
>
> No changes was made for supporting MediaWiki releases. Pywikibot 7 supports
> MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum
> Pywikibot
> version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4.
> (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
>
>
> Pywikibot site-package (PyPi):
> ------------------------------
>
> Installing Pywikibot as a site package (pip install pywikibot==7.0) comes
> with
> many innovations. pywikibot i18n messages bundle is available. Framework
> scripts listed above can be used with Pywikibot 7.The pwb wrapper script
> is the
> code entry point to invoke these scripts:
> pwb [global options] <framework script> [script options|global options]
>
>
>
> Pywikibot Tests:
> ----------------
>
> Pywikibot CI tests were moved from Travis to GitHub action. There are two
> actions implemented. The first runs up tp 20 workers in parallel and does
> the
> most tests whereas the second only runs login/logout tests but there are no
> parallel task which would lead to failures due to not logged-in test
> account.
>
>
> Module Changes:
> ---------------
>
> Support for API:Redirects was added in site and page module.
> User.is_locked()
> and APISite.is_locked() were added to determine whether a given user or
> user id
> is locked globally. The cached output functionality from compat release was
> re-implemented.
>
> families:
> Allow family files reside in `families` folder in base_dir by default.
> This is
> important if Pywikibot is used as a site-package. generate_family_file uses
> this new place to save the family file. Wikihow family file was added.
>
> i18n module:
> bundles() yields message bundle names or Path objects; known_languages()
> gives
> all languages we have localizations for.
>
> page module:
> isStaticRedirect() also considers transcluded __STATICREDIRECT__.
> put() and change_category() now has a show_diff parameter.
> User.is_blocked()
> can also detect range blocks.
>
> proofreadpage:
> Wikimedia OCR engine is supported.
>
> site module:
> APISite.get_globaluserinfo() method was added to retrieve globaluserinfo.
> New "maxlimit" property was added to determine the limits for API loads.
>
> textlib module:
> textlib comes with to_latin_digits() function as counterpart of
> to_local_digits() and case_escape() which gives an escaped regex pattern
> depending on 'first-letter' case.
>
> tools module:
> intersect_generators was rewritten which makes it running up to 10’000
> times
> faster.
>
>
> Scripts Changes:
> ----------------
>
> Scripts descriptions can be viewed at
> https://doc.wikimedia.org/pywikibot/master/scripts/index.html
>
> ConfigParserBot is provided for several scripts. Any option can be set
> within
> scripts.ini file.
>
> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
>
> add_text scripts provided -create and -createonly options. A CleanBot was
> added
> in category script which can be invoked by clean action option. It removes
> redundant grandchildren categories. The fixing_redirect script uses
> concurrent.futures to retrieve redirect targets; this decreases processing
> time
> by 90% if solving moved pages is enabled.
>
>
> Deprecations:
> -------------
>
> Most of the deprecated code parts were removed to start with a new proper
> and
> maintainable code without budensome overhead. The deprecations must be
> solved
> if you have your own code based on Pywikibot before you upgrade to the new
> release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html
> to see
> all changes. Run your scripts and focus on FutureWarnings about deprecated
> code
> parts. Refer the logs for deprecation warnings. Scripts must be run with
> global
> -log option; if your script does not use handle_args() function you can
> invoke
> it with the pwb wrapper script:
> pwb.py -log <yourscript> [your options]
>
>
>
> Thank you all, comments are welcome
>
> xqt
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
>
Hi Xqt,
I don't agree with the removal of the generators that someone decided to
deprecate [1]. We always had the principle that pagegenerators is the
point of entry to get pages to work on. The pagegenerators are a (thin)
layer of abstraction so that you don't have to directly use function in
the site objects. With this removal you're breaking one of the very
fundamental functions and principles of our framework.
Why did you remove it? Any arguments beside than the incorrect argument
that it's redundant?
Was the deprecation and removal discussed somewhere? Can't find it on
this list.
Maarten
[1]
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege…
On 12-02-2022 13:31, info(a)gno.de wrote:
>
> Hi all,
>
> a new stable release Pywikibot 7 will be deployed at the end of this
> month.
> There are a lot important points with this release, currently
> implemented in
> master branch:
>
>
> Python Plattform:
> -----------------
>
> Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum
> plattform
> required is Python 3.5.3 which is preinstalled at toolforge. On the
> other hand
> Pywikibot also works with new Python 3.11.0a and *new* supports PyPy
> 3. Most
> used plattform measured by PyPy download statistic is Python 3.8
> (T266984). For
> developers I highly propose to use Python 3.10 due to "Better error
> messages"
> (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
>
>
> Framework Scripts:
> ------------------
>
> The both scripts generate_family_file and generate_user_files were
> moved from
> framework root directory to pywikibot/scripts folder. The two scripts
> shell and
> version were moved from scripts folder to pywikibot/scripts. This
> arrangement
> was necessary to implement a code entry point for Pywikibot site-package.
> Possibly you have to change the path environment setting to use these
> scripts.
> But all of them are still available through the pwb.py wrapper script.
>
>
> Linux:
> ------
>
> Scripts hash bang was changed from python to python3. This enables to
> invoke
> scripts without preleading interpreter.
>
>
> PAWS:
> -----
>
> Pywikibot 7 is a stable release and becomes the Pywikibot framework
> base on
> PAWS.
>
>
> Toolforge:
> ----------
>
> After deployment of Pywikibot 7 this stable release will be available
> under
> /shared/pywikibot/stable and the current master can be accessed under
> /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge
> and the similar script search is functional.
> https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
>
>
> MediaWiki:
> ----------
>
> No changes was made for supporting MediaWiki releases. Pywikibot 7
> supports
> MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum
> Pywikibot
> version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4.
> (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
>
>
> Pywikibot site-package (PyPi):
> ------------------------------
>
> Installing Pywikibot as a site package (pip install pywikibot==7.0)
> comes with
> many innovations. pywikibot i18n messages bundle is available. Framework
> scripts listed above can be used with Pywikibot 7.The pwb wrapper
> script is the
> code entry point to invoke these scripts:
> pwb [global options] <framework script> [script options|global options]
>
>
>
> Pywikibot Tests:
> ----------------
>
> Pywikibot CI tests were moved from Travis to GitHub action. There are two
> actions implemented. The first runs up tp 20 workers in parallel and
> does the
> most tests whereas the second only runs login/logout tests but there
> are no
> parallel task which would lead to failures due to not logged-in test
> account.
>
>
> Module Changes:
> ---------------
>
> Support for API:Redirects was added in site and page module.
> User.is_locked()
> and APISite.is_locked() were added to determine whether a given user
> or user id
> is locked globally. The cached output functionality from compat
> release was
> re-implemented.
>
> families:
> Allow family files reside in `families` folder in base_dir by default.
> This is
> important if Pywikibot is used as a site-package. generate_family_file
> uses
> this new place to save the family file. Wikihow family file was added.
>
> i18n module:
> bundles() yields message bundle names or Path objects;
> known_languages() gives
> all languages we have localizations for.
>
> page module:
> isStaticRedirect() also considers transcluded __STATICREDIRECT__.
> put() and change_category() now has a show_diff parameter.
> User.is_blocked()
> can also detect range blocks.
>
> proofreadpage:
> Wikimedia OCR engine is supported.
>
> site module:
> APISite.get_globaluserinfo() method was added to retrieve globaluserinfo.
> New "maxlimit" property was added to determine the limits for API loads.
>
> textlib module:
> textlib comes with to_latin_digits() function as counterpart of
> to_local_digits() and case_escape() which gives an escaped regex pattern
> depending on 'first-letter' case.
>
> tools module:
> intersect_generators was rewritten which makes it running up to 10’000
> times
> faster.
>
>
> Scripts Changes:
> ----------------
>
> Scripts descriptions can be viewed at
> https://doc.wikimedia.org/pywikibot/master/scripts/index.html
>
> ConfigParserBot is provided for several scripts. Any option can be set
> within
> scripts.ini file.
> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
>
> add_text scripts provided -create and -createonly options. A CleanBot
> was added
> in category script which can be invoked by clean action option. It removes
> redundant grandchildren categories. The fixing_redirect script uses
> concurrent.futures to retrieve redirect targets; this decreases
> processing time
> by 90% if solving moved pages is enabled.
>
>
> Deprecations:
> -------------
>
> Most of the deprecated code parts were removed to start with a new
> proper and
> maintainable code without budensome overhead. The deprecations must be
> solved
> if you have your own code based on Pywikibot before you upgrade to the new
> release. Refer
> https://doc.wikimedia.org/pywikibot/master/changelog.html to see
> all changes. Run your scripts and focus on FutureWarnings about
> deprecated code
> parts. Refer the logs for deprecation warnings. Scripts must be run
> with global
> -log option; if your script does not use handle_args() function you
> can invoke
> it with the pwb wrapper script:
> pwb.py -log <yourscript> [your options]
>
>
>
> Thank you all, comments are welcome
>
> xqt
>
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
Hi Maarten,
thank you for you feedback. There is the same "redundancy" in some Page methods which calls the corresponding site method and I wouldn’t drop one over the other. Page methods have a higher tier than Site methods which calls api functions and finally the comms functions.
pagegenerators implements generators and filters for command line options and provides additional page generators which are outside of site methods like multiple site generators and other stuff.
But I don’t see any advantage using the same functionality from pagegenerators module over those site methods who are called by that function. The (now removed) functions had the form
def this_generator(total=None, site=None)
if not site: site = Pywikibot.Site()
site.this(total=total)
There is a disadvantage in that sense that site must not be determined and another disadvantage that most site parameters were not provided with the pg function.
Can you tell me your POV about advantage/disadvantage of those pg function?
There are about 50 generator functions in site.GeneratorMixin but pg had only a few of them.
I made a proposal in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would implement all site generators as pagegenerators function. Is this in your sense? Unfortunately this only works with Python 3.7+ and needs some additional work for older Pythons.
Best
xqt
> Am 13.02.2022 um 14:40 schrieb Maarten Dammers <maarten(a)mdammers.nl>:
>
>
> Hi Xqt,
>
> I don't agree with the removal of the generators that someone decided to deprecate [1]. We always had the principle that pagegenerators is the point of entry to get pages to work on. The pagegenerators are a (thin) layer of abstraction so that you don't have to directly use function in the site objects. With this removal you're breaking one of the very fundamental functions and principles of our framework.
>
> Why did you remove it? Any arguments beside than the incorrect argument that it's redundant?
> Was the deprecation and removal discussed somewhere? Can't find it on this list.
>
> Maarten
>
> [1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege…
>
>> On 12-02-2022 13:31, info(a)gno.de wrote:
>>
>> Hi all,
>>
>> a new stable release Pywikibot 7 will be deployed at the end of this month.
>> There are a lot important points with this release, currently implemented in
>> master branch:
>>
>>
>> Python Plattform:
>> -----------------
>>
>> Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum plattform
>> required is Python 3.5.3 which is preinstalled at toolforge. On the other hand
>> Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3. Most
>> used plattform measured by PyPy download statistic is Python 3.8 (T266984). For
>> developers I highly propose to use Python 3.10 due to "Better error messages"
>> (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
>>
>>
>> Framework Scripts:
>> ------------------
>>
>> The both scripts generate_family_file and generate_user_files were moved from
>> framework root directory to pywikibot/scripts folder. The two scripts shell and
>> version were moved from scripts folder to pywikibot/scripts. This arrangement
>> was necessary to implement a code entry point for Pywikibot site-package.
>> Possibly you have to change the path environment setting to use these scripts.
>> But all of them are still available through the pwb.py wrapper script.
>>
>>
>> Linux:
>> ------
>>
>> Scripts hash bang was changed from python to python3. This enables to invoke
>> scripts without preleading interpreter.
>>
>>
>> PAWS:
>> -----
>>
>> Pywikibot 7 is a stable release and becomes the Pywikibot framework base on
>> PAWS.
>>
>>
>> Toolforge:
>> ----------
>>
>> After deployment of Pywikibot 7 this stable release will be available under
>> /shared/pywikibot/stable and the current master can be accessed under
>> /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge
>> and the similar script search is functional.
>> https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
>>
>>
>> MediaWiki:
>> ----------
>>
>> No changes was made for supporting MediaWiki releases. Pywikibot 7 supports
>> MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum Pywikibot
>> version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4.
>> (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
>>
>>
>> Pywikibot site-package (PyPi):
>> ------------------------------
>>
>> Installing Pywikibot as a site package (pip install pywikibot==7.0) comes with
>> many innovations. pywikibot i18n messages bundle is available. Framework
>> scripts listed above can be used with Pywikibot 7.The pwb wrapper script is the
>> code entry point to invoke these scripts:
>> pwb [global options] <framework script> [script options|global options]
>>
>>
>>
>> Pywikibot Tests:
>> ----------------
>>
>> Pywikibot CI tests were moved from Travis to GitHub action. There are two
>> actions implemented. The first runs up tp 20 workers in parallel and does the
>> most tests whereas the second only runs login/logout tests but there are no
>> parallel task which would lead to failures due to not logged-in test account.
>>
>>
>> Module Changes:
>> ---------------
>>
>> Support for API:Redirects was added in site and page module. User.is_locked()
>> and APISite.is_locked() were added to determine whether a given user or user id
>> is locked globally. The cached output functionality from compat release was
>> re-implemented.
>>
>> families:
>> Allow family files reside in `families` folder in base_dir by default. This is
>> important if Pywikibot is used as a site-package. generate_family_file uses
>> this new place to save the family file. Wikihow family file was added.
>>
>> i18n module:
>> bundles() yields message bundle names or Path objects; known_languages() gives
>> all languages we have localizations for.
>>
>> page module:
>> isStaticRedirect() also considers transcluded __STATICREDIRECT__.
>> put() and change_category() now has a show_diff parameter. User.is_blocked()
>> can also detect range blocks.
>>
>> proofreadpage:
>> Wikimedia OCR engine is supported.
>>
>> site module:
>> APISite.get_globaluserinfo() method was added to retrieve globaluserinfo.
>> New "maxlimit" property was added to determine the limits for API loads.
>>
>> textlib module:
>> textlib comes with to_latin_digits() function as counterpart of
>> to_local_digits() and case_escape() which gives an escaped regex pattern
>> depending on 'first-letter' case.
>>
>> tools module:
>> intersect_generators was rewritten which makes it running up to 10’000 times
>> faster.
>>
>>
>> Scripts Changes:
>> ----------------
>>
>> Scripts descriptions can be viewed at
>> https://doc.wikimedia.org/pywikibot/master/scripts/index.html
>>
>> ConfigParserBot is provided for several scripts. Any option can be set within
>> scripts.ini file.
>> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
>>
>> add_text scripts provided -create and -createonly options. A CleanBot was added
>> in category script which can be invoked by clean action option. It removes
>> redundant grandchildren categories. The fixing_redirect script uses
>> concurrent.futures to retrieve redirect targets; this decreases processing time
>> by 90% if solving moved pages is enabled.
>>
>>
>> Deprecations:
>> -------------
>>
>> Most of the deprecated code parts were removed to start with a new proper and
>> maintainable code without budensome overhead. The deprecations must be solved
>> if you have your own code based on Pywikibot before you upgrade to the new
>> release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html to see
>> all changes. Run your scripts and focus on FutureWarnings about deprecated code
>> parts. Refer the logs for deprecation warnings. Scripts must be run with global
>> -log option; if your script does not use handle_args() function you can invoke
>> it with the pwb wrapper script:
>> pwb.py -log <yourscript> [your options]
>>
>>
>>
>> Thank you all, comments are welcome
>>
>> xqt
>>
>>
>> _______________________________________________
>> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
>> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
Seems this mail hasn’t passed.
xqt
> Am 19.02.2022 um 18:48
>
>
>
> Hi Maarten,
>
> I've restored all removed pagegenerators functions.
> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
> Other functions may follow possibly in such a way I examined already.
>
>
> > I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot.
>
> Structured Data was enabled with this patch
> https://gerrit.wikimedia.org/r/c/pywikibot/core/+/627243
> T223820 is still open. Not sure what is remainting to do there. But also "some horrible hacks to work around" would possibly help. Could you point it out?
>
> > Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly.
> I don't fully agree because a site can also be seen as a central point to work on. For example mwclient package is site oriented and Pywikibot looks more like a network: https://doc.wikimedia.org/pywikibot/master/index.html#framework-modules-ove… I think we can be flexible enough to provide different approaches. I am working on a site.Page() constructor method right now which is also able to upcast the correspondig BasePage class in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/763753
>
> > Downside is that we won't have any documentation for these generators at ...
> Must be defined in the rst file then.
>
> > Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ?
> Obviously this is appropriate, see https://phabricator.wikimedia.org/rPWBCd7719950d651277955ed804ad85fb729fcc5… as a sample.
>
> > Have we ever tried cross-referencing
> Yes we do that a lot, mostly automatically by type hints. But it can be improved anyway.
>
>
> Best
> xqt
> Von: Maarten Dammers <maarten(a)mdammers.nl>
> Gesendet: 14.02.2022 22:19
> An: <pywikibot(a)lists.wikimedia.org>
> Betreff: [pywikibot] Re: New Pywikibot 7 stable release
>
> Hi Xqt,
>
> On 13-02-2022 21:53, info(a)gno.de wrote:
>>
>> Hi Maarten,
>>
>> thank you for you feedback. There is the same "redundancy" in some Page methods which calls the corresponding site method and I wouldn’t drop one over the other. Page methods have a higher tier than Site methods which calls api functions and finally the comms functions.
> Exactly. I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot.
>
>>
>> pagegenerators implements generators and filters for command line options and provides additional page generators which are outside of site methods like multiple site generators and other stuff.
>>
>> But I don’t see any advantage using the same functionality from pagegenerators module over those site methods who are called by that function. The (now removed) functions had the form
>>
>> def this_generator(total=None, site=None)
>> if not site: site = Pywikibot.Site()
>> site.this(total=total)
>>
>> There is a disadvantage in that sense that site must not be determined and another disadvantage that most site parameters were not provided with the pg function.
>>
>> Can you tell me your POV about advantage/disadvantage of those pg function?
> Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly. The pagegenerators should invoke the needed code mostly from site, but maybe from other parts. I recognize the problem of the missing generators and missing parameters.
>>
>>
>> There are about 50 generator functions in site.GeneratorMixin but pg had only a few of them.
>>
>> I made a proposal in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would implement all site generators as pagegenerators function. Is this in your sense? Unfortunately this only works with Python 3.7+ and needs some additional work for older Pythons.
> Thanks for having a look at this. Interesting idea to create the generators like this. Would fix the example at https://www.mediawiki.org/wiki/Manual:Pywikibot/pagegenerators.py#Calls_fro… . Downside is that we won't have any documentation for these generators at https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#module-py… . Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ? Bit like we do in page at places like https://doc.wikimedia.org/pywikibot/master/_modules/pywikibot/page.html#Bas… . Does make me wonder how to link that with https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html#pywi… so we don't have to copy all the parameters. Each generator would be a minimal function with a short description and a link to the underlying function for detailed instructions. Have we ever tried cross-referencing as described at https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cr… ?
>
> Maarten
>
>>
>> Best
>> xqt
>>
>>>
>>> Am 13.02.2022 um 14:40 schrieb Maarten Dammers <maarten(a)mdammers.nl>:
>>>
>>> Hi Xqt,
>>>
>>> I don't agree with the removal of the generators that someone decided to deprecate [1]. We always had the principle that pagegenerators is the point of entry to get pages to work on. The pagegenerators are a (thin) layer of abstraction so that you don't have to directly use function in the site objects. With this removal you're breaking one of the very fundamental functions and principles of our framework.
>>>
>>> Why did you remove it? Any arguments beside than the incorrect argument that it's redundant?
>>> Was the deprecation and removal discussed somewhere? Can't find it on this list.
>>>
>>> Maarten
>>>
>>> [1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege…
>>>
>>> On 12-02-2022 13:31, info(a)gno.de wrote:
>
> Hi all,
>
> a new stable release Pywikibot 7 will be deployed at the end of this month.
> There are a lot important points with this release, currently implemented in
> master branch:
>
>
> Python Plattform:
> -----------------
>
> Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum plattform
> required is Python 3.5.3 which is preinstalled at toolforge. On the other hand
> Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3. Most
> used plattform measured by PyPy download statistic is Python 3.8 (T266984). For
> developers I highly propose to use Python 3.10 due to "Better error messages"
> (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
>
>
> Framework Scripts:
> ------------------
>
> The both scripts generate_family_file and generate_user_files were moved from
> framework root directory to pywikibot/scripts folder. The two scripts shell and
> version were moved from scripts folder to pywikibot/scripts. This arrangement
> was necessary to implement a code entry point for Pywikibot site-package.
> Possibly you have to change the path environment setting to use these scripts.
> But all of them are still available through the pwb.py wrapper script.
>
>
> Linux:
> ------
>
> Scripts hash bang was changed from python to python3. This enables to invoke
> scripts without preleading interpreter.
>
>
> PAWS:
> -----
>
> Pywikibot 7 is a stable release and becomes the Pywikibot framework base on
> PAWS.
>
>
> Toolforge:
> ----------
>
> After deployment of Pywikibot 7 this stable release will be available under
> /shared/pywikibot/stable and the current master can be accessed under
> /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge
> and the similar script search is functional.
> https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
>
>
> MediaWiki:
> ----------
>
> No changes was made for supporting MediaWiki releases. Pywikibot 7 supports
> MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum Pywikibot
> version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4.
> (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
>
>
> Pywikibot site-package (PyPi):
> ------------------------------
>
> Installing Pywikibot as a site package (pip install pywikibot==7.0) comes with
> many innovations. pywikibot i18n messages bundle is available. Framework
> scripts listed above can be used with Pywikibot 7.The pwb wrapper script is the
> code entry point to invoke these scripts:
> pwb [global options] <framework script> [script options|global options]
>
>
>
> Pywikibot Tests:
> ----------------
>
> Pywikibot CI tests were moved from Travis to GitHub action. There are two
> actions implemented. The first runs up tp 20 workers in parallel and does the
> most tests whereas the second only runs login/logout tests but there are no
> parallel task which would lead to failures due to not logged-in test account.
>
>
> Module Changes:
> ---------------
>
> Support for API:Redirects was added in site and page module. User.is_locked()
> and APISite.is_locked() were added to determine whether a given user or user id
> is locked globally. The cached output functionality from compat release was
> re-implemented.
>
> families:
> Allow family files reside in `families` folder in base_dir by default. This is
> important if Pywikibot is used as a site-package. generate_family_file uses
> this new place to save the family file. Wikihow family file was added.
>
> i18n module:
> bundles() yields message bundle names or Path objects; known_languages() gives
> all languages we have localizations for.
>
> page module:
> isStaticRedirect() also considers transcluded __STATICREDIRECT__.
> put() and change_category() now has a show_diff parameter. User.is_blocked()
> can also detect range blocks.
>
> proofreadpage:
> Wikimedia OCR engine is supported.
>
> site module:
> APISite.get_globaluserinfo() method was added to retrieve globaluserinfo.
> New "maxlimit" property was added to determine the limits for API loads.
>
> textlib module:
> textlib comes with to_latin_digits() function as counterpart of
> to_local_digits() and case_escape() which gives an escaped regex pattern
> depending on 'first-letter' case.
>
> tools module:
> intersect_generators was rewritten which makes it running up to 10’000 times
> faster.
>
>
> Scripts Changes:
> ----------------
>
> Scripts descriptions can be viewed at
> https://doc.wikimedia.org/pywikibot/master/scripts/index.html
>
> ConfigParserBot is provided for several scripts. Any option can be set within
> scripts.ini file.
> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
>
> add_text scripts provided -create and -createonly options. A CleanBot was added
> in category script which can be invoked by clean action option. It removes
> redundant grandchildren categories. The fixing_redirect script uses
> concurrent.futures to retrieve redirect targets; this decreases processing time
> by 90% if solving moved pages is enabled.
>
>
> Deprecations:
> -------------
>
> Most of the deprecated code parts were removed to start with a new proper and
> maintainable code without budensome overhead. The deprecations must be solved
> if you have your own code based on Pywikibot before you upgrade to the new
> release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html to see
> all changes. Run your scripts and focus on FutureWarnings about deprecated code
> parts. Refer the logs for deprecation warnings. Scripts must be run with global
> -log option; if your script does not use handle_args() function you can invoke
> it with the pwb wrapper script:
> pwb.py -log <yourscript> [your options]
>
>
>
> Thank you all, comments are welcome
>
> xqt
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
>
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org