Hi Xqt,
On 13-02-2022 21:53, info(a)gno.de wrote:
>
> Hi Maarten,
>
> thank you for you feedback. There is the same "redundancy" in some
> Page methods which calls the corresponding site method and I wouldn’t
> drop one over the other. Page methods have a higher tier than Site
> methods which calls api functions and finally the comms functions.
Exactly. I think I'm only using the site object directly in some
horrible hacks to work around the lack of structured data on Commons
support in Pywikibot.
>
> pagegenerators implements generators and filters for command line
> options and provides additional page generators which are outside of
> site methods like multiple site generators and other stuff.
>
> But I don’t see any advantage using the same functionality from
> pagegenerators module over those site methods who are called by that
> function. The (now removed) functions had the form
>
> def this_generator(total=None, site=None)
> if not site: site = Pywikibot.Site()
> site.this(total=total)
>
> There is a disadvantage in that sense that site must not be determined
> and another disadvantage that most site parameters were not provided
> with the pg function.
>
> Can you tell me your POV about advantage/disadvantage of those pg
> function?
Just like page being the central point of entry from doing things with
pages, I see the pagegenerators as the central point of entry to getting
a set of pages. Site functions shouldn't be used directly. The
pagegenerators should invoke the needed code mostly from site, but maybe
from other parts. I recognize the problem of the missing generators and
missing parameters.
>
> There are about 50 generator functions in site.GeneratorMixin but pg
> had only a few of them.
>
> I made a proposal in
> https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would
> implement all site generators as pagegenerators function. Is this in
> your sense? Unfortunately this only works with Python 3.7+ and needs
> some additional work for older Pythons.
Thanks for having a look at this. Interesting idea to create the
generators like this. Would fix the example at
https://www.mediawiki.org/wiki/Manual:Pywikibot/pagegenerators.py#Calls_fro…
. Downside is that we won't have any documentation for these generators
at
https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#module-py…
. Maybe middle ground to have a function that gives all the parameters
to the site function using **kwargs ? Bit like we do in page at places
like
https://doc.wikimedia.org/pywikibot/master/_modules/pywikibot/page.html#Bas…
. Does make me wonder how to link that with
https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html#pywi…
so we don't have to copy all the parameters. Each generator would be a
minimal function with a short description and a link to the underlying
function for detailed instructions. Have we ever tried cross-referencing
as described at
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cr…
?
Maarten
>
> Best
> xqt
>
>> Am 13.02.2022 um 14:40 schrieb Maarten Dammers <maarten(a)mdammers.nl>:
>>
>>
>>
>> Hi Xqt,
>>
>> I don't agree with the removal of the generators that someone decided
>> to deprecate [1]. We always had the principle that pagegenerators is
>> the point of entry to get pages to work on. The pagegenerators are a
>> (thin) layer of abstraction so that you don't have to directly use
>> function in the site objects. With this removal you're breaking one
>> of the very fundamental functions and principles of our framework.
>>
>> Why did you remove it? Any arguments beside than the incorrect
>> argument that it's redundant?
>> Was the deprecation and removal discussed somewhere? Can't find it on
>> this list.
>>
>> Maarten
>>
>> [1]
>> https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege…
>>
>> On 12-02-2022 13:31, info(a)gno.de wrote:
>>>
>>> Hi all,
>>>
>>> a new stable release Pywikibot 7 will be deployed at the end of this
>>> month.
>>> There are a lot important points with this release, currently
>>> implemented in
>>> master branch:
>>>
>>>
>>> Python Plattform:
>>> -----------------
>>>
>>> Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The
>>> minimum plattform
>>> required is Python 3.5.3 which is preinstalled at toolforge. On the
>>> other hand
>>> Pywikibot also works with new Python 3.11.0a and *new* supports PyPy
>>> 3. Most
>>> used plattform measured by PyPy download statistic is Python 3.8
>>> (T266984). For
>>> developers I highly propose to use Python 3.10 due to "Better error
>>> messages"
>>> (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
>>>
>>>
>>> Framework Scripts:
>>> ------------------
>>>
>>> The both scripts generate_family_file and generate_user_files were
>>> moved from
>>> framework root directory to pywikibot/scripts folder. The two
>>> scripts shell and
>>> version were moved from scripts folder to pywikibot/scripts. This
>>> arrangement
>>> was necessary to implement a code entry point for Pywikibot
>>> site-package.
>>> Possibly you have to change the path environment setting to use
>>> these scripts.
>>> But all of them are still available through the pwb.py wrapper script.
>>>
>>>
>>> Linux:
>>> ------
>>>
>>> Scripts hash bang was changed from python to python3. This enables
>>> to invoke
>>> scripts without preleading interpreter.
>>>
>>>
>>> PAWS:
>>> -----
>>>
>>> Pywikibot 7 is a stable release and becomes the Pywikibot framework
>>> base on
>>> PAWS.
>>>
>>>
>>> Toolforge:
>>> ----------
>>>
>>> After deployment of Pywikibot 7 this stable release will be
>>> available under
>>> /shared/pywikibot/stable and the current master can be accessed under
>>> /shared/pywikibot/core. The pwb.py wrapper script can be used at
>>> Tooforge
>>> and the similar script search is functional.
>>> https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
>>>
>>>
>>> MediaWiki:
>>> ----------
>>>
>>> No changes was made for supporting MediaWiki releases. Pywikibot 7
>>> supports
>>> MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum
>>> Pywikibot
>>> version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires
>>> PWB 4.
>>> (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
>>>
>>>
>>> Pywikibot site-package (PyPi):
>>> ------------------------------
>>>
>>> Installing Pywikibot as a site package (pip install pywikibot==7.0)
>>> comes with
>>> many innovations. pywikibot i18n messages bundle is available.
>>> Framework
>>> scripts listed above can be used with Pywikibot 7.The pwb wrapper
>>> script is the
>>> code entry point to invoke these scripts:
>>> pwb [global options] <framework script> [script options|global options]
>>>
>>>
>>>
>>> Pywikibot Tests:
>>> ----------------
>>>
>>> Pywikibot CI tests were moved from Travis to GitHub action. There
>>> are two
>>> actions implemented. The first runs up tp 20 workers in parallel and
>>> does the
>>> most tests whereas the second only runs login/logout tests but there
>>> are no
>>> parallel task which would lead to failures due to not logged-in test
>>> account.
>>>
>>>
>>> Module Changes:
>>> ---------------
>>>
>>> Support for API:Redirects was added in site and page module.
>>> User.is_locked()
>>> and APISite.is_locked() were added to determine whether a given user
>>> or user id
>>> is locked globally. The cached output functionality from compat
>>> release was
>>> re-implemented.
>>>
>>> families:
>>> Allow family files reside in `families` folder in base_dir by
>>> default. This is
>>> important if Pywikibot is used as a site-package.
>>> generate_family_file uses
>>> this new place to save the family file. Wikihow family file was added.
>>>
>>> i18n module:
>>> bundles() yields message bundle names or Path objects;
>>> known_languages() gives
>>> all languages we have localizations for.
>>>
>>> page module:
>>> isStaticRedirect() also considers transcluded __STATICREDIRECT__.
>>> put() and change_category() now has a show_diff parameter.
>>> User.is_blocked()
>>> can also detect range blocks.
>>>
>>> proofreadpage:
>>> Wikimedia OCR engine is supported.
>>>
>>> site module:
>>> APISite.get_globaluserinfo() method was added to retrieve
>>> globaluserinfo.
>>> New "maxlimit" property was added to determine the limits for API loads.
>>>
>>> textlib module:
>>> textlib comes with to_latin_digits() function as counterpart of
>>> to_local_digits() and case_escape() which gives an escaped regex pattern
>>> depending on 'first-letter' case.
>>>
>>> tools module:
>>> intersect_generators was rewritten which makes it running up to
>>> 10’000 times
>>> faster.
>>>
>>>
>>> Scripts Changes:
>>> ----------------
>>>
>>> Scripts descriptions can be viewed at
>>> https://doc.wikimedia.org/pywikibot/master/scripts/index.html
>>>
>>> ConfigParserBot is provided for several scripts. Any option can be
>>> set within
>>> scripts.ini file.
>>> https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
>>>
>>> add_text scripts provided -create and -createonly options. A
>>> CleanBot was added
>>> in category script which can be invoked by clean action option. It
>>> removes
>>> redundant grandchildren categories. The fixing_redirect script uses
>>> concurrent.futures to retrieve redirect targets; this decreases
>>> processing time
>>> by 90% if solving moved pages is enabled.
>>>
>>>
>>> Deprecations:
>>> -------------
>>>
>>> Most of the deprecated code parts were removed to start with a new
>>> proper and
>>> maintainable code without budensome overhead. The deprecations must
>>> be solved
>>> if you have your own code based on Pywikibot before you upgrade to
>>> the new
>>> release. Refer
>>> https://doc.wikimedia.org/pywikibot/master/changelog.html to see
>>> all changes. Run your scripts and focus on FutureWarnings about
>>> deprecated code
>>> parts. Refer the logs for deprecation warnings. Scripts must be run
>>> with global
>>> -log option; if your script does not use handle_args() function you
>>> can invoke
>>> it with the pwb wrapper script:
>>> pwb.py -log <yourscript> [your options]
>>>
>>>
>>>
>>> Thank you all, comments are welcome
>>>
>>> xqt
>>>
>>> _______________________________________________
>>> pywikibot mailing list --pywikibot(a)lists.wikimedia.org
>>> To unsubscribe send an email topywikibot-leave(a)lists.wikimedia.org
>> _______________________________________________
>> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
>> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
>
> _______________________________________________
> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org