Hi Xqt,
On 13-02-2022 21:53, info(a)gno.de wrote:
Hi Maarten,
thank you for you feedback. There is the same "redundancy" in some
Page methods which calls the corresponding site method and I wouldn’t
drop one over the other. Page methods have a higher tier than Site
methods which calls api functions and finally the comms functions.
Exactly. I think I'm only using the site object directly in some
horrible hacks to work around the lack of structured data on Commons
support in Pywikibot.
pagegenerators implements generators and filters for command line
options and provides additional page generators which are outside of
site methods like multiple site generators and other stuff.
But I don’t see any advantage using the same functionality from
pagegenerators module over those site methods who are called by that
function. The (now removed) functions had the form
def this_generator(total=None, site=None)
if not site: site = Pywikibot.Site()
site.this(total=total)
There is a disadvantage in that sense that site must not be determined
and another disadvantage that most site parameters were not provided
with the pg function.
Can you tell me your POV about advantage/disadvantage of those pg
function?
Just like page being the central point of entry from doing things with
pages, I see the pagegenerators as the central point of entry to getting
a set of pages. Site functions shouldn't be used directly. The
pagegenerators should invoke the needed code mostly from site, but maybe
from other parts. I recognize the problem of the missing generators and
missing parameters.
There are about 50 generator functions in site.GeneratorMixin but pg
had only a few of them.
I made a proposal in
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would
implement all site generators as pagegenerators function. Is this in
your sense? Unfortunately this only works with Python 3.7+ and needs
some additional work for older Pythons.
Thanks for having a look at this. Interesting idea to create the
generators like this. Would fix the example at
https://www.mediawiki.org/wiki/Manual:Pywikibot/pagegenerators.py#Calls_fro…
. Downside is that we won't have any documentation for these generators
at
https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#module-py…
. Maybe middle ground to have a function that gives all the parameters
to the site function using **kwargs ? Bit like we do in page at places
like
https://doc.wikimedia.org/pywikibot/master/_modules/pywikibot/page.html#Bas…
. Does make me wonder how to link that with
https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html#pywi…
so we don't have to copy all the parameters. Each generator would be a
minimal function with a short description and a link to the underlying
function for detailed instructions. Have we ever tried cross-referencing
as described at
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cr…
?
Maarten
Best
xqt
Am 13.02.2022 um 14:40 schrieb Maarten Dammers
<maarten(a)mdammers.nl>nl>:
Hi Xqt,
I don't agree with the removal of the generators that someone decided
to deprecate [1]. We always had the principle that pagegenerators is
the point of entry to get pages to work on. The pagegenerators are a
(thin) layer of abstraction so that you don't have to directly use
function in the site objects. With this removal you're breaking one
of the very fundamental functions and principles of our framework.
Why did you remove it? Any arguments beside than the incorrect
argument that it's redundant?
Was the deprecation and removal discussed somewhere? Can't find it on
this list.
Maarten
[1]
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege…
On 12-02-2022 13:31, info(a)gno.de wrote:
Hi all,
a new stable release Pywikibot 7 will be deployed at the end of this
month.
There are a lot important points with this release, currently
implemented in
master branch:
Python Plattform:
-----------------
Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The
minimum plattform
required is Python 3.5.3 which is preinstalled at toolforge. On the
other hand
Pywikibot also works with new Python 3.11.0a and *new* supports PyPy
3. Most
used plattform measured by PyPy download statistic is Python 3.8
(T266984). For
developers I highly propose to use Python 3.10 due to "Better error
messages"
(
https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages).
Framework Scripts:
------------------
The both scripts generate_family_file and generate_user_files were
moved from
framework root directory to pywikibot/scripts folder. The two
scripts shell and
version were moved from scripts folder to pywikibot/scripts. This
arrangement
was necessary to implement a code entry point for Pywikibot
site-package.
Possibly you have to change the path environment setting to use
these scripts.
But all of them are still available through the pwb.py wrapper script.
Linux:
------
Scripts hash bang was changed from python to python3. This enables
to invoke
scripts without preleading interpreter.
PAWS:
-----
Pywikibot 7 is a stable release and becomes the Pywikibot framework
base on
PAWS.
Toolforge:
----------
After deployment of Pywikibot 7 this stable release will be
available under
/shared/pywikibot/stable and the current master can be accessed under
/shared/pywikibot/core. The pwb.py wrapper script can be used at
Tooforge
and the similar script search is functional.
https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb
MediaWiki:
----------
No changes was made for supporting MediaWiki releases. Pywikibot 7
supports
MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum
Pywikibot
version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires
PWB 4.
(
https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility)
Pywikibot site-package (PyPi):
------------------------------
Installing Pywikibot as a site package (pip install pywikibot==7.0)
comes with
many innovations. pywikibot i18n messages bundle is available.
Framework
scripts listed above can be used with Pywikibot 7.The pwb wrapper
script is the
code entry point to invoke these scripts:
pwb [global options] <framework script> [script options|global options]
Pywikibot Tests:
----------------
Pywikibot CI tests were moved from Travis to GitHub action. There
are two
actions implemented. The first runs up tp 20 workers in parallel and
does the
most tests whereas the second only runs login/logout tests but there
are no
parallel task which would lead to failures due to not logged-in test
account.
Module Changes:
---------------
Support for API:Redirects was added in site and page module.
User.is_locked()
and APISite.is_locked() were added to determine whether a given user
or user id
is locked globally. The cached output functionality from compat
release was
re-implemented.
families:
Allow family files reside in `families` folder in base_dir by
default. This is
important if Pywikibot is used as a site-package.
generate_family_file uses
this new place to save the family file. Wikihow family file was added.
i18n module:
bundles() yields message bundle names or Path objects;
known_languages() gives
all languages we have localizations for.
page module:
isStaticRedirect() also considers transcluded __STATICREDIRECT__.
put() and change_category() now has a show_diff parameter.
User.is_blocked()
can also detect range blocks.
proofreadpage:
Wikimedia OCR engine is supported.
site module:
APISite.get_globaluserinfo() method was added to retrieve
globaluserinfo.
New "maxlimit" property was added to determine the limits for API loads.
textlib module:
textlib comes with to_latin_digits() function as counterpart of
to_local_digits() and case_escape() which gives an escaped regex pattern
depending on 'first-letter' case.
tools module:
intersect_generators was rewritten which makes it running up to
10’000 times
faster.
Scripts Changes:
----------------
Scripts descriptions can be viewed at
https://doc.wikimedia.org/pywikibot/master/scripts/index.html
ConfigParserBot is provided for several scripts. Any option can be
set within
scripts.ini file.
https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot…
add_text scripts provided -create and -createonly options. A
CleanBot was added
in category script which can be invoked by clean action option. It
removes
redundant grandchildren categories. The fixing_redirect script uses
concurrent.futures to retrieve redirect targets; this decreases
processing time
by 90% if solving moved pages is enabled.
Deprecations:
-------------
Most of the deprecated code parts were removed to start with a new
proper and
maintainable code without budensome overhead. The deprecations must
be solved
if you have your own code based on Pywikibot before you upgrade to
the new
release. Refer
https://doc.wikimedia.org/pywikibot/master/changelog.html to see
all changes. Run your scripts and focus on FutureWarnings about
deprecated code
parts. Refer the logs for deprecation warnings. Scripts must be run
with global
-log option; if your script does not use handle_args() function you
can invoke
it with the pwb wrapper script:
pwb.py -log <yourscript> [your options]
Thank you all, comments are welcome
xqt
_______________________________________________
pywikibot mailing list --pywikibot(a)lists.wikimedia.org
To unsubscribe send an email topywikibot-leave(a)lists.wikimedia.org
_______________________________________________
pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org
_______________________________________________
pywikibot mailing list -- pywikibot(a)lists.wikimedia.org
To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org