pywikibot February 2022

pywikibot@lists.wikimedia.org

8 participants
8 discussions

python pwb.py replace.py not working
by Basudev Krishna 11 Jan '23

11 Jan '23

Traceback (most recent call last): File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 399, in <module> if not main(): File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 391, in main run_python_file(filename, File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 106, in run_python_file exec(compile(source, filename, 'exec', dont_inherit=True), File "./scripts/replace.py", line 1075, in <module> main() File "./scripts/replace.py", line 929, in main single_summary = i18n.twtranslate( File "/data/data/com.termux/files/home/vikaspy/pywikibot/tools/_deprecate.py", line 404, in wrapper return obj(*__args, **__kw) File "/data/data/com.termux/files/home/vikaspy/pywikibot/i18n.py", line 700, in twtranslate raise pywikibot.exceptions.TranslationError( pywikibot.exceptions.TranslationError: Unable to load messages package scripts.i18n for bundle replace-replacing It can happen due to lack of i18n submodule or files. See https://www.mediawiki.org/wiki/Manual:Pywikibot/i18n CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.TranslationError'> $

3 8

i18n files and pip
by André Costa 08 Sep '22

08 Sep '22

Hi all, I'm building a small pywikibot tool[1] which is designed to be installed via pip (and in turn installs Pywikibot via pip). The tool uses the page.touch() function which is where I get a pywikibot.i18n.TranslationError when I run it. page.touch() gets it's edit summary from i18n.twtranslate(self.site, 'pywikibot-touch') which in turn is defined in /scripts/i18n/pywikibot/. Unless I'm confused the Error occurs because the pip distribution does not include the /scripts folder or the i18n submodule. So my first question is am I just doing something obviously wrong and the i18n submodule should have been available over pip as well? If it's not just me then would it not make sense to have any i18n files necessary to the Pywikibot *library* to also be distributed via the same pip package? (i18n for scripts is another issue since for scripts you cannot use pip). Cheers, André / Lokal_Profil [1] https://github.com/lokal-profil/pywikibot-sdc André Costa | Chief Operating Officer, Wikimedia Sverige | Andre.Costa(a)wikimedia.se | +46 (0)733-964574 Stöd fri kunskap, bli medlem i Wikimedia Sverige. Läs mer på blimedlem.wikimedia.se ------ sent from my mobile, all typos are due to autocorrect ;)

8 19

Contributing to the pywikibot project for the Google Summer of Code Programme
by Fortunate Omoleye 28 Feb '22

28 Feb '22

Hey Folks, How are you doing today and I hope you are safe and well wherever you are. My name is Omoleye Julius Fortunate and I'm a year three statistics student of the federal University of Agriculture Abeokuta Ogun State Nigeria. I would like to contribute to the Wikimedia pywikibot project for the 2022 google summer of code programme , and would really love to get an headstart on how to get started. I have skills using the Python Django framework, weak machine learning skills , also some basic site reliability Engineering skills too. I would be expecting a positive response from anyone that can help soon. Thank you for your time. Omoleye Julius

1 0

NotImplementedError: page_from_repository method
by Ugo Chima 28 Feb '22

28 Feb '22

Good morning everyone Please anyone have any Idea why I receive this error "*NotImplementedError: page_from_repository method is not implemented for Wikibase **site:lang" * when running the *newitem.py* script? Chima.

2 7

Re: New Pywikibot 7 stable release
by info＠gno.de 24 Feb '22

24 Feb '22

Seems this mail hasn’t passed. xqt > Am 19.02.2022 um 18:48 > > > > Hi Maarten, > > I've restored all removed pagegenerators functions. > https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot… > Other functions may follow possibly in such a way I examined already. > > > > I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot. > > Structured Data was enabled with this patch > https://gerrit.wikimedia.org/r/c/pywikibot/core/+/627243 > T223820 is still open. Not sure what is remainting to do there. But also "some horrible hacks to work around" would possibly help. Could you point it out? > > > Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly. > I don't fully agree because a site can also be seen as a central point to work on. For example mwclient package is site oriented and Pywikibot looks more like a network: https://doc.wikimedia.org/pywikibot/master/index.html#framework-modules-ove… I think we can be flexible enough to provide different approaches. I am working on a site.Page() constructor method right now which is also able to upcast the correspondig BasePage class in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/763753 > > > Downside is that we won't have any documentation for these generators at ... > Must be defined in the rst file then. > > > Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ? > Obviously this is appropriate, see https://phabricator.wikimedia.org/rPWBCd7719950d651277955ed804ad85fb729fcc5… as a sample. > > > Have we ever tried cross-referencing > Yes we do that a lot, mostly automatically by type hints. But it can be improved anyway. > > > Best > xqt > Von: Maarten Dammers <maarten(a)mdammers.nl> > Gesendet: 14.02.2022 22:19 > An: <pywikibot(a)lists.wikimedia.org> > Betreff: [pywikibot] Re: New Pywikibot 7 stable release > > Hi Xqt, > > On 13-02-2022 21:53, info(a)gno.de wrote: >> >> Hi Maarten, >> >> thank you for you feedback. There is the same "redundancy" in some Page methods which calls the corresponding site method and I wouldn’t drop one over the other. Page methods have a higher tier than Site methods which calls api functions and finally the comms functions. > Exactly. I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot. > >> >> pagegenerators implements generators and filters for command line options and provides additional page generators which are outside of site methods like multiple site generators and other stuff. >> >> But I don’t see any advantage using the same functionality from pagegenerators module over those site methods who are called by that function. The (now removed) functions had the form >> >> def this_generator(total=None, site=None) >> if not site: site = Pywikibot.Site() >> site.this(total=total) >> >> There is a disadvantage in that sense that site must not be determined and another disadvantage that most site parameters were not provided with the pg function. >> >> Can you tell me your POV about advantage/disadvantage of those pg function? > Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly. The pagegenerators should invoke the needed code mostly from site, but maybe from other parts. I recognize the problem of the missing generators and missing parameters. >> >> >> There are about 50 generator functions in site.GeneratorMixin but pg had only a few of them. >> >> I made a proposal in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would implement all site generators as pagegenerators function. Is this in your sense? Unfortunately this only works with Python 3.7+ and needs some additional work for older Pythons. > Thanks for having a look at this. Interesting idea to create the generators like this. Would fix the example at https://www.mediawiki.org/wiki/Manual:Pywikibot/pagegenerators.py#Calls_fro… . Downside is that we won't have any documentation for these generators at https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#module-py… . Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ? Bit like we do in page at places like https://doc.wikimedia.org/pywikibot/master/_modules/pywikibot/page.html#Bas… . Does make me wonder how to link that with https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html#pywi… so we don't have to copy all the parameters. Each generator would be a minimal function with a short description and a link to the underlying function for detailed instructions. Have we ever tried cross-referencing as described at https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cr… ? > > Maarten > >> >> Best >> xqt >> >>> >>> Am 13.02.2022 um 14:40 schrieb Maarten Dammers <maarten(a)mdammers.nl>: >>> >>> Hi Xqt, >>> >>> I don't agree with the removal of the generators that someone decided to deprecate [1]. We always had the principle that pagegenerators is the point of entry to get pages to work on. The pagegenerators are a (thin) layer of abstraction so that you don't have to directly use function in the site objects. With this removal you're breaking one of the very fundamental functions and principles of our framework. >>> >>> Why did you remove it? Any arguments beside than the incorrect argument that it's redundant? >>> Was the deprecation and removal discussed somewhere? Can't find it on this list. >>> >>> Maarten >>> >>> [1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege… >>> >>> On 12-02-2022 13:31, info(a)gno.de wrote: > > Hi all, > > a new stable release Pywikibot 7 will be deployed at the end of this month. > There are a lot important points with this release, currently implemented in > master branch: > > > Python Plattform: > ----------------- > > Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum plattform > required is Python 3.5.3 which is preinstalled at toolforge. On the other hand > Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3. Most > used plattform measured by PyPy download statistic is Python 3.8 (T266984). For > developers I highly propose to use Python 3.10 due to "Better error messages" > (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages). > > > Framework Scripts: > ------------------ > > The both scripts generate_family_file and generate_user_files were moved from > framework root directory to pywikibot/scripts folder. The two scripts shell and > version were moved from scripts folder to pywikibot/scripts. This arrangement > was necessary to implement a code entry point for Pywikibot site-package. > Possibly you have to change the path environment setting to use these scripts. > But all of them are still available through the pwb.py wrapper script. > > > Linux: > ------ > > Scripts hash bang was changed from python to python3. This enables to invoke > scripts without preleading interpreter. > > > PAWS: > ----- > > Pywikibot 7 is a stable release and becomes the Pywikibot framework base on > PAWS. > > > Toolforge: > ---------- > > After deployment of Pywikibot 7 this stable release will be available under > /shared/pywikibot/stable and the current master can be accessed under > /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge > and the similar script search is functional. > https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb > > > MediaWiki: > ---------- > > No changes was made for supporting MediaWiki releases. Pywikibot 7 supports > MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum Pywikibot > version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4. > (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility) > > > Pywikibot site-package (PyPi): > ------------------------------ > > Installing Pywikibot as a site package (pip install pywikibot==7.0) comes with > many innovations. pywikibot i18n messages bundle is available. Framework > scripts listed above can be used with Pywikibot 7.The pwb wrapper script is the > code entry point to invoke these scripts: > pwb [global options] <framework script> [script options|global options] > > > > Pywikibot Tests: > ---------------- > > Pywikibot CI tests were moved from Travis to GitHub action. There are two > actions implemented. The first runs up tp 20 workers in parallel and does the > most tests whereas the second only runs login/logout tests but there are no > parallel task which would lead to failures due to not logged-in test account. > > > Module Changes: > --------------- > > Support for API:Redirects was added in site and page module. User.is_locked() > and APISite.is_locked() were added to determine whether a given user or user id > is locked globally. The cached output functionality from compat release was > re-implemented. > > families: > Allow family files reside in `families` folder in base_dir by default. This is > important if Pywikibot is used as a site-package. generate_family_file uses > this new place to save the family file. Wikihow family file was added. > > i18n module: > bundles() yields message bundle names or Path objects; known_languages() gives > all languages we have localizations for. > > page module: > isStaticRedirect() also considers transcluded __STATICREDIRECT__. > put() and change_category() now has a show_diff parameter. User.is_blocked() > can also detect range blocks. > > proofreadpage: > Wikimedia OCR engine is supported. > > site module: > APISite.get_globaluserinfo() method was added to retrieve globaluserinfo. > New "maxlimit" property was added to determine the limits for API loads. > > textlib module: > textlib comes with to_latin_digits() function as counterpart of > to_local_digits() and case_escape() which gives an escaped regex pattern > depending on 'first-letter' case. > > tools module: > intersect_generators was rewritten which makes it running up to 10’000 times > faster. > > > Scripts Changes: > ---------------- > > Scripts descriptions can be viewed at > https://doc.wikimedia.org/pywikibot/master/scripts/index.html > > ConfigParserBot is provided for several scripts. Any option can be set within > scripts.ini file. > https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot… > > add_text scripts provided -create and -createonly options. A CleanBot was added > in category script which can be invoked by clean action option. It removes > redundant grandchildren categories. The fixing_redirect script uses > concurrent.futures to retrieve redirect targets; this decreases processing time > by 90% if solving moved pages is enabled. > > > Deprecations: > ------------- > > Most of the deprecated code parts were removed to start with a new proper and > maintainable code without budensome overhead. The deprecations must be solved > if you have your own code based on Pywikibot before you upgrade to the new > release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html to see > all changes. Run your scripts and focus on FutureWarnings about deprecated code > parts. Refer the logs for deprecation warnings. Scripts must be run with global > -log option; if your script does not use handle_args() function you can invoke > it with the pwb wrapper script: > pwb.py -log <yourscript> [your options] > > > > Thank you all, comments are welcome > > xqt > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org > > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org

1 0

Bulk uploads to Commons using PWB and data_ingestion script. Is it possible?
by Ian Watt 15 Feb '22

15 Feb '22

Hi, I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked). I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons. I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them. I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick. I see that it is in /scripts/archive/ Is this still a viable script - or is it deprecated in some way? Does anyone have a guide for using it beyond the comments at the top of the script? I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually. Any suggestions (including ‘don’t do this’) with explanations would be welcome please. Many thanks Ian Ian Watt ianwatt(a)gmail.com watty62

2 8

New Pywikibot 7 stable release
by info＠gno.de 14 Feb '22

14 Feb '22

3 4

pywikibot vs mwclient?
by Roy Smith 06 Feb '22

06 Feb '22

I've been using mwclient for eons. I'm wondering if it's worth my switching to pywikibot. What's different/better about pywikibot that would justify the switch? Is pywikibot a fork of mwclient or a clean start? I found some comparisons on https://www.mediawiki.org/wiki/API:Client_code#Python <https://www.mediawiki.org/wiki/API:Client_code#Python>, but most of that hasn't been updated in years.

4 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot February 2022