pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

Contributing to the pywikibot project for the Google Summer of Code Programme
by Fortunate Omoleye 28 Feb '22

28 Feb '22

Hey Folks, How are you doing today and I hope you are safe and well wherever you are. My name is Omoleye Julius Fortunate and I'm a year three statistics student of the federal University of Agriculture Abeokuta Ogun State Nigeria. I would like to contribute to the Wikimedia pywikibot project for the 2022 google summer of code programme , and would really love to get an headstart on how to get started. I have skills using the Python Django framework, weak machine learning skills , also some basic site reliability Engineering skills too. I would be expecting a positive response from anyone that can help soon. Thank you for your time. Omoleye Julius

1 0

NotImplementedError: page_from_repository method
by Ugo Chima 28 Feb '22

28 Feb '22

Good morning everyone Please anyone have any Idea why I receive this error "*NotImplementedError: page_from_repository method is not implemented for Wikibase **site:lang" * when running the *newitem.py* script? Chima.

2 7

Re: New Pywikibot 7 stable release
by info＠gno.de 24 Feb '22

24 Feb '22

Seems this mail hasn’t passed. xqt > Am 19.02.2022 um 18:48 > > > > Hi Maarten, > > I've restored all removed pagegenerators functions. > https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot… > Other functions may follow possibly in such a way I examined already. > > > > I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot. > > Structured Data was enabled with this patch > https://gerrit.wikimedia.org/r/c/pywikibot/core/+/627243 > T223820 is still open. Not sure what is remainting to do there. But also "some horrible hacks to work around" would possibly help. Could you point it out? > > > Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly. > I don't fully agree because a site can also be seen as a central point to work on. For example mwclient package is site oriented and Pywikibot looks more like a network: https://doc.wikimedia.org/pywikibot/master/index.html#framework-modules-ove… I think we can be flexible enough to provide different approaches. I am working on a site.Page() constructor method right now which is also able to upcast the correspondig BasePage class in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/763753 > > > Downside is that we won't have any documentation for these generators at ... > Must be defined in the rst file then. > > > Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ? > Obviously this is appropriate, see https://phabricator.wikimedia.org/rPWBCd7719950d651277955ed804ad85fb729fcc5… as a sample. > > > Have we ever tried cross-referencing > Yes we do that a lot, mostly automatically by type hints. But it can be improved anyway. > > > Best > xqt > Von: Maarten Dammers <maarten(a)mdammers.nl> > Gesendet: 14.02.2022 22:19 > An: <pywikibot(a)lists.wikimedia.org> > Betreff: [pywikibot] Re: New Pywikibot 7 stable release > > Hi Xqt, > > On 13-02-2022 21:53, info(a)gno.de wrote: >> >> Hi Maarten, >> >> thank you for you feedback. There is the same "redundancy" in some Page methods which calls the corresponding site method and I wouldn’t drop one over the other. Page methods have a higher tier than Site methods which calls api functions and finally the comms functions. > Exactly. I think I'm only using the site object directly in some horrible hacks to work around the lack of structured data on Commons support in Pywikibot. > >> >> pagegenerators implements generators and filters for command line options and provides additional page generators which are outside of site methods like multiple site generators and other stuff. >> >> But I don’t see any advantage using the same functionality from pagegenerators module over those site methods who are called by that function. The (now removed) functions had the form >> >> def this_generator(total=None, site=None) >> if not site: site = Pywikibot.Site() >> site.this(total=total) >> >> There is a disadvantage in that sense that site must not be determined and another disadvantage that most site parameters were not provided with the pg function. >> >> Can you tell me your POV about advantage/disadvantage of those pg function? > Just like page being the central point of entry from doing things with pages, I see the pagegenerators as the central point of entry to getting a set of pages. Site functions shouldn't be used directly. The pagegenerators should invoke the needed code mostly from site, but maybe from other parts. I recognize the problem of the missing generators and missing parameters. >> >> >> There are about 50 generator functions in site.GeneratorMixin but pg had only a few of them. >> >> I made a proposal in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/762132 which would implement all site generators as pagegenerators function. Is this in your sense? Unfortunately this only works with Python 3.7+ and needs some additional work for older Pythons. > Thanks for having a look at this. Interesting idea to create the generators like this. Would fix the example at https://www.mediawiki.org/wiki/Manual:Pywikibot/pagegenerators.py#Calls_fro… . Downside is that we won't have any documentation for these generators at https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#module-py… . Maybe middle ground to have a function that gives all the parameters to the site function using **kwargs ? Bit like we do in page at places like https://doc.wikimedia.org/pywikibot/master/_modules/pywikibot/page.html#Bas… . Does make me wonder how to link that with https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.site.html#pywi… so we don't have to copy all the parameters. Each generator would be a minimal function with a short description and a link to the underlying function for detailed instructions. Have we ever tried cross-referencing as described at https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cr… ? > > Maarten > >> >> Best >> xqt >> >>> >>> Am 13.02.2022 um 14:40 schrieb Maarten Dammers <maarten(a)mdammers.nl>: >>> >>> Hi Xqt, >>> >>> I don't agree with the removal of the generators that someone decided to deprecate [1]. We always had the principle that pagegenerators is the point of entry to get pages to work on. The pagegenerators are a (thin) layer of abstraction so that you don't have to directly use function in the site objects. With this removal you're breaking one of the very fundamental functions and principles of our framework. >>> >>> Why did you remove it? Any arguments beside than the incorrect argument that it's redundant? >>> Was the deprecation and removal discussed somewhere? Can't find it on this list. >>> >>> Maarten >>> >>> [1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/761395/5/pywikibot/pagege… >>> >>> On 12-02-2022 13:31, info(a)gno.de wrote: > > Hi all, > > a new stable release Pywikibot 7 will be deployed at the end of this month. > There are a lot important points with this release, currently implemented in > master branch: > > > Python Plattform: > ----------------- > > Pywkibot support of Python 3.5.0 - 3.5.2 will be dropped. The minimum plattform > required is Python 3.5.3 which is preinstalled at toolforge. On the other hand > Pywikibot also works with new Python 3.11.0a and *new* supports PyPy 3. Most > used plattform measured by PyPy download statistic is Python 3.8 (T266984). For > developers I highly propose to use Python 3.10 due to "Better error messages" > (https://docs.python.org/3.10/whatsnew/3.10.html#better-error-messages). > > > Framework Scripts: > ------------------ > > The both scripts generate_family_file and generate_user_files were moved from > framework root directory to pywikibot/scripts folder. The two scripts shell and > version were moved from scripts folder to pywikibot/scripts. This arrangement > was necessary to implement a code entry point for Pywikibot site-package. > Possibly you have to change the path environment setting to use these scripts. > But all of them are still available through the pwb.py wrapper script. > > > Linux: > ------ > > Scripts hash bang was changed from python to python3. This enables to invoke > scripts without preleading interpreter. > > > PAWS: > ----- > > Pywikibot 7 is a stable release and becomes the Pywikibot framework base on > PAWS. > > > Toolforge: > ---------- > > After deployment of Pywikibot 7 this stable release will be available under > /shared/pywikibot/stable and the current master can be accessed under > /shared/pywikibot/core. The pwb.py wrapper script can be used at Tooforge > and the similar script search is functional. > https://doc.wikimedia.org/pywikibot/master/utilities/index.html#module-pwb > > > MediaWiki: > ---------- > > No changes was made for supporting MediaWiki releases. Pywikibot 7 supports > MediaWiki 1.23-1.37. For current MediaWiki release 1.37 the minimum Pywikibot > version must be 6.6.1. MW 1.19 requires PWB 5 and MW 1.14 requires PWB 4. > (https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility) > > > Pywikibot site-package (PyPi): > ------------------------------ > > Installing Pywikibot as a site package (pip install pywikibot==7.0) comes with > many innovations. pywikibot i18n messages bundle is available. Framework > scripts listed above can be used with Pywikibot 7.The pwb wrapper script is the > code entry point to invoke these scripts: > pwb [global options] <framework script> [script options|global options] > > > > Pywikibot Tests: > ---------------- > > Pywikibot CI tests were moved from Travis to GitHub action. There are two > actions implemented. The first runs up tp 20 workers in parallel and does the > most tests whereas the second only runs login/logout tests but there are no > parallel task which would lead to failures due to not logged-in test account. > > > Module Changes: > --------------- > > Support for API:Redirects was added in site and page module. User.is_locked() > and APISite.is_locked() were added to determine whether a given user or user id > is locked globally. The cached output functionality from compat release was > re-implemented. > > families: > Allow family files reside in `families` folder in base_dir by default. This is > important if Pywikibot is used as a site-package. generate_family_file uses > this new place to save the family file. Wikihow family file was added. > > i18n module: > bundles() yields message bundle names or Path objects; known_languages() gives > all languages we have localizations for. > > page module: > isStaticRedirect() also considers transcluded __STATICREDIRECT__. > put() and change_category() now has a show_diff parameter. User.is_blocked() > can also detect range blocks. > > proofreadpage: > Wikimedia OCR engine is supported. > > site module: > APISite.get_globaluserinfo() method was added to retrieve globaluserinfo. > New "maxlimit" property was added to determine the limits for API loads. > > textlib module: > textlib comes with to_latin_digits() function as counterpart of > to_local_digits() and case_escape() which gives an escaped regex pattern > depending on 'first-letter' case. > > tools module: > intersect_generators was rewritten which makes it running up to 10’000 times > faster. > > > Scripts Changes: > ---------------- > > Scripts descriptions can be viewed at > https://doc.wikimedia.org/pywikibot/master/scripts/index.html > > ConfigParserBot is provided for several scripts. Any option can be set within > scripts.ini file. > https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html#pywikibot… > > add_text scripts provided -create and -createonly options. A CleanBot was added > in category script which can be invoked by clean action option. It removes > redundant grandchildren categories. The fixing_redirect script uses > concurrent.futures to retrieve redirect targets; this decreases processing time > by 90% if solving moved pages is enabled. > > > Deprecations: > ------------- > > Most of the deprecated code parts were removed to start with a new proper and > maintainable code without budensome overhead. The deprecations must be solved > if you have your own code based on Pywikibot before you upgrade to the new > release. Refer https://doc.wikimedia.org/pywikibot/master/changelog.html to see > all changes. Run your scripts and focus on FutureWarnings about deprecated code > parts. Refer the logs for deprecation warnings. Scripts must be run with global > -log option; if your script does not use handle_args() function you can invoke > it with the pwb wrapper script: > pwb.py -log <yourscript> [your options] > > > > Thank you all, comments are welcome > > xqt > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org > > _______________________________________________ > pywikibot mailing list -- pywikibot(a)lists.wikimedia.org > To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org

1 0

Bulk uploads to Commons using PWB and data_ingestion script. Is it possible?
by Ian Watt 16 Feb '22

16 Feb '22

Hi, I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked). I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons. I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them. I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick. I see that it is in /scripts/archive/ Is this still a viable script - or is it deprecated in some way? Does anyone have a guide for using it beyond the comments at the top of the script? I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually. Any suggestions (including ‘don’t do this’) with explanations would be welcome please. Many thanks Ian Ian Watt ianwatt(a)gmail.com watty62

2 8

New Pywikibot 7 stable release
by info＠gno.de 15 Feb '22

15 Feb '22

3 4

pywikibot vs mwclient?
by Roy Smith 07 Feb '22

07 Feb '22

I've been using mwclient for eons. I'm wondering if it's worth my switching to pywikibot. What's different/better about pywikibot that would justify the switch? Is pywikibot a fork of mwclient or a clean start? I found some comparisons on https://www.mediawiki.org/wiki/API:Client_code#Python <https://www.mediawiki.org/wiki/API:Client_code#Python>, but most of that hasn't been updated in years.

4 3

Re: Deprecation policy?
by info＠gno.de 20 Dec '21

20 Dec '21

1 0

Error on PWB
by Ugo Chima 05 Dec '21

05 Dec '21

Good day everyone, please I have an issue with my PWB, I am running a script on a site that uses Wikibase, and I want the script to create new items for new pages from the client wiki on the repository wiki. But unfortunately, I get this error when I run the script, please I have tried fixing it on my own but to no avail, I'm sure someone else has gotten this typa error before, Please help me. It says *pywikibot.exceptions.EntityTypeUnknownError: DataSite("en", "mydatafamilyfile") does not support entity type "item".* Please help Full error that it shows 👇👇 Traceback (most recent call last): File "C:\Users\Username\pywikibot\curate.newpages.en.py", line 380, in <module> main() File "C:\Users\Username\pywikibot\curate.newpages.en.py", line 367, in main newitem = pywikibot.ItemPage(repo) File "C:\Users\Username\pywikibot\pywikibot\page\__init__.py", line 3901, in __init__ ns = site.item_namespace File "C:\Users\Username\pywikibot\pywikibot\site\_datasite.py", line 91, in item_namespace self._item_namespace = self.get_namespace_for_entity_type( File "C:\Users\Username\pywikibot\pywikibot\site\_datasite.py", line 78, in get_namespace_for_entity_type raise EntityTypeUnknownError( pywikibot.exceptions.EntityTypeUnknownError: DataSite("en", "mydatafamilyfile") does not support entity type "item" -Joseph.

2 2

Using noreferences.py within PAWS ipynb
by goethe.wiki＠gmail.com 03 Dec '21

03 Dec '21

Hi, I am trying to build a script to add a reference to a few hundred articles on pt.wikipedia. I would like to use the noreferences.py script that is part of core, to add a reference section in the same run, if it is not already present in the article. I have tried to import the script, the NoReferencesBot class, but I must be doing something wrong. It says that the function hasn't been defined. You can check what I have been trying to do here: https://public.paws.wmcloud.org/User:GoEThe/IUCN%20status.ipynb Any help will be greatly appreciated! Regards, GoEThe

2 2

Deprecation policy?
by Damian Johnson 15 Nov '21

15 Nov '21

Hi lovely pywikibot folks. I'd like to clean up our deprecated code since we have some that's six years old.... % grep -RIoh "since='[^']\+'" ./* | sort | head -n 5 since='20150206' since='20150515' since='20150615' since='20150617' since='20150619' % grep -RIoh "since='[^']\+'" ./* | wc -l 130 What is pywikibot's policy regarding code deprecation? Can we remove it after a set duration and, if so, what is it? Thanks! -Damian

6 8

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot