pywikibot

pywikibot@lists.wikimedia.org

1 participants
6836 discussions

Training materials for Small wiki toolkits Pywikibot workshops
by Kamil Bach 13 Apr '23

13 Apr '23

Hello everyone, *Summary* The Developer Advocacy team is excited to share with you a new set of educational materials you can use to learn and teach others about Pywikibot [1]. These new materials are based on content originally presented as part of the Small wiki toolkits initiative [2]. Please read and share these materials, give feedback in Phabricator [3] or on Talk pages, or edit the pages directly if you find errors. This will help improve the materials for these workshops as well as workshops planned for the future. *Long version* Last year, the Small wiki toolkits initiative held monthly technical workshops about using, building, and hosting bots [4]. In response to feedback received from participants [5], this year the Developer Advocacy team launched an initiative (Phabricator task: [6]) to develop on-wiki learning and training materials based on the previous workshops. The goal of these materials is to make it easier for you to learn about Pywikibot and teach others in your community about it. Drafts for two workshops [7] are now ready for your review: - How to run basic scripts using Pywikibot - How to host a bot on Toolforge Materials for each workshop come in two formats: 1. Self-study guide - explains how to learn a given subject on your own - links to different resources on wikis and other sites without duplicating them. 2. Workshop organizer's handbook - teaches you how to teach the subject to others - explains how to prepare a workshop that you can run during a hackathon or a conference. If you've ever wanted to give a talk about Pywikibot but didn't have the time to design it, the handbook should help. *How to get involved* Please read the linked materials and provide feedback in Phabricator or on talk pages. Is this the right format for learners and trainers? Did we miss anything? Are there sections that are difficult to follow or make no sense? Please let us know. Your input will help inform the next steps for the development of the remaining materials. If you want to contribute any changes or corrections, you can do so either directly on the pages, or by leaving comments on talk pages or in Phabricator. Thank you for your help! On behalf of the Developer Advocacy team, Kamil Bach [1] https://www.mediawiki.org/wiki/Manual:Pywikibot/Workshop_Materials [2] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops [3] https://phabricator.wikimedia.org/T327282 [4] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops [5] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops/Report [6] https://phabricator.wikimedia.org/T327282 [7] https://www.mediawiki.org/wiki/Manual:Pywikibot/Workshop_Materials -- Kamil Bach (they/them) Technical Writer Wikimedia Foundation

1 0

Limit cosmetic changes by a function or generator
by Bináris 02 Apr '23

02 Apr '23

Hi, I wrote a module which handles Flagged Revisions in huwiki. https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:BinBot/huwiki/flaggedrevs.py It has a function called cosmeticable(page) to tell if a page is eligible for cosmetic changes according to huwiki rules. It also has a CosmeticableFilterPageGenerator which yields only cosmeticable pages. But it may not have a command-line argument as it is not in the framework. Now, what is the way to tell Pywikibot that cosmetic changes should be done only on pages where the function returns True or which are from this pagegenerator? I don't want to modify the core system on my device, because it will be disrupted when refreshing and is not distributable to other users. In eneral, is there a way to use own _global_ command-line arguments? -- Bináris

1 0

Place for Wikipedia in the framework
by Bináris 31 Mar '23

31 Mar '23

Tl;dr: is there a place for Wikipedia-related code? What do we do for code reusability? I proposed a Page method in https://phabricator.wikimedia.org/T328769. This would have shown if an article is a biography (it is about a person). My idea was opposed because the codebase of BasePage and APIPage is not Wikipedia-related, and some developers don't want to see any Wikipedia-specific code in it. OK, let's say, this is a valid argument. Let's call it *code purity*. On the other hand, Pywikibot is mostly developed by Wikipedians, mostly used in Wikipedia, and biographies form a primary scope of Wikipedia. I definitely think that biographies SHOULD be subclassed to have a lot of methods that are useful for a lot of Wikipedias and bot owners. If they write this code for themselves again and again, that's a waste. Now I definitely will use this feature. What can I do? First idea is to subclass Page in my code. That is the natural solution. Of course, when I publish my scripts, others won't be able to use them, and I won't be able to use others' scripts as they are, because I need this subclass. Let's call this point od view *code reusability* which is generally kept an important thing in the world of programming. Now, what we do (see the above task) is that we throw away code reusability for the sake of code purity. Is that OK? I am honestly curious, where the place of Wikipedia is in this framework. What I WILL do in the present situation: write a module called huwiki and don't bother other Wikipedias and place my code into this module. It will contain functions that take the page as parameter (not nice, not natural, I don't like it, but this is the only way if I cannot subclass) and nice pagegenerators, all for Hungarian Wikipedia alone, because I need them. Is that really what we want? Please help to find the place of Wikipedia-related code. -- Bináris

3 3

Re: Parallel gets?
by Roy Smith 29 Mar '23

29 Mar '23

Without the screenshot this time.... > On Mar 28, 2023, at 2:01 PM, Roy Smith <roy(a)panix.com> wrote: > > Hmmm. What I'm doing requires Page.expand_text(), which looks like it does a Page.get() followed by a Site.expand_text(), and it's the later which actually takes most of the time. That becomes an action=expandtemplates API call <https://www.mediawiki.org/w/api.php?action=help&modules=expandtemplates>, which I don't see any way to batch. > > > > > <Screen Shot 2023-03-28 at 1.55.55 PM.png> > >> On Mar 28, 2023, at 1:04 PM, Kunal Mehta <legoktm(a)debian.org <mailto:legoktm@debian.org>> wrote: >> >> Hi, >> >> On 3/27/23 15:57, Roy Smith wrote: >>> I need to issue a bunch of Page.get() requests in parallel. >> >> Please don't. From <https://www.mediawiki.org/wiki/API:Etiquette#Request_limit <https://www.mediawiki.org/wiki/API:Etiquette#Request_limit>>: >> >> "Making your requests in series rather than in parallel, by waiting for one request to finish before sending a new request, should result in a safe request rate." >> >> Instead of making parallel requests, you should make batched requests, which is how the preloading stuff Xqt mentioned works. >> >> -- Kunal / Legoktm >> _______________________________________________ >> pywikibot mailing list -- pywikibot(a)lists.wikimedia.org <mailto:pywikibot@lists.wikimedia.org> >> Public archives at https://lists.wikimedia.org/hyperkitty/list/pywikibot@lists.wikimedia.org/m… <https://lists.wikimedia.org/hyperkitty/list/pywikibot@lists.wikimedia.org/m…> >> To unsubscribe send an email to pywikibot-leave(a)lists.wikimedia.org <mailto:pywikibot-leave@lists.wikimedia.org> >> >

3 4

OFF: Idle 80
by Bináris 29 Mar '23

29 Mar '23

Wikipedia says me that Eric Idle (after whom IDLE is named) is 80 today. Well, on a Pythonic mailing list this may be interesting. :-) -- Bináris

1 0

Parallel gets?
by Roy Smith 29 Mar '23

29 Mar '23

I need to issue a bunch of Page.get() requests in parallel. My understanding is that pywikibot uses the requests library which is incompatible with async_io, so that's out. So what do people use? Threading <https://docs.python.org/3.9/library/threading.html>? Or, I see there's an async_io friendly requests port <https://github.com/rdbhost/yieldfromRequests>. Is there a way to make pywikibot use that?

4 6

Huwiki has new namespaces today, Pywikibot does not handle them
by Bináris 28 Mar '23

28 Mar '23

Today in wikipedia:hu namespaces 118, 119 were set. See T33308. https://phabricator.wikimedia.org/T333083 Now I have problems with the new namespaces. Message: KeyError: '118 is not a known namespace. Maybe you should clear the api cache.' Source: Pywikibot\pywikibot\site\_namespace.py >>> list(site.namespaces()) [-2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 90, 91, 92, 93, 100, 101, 710, 711, 828, 829, 2300, 2301, 2302, 2303] How to clear the API cache? Should Pywikibot automatically recognaize the new namespaces or shall I alter the code somewhere? -- Bináris

2 2

How to get enclosing section with mwparserfromhell?
by Roy Smith 24 Mar '23

24 Mar '23

To my surprise, wikicode.get_parent() does not get you the section a node is part of: > import mwparserfromhell as mwp > > text = """==foo== > {{Template:Foo}} > """ > wikicode = mwp.parse(text) > print(wikicode.get_tree()) > > print('++++++++++') > > node = wikicode.nodes[-2] > print(f"{node=}") > print(f"{wikicode.get_parent(node)=}") prints: > == > foo > == > \n > {{ > Template:Foo > }} > \n > ++++++++++ > node='{{Template:Foo}}' > wikicode.get_parent(node)=None Am I just doing this wrong?

2 1

How to avoid leaving extra blank lines when removing templates?
by Roy Smith 22 Mar '23

22 Mar '23

I've got some code which is essentially: > wikicode = mwp.parse(self.page.get()) > for node in wikicode.filter_templates(recursive=False, matches=title): > wikicode.remove(node) > self.page.text = str(wikicode) > self.page.save() which works, but it leaves an extra blank line behind where the template used to be. This is intended to be run on [[:en:Template talk:Did you know/Approved]], i.e. one template per line. What's the best way to get rid of the blank lines? I'm trying to avoid just running a regex replacement on the raw text because that's fragile, but maybe theres really no good alternative here?

3 3

Failing unit tests?
by Roy Smith 12 Mar '23

12 Mar '23

I'm gearing up to do some work (hopefully dive into fixing https://phabricator.wikimedia.org/T326650). I've gotten as far as closing the repo and running the existing unit tests. I get 4 failures: FAILED tests/make_dist_tests.py::TestMakeDist::test_handle_args - AssertionError: '/Users/roy/pywikibot/pywikibot-git/tests/make_dist_tests.py' != '/Users/roy/pywikibot/venv/bin/pytest' FAILED tests/make_dist_tests.py::TestMakeDist::test_handle_args_empty - AssertionError: '/Users/roy/pywikibot/pywikibot-git/tests/make_dist_tests.py' != '/Users/roy/pywikibot/venv/bin/pytest' FAILED tests/make_dist_tests.py::TestMakeDist::test_handle_args_nodist - AssertionError: '/Users/roy/pywikibot/pywikibot-git/tests/make_dist_tests.py' != '/Users/roy/pywikibot/venv/bin/pytest' FAILED tests/site_detect_tests.py::MediaWikiSiteTestCase::test_proofreadwiki - RuntimeError: Unsupported url: https://www.proofwiki.org/wiki/ <https://www.proofwiki.org/wiki/> Are these known issues? Or something wrong with my environment? I'm working on MacOS Monterey, with Python 3.9.

1 0

← Newer
1
2
3
4
5
6
7
8
...
684
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot