pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

How to ignore all the non-visible stuff?
by Roy Smith 04 Feb '23

04 Feb '23

[[:en:Template:Did you know/Queue/NextPrep]] contains: > 4<noinclude>  > {{documentation|content=This number indicates the next DYK prep set to move into the queue.}} > </noinclude> What I want to get is just the "4". Is Page.extract() what I'm looking for? Experimentally, it does what I want, but it's not clear if this is actually the intended use case.

2 1

Text between two comments?
by Roy Smith 03 Feb '23

03 Feb '23

I'm trying to parse DYK prep area templates, for example Template:Did you know/Preparation area 3 <https://en.wikipedia.org/wiki/Template:Did_you_know/Preparation_area_3>. Unfortunately, these are more like flat text files than any kind of nicely structured data. The stuff of interest is everything between two HTML comments: >  > {{main page image/DYK|image=Melissa Ong.webp|caption=Selfie of Ong, commonly replicated by the Step Chickens}} > * ... that "Step Chickens" on TikTok replace their profile pictures with an image ''(shown)'' of '''[[Melissa Ong]]''', whom they call "Mother Hen"? > * ... that '''[[interfaith greetings in Indonesia]]''' include phrases from Islam, Christianity, Hinduism, Buddhism, and Confucianism? > * ... that '''[[Kimmo Leinonen]]''' helped establish both the [[Finnish Hockey Hall of Fame]] and the [[IIHF Hall of Fame]]? > * ... that the [[Pulitzer Prize for Fiction|Pulitzer Prize]]-winning novel '''''[[All the Light We Cannot See]]''''' contains a sympathetic [[Nazism|Nazi]]? > * ... that a {{Convert|10|ft|m|adj=mid|-tall|0}} '''[[Lady Rainier|statue of a woman]]''' in [[Seattle]] was commissioned by a local brewery in 1903? > * ... that ... > * ... that prior to entering politics, '''[[Herbert Salvatierra]]''' led a troupe of [[carnival]] ''[[comparsa]]s''? > * ... that [[Winston Churchill]] published '''[[Are There Men on the Moon?|an essay on extraterrestrial life]]''' during the Second World War? >  I can find the comments with Wikicode.filter_comments(). But once I've found the two delimiting comments, how do I grab the text between them? Or is the parser the wrong tool? Would I do better to treat the content of the page as flat text and just iterate over it line by line, teasing it apart with regexes?

4 4

How to exclude HTML comments
by Bináris 31 Jan '23

31 Jan '23

Given a piece of text, I want to exclude  when searching a template. Is there a ready-to-use, nice, canonical method for it? -- Bináris

1 1

Templates in archivebot.py
by Bináris 31 Jan '23

31 Jan '23

Hi, In archivebot.py there is a line: # TODO: handle marked with template This means resolved and unresolved sections. Unresolved sections are not to be archived even if they are old enough, resolved could be archived faster. I coded unresolved for Hungarian Wikipedia (1), and I will do the resolved, too. But this way we use a separate branch. If we want to do it in general, we should put the templates somewhere. Templates depend on project, not language. There may also be a set of templates to trigger the same behaviour, see e.g. (1) for variety of templates. So we want a set of "resolved" and another set of "unresolved" templates per project. What is the nice architecture? Where would you put them? Of course, the easiest way is to hardcode in the script, but I think we don't do such things nowadays. [1] https://hu.wikipedia.org/w/index.php?title=Szerkeszt%C5%91:Atobot/archivebo… [2] https://en.wikipedia.org/wiki/Template:Unresolved -- Bináris

1 0

Tortoise SVN installs an old version
by Bináris 30 Jan '23

30 Jan '23

Sorry for the previous, but this one seems to be real. I followed https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation/SVN#Download_P… c:\Pywikibot>python pwb.py version WARNING: Http response status 404 Pywikibot: pywikibot/__init__.py (, -1 (unknown), 2023/01/30, 13:59:21, n/a) Release version: *3.1.dev0* requests version: 2.25.1 cacerts: C:\Python37\lib\site-packages\certifi\cacert.pem certificate test: ok Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] -- Bináris

3 5

SVN install vs i18n
by Bináris 30 Jan '23

30 Jan '23

Hi, I followed https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation/SVN#Download_P… At the last step I get SVN error: C:\Pywikibot\scripts\i18n The node 'C:\Pywikibot\scripts\i18n' was not found. What i the next step? -- Bináris

1 1

New Pywikibot 8 relase
by info＠gno.de 21 Jan '23

21 Jan '23

1 0

Pruning category search?
by Roy Smith 20 Jan '23

20 Jan '23

Is there a way to recurse through a category, but excluding specific sub-cats? For example, I want to find all the templates in [[Category:People and person infobox templates]], except that I don't want to recurse into [[Category:Styles infobox templates]].

2 1

Re: Viztracer
by Roy Smith 14 Jan '23

14 Jan '23

I see my original message was held up in moderation due to the large attachment. I've uploaded the image to File:Screenshot of viztracer output.png <https://commons.wikimedia.org/wiki/File:Screenshot_of_viztracer_output.png> > On Jan 13, 2023, at 6:38 PM, Roy Smith <roy(a)panix.com> wrote: > > > I just discovered viztracer. https://viztracer.readthedocs.io/en/stable/index.html <https://viztracer.readthedocs.io/en/stable/index.html> > > I've been trying to figure out why my pywikibot app is so slow. It took me about 1 minute to instrument my code (see diff), I hit my URL in a browser, and loaded up the resulting json file into visviewer. Now I'm scrolling around and drilling down into a total execution trace of my code, although it really only took a moment to see that most of the time is in 11 serialized API calls. This is easily the coolest performance analysis tool I've ever used. > > <Screen Shot 2023-01-13 at 6.32.31 PM.png> > > diff --git a/dyk_web/core.py b/dyk_web/core.py > index 31758dc..1e58203 100644 > --- a/dyk_web/core.py > +++ b/dyk_web/core.py > @@ -28,11 +28,16 @@ def get_pending_nominations(): > return titles > > > +from viztracer import VizTracer > +from pathlib import Path > + > + > @bp.route("/display") > def display(): > """template_name query arg is the DYK nomination template, including the Template: prefix.""" > - current_app.logger.info <http://current_app.logger.info/>("Running on %s", os.uname().nodename) > - page = Page(g.site <http://g.site/>, request.args["template_name"]) > - nomination = Nomination(page) > - nomination_data = NominationData.from_nomination(nomination) > - return render_template("display.html", nomination=nomination_data) > + with VizTracer(output_file=str(Path.home() / "viztracer.json")): > + current_app.logger.info <http://current_app.logger.info/>("Running on %s", os.uname().nodename) > + page = Page(g.site <http://g.site/>, request.args["template_name"]) > + nomination = Nomination(page) > + nomination_data = NominationData.from_nomination(nomination) > + return render_template("display.html", nomination=nomination_data) >

1 0

python pwb.py replace.py not working
by Basudev Krishna 11 Jan '23

11 Jan '23

Traceback (most recent call last): File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 399, in <module> if not main(): File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 391, in main run_python_file(filename, File "/data/data/com.termux/files/home/vikaspy/pwb.py", line 106, in run_python_file exec(compile(source, filename, 'exec', dont_inherit=True), File "./scripts/replace.py", line 1075, in <module> main() File "./scripts/replace.py", line 929, in main single_summary = i18n.twtranslate( File "/data/data/com.termux/files/home/vikaspy/pywikibot/tools/_deprecate.py", line 404, in wrapper return obj(*__args, **__kw) File "/data/data/com.termux/files/home/vikaspy/pywikibot/i18n.py", line 700, in twtranslate raise pywikibot.exceptions.TranslationError( pywikibot.exceptions.TranslationError: Unable to load messages package scripts.i18n for bundle replace-replacing It can happen due to lack of i18n submodule or files. See https://www.mediawiki.org/wiki/Manual:Pywikibot/i18n CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.TranslationError'> $

3 8

← Newer
1
...
4
5
6
7
8
9
10
...
683
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot