pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

[Announcement] Pywikibot Core can now be Indexed/Searched by Codesearch
by Derick N. Alangi 09 Oct '18

09 Oct '18

Hello Everyone, Few days ago while helping out with some few contributions on the Pywikibot project, it came to my notice that Pywikibot repo is not available in Legoktm's codesearch <https://www.mediawiki.org/wiki/Codesearch> tool. Since the contributions I made are mostly cleaning up code etc, I had to use text editor based search which (ugh, is very slow). So I remembered that ah, there is a code search tool somewhere that I can use to search what is online (even faster) :D, but Pywikibot was not available in the tool yet so I made this patch to add it: https://gerrit.wikimedia.org/r/c/labs/codesearch/+/463719. Thanks to @Xqt for review and a design feedback and also thanks to @Legoktm for reviewing, merging and deploying the patch. Now we have Pywikibot in Codesearch: https://codesearch.wmflabs.org/pywikibot/ Lego's comments; > Merged and deployed :-) > > Pywikibot is included in the main "search" profile, and the separate > Pywikibot profile. > P.S.: I'm Derick Alangi <https://phabricator.wikimedia.org/p/D3r1ck01/> from Cameroon, and I'm new to the project and it's community. I like the project and want to make some useful contributions that matter. Thanks! :) *--* *Regards, Derick .N.,* *V. Developer {at} Wikimedia* *See also: https://bit.ly/2xUjuLT <https://bit.ly/2xUjuLT>*

3 4

Re: [pywikibot] [Announcement] Pywikibot Core can now be Indexed/Searched by Codesearch
by billinghurst 07 Oct '18

07 Oct '18

I have added the codesearch link to https://www.mediawiki.org/wiki/Manual:Pywikibot and set up that component for translation. -- billinghurst ------ Original Message ------ From: "Derick N. Alangi" <alangiderick(a)gmail.com> To: billinghurstwiki(a)gmail.com Sent: 7/10/2018 11:39:50 PM Subject: Re: [pywikibot] [Announcement] Pywikibot Core can now be Indexed/Searched by Codesearch >Thank you very much! :) > >-- >Regards, Derick .N., >V. Developer {at} Wikimedia >See also: https://bit.ly/2xUjuLT > > >On Sun, Oct 7, 2018 at 1:32 PM billinghurst ><billinghurstwiki(a)gmail.com> wrote: >>Job well done. Thanks. >> >>-- billinghurst >> >>------ Original Message ------ >>From: "Derick N. Alangi" <alangiderick(a)gmail.com> >>To: pywikibot-announce(a)lists.wikimedia.org; >>pywikibot(a)lists.wikimedia.org >>Sent: 7/10/2018 8:58:11 AM >>Subject: [pywikibot] [Announcement] Pywikibot Core can now be >>Indexed/Searched by Codesearch >> >>>Hello Everyone, >>> >>>Few days ago while helping out with some few contributions on the >>>Pywikibot project, it came to my notice that Pywikibot repo is not >>>available in Legoktm's codesearch >>><https://www.mediawiki.org/wiki/Codesearch> tool. >>> >>>Since the contributions I made are mostly cleaning up code etc, I had >>>to use text editor based search which (ugh, is very slow). So I >>>remembered that ah, there is a code search tool somewhere that I can >>>use to search what is online (even faster) :D, but Pywikibot was not >>>available in the tool yet so I made this patch to add it: >>>https://gerrit.wikimedia.org/r/c/labs/codesearch/+/463719. >>> >>>Thanks to @Xqt for review and a design feedback and also thanks to >>>@Legoktm for reviewing, merging and deploying the patch. Now we have >>>Pywikibot in Codesearch: >>> >>>https://codesearch.wmflabs.org/pywikibot/ >>> >>>Lego's comments; >>>>Merged and deployed :-) >>>> >>>>Pywikibot is included in the main "search" profile, and the separate >>>>Pywikibot profile. >>>> >>> >>>P.S.: I'm Derick Alangi >>><https://phabricator.wikimedia.org/p/D3r1ck01/> from Cameroon, and >>>I'm new to the project and it's community. I like the project and >>>want to make some useful contributions that matter. Thanks! :) >>> >>>-- >>>Regards, Derick .N., >>>V. Developer {at} Wikimedia >>>See also: https://bit.ly/2xUjuLT

2 1

Helper script for Check Wikipedia Project
by Bináris 27 Sep '18

27 Sep '18

For those who want to work with Check Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Check_Wikipedia and https://tools.wmflabs.org/checkwiki/cgi-bin/checkwiki.cgi I created a script that will download the page list to a certain error type. It can be used in 3 ways: * Save to a file, then use Pywikibot with -file. * Upload the contents of the file to a subpage, then use Pywikibot with -links. * Import as a pagegenerator. Check Wikipedia is something similar to Special:LInterrors, but not the same. As it provides plain title lists for bots and no API, I wrote an HTML-scraper, but its output is prepared for use as a pagegenerator in Pywikibot. Have fun! https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:BinBot/checkwiki.py -- Bináris

1 0

Replace.py: very slow reading from XML dump
by Bináris 17 Sep '18

17 Sep '18

Hi folks, I still use trunk/compat for many reasons, but as I see the new code at https://github.com/wikimedia/pywikibot/blob/master/scripts/replace.py, the core version must suffer from the same problem. If we use -namespace for namespace filtering, class XmlDumpReplacePageGenerator will go through ALL pages, THEN the result is filtered by a namespace generator. This may MULTIPLY the running time in some cases and this may cost hours or even days for a fix of complicated, slow regexes. I have just checked, that dump does contain namespace informátion. So why don't we filter during the scan? I made an experiment. I modified my copy to display count of articles and count of matching pages. The replacement was: (ur'(\d)\s*%', ur'\1%'), which seems pretty slow. :-( The bot scanned the latest huwiki dump for 14 hours(!). (Not the whole dump, I used -xmlstart.) It went through 820 thousand pages and found 240+ matches (I displayed every 10th match). Then the bot worked further 30-40 minutes to check the actual pages from live wiki, this time with namespace filtering on. (I don't replace in this phase, just save the list, so no human interaction is implied in this time.) Guess the result! 62 out of 240 remained. This means that the bigger part of these 14 hours went into /dev/null. Now I realize how much time I wasted in the past 10 years. :-( I am sure that passing namespaces to XmlDumpReplacePageGenerator is worth. -- Bináris

2 5

asynchronous put on core
by Steen Thomassen 17 Sep '18

17 Sep '18

Hi In core is how many edits I can set for later save is only 2-3. How can it increase the number of edits that are waiting to be saved? In compat the number was up to 60. /Steen

4 3

[Pywikipedia-l] Urlencoded section titles
by Bináris 13 Sep '18

13 Sep '18

Happy Monday, There are strange people who make such links (kindof urlencoded?): [[Második világháború#Partrasz.C3.A1ll.C3.A1s Szic.C3.ADli.C3.A1ban .28Huskey hadm.C5.B1velet.29|Huskey hadműveletben]] So the section title must have been copied from the URL. Do we have a ready tool to fix these? -- Bináris

3 11

Google Code-in 2018: Mentor some smaller tasks for young contributors!
by Andre Klapper 07 Sep '18

07 Sep '18

Also see https://phabricator.wikimedia.org/T203122 <tl;dr>: Read https://www.mediawiki.org/wiki/Google_Code-in/Mentors and add your name to the mentors table and start tagging #GCI-2018 tasks. We'll need MANY mentors and MANY tasks, otherwise we cannot make it. Google Code-in is an annual contest for 13-17 year old students. It will take place from Oct23 to Dec13. It's not only about coding: we also need tasks about design, docs, outreach/research, QA. Last year, 300 students worked on 760 tasks supported by 51 mentors. For some achievements from last round, see https://blog.wikimedia.org/2018/03/20/wikimedia-google-code-in-2017/ While we wait whether Wikimedia will get accepted: * You have small, self-contained bugs you'd like to see fixed? * Your documentation needs specific improvements? * Your user interface has some smaller design issues? * Your Outreachy/Summer of Code project welcomes small tweaks? * You'd enjoy helping someone port your template to Lua? * Your gadget code uses some deprecated API calls? * You have tasks in mind that welcome some research? Note that "beginner tasks" (e.g. "Set up Vagrant") and generic tasks are very welcome (like "Choose and fix 2 PHP7 issues from the list in https://phabricator.wikimedia.org/T120336" style). We also have more than 400 unassigned open #easy tasks listed: https://phabricator.wikimedia.org/maniphest/query/HCyOonSbFn.z/#R Can you mentor some of those tasks in your area? Please take a moment to find / update [Phabricator etc.] tasks in your project(s) which would take an experienced contributor 2-3 hours. Read https://www.mediawiki.org/wiki/Google_Code-in/Mentors , ask if you have any questions, and add your name to https://www.mediawiki.org/wiki/Google_Code-in/2018#List_of_Wikimedia_mentors (If you have mentored before and have a good overview of our infrastructure: We also need more organization admins! See https://www.mediawiki.org/wiki/Google_Code-in/Admins ) Thanks (as we cannot run this without your help), andre -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

1 0

Re: [pywikibot] Bug on cosmetic changes
by Dalba 23 Aug '18

23 Aug '18

The crash on header comments issue is now resolved.[1] Regarding the removal of entire sections not being a cosmetic change, have a look at the comments on the original task[2] where this was introduced and if you still think that it should not be part of CC toolkit, please consider leaving a comment there, reopening the the task, or creating a new one. I guess we could also make it an opt-in/opt-out option via user-config.py. [1]: https://gerrit.wikimedia.org/r/c/pywikibot/core/+/453302 [2]: https://phabricator.wikimedia.org/T140570

2 2

Bug on cosmetic changes
by Hercule Antoine 16 Aug '18

16 Aug '18

Hello, There is a big issue with the removal of empty sections by cosmetic changes. https://gerrit.wikimedia.org/r/#/c/433914/6/pywikibot/cosmetic_changes.py It removes also non-empty sections, that starts and ends with HTML comments See : https://fr.wikipedia.org/w/index.php?title=S%C3%A9ries_t%C3% A9l%C3%A9vis%C3%A9es_diffus%C3%A9es_sur_American_Broadcast ing_Company&diff=prev&oldid=148771218 This bug is, I think, critical because all changes with CC activated must be verified. I don't think that CC should remove comments added by editors. I tried to open a bug on phabricator but don't understand how to do it. Regard

3 7

Fwd: MultiContent Revisions and changes to the XML dumps
by Bináris 02 Aug '18

02 Aug '18

This may be a breaking change. ---------- Forwarded message ---------- From: Ariel Glenn WMF <ariel(a)wikimedia.org> Date: 2018-08-02 16:40 GMT+02:00 Subject: [Wikitech-l] MultiContent Revisions and changes to the XML dumps To: Wikipedia Xmldatadumps-l <Xmldatadumps-l(a)lists.wikimedia.org>, Wikimedia developers <wikitech-l(a)lists.wikimedia.org> As many of you may know, MultiContent Revisions are coming soon (October?) to a wiki near you. This means that we need changes to the XML dumps schema; these changes will likely NOT be backwards compatible. Initial discussion will take place here: https://phabricator.wikimedia.org/T199121 For background on MultiContent Revisions and their use on e.g. Commons or WikiData, see: https://phabricator.wikimedia.org/T200903 (Commons media medata) https://phabricator.wikimedia.org/T194729 (Wikidata entites) https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions (MCR generally) There may be other, better tickets/pages for background; feel free to supplement this list if you have such links. Ariel _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Bináris

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot