pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

Re: [Pywikipedia-l] Was feeling lucky- "Not" - still error onother wiki
by info＠gno.de 28 Jan '12

28 Jan '12

Let us explore in that issue, please send us the dump file too: ApiGetDataParse_isoggbot_en__Thu_Jan_26_07-58-17_2012.dump Regards xqt

3 9

[Pywikipedia-l] Was feeling lucky- "Not" - still error on other wiki
by Tom Hutchison 28 Jan '12

28 Jan '12

I was feeling lucky with the update and the one wiki I use Pywikipedia on. So I ran generate_family_file.py, created the family file to it. Updated my config file. Did everything the same as I did on the other wiki. I even used a different file folder to keep them entirely separate. Error, which was what the problem has always been with this wiki. C:\ISOOGBot>version.py Pywikipedia [http] trunk/pywikipedia (r9842, 2012/01/22, 00:51:55) Python 2.7 (r27:82525, Jul 4 2010, 09:01:59) [MSC v.1500 32 bit (Intel)] config-settings: use_api = True use_api_login = True unicode test: triggers problem #3081100 C:\ISOOGBot>login.py unicode test: triggers problem #3081100 Password for user ISOGGWikiSuperbot on isoggbot:en: Logging in to isoggbot:en as ISOGGWikiSuperbot via API. Error downloading data: invalid literal for int() with base 10: '147, 147' Request en:/w/api.php? Retrying in 1 minutes... I never could get this wiki to work with Pywikipedia, save for a very small time, about a month, 1 ½ years ago. It suddenly stopped working. Could never login to it again. Ideas on why this wiki wont play with Pywikipedia? Thanks Tom

4 5

Re: [Pywikipedia-l] Was feeling lucky- "Not" - still error on other wiki
by info＠gno.de 26 Jan '12

26 Jan '12

Additional to Merlijns mail please start again and wait until the bots api request is finished. It tries I guess 5 times and should then raise an error with trace informations. Also it creates a dump file named "ApiGetDataParse<something_else>". Please give us that dump file (content). Regards xqt ----- Original Nachricht ---- Von: Merlijn van Deen <valhallasw(a)arctus.nl> An: Pywikipedia discussion list <pywikipedia-l(a)lists.wikimedia.org> Datum: 26.01.2012 08:10 Betreff: Re: [Pywikipedia-l] Was feeling lucky- "Not" - still error on other wiki > > > > Error downloading data: invalid literal for int() with base 10: '147, > 147' > > **** > > > > > > > could you try running login.py with the -v flag? This will give some more > debug output, among others about the exact api.php queries and their > responses. > > Secondary to that, if you could give the url to the wiki (i.e. it's not a > private or local wiki), that would help with debugging /a lot/. > > Best, > Merlijn > > > -------------------------------- > > _______________________________________________ > Pywikipedia-l mailing list > Pywikipedia-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l >

2 1

Re: [Pywikipedia-l] Does Mediawiki 1.17 break old version pywikiapedia?
by info＠gno.de 25 Jan '12

25 Jan '12

Please update your bot first. Current release is r9843 and we couldn't examine that old stuff. As login.py worked as well, API is on. Please again create a new family file with the new release and try the command login.py -test This should determine whether the not is still logged in. If this does nothing, try login.py -test -v which prints additional I formations and give us the results and the new results of version.py too. Regards xqt

2 1

[Pywikipedia-l] Does Mediawiki break old version pywikiapedia?
by Tom 25 Jan '12

25 Jan '12

I have been running pywikiapedia on 1.16 for past year and a 1/2. Upgraded to 1.17 and it is now returning errors. Do I need to upgrade pywikiapedia, what has changed in the API to break it? Thanks Tom

4 6

[Pywikipedia-l] Need help for: Page rename / insert text / update links
by Eric K 24 Jan '12

24 Jan '12

Hi guys, I just installed the pywikipedia bot on my wiki yesterday. I'm new to Python but I can try learn it since I'm familiar with PHP. It would take me a while though to make this first bot since I'm new to the language. The tasks are pretty straightforward. I would like the bot to run without any user input and do all of this by itself: 1. For every page on the wiki, check if it has these three characters: ( , ) , : . Any page containing any of these characters (curly brackets and colon) will be moved to a new title. The original title is var_1. 2. For the new title, brackets are simply deleted, and the : (colon) is replaced with a " - " (a dash with a space on each side). The new title generated is var_2. 3. Insert this text at the top of this page: {{page_rename|var_1}}, and save page. 4. Find any existing links on the site to this page which would be in the format of [[var_1]], and change them to [[var_2|var_1]]. I don't need any menus or other functionality. Is something something pretty straightforward to make? I would appreciate any tips/help and if its something that can be made pretty easily, I would be really thankful if someone could do this for me or give me a good start. I've looked at some of existing pywikipedia bot scripts (basic.py, movepages.py) but none of them would work for me and being new to Python, it would take me a long time to do what I need but in any case I will learn a lot in this first attempt. thanks Eric

3 5

[Pywikipedia-l] noreferences.py
by Michael Movchin 21 Jan '12

21 Jan '12

Hello, I just have a question: Where is it possibly to add or change the summary on noreferences.py? Thanks, Michael

2 1

[Pywikipedia-l] toolserver interwiki project (was: interwiki.py)
by Merlijn van Deen 21 Jan '12

21 Jan '12

For the pywikipedia-l listeners just tuning in: the toolserver has an overload of interwiki bots, and we want to reduce this. As such, we want to switch to a single bot that runs all the interwiki updates from the toolserver. On 16 January 2012 09:19, Merlijn van Deen <valhallasw(a)arctus.nl> wrote: > The only reasonable action we can take to reduce the memory > consumption is to let the OS do its job in freeing memory: using one > process to track pages that have to be corrected (using the database, > if possible), and one process to do the actual fixing (interwiki.py). > This should be reasonably easy to implement (i.e. use a pywikibot page > generator to generate a list of pages, use a database layer to track > interlanguage links and popen('interwiki.py <page>') if this is a > fixable situation) > > I took some time yesterday to work out some details on this - see http://piratepad.net/T29Uj4j1U4 . It boils down to this: 1) generation of a list of pages to work on: from the database, if possible 2) dispatching interwiki.py based on that list of pages and handling logging 3) interwiki.py itself My suggestion is to split these tasks, and creating a simple interface (e.g. WSGI) between 1) and 2), and using subprocesses for 2) to 3). Yesterday, I have been working (mainly) on speeding up the startup of interwiki.py, so that we can spawn one process per Page. On the Toolserver side, I would appreciate any comments/work/existing work on the creation of an interwiki graph from the database - there are already tools that suggest images based on interwiki links, so this code should be around - and hopefully be adaptable. The only goal for this process would be to create a list of starting pages interwiki.py can use - i.e. graphs with one or more missing links, but without any double links. On the Pywikipedia side, some thoughts on running interwiki.py in a new process would be welcome. e.g. how can we improve startup time ('kill all the regexps!') and effectively spawn multiple processes to run. What parameters (throttles?) should be tuned, et cetera. Best, Merlijn

1 0

[Pywikipedia-l] snapshot of link graph at a given date
by John R. Frank 21 Jan '12

21 Jan '12

Hello Pywikipedians, As part of the TREC Knowledge Base Acceleration[1] eval for 2012, I want to generate snapshots of wikipedia's link graph around certain pages as of a given date. Does anyone have advice about this? Anyone doing something similar that I can learn from? For example, given a wikipedia.Page() instance for urlname=Takashi_Murakami, call it "central node," I want to iterate through the Pages returned by getReferences() and only keep those that ref'ed my central node on a date in the past, like November 30, 2011. Is the best (only?) way to do this to iterate through previous revisions of those Pages and verify that they ref'ed the central node before that date? What about pages that did ref and no longer do? (probly rare?) Thanks for any pointers! John 1 - http://www.mit.edu/~jrf/knowledge-base-acceleration/ -- ___________________________ John R. Frank <jrf(a)mit.edu>

3 2

Re: [Pywikipedia-l] [Pywikipedia-svn] SVN: [9196] trunk/pywikipedia
by Merlijn van Deen 19 Jan '12

19 Jan '12

Whoo! Great work :-) Tests always are good contributions :-) On a sidenote - is there a reason you're implementing these in 'trunk' and not in 'rewrite'? Of course, these contributions are very welcome in the trunk, but I still think it would be good to push the rewrite branch. Best regards, Merlijn On 24 April 2011 07:41, <jayvdb(a)svn.wikimedia.org> wrote: > http://www.mediawiki.org/wiki/Special:Code/pywikipedia/9196 > > Revision: 9196 > Author: jayvdb > Date: 2011-04-24 05:40:59 +0000 (Sun, 24 Apr 2011) > Log Message: > ----------- > Allow lists of Page and User objects to be interogated > > Modified Paths: > -------------- > trunk/pywikipedia/query.py > trunk/pywikipedia/tests/test_query.py > > Modified: trunk/pywikipedia/query.py > =================================================================== > --- trunk/pywikipedia/query.py 2011-04-24 04:23:12 UTC (rev 9195) > +++ trunk/pywikipedia/query.py 2011-04-24 05:40:59 UTC (rev 9196) > @@ -263,10 +263,21 @@ > > encList = '' > # items may not have one symbol - '|' > - for l in list: > - if type(l) == str and u'|' in l: > - raise wikipedia.Error("item '%s' contains '|' symbol" % l ) > - encList += ToUtf8(l) + u'|' > + for item in list: > + if isinstance(item,basestring): > + if u'|' in item: > + raise wikipedia.Error(u"item '%s' contains '|' symbol" % > item ) > + encList += ToUtf8(item) + u'|' > + elif isinstance(item,wikipedia.Page): > + encList += ToUtf8(item.title()) + u'|' > + elif item.__class__.__name__ == 'User': > + # delay loading this until it is needed > + import userlib > + encList += ToUtf8(item.name()) + u'|' > + else: > + raise wikipedia.Error(u'unknown item class %s' % > item.__class__.__name__) > + > + # strip trailing '|' before returning > return encList[:-1] > > def ToUtf8(s): > > Modified: trunk/pywikipedia/tests/test_query.py > =================================================================== > --- trunk/pywikipedia/tests/test_query.py 2011-04-24 04:23:12 UTC > (rev 9195) > +++ trunk/pywikipedia/tests/test_query.py 2011-04-24 05:40:59 UTC > (rev 9196) > @@ -7,6 +7,8 @@ > import unittest > import tests.test_pywiki > > +import wikipedia as pywikibot > +import catlib, userlib > import query > > > @@ -74,5 +76,72 @@ > ]} > self.assertEqualQueryResult(params, expectedresult) > > + def test_titles_Page(self): > + params = { > + 'action': 'query', > + 'list': 'users', > + 'usprop': ['registration'], > + 'ususers': [pywikibot.Page(self.site, u'Example'), > + pywikibot.Page(self.site, u'Example2')], > + } > + expectedresult = {u'users': [ > + { > + u'userid': 215131, > + u'name': u'Example', > + u'registration': u'2005-03-19T00:17:19Z' > + }, > + { > + u'userid': 5176706, > + u'name': u'Example2', > + u'registration': u'2007-08-26T02:13:33Z' > + }, > + ]} > + self.assertEqualQueryResult(params, expectedresult) > + > + def test_titles_User(self): > + params = { > + 'action': 'query', > + 'list': 'users', > + 'usprop': ['registration'], > + 'ususers': [userlib.User(self.site, u'Example'), > + userlib.User(self.site, u'Example2')], > + } > + expectedresult = {u'users': [ > + { > + u'userid': 215131, > + u'name': u'Example', > + u'registration': u'2005-03-19T00:17:19Z' > + }, > + { > + u'userid': 5176706, > + u'name': u'Example2', > + u'registration': u'2007-08-26T02:13:33Z' > + }, > + ]} > + self.assertEqualQueryResult(params, expectedresult) > + > + def test_titles_Category(self): > + params = { > + 'action': 'query', > + 'prop': 'revisions', > + 'rvprop': ['ids', 'timestamp', 'user'], > + 'rvdir': 'newer', > + 'rvlimit': 1, > + 'titles': [catlib.Category(self.site, > u'Category:Categories')], > + } > + expectedresult = {u'pages': {u'794823': > + { > + u'ns': 14, > + u'pageid': 794823, > + u'revisions': [{ > + u'revid': 4494485, > + u'user': u'SEWilco', > + u'timestamp': u'2004-07-07T18:45:50Z', > + }], > + u'title': u'Category:Categories', > + }, > + }} > + self.assertEqualQueryResult(params, expectedresult) > + > if __name__ == "__main__": > unittest.main() > > > _______________________________________________ > Pywikipedia-svn mailing list > Pywikipedia-svn(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikipedia-svn >

8 13

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot