[Pywikipedia-l] replace.py -catr: max depth error

[Bináris]

I think I HAVE FOUND THE BUG!
(See the original problem below in quotation.)

First, I show you my experiment. This script works on the below mentioned
category that possibly has a loop.

  kat=u"Hungría"
  site=pywikibot.getSite('es')
  katpage=catlib.Category(site, kat)
  katlista=catlib.unique(list(katpage.subcategories(recurse=True,
cacheResults=True))) *#1*
  # katlista=katpage.subcategoriesList(recurse=True)#, cacheResults=True) *
#2*

This script with #1 working and #2 commented runs 12 minutes and gathers 655
categories.
With #2 working and #1 commented out runs very-very long and then exits with
the same Python (not pywiki) error as mentioned below -- maximum recursion
depth exceeded.
The only difference between them is that subcategoriesList has no *
cacheResults* parameter. I use subcategories instead with cacheResults, then
I make a unique list as subcategoriesList does.

Now, when I use the command *replace.py -catr:Hungría*, -catr in
pagenerators uses getCategoryGen, and getCategoryGen calls
CategorizedPageGenerator, and CategorizedPageGenerator calls
category.articles from catlib.py. Category.articles does have a parameter
called cacheResults just as subcategories does, but getCategoryGen and
getCategoryGen cannot handle it. So I think this should be hacked that -catr
can use category.articles with cache, and thus the generator could avoid
infinite loops in categories.

-- 
Bináris

2011/2/27 Bináris <wikiposta at gmail.com>

> Hi folks!
>
> I create pages for Hungarian Wikipedia like
> http://hu.wikipedia.org/wiki/Wikipédia:Kért_cikkek/fr,
> http://hu.wikipedia.org/wiki/Wikipédia:Kért_cikkek/en etc. These collect
> Hungary-related articles from other Wikipedias that have no Hungarian
> interwiki. Either they must be supplied with an iw or they are a good idea
> to write new articles.
>
> First I collect all the pages with replace.py, then I upload them and
> process the list with a newly developed script which I will soon offer for
> Pywikipedia because it can be used in other Wikipedias.
>
> Itt successfully ran in en, fr, ro wikis but stopped in eswiki.
> My command:
> *replace.py -catr:Hungría . @ -lang:es -excepttext:"[[hu:"
> -savenew:magyarok.txt -always*
>
> The error message follows here. As far as I understand it comes from Python
> rather than pywiki, but could we somehow handle it?
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 167, in
> _getContentsNaive
>     for item in page._getContentsNaive(recurse=True):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 164, in
> _getContentsNaive
>     for tag, page in self._parseCategory(startFrom=startFrom):
>   File "C:\Program Files\Pywikipedia\catlib.py", line 215, in
> _parseCategory
>     data = query.GetData(params, self.site())
>   File "C:\Program Files\Pywikipedia\query.py", line 132, in GetData
>     jsontext = json.loads( jsontext )
>   File "C:\Program Files\Pywikipedia\simplejson\__init__.py", line 262, in
> loads
>
>     return _default_decoder.decode(s)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 251, in
> decode
>
>     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 268, in
> raw_de
> code
>     obj, end = self._scanner.iterscan(s, **kw).next()
>   File "C:\Program Files\Pywikipedia\simplejson\scanner.py", line 50, in
> itersca
> n
>     rval, next_pos = action(m, context)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 138, in
> JSONOb
> ject
>     value, end = iterscan(s, idx=end, context=context).next()
>   File "C:\Program Files\Pywikipedia\simplejson\scanner.py", line 50, in
> itersca
> n
>     rval, next_pos = action(m, context)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 138, in
> JSONOb
> ject
>     value, end = iterscan(s, idx=end, context=context).next()
>   File "C:\Program Files\Pywikipedia\simplejson\scanner.py", line 50, in
> itersca
> n
>     rval, next_pos = action(m, context)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 171, in
> JSONAr
> ray
>     value, end = iterscan(s, idx=end, context=context).next()
>   File "C:\Program Files\Pywikipedia\simplejson\scanner.py", line 50, in
> itersca
> n
>     rval, next_pos = action(m, context)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 138, in
> JSONOb
> ject
>     value, end = iterscan(s, idx=end, context=context).next()
>   File "C:\Program Files\Pywikipedia\simplejson\scanner.py", line 50, in
> itersca
> n
>     rval, next_pos = action(m, context)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 113, in
> JSONSt
> ring
>     return scanstring(match.string, match.end(), encoding)
>   File "C:\Program Files\Pywikipedia\simplejson\decoder.py", line 85, in
> scanstr
> ing
>     if terminator == '"':
> RuntimeError: maximum recursion depth exceeded in cmp
> maximum recursion depth exceeded in cmp
> 935 titles were saved.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/pywikipedia-l/attachments/20110309/7e9ed97f/attachment-0001.htm