[Pywikipedia-l] SVN: [6477] branches/rewrite/pywikibot

Revision: 6477 Author: nicdumz Date: 2009-03-03 10:10:50 +0000 (Tue, 03 Mar 2009) Log Message: ----------- svn propset svn:eol-style native -R . Modified Paths: -------------- branches/rewrite/pywikibot/README-conversion.txt branches/rewrite/pywikibot/__init__.py branches/rewrite/pywikibot/bot.py branches/rewrite/pywikibot/comms/http.py branches/rewrite/pywikibot/epydoc.cfg branches/rewrite/pywikibot/scripts/__init__.py branches/rewrite/pywikibot/scripts/touch.py branches/rewrite/pywikibot/tests/api_tests.py branches/rewrite/pywikibot/tests/page_tests.py branches/rewrite/pywikibot/tests/site_tests.py Property Changed: ---------------- branches/rewrite/pywikibot/README-conversion.txt branches/rewrite/pywikibot/__init__.py branches/rewrite/pywikibot/bot.py branches/rewrite/pywikibot/comms/__init__.py branches/rewrite/pywikibot/comms/http.py branches/rewrite/pywikibot/comms/threadedhttp.py branches/rewrite/pywikibot/data/logentries.py branches/rewrite/pywikibot/epydoc.cfg branches/rewrite/pywikibot/families/__init__.py branches/rewrite/pywikibot/families/osm_family.py branches/rewrite/pywikibot/scripts/__init__.py branches/rewrite/pywikibot/scripts/touch.py branches/rewrite/pywikibot/tests/__init__.py branches/rewrite/pywikibot/tests/api_tests.py branches/rewrite/pywikibot/tests/page_tests.py branches/rewrite/pywikibot/tests/site_tests.py branches/rewrite/pywikibot/userinterfaces/__init__.py Modified: branches/rewrite/pywikibot/README-conversion.txt =================================================================== --- branches/rewrite/pywikibot/README-conversion.txt 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/README-conversion.txt 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,110 +1,110 @@ -This is a guide to converting bot scripts from version 1 of the -Pywikipediabot framework to version 2. - -Most importantly, note that the version 2 framework *only* supports wikis -using MediaWiki v.1.14 or higher software. If you need to access a wiki that -uses older software, you should continue using version 1 for this purpose. - -The root namespace used in the project has changed from "wikipedia" -to "pywikibot". References to wikipedia need to be changed globally to -pywikibot. Unless noted in this document, other names have not changed; for -example, wikipedia.Page can be replaced by pywikibot.Page throughout any -bot. An effort has been made to design the interface to be as backwards- -compatible as possible, so that in most cases it should be possible to convert -scripts to the new interface simply by changing import statements and doing -global search-and-replace on module names, as discussed in this document. - -With pywikipedia scripts were importing "wikipedia" or "pagegenerators" -libraries; pywikibot is now written as a standard package, and other modules -are contained within it (e.g., pywikibot.site contains Site classes). However, -most commonly-used names are imported into the pywikibot namespace, so that -module names don't need to be used unless specified in the documentation. - -Make sure that the directory that contains the "pywikibot" subdirectory (or -folder) is in sys.path. - -The following changes, at a minimum, need to be made to allow scripts to run: - - change "import wikipedia" to "import pywikibot" - change "import pagegenerators" to "from pywikibot import pagegenerators" - change "import config" to "from pywikibot import config" - change "import catlib" to "from pywikibot import catlib" - change "wikipedia." to "pywikibot." - -wikipedia.setAction() no longer works; you must revise the script to pass an -explicit edit summary message on each put() or put_async() call. - -== Python librairies == - -[Note: the goal will be to package pywikibot with setuptools easy_install, -so that these dependencies will be loaded automatically when the package is -installed, and users won't need to worry about this...] - -To run pywikibot, you will need the httplib2, simplejson, and setuptools -packages-- -* httplib2 : http://code.google.com/p/httplib2/ -* setuptools : http://pypi.python.org/pypi/setuptools/ -* simplejson : http://svn.red-bean.com/bob/simplejson/tags/simplejson-1.7.1/docs/index.html - -or, if you already have setuptools installed, just execute -'easy_install httplib2' and 'easy_install simplejson' - -If you run into errors involving httplib2.urlnorm, update httplib2 to 0.4.0 -(Ubuntu package python-httlib2, for example, is outdated). Note that -httplib2 will run under Python 2.6, but will emit DeprecationWarnings (which -are annoying but don't affect the ability to use the package). - -== Page objects == - -The constructor syntax for Pages has been modified; existing calls in the -format of Page(site, title) will still work, and this is still the preferred -way of creating a Page object from data retrieved from the MediaWiki API -(because the API will have parsed and normalized the title). However, for -titles input by a user or scraped from wikitext, it is preferred to use the -alternative syntax Page(Link(site, wikitext)), where "wikitext" is the -string found between [[ and ]] delimiters. The new Link object (more on -this below) handles link parsing and interpretation that doesn't require -access to the wiki server. - -A third syntax allows easy conversion from a Page object to an ImagePage or -Category, or vice versa: e.g., Category(pageobj) converts a Page to a -Category, as long as the page is in the category namespace. - -The following methods of the Page object have been deprecated (deprecated -methods still work, but print a warning message in debug mode): - -- urlname(): replaced by Page.title(asUrl=True) -- titleWithoutNamespace(): replaced by Page.title(withNamespace=False) -- sectionFreeTitle(): replaced by Page.title(withSection=False) -- aslink(): replaced by Page.title(asLink=True) -- encoding(): replaced by Page.site().encoding() - -The following methods of the Page object have been obsoleted and no longer -work (but these methods don't appear to be used anywhere in the code -distributed with the bot framework). The functionality of the two obsolete -methods is easily replaced by using standard search-and-replace techniques. -If you call them, they will print a warning and do nothing else: - -- removeImage() -- replaceImage() - -=== ImagePage objects === - -For ImagePage objects, the getFileMd5Sum() method is deprecated; it is -recommended to replace it with getFileSHA1Sum(), because MediaWiki now -stores the SHA1 hash of images. - -=== Category objects === - -The Category object has been moved from the catlib module to the pywikibot -namespace. Any references to "catlib.Category" can be replaced by -"pywikibot.Category", but the old form is retained for backwards-compatibility. - -For Category objects, the following methods are deprecated: - -- subcategoriesList: use, for example, list(self.subcategories()) instead -- articlesList: use, for example, list(self.articles()) instead -- supercategories: use self.categories() instead -- supercategoriesList: use, for example, list(self.categories()) instead - -# MORE TO COME # +This is a guide to converting bot scripts from version 1 of the +Pywikipediabot framework to version 2. + +Most importantly, note that the version 2 framework *only* supports wikis +using MediaWiki v.1.14 or higher software. If you need to access a wiki that +uses older software, you should continue using version 1 for this purpose. + +The root namespace used in the project has changed from "wikipedia" +to "pywikibot". References to wikipedia need to be changed globally to +pywikibot. Unless noted in this document, other names have not changed; for +example, wikipedia.Page can be replaced by pywikibot.Page throughout any +bot. An effort has been made to design the interface to be as backwards- +compatible as possible, so that in most cases it should be possible to convert +scripts to the new interface simply by changing import statements and doing +global search-and-replace on module names, as discussed in this document. + +With pywikipedia scripts were importing "wikipedia" or "pagegenerators" +libraries; pywikibot is now written as a standard package, and other modules +are contained within it (e.g., pywikibot.site contains Site classes). However, +most commonly-used names are imported into the pywikibot namespace, so that +module names don't need to be used unless specified in the documentation. + +Make sure that the directory that contains the "pywikibot" subdirectory (or +folder) is in sys.path. + +The following changes, at a minimum, need to be made to allow scripts to run: + + change "import wikipedia" to "import pywikibot" + change "import pagegenerators" to "from pywikibot import pagegenerators" + change "import config" to "from pywikibot import config" + change "import catlib" to "from pywikibot import catlib" + change "wikipedia." to "pywikibot." + +wikipedia.setAction() no longer works; you must revise the script to pass an +explicit edit summary message on each put() or put_async() call. + +== Python librairies == + +[Note: the goal will be to package pywikibot with setuptools easy_install, +so that these dependencies will be loaded automatically when the package is +installed, and users won't need to worry about this...] + +To run pywikibot, you will need the httplib2, simplejson, and setuptools +packages-- +* httplib2 : http://code.google.com/p/httplib2/ +* setuptools : http://pypi.python.org/pypi/setuptools/ +* simplejson : http://svn.red-bean.com/bob/simplejson/tags/simplejson-1.7.1/docs/index.html + +or, if you already have setuptools installed, just execute +'easy_install httplib2' and 'easy_install simplejson' + +If you run into errors involving httplib2.urlnorm, update httplib2 to 0.4.0 +(Ubuntu package python-httlib2, for example, is outdated). Note that +httplib2 will run under Python 2.6, but will emit DeprecationWarnings (which +are annoying but don't affect the ability to use the package). + +== Page objects == + +The constructor syntax for Pages has been modified; existing calls in the +format of Page(site, title) will still work, and this is still the preferred +way of creating a Page object from data retrieved from the MediaWiki API +(because the API will have parsed and normalized the title). However, for +titles input by a user or scraped from wikitext, it is preferred to use the +alternative syntax Page(Link(site, wikitext)), where "wikitext" is the +string found between [[ and ]] delimiters. The new Link object (more on +this below) handles link parsing and interpretation that doesn't require +access to the wiki server. + +A third syntax allows easy conversion from a Page object to an ImagePage or +Category, or vice versa: e.g., Category(pageobj) converts a Page to a +Category, as long as the page is in the category namespace. + +The following methods of the Page object have been deprecated (deprecated +methods still work, but print a warning message in debug mode): + +- urlname(): replaced by Page.title(asUrl=True) +- titleWithoutNamespace(): replaced by Page.title(withNamespace=False) +- sectionFreeTitle(): replaced by Page.title(withSection=False) +- aslink(): replaced by Page.title(asLink=True) +- encoding(): replaced by Page.site().encoding() + +The following methods of the Page object have been obsoleted and no longer +work (but these methods don't appear to be used anywhere in the code +distributed with the bot framework). The functionality of the two obsolete +methods is easily replaced by using standard search-and-replace techniques. +If you call them, they will print a warning and do nothing else: + +- removeImage() +- replaceImage() + +=== ImagePage objects === + +For ImagePage objects, the getFileMd5Sum() method is deprecated; it is +recommended to replace it with getFileSHA1Sum(), because MediaWiki now +stores the SHA1 hash of images. + +=== Category objects === + +The Category object has been moved from the catlib module to the pywikibot +namespace. Any references to "catlib.Category" can be replaced by +"pywikibot.Category", but the old form is retained for backwards-compatibility. + +For Category objects, the following methods are deprecated: + +- subcategoriesList: use, for example, list(self.subcategories()) instead +- articlesList: use, for example, list(self.articles()) instead +- supercategories: use self.categories() instead +- supercategoriesList: use, for example, list(self.categories()) instead + +# MORE TO COME # Property changes on: branches/rewrite/pywikibot/README-conversion.txt ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/__init__.py =================================================================== --- branches/rewrite/pywikibot/__init__.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/__init__.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,248 +1,248 @@ -# -*- coding: utf-8 -*- -""" -The initialization file for the Pywikibot framework. -""" -# -# (C) Pywikipedia bot team, 2008 -# -# Distributed under the terms of the MIT license. -# -__version__ = '$Id$' - -import difflib -import logging -import re -import sys - -import config2 as config -from bot import * -from exceptions import * -from textlib import * - -logging.basicConfig(fmt="%(message)s") - - -def deprecated(instead=None): - """Outputs a method deprecation warning. - Uses the stack to determine class and name of calling method - - @param instead: if provided, will be used to specify the replacement - @type instead: string - """ - f = sys._getframe(1) - classname = f.f_locals['self'].__class__.__name__ - funcname = f.f_code.co_name - if instead: - output(u"%s.%s is DEPRECATED, use %s instead." - % (classname, funcname, instead), - level=WARNING) - else: - output(u"%s.%s is DEPRECATED." % (classname, funcname), - level=WARNING) - -def deprecate_arg(old_arg, new_arg): - """Decorator to declare old_arg deprecated and replace it with new_arg""" - logger = logging.getLogger() - def decorator(method): - def wrapper(*__args, **__kw): - meth_name = method.__name__ - if old_arg in __kw: - if new_arg: - if new_arg in __kw: - pywikibot.output( -"%(new_arg)s argument of %(meth_name)s replaces %(old_arg)s; cannot use both." - % locals(), level=WARNING) - else: - pywikibot.output( -"%(old_arg)s argument of %(meth_name)s is deprecated; use %(new_arg)s instead." - % locals(), level=DEBUG) - __kw[new_arg] = __kw[old_arg] - else: - pywikibot.output( - "%(old_arg)s argument of %(meth_name)s is deprecated." - % locals(), level=DEBUG) - del __kw[old_arg] - return method(*__args, **__kw) - wrapper.__doc__ = method.__doc__ - wrapper.__name__ = method.__name__ - return wrapper - return decorator - - -_sites = {} - -@deprecate_arg("persistent_http", None) -def Site(code=None, fam=None, user=None, sysop=None, interface=None): - """Return the specified Site object. - - Returns a cached object if possible, otherwise instantiates a new one. - - @param code: language code - @type code: string - @param fam: family name or object - @type fam: string or Family - @param user: bot user name to use on this site - @type user: unicode - - """ - logger = logging.getLogger("wiki") - - if code is None: - code = config.mylang - if fam is None: - fam = config.family - if user is None: - try: - user = config.usernames[fam][code] - except KeyError: - user = None - if sysop is None: - try: - sysop = config.sysopnames[fam][code] - except KeyError: - sysop = None - if interface is None: - interface = config.site_interface - try: - exec "from site import %s as __Site" % interface - except ImportError: - raise ValueError("Invalid interface name '%(interface)s'" % locals()) - key = '%s:%s:%s' % (fam, code, user) - if not key in _sites: - _sites[key] = __Site(code=code, fam=fam, user=user, sysop=sysop) - logging.debug(u"Instantiating Site object '%(site)s'" - % {'site': _sites[key]}) - return _sites[key] - -getSite = Site # alias for backwards-compability - - -from page import Page, ImagePage, Category, Link - - -link_regex = re.compile(r'\[\[(?P<title>[^\]|[#<>{}]*)(\|.*?)?\]\]') - - -def setAction(s): - """Set a summary to use for changed page submissions""" - config.default_edit_summary = s - - -def set_debug(layer): - """Set the logger for specified layer to DEBUG level. - - The framework has four layers (by default, others can be added), each - designated by a string -- - - 1. "comm": the communication layer (http requests, etc.) - 2. "data": the raw data layer (API requests, XML dump parsing) - 3. "wiki": the wiki content representation layer (Page and Site objects) - 4. "bot": the application layer - - This method sets the logger for any specified layer to the DEBUG level, - causing it to output extensive debugging information. If this method is - not called for a layer, the default logging setting is the INFO level. - - This method does not check the 'layer' argument for validity. - - """ - logging.getLogger(layer).setLevel(DEBUG) - - -def showDiff(oldtext, newtext): - """ - Output a string showing the differences between oldtext and newtext. - The differences are highlighted (only on compatible systems) to show which - changes were made. - - """ - # This is probably not portable to non-terminal interfaces.... - # For information on difflib, see http://pydoc.org/2.3/difflib.html - color = { - '+': 'lightgreen', - '-': 'lightred', - } - diff = u'' - colors = [] - # This will store the last line beginning with + or -. - lastline = None - # For testing purposes only: show original, uncolored diff - # for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()): - # print line - for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()): - if line.startswith('?'): - # initialize color vector with None, which means default color - lastcolors = [None for c in lastline] - # colorize the + or - sign - lastcolors[0] = color[lastline[0]] - # colorize changed parts in red or green - for i in range(min(len(line), len(lastline))): - if line[i] != ' ': - lastcolors[i] = color[lastline[0]] - diff += lastline + '\n' - # append one None (default color) for the newline character - colors += lastcolors + [None] - elif lastline: - diff += lastline + '\n' - # colorize the + or - sign only - lastcolors = [None for c in lastline] - lastcolors[0] = color[lastline[0]] - colors += lastcolors + [None] - lastline = None - if line[0] in ('+', '-'): - lastline = line - # there might be one + or - line left that wasn't followed by a ? line. - if lastline: - diff += lastline + '\n' - # colorize the + or - sign only - lastcolors = [None for c in lastline] - lastcolors[0] = color[lastline[0]] - colors += lastcolors + [None] - - result = u'' - lastcolor = None - for i in range(len(diff)): - if colors[i] != lastcolor: - if lastcolor is None: - result += '\03{%s}' % colors[i] - else: - result += '\03{default}' - lastcolor = colors[i] - result += diff[i] - output(result) - - -# Throttle and thread handling - -threadpool = [] # add page-putting threads to this list as they are created -stopped = False - -def stopme(): - """Drop this process from the throttle log, after pending threads finish. - - Can be called manually if desired, but if not, will be called automatically - at Python exit. - - """ - global stopped - logger = logging.getLogger("wiki") - - if not stopped: - logging.debug("stopme() called") - count = sum(1 for thd in threadpool if thd.isAlive()) - if count: - pywikibot.output(u"Waiting for about %(count)s pages to be saved." - % locals()) - for thd in threadpool: - if thd.isAlive(): - thd.join() - stopped = True - # only need one drop() call because all throttles use the same global pid - try: - _sites[_sites.keys()[0]].throttle.drop() - pywikibot.output(u"Dropped throttle(s).", level=VERBOSE) - except IndexError: - pass - -import atexit -atexit.register(stopme) +# -*- coding: utf-8 -*- +""" +The initialization file for the Pywikibot framework. +""" +# +# (C) Pywikipedia bot team, 2008 +# +# Distributed under the terms of the MIT license. +# +__version__ = '$Id$' + +import difflib +import logging +import re +import sys + +import config2 as config +from bot import * +from exceptions import * +from textlib import * + +logging.basicConfig(fmt="%(message)s") + + +def deprecated(instead=None): + """Outputs a method deprecation warning. + Uses the stack to determine class and name of calling method + + @param instead: if provided, will be used to specify the replacement + @type instead: string + """ + f = sys._getframe(1) + classname = f.f_locals['self'].__class__.__name__ + funcname = f.f_code.co_name + if instead: + output(u"%s.%s is DEPRECATED, use %s instead." + % (classname, funcname, instead), + level=WARNING) + else: + output(u"%s.%s is DEPRECATED." % (classname, funcname), + level=WARNING) + +def deprecate_arg(old_arg, new_arg): + """Decorator to declare old_arg deprecated and replace it with new_arg""" + logger = logging.getLogger() + def decorator(method): + def wrapper(*__args, **__kw): + meth_name = method.__name__ + if old_arg in __kw: + if new_arg: + if new_arg in __kw: + pywikibot.output( +"%(new_arg)s argument of %(meth_name)s replaces %(old_arg)s; cannot use both." + % locals(), level=WARNING) + else: + pywikibot.output( +"%(old_arg)s argument of %(meth_name)s is deprecated; use %(new_arg)s instead." + % locals(), level=DEBUG) + __kw[new_arg] = __kw[old_arg] + else: + pywikibot.output( + "%(old_arg)s argument of %(meth_name)s is deprecated." + % locals(), level=DEBUG) + del __kw[old_arg] + return method(*__args, **__kw) + wrapper.__doc__ = method.__doc__ + wrapper.__name__ = method.__name__ + return wrapper + return decorator + + +_sites = {} + +@deprecate_arg("persistent_http", None) +def Site(code=None, fam=None, user=None, sysop=None, interface=None): + """Return the specified Site object. + + Returns a cached object if possible, otherwise instantiates a new one. + + @param code: language code + @type code: string + @param fam: family name or object + @type fam: string or Family + @param user: bot user name to use on this site + @type user: unicode + + """ + logger = logging.getLogger("wiki") + + if code is None: + code = config.mylang + if fam is None: + fam = config.family + if user is None: + try: + user = config.usernames[fam][code] + except KeyError: + user = None + if sysop is None: + try: + sysop = config.sysopnames[fam][code] + except KeyError: + sysop = None + if interface is None: + interface = config.site_interface + try: + exec "from site import %s as __Site" % interface + except ImportError: + raise ValueError("Invalid interface name '%(interface)s'" % locals()) + key = '%s:%s:%s' % (fam, code, user) + if not key in _sites: + _sites[key] = __Site(code=code, fam=fam, user=user, sysop=sysop) + logging.debug(u"Instantiating Site object '%(site)s'" + % {'site': _sites[key]}) + return _sites[key] + +getSite = Site # alias for backwards-compability + + +from page import Page, ImagePage, Category, Link + + +link_regex = re.compile(r'\[\[(?P<title>[^\]|[#<>{}]*)(\|.*?)?\]\]') + + +def setAction(s): + """Set a summary to use for changed page submissions""" + config.default_edit_summary = s + + +def set_debug(layer): + """Set the logger for specified layer to DEBUG level. + + The framework has four layers (by default, others can be added), each + designated by a string -- + + 1. "comm": the communication layer (http requests, etc.) + 2. "data": the raw data layer (API requests, XML dump parsing) + 3. "wiki": the wiki content representation layer (Page and Site objects) + 4. "bot": the application layer + + This method sets the logger for any specified layer to the DEBUG level, + causing it to output extensive debugging information. If this method is + not called for a layer, the default logging setting is the INFO level. + + This method does not check the 'layer' argument for validity. + + """ + logging.getLogger(layer).setLevel(DEBUG) + + +def showDiff(oldtext, newtext): + """ + Output a string showing the differences between oldtext and newtext. + The differences are highlighted (only on compatible systems) to show which + changes were made. + + """ + # This is probably not portable to non-terminal interfaces.... + # For information on difflib, see http://pydoc.org/2.3/difflib.html + color = { + '+': 'lightgreen', + '-': 'lightred', + } + diff = u'' + colors = [] + # This will store the last line beginning with + or -. + lastline = None + # For testing purposes only: show original, uncolored diff + # for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()): + # print line + for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()): + if line.startswith('?'): + # initialize color vector with None, which means default color + lastcolors = [None for c in lastline] + # colorize the + or - sign + lastcolors[0] = color[lastline[0]] + # colorize changed parts in red or green + for i in range(min(len(line), len(lastline))): + if line[i] != ' ': + lastcolors[i] = color[lastline[0]] + diff += lastline + '\n' + # append one None (default color) for the newline character + colors += lastcolors + [None] + elif lastline: + diff += lastline + '\n' + # colorize the + or - sign only + lastcolors = [None for c in lastline] + lastcolors[0] = color[lastline[0]] + colors += lastcolors + [None] + lastline = None + if line[0] in ('+', '-'): + lastline = line + # there might be one + or - line left that wasn't followed by a ? line. + if lastline: + diff += lastline + '\n' + # colorize the + or - sign only + lastcolors = [None for c in lastline] + lastcolors[0] = color[lastline[0]] + colors += lastcolors + [None] + + result = u'' + lastcolor = None + for i in range(len(diff)): + if colors[i] != lastcolor: + if lastcolor is None: + result += '\03{%s}' % colors[i] + else: + result += '\03{default}' + lastcolor = colors[i] + result += diff[i] + output(result) + + +# Throttle and thread handling + +threadpool = [] # add page-putting threads to this list as they are created +stopped = False + +def stopme(): + """Drop this process from the throttle log, after pending threads finish. + + Can be called manually if desired, but if not, will be called automatically + at Python exit. + + """ + global stopped + logger = logging.getLogger("wiki") + + if not stopped: + logging.debug("stopme() called") + count = sum(1 for thd in threadpool if thd.isAlive()) + if count: + pywikibot.output(u"Waiting for about %(count)s pages to be saved." + % locals()) + for thd in threadpool: + if thd.isAlive(): + thd.join() + stopped = True + # only need one drop() call because all throttles use the same global pid + try: + _sites[_sites.keys()[0]].throttle.drop() + pywikibot.output(u"Dropped throttle(s).", level=VERBOSE) + except IndexError: + pass + +import atexit +atexit.register(stopme) Property changes on: branches/rewrite/pywikibot/__init__.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/bot.py =================================================================== --- branches/rewrite/pywikibot/bot.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/bot.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,435 +1,435 @@ -# -*- coding: utf-8 -*- -""" -User-interface related functions for building bots -""" -# -# (C) Pywikipedia bot team, 2008 -# -# Distributed under the terms of the MIT license. -# -__version__ = '$Id$' - -# Note: the intention is to develop this module (at some point) into a Bot -# class definition that can be subclassed to create new, functional bot -# scripts, instead of writing each one from scratch. - - -import logging, logging.handlers - # all output goes thru python std library "logging" module -import os -import os.path -import sys - -# logging levels -logger = logging.getLogger("bot") - -from logging import DEBUG, INFO, WARNING, ERROR, CRITICAL -STDOUT = 16 -VERBOSE = 18 -INPUT = 25 - -import pywikibot -from pywikibot import config - - -class MaxLevelFilter(logging.Filter): - """Filter that only passes records at or below a specific level. - - (setting handler level only passes records at or *above* a specified level, - so this provides the opposite functionality) - - """ - def __init__(self, level=None): - self.level = level - - def filter(self, record): - if self.level: - return record.levelno <= self.level - else: - return True - - -# User interface initialization -# search for user interface module in the 'userinterfaces' subdirectory -exec ("import pywikibot.userinterfaces.%s_interface as uiModule" - % config.userinterface) -ui = uiModule.UI() -TerminalHandler = uiModule.TerminalHandler - - -class RotatingFileHandler(logging.handlers.RotatingFileHandler): - """Strip trailing newlines before outputting text to file""" - def format(self, record): - text = logging.handlers.RotatingFileHandler.format(self, record) - return text.rstrip("\r\n") - - -def output(text, decoder=None, newline=True, toStdout=False, level=INFO): - """Output a message to the user via the userinterface. - - Works like print, but uses the encoding used by the user's console - (console_encoding in the configuration file) instead of ASCII. - If decoder is None, text should be a unicode string. Otherwise it - should be encoded in the given encoding. - - If newline is True, a linebreak will be added after printing the text. - - If toStdout is True, the text will be sent to standard output, - so that it can be piped to another process. All other text will - be sent to stderr. See: http://en.wikipedia.org/wiki/Pipeline_%28Unix%29 - - text can contain special sequences to create colored output. These - consist of the escape character \03 and the color name in curly braces, - e. g. \03{lightpurple}. \03{default} resets the color. - - @param level: output level for logging module; use VERBOSE for optional - messages, INPUT for prompts requiring user reponse (not yet fully - implemented) - - """ - # make sure logging system has been initialized - root = logging.getLogger() - if root.level == 30: # init_handlers sets this level - init_handlers() - - if decoder: - text = unicode(text, decoder) - elif not isinstance(text, unicode): -## import traceback -## pywikibot.output( -## u"Non-unicode (%s) passed to wikipedia.output without decoder!\n" -## % type(text), -## level=VERBOSE -## ) - if not isinstance(text, str): - # looks like text is a non-text object. - # Maybe it has a __unicode__ builtin ? - # (allows to print Page, Site...) - text = unicode(text) - else: - try: - text = unicode(text, 'utf-8') - except UnicodeDecodeError: - text = unicode(text, 'iso8859-1') - if newline: - text += "\n" - if toStdout: - level = STDOUT - ui.output(text, level=level) - -def input(question, password=False): - """Ask the user a question, return the user's answer. - - Parameters: - * question - a unicode string that will be shown to the user. Don't add a - space after the question mark/colon, this method will do this - for you. - * password - if True, hides the user's input (for password entry). - - Returns a unicode string. - - """ - # make sure logging system has been initialized - root = logging.getLogger() - if root.level == 30: # init_handlers sets this level - init_handlers() - - data = ui.input(question, password) - return data - -def inputChoice(question, answers, hotkeys, default=None): - """Ask the user a question with several options, return the user's choice. - - The user's input will be case-insensitive, so the hotkeys should be - distinctive case-insensitively. - - Parameters: - * question - a unicode string that will be shown to the user. Don't add a - space after the question mark, this method will do this - for you. - * answers - a list of strings that represent the options. - * hotkeys - a list of one-letter strings, one for each answer. - * default - an element of hotkeys, or None. The default choice that will - be returned when the user just presses Enter. - - Returns a one-letter string in lowercase. - - """ - # make sure logging system has been initialized - root = logging.getLogger() - if root.level == 30: # init_handlers sets this level - init_handlers() - - data = ui.inputChoice(question, answers, hotkeys, default).lower() - return data - - -def init_handlers(): - """Initialize logging system for terminal-based bots""" - - # All user output is routed through the logging module. - # Each type of output is handled by an appropriate handler object. - # This structure is used to permit eventual development of other - # user interfaces (GUIs) without modifying the core bot code. - # The following output levels are defined: - # DEBUG - only for file logging; debugging messages - # STDOUT - output that must be sent to sys.stdout (for bots that may - # have their output redirected to a file or other destination) - # VERBOSE - optional progress information for display to user - # INFO - normal (non-optional) progress information for display to user - # INPUT - prompts requiring user response - # WARN - user warning messages - # ERROR - user error messages - # CRITICAL - fatal error messages - # Accordingly, do ''not'' use print statements in bot code; instead, - # use pywikibot.output function. - - moduleName = calledModuleName() - if not moduleName: - moduleName = "terminal-interface" - - logging.addLevelName(VERBOSE, "VERBOSE") - # for messages to be displayed on terminal at "verbose" setting - # use INFO for messages to be displayed even on non-verbose setting - logging.addLevelName(STDOUT, "STDOUT") - # for messages to be displayed to stdout - logging.addLevelName(INPUT, "INPUT") - # for prompts requiring user response - - root_logger = logging.getLogger() - root_logger.handlers = [] # get rid of default handler - root_logger.setLevel(DEBUG+1) # all records except DEBUG go to logger - - # configure default handler for VERBOSE and INFO levels - default_handler = TerminalHandler(strm=sys.stderr) - if config.verbose_output: - default_handler.setLevel(VERBOSE) - else: - default_handler.setLevel(INFO) - default_handler.addFilter(MaxLevelFilter(INPUT)) - default_handler.setFormatter(logging.Formatter(fmt="%(message)s")) - root_logger.addHandler(default_handler) - - # if user has enabled file logging, configure file handler - if moduleName in config.log or '*' in config.log: - if config.logfilename: - logfile = config.datafilepath(config.logfilename) - else: - logfile = config.datafilepath("%s-bot.log" % moduleName) - file_handler = RotatingFileHandler( - filename=logfile, maxBytes=2 << 20, backupCount=5) - - file_handler.setLevel(DEBUG) - form = logging.Formatter( - fmt="%(asctime)s %(filename)18s, %(lineno)d: " - "%(levelname)-8s %(message)s", - datefmt="%Y-%m-%d %H:%M:%S" - ) - file_handler.setFormatter(form) - root_logger.addHandler(file_handler) - for component in config.debug_log: - debuglogger = logging.getLogger(component) - debuglogger.setLevel(DEBUG) - debuglogger.addHandler(file_handler) - - # handler for level STDOUT - output_handler = TerminalHandler(strm=sys.stdout) - output_handler.setLevel(STDOUT) - output_handler.addFilter(MaxLevelFilter(STDOUT)) - output_handler.setFormatter(logging.Formatter(fmt="%(message)s")) - root_logger.addHandler(output_handler) - - # handler for levels WARNING and higher - warning_handler = TerminalHandler(strm=sys.stderr) - warning_handler.setLevel(logging.WARNING) - warning_handler.setFormatter( - logging.Formatter(fmt="%(levelname)s: %(message)s")) - root_logger.addHandler(warning_handler) - - -# Command line parsing and help - -def calledModuleName(): - """Return the name of the module calling this function. - - This is required because the -help option loads the module's docstring - and because the module name will be used for the filename of the log. - - """ - # get commandline arguments - called = sys.argv[0].strip() - if ".py" in called: # could end with .pyc, .pyw, etc. on some platforms - called = called[ : called.rindex(".py")] - return os.path.basename(called) - -def _decodeArg(arg): - if sys.platform=='win32': - if config.console_encoding in ("cp437", 'cp850'): - # Western Windows versions give parameters encoded as windows-1252 - # even though the console encoding is cp850 or cp437. - return unicode(arg, 'windows-1252') - elif config.console_encoding == 'cp852': - # Central/Eastern European Windows versions give parameters encoded - # as windows-1250 even though the console encoding is cp852. - return unicode(arg, 'windows-1250') - else: - return unicode(arg, config.console_encoding) - else: - # Linux uses the same encoding for both. - # I don't know how non-Western Windows versions behave. - return unicode(arg, config.console_encoding) - -def handleArgs(*args): - """Handle standard command line arguments, return the rest as a list. - - Takes the commandline arguments, converts them to Unicode, processes all - global parameters such as -lang or -log. Returns a list of all arguments - that are not global. This makes sure that global arguments are applied - first, regardless of the order in which the arguments were given. - - args may be passed as an argument, thereby overriding sys.argv - - """ - # get commandline arguments if necessary - if not args: - args = sys.argv[1:] - # get the name of the module calling this function. This is - # required because the -help option loads the module's docstring and because - # the module name will be used for the filename of the log. - moduleName = calledModuleName() - if not moduleName: - moduleName = "terminal-interface" - nonGlobalArgs = [] - username = None - do_help = False - for arg in args: - arg = _decodeArg(arg) - if arg == '-help': - do_help = True - elif arg.startswith('-family:'): - config.family = arg[len("-family:") : ] - elif arg.startswith('-lang:'): - config.mylang = arg[len("-lang:") : ] - elif arg.startswith("-user:"): - username = arg[len("-user:") : ] - elif arg.startswith('-putthrottle:'): - config.put_throttle = int(arg[len("-putthrottle:") : ]) - elif arg.startswith('-pt:'): - config.put_throttle = int(arg[len("-pt:") : ]) - elif arg.startswith("-maxlag:"): - config.maxlag = int(arg[len("-maxlag:") : ]) - elif arg == '-log': - if moduleName not in config.log: - config.log.append(moduleName) - elif arg.startswith('-log:'): - if moduleName not in config.log: - config.log.append(moduleName) - config.logfilename = arg[len("-log:") : ] - elif arg == '-nolog': - if moduleName in config.log: - config.log.remove(moduleName) - elif arg == "-debug": - if moduleName not in config.log: - config.log.append(moduleName) - if "" not in config.debug_log: - config.debug_log.append("") - elif arg.startswith("-debug:"): - if moduleName not in config.log: - config.log.append(moduleName) - component = arg[len("-debug:") : ] - if component not in config.debug_log: - config.debug_log.append(component) - elif arg == '-verbose' or arg == "-v": - config.verbose_output += 1 - elif arg == '-daemonize': - import daemonize - daemonize.daemonize() - elif arg.startswith('-daemonize:'): - import daemonize - daemonize.daemonize(redirect_std = arg[11:]) - else: - # the argument is not global. Let the specific bot script care - # about it. - nonGlobalArgs.append(arg) - - if username: - config.usernames[config.family][config.mylang] = username - - init_handlers() - - if config.verbose_output: - import re - ver = pywikibot.__version__ # probably can be improved on - m = re.search(r"\$Id: .* (\d+ \d+-\d+-\d+ \d+:\d+:\d+Z) .*\$", ver) - pywikibot.output(u'Pywikipediabot r%s' % m.group(1)) - pywikibot.output(u'Python %s' % sys.version) - - if do_help: - showHelp() - sys.exit(0) - logger.debug(u"handleArgs() completed.") - return nonGlobalArgs - - -def showHelp(name=""): - # argument, if given, is ignored - modname = calledModuleName() - if not modname: - try: - modname = sys.modules['__main__'].main.__module__ - except NameError: - modname = "no_module" - - globalHelp =u'''\ -Global arguments available for all bots: - --dir:PATH Read the bot's configuration data from directory given by - PATH, instead of from the default directory. - --lang:xx Set the language of the wiki you want to work on, overriding - the configuration in user-config.py. xx should be the - language code. - --family:xyz Set the family of the wiki you want to work on, e.g. - wikipedia, wiktionary, wikitravel, ... - This will override the configuration in user-config.py. - --user:xyz Log in as user 'xyz' instead of the default username. - --daemonize:xyz Immediately return control to the terminal and redirect - stdout and stderr to xyz (only use for bots that require - no input from stdin). - --help Show this help text. - --log Enable the logfile, using the default filename - '%s-bot.log' - --log:xyz Enable the logfile, using 'xyz' as the filename. - --nolog Disable the logfile (if it is enabled by default). - --debug:item Enable the logfile and include extensive debugging data --debug for component "item" (or all components if the second form - is used). - --putthrottle:n Set the minimum time (in seconds) the bot will wait between --pt:n saving pages. - --verbose Have the bot provide additional console output that may be --v useful in debugging. - -''' % modname - try: - exec('import %s as module' % modname) - helpText = module.__doc__.decode('utf-8') - if hasattr(module, 'docuReplacements'): - for key, value in module.docuReplacements.iteritems(): - helpText = helpText.replace(key, value.strip('\n\r')) - pywikibot.output(helpText, level=pywikibot.STDOUT) # output to STDOUT - except Exception: - if modname: - pywikibot.output(u'Sorry, no help available for %s' % modname, - level=pywikibot.STDOUT) - logging.exception('showHelp:') - pywikibot.output(globalHelp, level=pywikibot.STDOUT) +# -*- coding: utf-8 -*- +""" +User-interface related functions for building bots +""" +# +# (C) Pywikipedia bot team, 2008 +# +# Distributed under the terms of the MIT license. +# +__version__ = '$Id$' + +# Note: the intention is to develop this module (at some point) into a Bot +# class definition that can be subclassed to create new, functional bot +# scripts, instead of writing each one from scratch. + + +import logging, logging.handlers + # all output goes thru python std library "logging" module +import os +import os.path +import sys + +# logging levels +logger = logging.getLogger("bot") + +from logging import DEBUG, INFO, WARNING, ERROR, CRITICAL +STDOUT = 16 +VERBOSE = 18 +INPUT = 25 + +import pywikibot +from pywikibot import config + + +class MaxLevelFilter(logging.Filter): + """Filter that only passes records at or below a specific level. + + (setting handler level only passes records at or *above* a specified level, + so this provides the opposite functionality) + + """ + def __init__(self, level=None): + self.level = level + + def filter(self, record): + if self.level: + return record.levelno <= self.level + else: + return True + + +# User interface initialization +# search for user interface module in the 'userinterfaces' subdirectory +exec ("import pywikibot.userinterfaces.%s_interface as uiModule" + % config.userinterface) +ui = uiModule.UI() +TerminalHandler = uiModule.TerminalHandler + + +class RotatingFileHandler(logging.handlers.RotatingFileHandler): + """Strip trailing newlines before outputting text to file""" + def format(self, record): + text = logging.handlers.RotatingFileHandler.format(self, record) + return text.rstrip("\r\n") + + +def output(text, decoder=None, newline=True, toStdout=False, level=INFO): + """Output a message to the user via the userinterface. + + Works like print, but uses the encoding used by the user's console + (console_encoding in the configuration file) instead of ASCII. + If decoder is None, text should be a unicode string. Otherwise it + should be encoded in the given encoding. + + If newline is True, a linebreak will be added after printing the text. + + If toStdout is True, the text will be sent to standard output, + so that it can be piped to another process. All other text will + be sent to stderr. See: http://en.wikipedia.org/wiki/Pipeline_%28Unix%29 + + text can contain special sequences to create colored output. These + consist of the escape character \03 and the color name in curly braces, + e. g. \03{lightpurple}. \03{default} resets the color. + + @param level: output level for logging module; use VERBOSE for optional + messages, INPUT for prompts requiring user reponse (not yet fully + implemented) + + """ + # make sure logging system has been initialized + root = logging.getLogger() + if root.level == 30: # init_handlers sets this level + init_handlers() + + if decoder: + text = unicode(text, decoder) + elif not isinstance(text, unicode): +## import traceback +## pywikibot.output( +## u"Non-unicode (%s) passed to wikipedia.output without decoder!\n" +## % type(text), +## level=VERBOSE +## ) + if not isinstance(text, str): + # looks like text is a non-text object. + # Maybe it has a __unicode__ builtin ? + # (allows to print Page, Site...) + text = unicode(text) + else: + try: + text = unicode(text, 'utf-8') + except UnicodeDecodeError: + text = unicode(text, 'iso8859-1') + if newline: + text += "\n" + if toStdout: + level = STDOUT + ui.output(text, level=level) + +def input(question, password=False): + """Ask the user a question, return the user's answer. + + Parameters: + * question - a unicode string that will be shown to the user. Don't add a + space after the question mark/colon, this method will do this + for you. + * password - if True, hides the user's input (for password entry). + + Returns a unicode string. + + """ + # make sure logging system has been initialized + root = logging.getLogger() + if root.level == 30: # init_handlers sets this level + init_handlers() + + data = ui.input(question, password) + return data + +def inputChoice(question, answers, hotkeys, default=None): + """Ask the user a question with several options, return the user's choice. + + The user's input will be case-insensitive, so the hotkeys should be + distinctive case-insensitively. + + Parameters: + * question - a unicode string that will be shown to the user. Don't add a + space after the question mark, this method will do this + for you. + * answers - a list of strings that represent the options. + * hotkeys - a list of one-letter strings, one for each answer. + * default - an element of hotkeys, or None. The default choice that will + be returned when the user just presses Enter. + + Returns a one-letter string in lowercase. + + """ + # make sure logging system has been initialized + root = logging.getLogger() + if root.level == 30: # init_handlers sets this level + init_handlers() + + data = ui.inputChoice(question, answers, hotkeys, default).lower() + return data + + +def init_handlers(): + """Initialize logging system for terminal-based bots""" + + # All user output is routed through the logging module. + # Each type of output is handled by an appropriate handler object. + # This structure is used to permit eventual development of other + # user interfaces (GUIs) without modifying the core bot code. + # The following output levels are defined: + # DEBUG - only for file logging; debugging messages + # STDOUT - output that must be sent to sys.stdout (for bots that may + # have their output redirected to a file or other destination) + # VERBOSE - optional progress information for display to user + # INFO - normal (non-optional) progress information for display to user + # INPUT - prompts requiring user response + # WARN - user warning messages + # ERROR - user error messages + # CRITICAL - fatal error messages + # Accordingly, do ''not'' use print statements in bot code; instead, + # use pywikibot.output function. + + moduleName = calledModuleName() + if not moduleName: + moduleName = "terminal-interface" + + logging.addLevelName(VERBOSE, "VERBOSE") + # for messages to be displayed on terminal at "verbose" setting + # use INFO for messages to be displayed even on non-verbose setting + logging.addLevelName(STDOUT, "STDOUT") + # for messages to be displayed to stdout + logging.addLevelName(INPUT, "INPUT") + # for prompts requiring user response + + root_logger = logging.getLogger() + root_logger.handlers = [] # get rid of default handler + root_logger.setLevel(DEBUG+1) # all records except DEBUG go to logger + + # configure default handler for VERBOSE and INFO levels + default_handler = TerminalHandler(strm=sys.stderr) + if config.verbose_output: + default_handler.setLevel(VERBOSE) + else: + default_handler.setLevel(INFO) + default_handler.addFilter(MaxLevelFilter(INPUT)) + default_handler.setFormatter(logging.Formatter(fmt="%(message)s")) + root_logger.addHandler(default_handler) + + # if user has enabled file logging, configure file handler + if moduleName in config.log or '*' in config.log: + if config.logfilename: + logfile = config.datafilepath(config.logfilename) + else: + logfile = config.datafilepath("%s-bot.log" % moduleName) + file_handler = RotatingFileHandler( + filename=logfile, maxBytes=2 << 20, backupCount=5) + + file_handler.setLevel(DEBUG) + form = logging.Formatter( + fmt="%(asctime)s %(filename)18s, %(lineno)d: " + "%(levelname)-8s %(message)s", + datefmt="%Y-%m-%d %H:%M:%S" + ) + file_handler.setFormatter(form) + root_logger.addHandler(file_handler) + for component in config.debug_log: + debuglogger = logging.getLogger(component) + debuglogger.setLevel(DEBUG) + debuglogger.addHandler(file_handler) + + # handler for level STDOUT + output_handler = TerminalHandler(strm=sys.stdout) + output_handler.setLevel(STDOUT) + output_handler.addFilter(MaxLevelFilter(STDOUT)) + output_handler.setFormatter(logging.Formatter(fmt="%(message)s")) + root_logger.addHandler(output_handler) + + # handler for levels WARNING and higher + warning_handler = TerminalHandler(strm=sys.stderr) + warning_handler.setLevel(logging.WARNING) + warning_handler.setFormatter( + logging.Formatter(fmt="%(levelname)s: %(message)s")) + root_logger.addHandler(warning_handler) + + +# Command line parsing and help + +def calledModuleName(): + """Return the name of the module calling this function. + + This is required because the -help option loads the module's docstring + and because the module name will be used for the filename of the log. + + """ + # get commandline arguments + called = sys.argv[0].strip() + if ".py" in called: # could end with .pyc, .pyw, etc. on some platforms + called = called[ : called.rindex(".py")] + return os.path.basename(called) + +def _decodeArg(arg): + if sys.platform=='win32': + if config.console_encoding in ("cp437", 'cp850'): + # Western Windows versions give parameters encoded as windows-1252 + # even though the console encoding is cp850 or cp437. + return unicode(arg, 'windows-1252') + elif config.console_encoding == 'cp852': + # Central/Eastern European Windows versions give parameters encoded + # as windows-1250 even though the console encoding is cp852. + return unicode(arg, 'windows-1250') + else: + return unicode(arg, config.console_encoding) + else: + # Linux uses the same encoding for both. + # I don't know how non-Western Windows versions behave. + return unicode(arg, config.console_encoding) + +def handleArgs(*args): + """Handle standard command line arguments, return the rest as a list. + + Takes the commandline arguments, converts them to Unicode, processes all + global parameters such as -lang or -log. Returns a list of all arguments + that are not global. This makes sure that global arguments are applied + first, regardless of the order in which the arguments were given. + + args may be passed as an argument, thereby overriding sys.argv + + """ + # get commandline arguments if necessary + if not args: + args = sys.argv[1:] + # get the name of the module calling this function. This is + # required because the -help option loads the module's docstring and because + # the module name will be used for the filename of the log. + moduleName = calledModuleName() + if not moduleName: + moduleName = "terminal-interface" + nonGlobalArgs = [] + username = None + do_help = False + for arg in args: + arg = _decodeArg(arg) + if arg == '-help': + do_help = True + elif arg.startswith('-family:'): + config.family = arg[len("-family:") : ] + elif arg.startswith('-lang:'): + config.mylang = arg[len("-lang:") : ] + elif arg.startswith("-user:"): + username = arg[len("-user:") : ] + elif arg.startswith('-putthrottle:'): + config.put_throttle = int(arg[len("-putthrottle:") : ]) + elif arg.startswith('-pt:'): + config.put_throttle = int(arg[len("-pt:") : ]) + elif arg.startswith("-maxlag:"): + config.maxlag = int(arg[len("-maxlag:") : ]) + elif arg == '-log': + if moduleName not in config.log: + config.log.append(moduleName) + elif arg.startswith('-log:'): + if moduleName not in config.log: + config.log.append(moduleName) + config.logfilename = arg[len("-log:") : ] + elif arg == '-nolog': + if moduleName in config.log: + config.log.remove(moduleName) + elif arg == "-debug": + if moduleName not in config.log: + config.log.append(moduleName) + if "" not in config.debug_log: + config.debug_log.append("") + elif arg.startswith("-debug:"): + if moduleName not in config.log: + config.log.append(moduleName) + component = arg[len("-debug:") : ] + if component not in config.debug_log: + config.debug_log.append(component) + elif arg == '-verbose' or arg == "-v": + config.verbose_output += 1 + elif arg == '-daemonize': + import daemonize + daemonize.daemonize() + elif arg.startswith('-daemonize:'): + import daemonize + daemonize.daemonize(redirect_std = arg[11:]) + else: + # the argument is not global. Let the specific bot script care + # about it. + nonGlobalArgs.append(arg) + + if username: + config.usernames[config.family][config.mylang] = username + + init_handlers() + + if config.verbose_output: + import re + ver = pywikibot.__version__ # probably can be improved on + m = re.search(r"\$Id: .* (\d+ \d+-\d+-\d+ \d+:\d+:\d+Z) .*\$", ver) + pywikibot.output(u'Pywikipediabot r%s' % m.group(1)) + pywikibot.output(u'Python %s' % sys.version) + + if do_help: + showHelp() + sys.exit(0) + logger.debug(u"handleArgs() completed.") + return nonGlobalArgs + + +def showHelp(name=""): + # argument, if given, is ignored + modname = calledModuleName() + if not modname: + try: + modname = sys.modules['__main__'].main.__module__ + except NameError: + modname = "no_module" + + globalHelp =u'''\ +Global arguments available for all bots: + +-dir:PATH Read the bot's configuration data from directory given by + PATH, instead of from the default directory. + +-lang:xx Set the language of the wiki you want to work on, overriding + the configuration in user-config.py. xx should be the + language code. + +-family:xyz Set the family of the wiki you want to work on, e.g. + wikipedia, wiktionary, wikitravel, ... + This will override the configuration in user-config.py. + +-user:xyz Log in as user 'xyz' instead of the default username. + +-daemonize:xyz Immediately return control to the terminal and redirect + stdout and stderr to xyz (only use for bots that require + no input from stdin). + +-help Show this help text. + +-log Enable the logfile, using the default filename + '%s-bot.log' + +-log:xyz Enable the logfile, using 'xyz' as the filename. + +-nolog Disable the logfile (if it is enabled by default). + +-debug:item Enable the logfile and include extensive debugging data +-debug for component "item" (or all components if the second form + is used). + +-putthrottle:n Set the minimum time (in seconds) the bot will wait between +-pt:n saving pages. + +-verbose Have the bot provide additional console output that may be +-v useful in debugging. + +''' % modname + try: + exec('import %s as module' % modname) + helpText = module.__doc__.decode('utf-8') + if hasattr(module, 'docuReplacements'): + for key, value in module.docuReplacements.iteritems(): + helpText = helpText.replace(key, value.strip('\n\r')) + pywikibot.output(helpText, level=pywikibot.STDOUT) # output to STDOUT + except Exception: + if modname: + pywikibot.output(u'Sorry, no help available for %s' % modname, + level=pywikibot.STDOUT) + logging.exception('showHelp:') + pywikibot.output(globalHelp, level=pywikibot.STDOUT) Property changes on: branches/rewrite/pywikibot/bot.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/comms/__init__.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/comms/http.py =================================================================== --- branches/rewrite/pywikibot/comms/http.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/comms/http.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,111 +1,111 @@ -# -*- coding: utf-8 -*- -""" -Basic HTTP access interface. - -This module handles communication between the bot and the HTTP threads. - -This module is responsible for - - Setting up a connection pool - - Providing a (blocking) interface for HTTP requests - - Translate site objects with query strings into urls - - Urlencoding all data - - Basic HTTP error handling -""" - -# -# (C) Pywikipedia bot team, 2007 -# -# Distributed under the terms of the MIT license. -# - -__version__ = '$Id: $' -__docformat__ = 'epytext' - -import Queue -import urllib -import urlparse -import logging -import atexit - -from pywikibot import config -import pywikibot -import cookielib -import threadedhttp - -logger = logging.getLogger("comms.http") - - -# global variables - -useragent = 'Pywikipediabot/2.0' # This should include some global version string -numthreads = 1 -threads = [] - -connection_pool = threadedhttp.ConnectionPool() -http_queue = Queue.Queue() - -cookie_jar = threadedhttp.LockableCookieJar( - config.datafilepath("pywikibot.lwp")) -try: - cookie_jar.load() -except (IOError, cookielib.LoadError): - logger.debug("Loading cookies failed.") -else: - logger.debug("Loaded cookies from file.") - - -# Build up HttpProcessors -pywikibot.output('Starting %(numthreads)i threads...' % locals(), - level=pywikibot.VERBOSE) -for i in range(numthreads): - proc = threadedhttp.HttpProcessor(http_queue, cookie_jar, connection_pool) - proc.setDaemon(True) - threads.append(proc) - proc.start() - -# Prepare flush on quit -def _flush(): - for i in threads: - http_queue.put(None) - pywikibot.output(u'Waiting for threads to finish... ', - level=pywikibot.VERBOSE) - for i in threads: - i.join() - logger.debug('All threads finished.') -atexit.register(_flush) - -# export cookie_jar to global namespace -import pywikibot -pywikibot.cookie_jar = cookie_jar - -def request(site, uri, *args, **kwargs): - """Queue a request to be submitted to Site. - - All parameters not listed below are the same as - L{httplib2.Http.request}, but the uri is relative - - @param site: The Site to connect to - @return: The received data (a unicode string). - """ - baseuri = "%s://%s/" % (site.protocol(), site.hostname()) - uri = urlparse.urljoin(baseuri, uri) - - # set default user-agent string - kwargs.setdefault("headers", {}) - kwargs["headers"].setdefault("user-agent", useragent) - request = threadedhttp.HttpRequest(uri, *args, **kwargs) - http_queue.put(request) - request.lock.acquire() - - #TODO: do some error correcting stuff - - #if all else fails - if isinstance(request.data, Exception): - raise request.data - - if request.data[0].status != 200: - pywikibot.output(u"Http response status %(status)s" - % {'status': request.data[0].status}, - level=pywikibot.WARNING) - - return request.data[1] +# -*- coding: utf-8 -*- +""" +Basic HTTP access interface. + +This module handles communication between the bot and the HTTP threads. + +This module is responsible for + - Setting up a connection pool + - Providing a (blocking) interface for HTTP requests + - Translate site objects with query strings into urls + - Urlencoding all data + - Basic HTTP error handling +""" + +# +# (C) Pywikipedia bot team, 2007 +# +# Distributed under the terms of the MIT license. +# + +__version__ = '$Id: $' +__docformat__ = 'epytext' + +import Queue +import urllib +import urlparse +import logging +import atexit + +from pywikibot import config +import pywikibot +import cookielib +import threadedhttp + +logger = logging.getLogger("comms.http") + + +# global variables + +useragent = 'Pywikipediabot/2.0' # This should include some global version string +numthreads = 1 +threads = [] + +connection_pool = threadedhttp.ConnectionPool() +http_queue = Queue.Queue() + +cookie_jar = threadedhttp.LockableCookieJar( + config.datafilepath("pywikibot.lwp")) +try: + cookie_jar.load() +except (IOError, cookielib.LoadError): + logger.debug("Loading cookies failed.") +else: + logger.debug("Loaded cookies from file.") + + +# Build up HttpProcessors +pywikibot.output('Starting %(numthreads)i threads...' % locals(), + level=pywikibot.VERBOSE) +for i in range(numthreads): + proc = threadedhttp.HttpProcessor(http_queue, cookie_jar, connection_pool) + proc.setDaemon(True) + threads.append(proc) + proc.start() + +# Prepare flush on quit +def _flush(): + for i in threads: + http_queue.put(None) + pywikibot.output(u'Waiting for threads to finish... ', + level=pywikibot.VERBOSE) + for i in threads: + i.join() + logger.debug('All threads finished.') +atexit.register(_flush) + +# export cookie_jar to global namespace +import pywikibot +pywikibot.cookie_jar = cookie_jar + +def request(site, uri, *args, **kwargs): + """Queue a request to be submitted to Site. + + All parameters not listed below are the same as + L{httplib2.Http.request}, but the uri is relative + + @param site: The Site to connect to + @return: The received data (a unicode string). + """ + baseuri = "%s://%s/" % (site.protocol(), site.hostname()) + uri = urlparse.urljoin(baseuri, uri) + + # set default user-agent string + kwargs.setdefault("headers", {}) + kwargs["headers"].setdefault("user-agent", useragent) + request = threadedhttp.HttpRequest(uri, *args, **kwargs) + http_queue.put(request) + request.lock.acquire() + + #TODO: do some error correcting stuff + + #if all else fails + if isinstance(request.data, Exception): + raise request.data + + if request.data[0].status != 200: + pywikibot.output(u"Http response status %(status)s" + % {'status': request.data[0].status}, + level=pywikibot.WARNING) + + return request.data[1] Property changes on: branches/rewrite/pywikibot/comms/http.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/comms/threadedhttp.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/data/logentries.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/epydoc.cfg =================================================================== --- branches/rewrite/pywikibot/epydoc.cfg 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/epydoc.cfg 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,84 +1,84 @@ -[epydoc] # Epydoc section marker (required by ConfigParser) - -# modules -# The list of objects to document. Objects can be named using -# dotted names, module filenames, or package directory names. -# Alases for this option include "objects" and "values". -modules: data.threadedhttp - -# output -# The type of output that should be generated. Should be one -# of: html, text, latex, dvi, ps, pdf. -output: html - -# target -# The path to the output directory. May be relative or absolute. -target: doc/ - -# css -# The CSS stylesheet for HTML output. Can be the name of a builtin -# stylesheet, or the name of a file. -css: white - -# name -# The documented project's name. -name: Python Mediawiki Framework - -# url -# The documented project's URL. -url: http://pywikipediabot.sourceforge.net - -# frames -# Whether or not to include a frames-based table of contents. -frames: yes - -# private -# Whether or not to inclue private variables. (Even if included, -# private variables will be hidden by default.) -private: yes - -# imports -# Whether or not to list each module's imports. -imports: yes - -# verbosity -# An integer indicating how verbose epydoc should be. The default -# value is 0; negative values will supress warnings and errors; -# positive values will give more verbose output. -verbosity: 0 - -# parse -# Whether or not parsing should be used to examine objects. -parse: yes - -# introspect -# Whether or not introspection should be used to examine objects. -introspect: yes - -# graph -# The list of graph types that should be automatically included -# in the output. Graphs are generated using the Graphviz "dot" -# executable. Graph types include: "classtree", "callgraph", -# "umlclass". Use "all" to include all graph types -graph: all - -# dotpath -# The path to the Graphviz "dot" executable, used to generate -# graphs. -dotpath: /usr/bin/dot - -# sourcecode -# Whether or not to include syntax highlighted source code in -# the output (HTML only). -sourcecode: no - -# pstat -# The name of one or more pstat files (generated by the profile -# or hotshot module). These are used to generate call graphs. -pstat: profile.out - -# separate-classes -# Whether each class should be listed in its own section when -# generating LaTeX or PDF output. -separate-classes: no - +[epydoc] # Epydoc section marker (required by ConfigParser) + +# modules +# The list of objects to document. Objects can be named using +# dotted names, module filenames, or package directory names. +# Alases for this option include "objects" and "values". +modules: data.threadedhttp + +# output +# The type of output that should be generated. Should be one +# of: html, text, latex, dvi, ps, pdf. +output: html + +# target +# The path to the output directory. May be relative or absolute. +target: doc/ + +# css +# The CSS stylesheet for HTML output. Can be the name of a builtin +# stylesheet, or the name of a file. +css: white + +# name +# The documented project's name. +name: Python Mediawiki Framework + +# url +# The documented project's URL. +url: http://pywikipediabot.sourceforge.net + +# frames +# Whether or not to include a frames-based table of contents. +frames: yes + +# private +# Whether or not to inclue private variables. (Even if included, +# private variables will be hidden by default.) +private: yes + +# imports +# Whether or not to list each module's imports. +imports: yes + +# verbosity +# An integer indicating how verbose epydoc should be. The default +# value is 0; negative values will supress warnings and errors; +# positive values will give more verbose output. +verbosity: 0 + +# parse +# Whether or not parsing should be used to examine objects. +parse: yes + +# introspect +# Whether or not introspection should be used to examine objects. +introspect: yes + +# graph +# The list of graph types that should be automatically included +# in the output. Graphs are generated using the Graphviz "dot" +# executable. Graph types include: "classtree", "callgraph", +# "umlclass". Use "all" to include all graph types +graph: all + +# dotpath +# The path to the Graphviz "dot" executable, used to generate +# graphs. +dotpath: /usr/bin/dot + +# sourcecode +# Whether or not to include syntax highlighted source code in +# the output (HTML only). +sourcecode: no + +# pstat +# The name of one or more pstat files (generated by the profile +# or hotshot module). These are used to generate call graphs. +pstat: profile.out + +# separate-classes +# Whether each class should be listed in its own section when +# generating LaTeX or PDF output. +separate-classes: no + Property changes on: branches/rewrite/pywikibot/epydoc.cfg ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/families/__init__.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/families/osm_family.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/scripts/__init__.py =================================================================== --- branches/rewrite/pywikibot/scripts/__init__.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/scripts/__init__.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1 +1 @@ -# THIS DIRECTORY IS TO HOLD BOT SCRIPTS FOR THE NEW FRAMEWORK +# THIS DIRECTORY IS TO HOLD BOT SCRIPTS FOR THE NEW FRAMEWORK Property changes on: branches/rewrite/pywikibot/scripts/__init__.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/scripts/touch.py =================================================================== --- branches/rewrite/pywikibot/scripts/touch.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/scripts/touch.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,92 +1,92 @@ -#!/usr/bin/python -# -*- coding: utf-8 -*- - -"""This bot goes over multiple pages of a wiki, and edits them without -changing. This is for example used to get category links in templates -working. - -This script understands various command-line arguments: - -&params; - --redir specifies that the robot should touch redirect pages; - otherwise, they will be skipped. - -All other parameters will be regarded as a page title; in this case, the bot -will only touch a single page. -""" - -__version__='$Id: touch.py,v 1.13 2006/03/01 14:07:06 russblau Exp $' - -import pywikibot -from pywikibot import pagegenerators, catlib, config -import sys - -docuReplacements = {'&params;': pagegenerators.parameterHelp} - - -class TouchBot: - def __init__(self, generator, touch_redirects): - self.generator = generator - self.touch_redirects = touch_redirects - - def run(self): - for page in self.generator: - try: - # get the page, and save it using the unmodified text. - # whether or not getting a redirect throws an exception - # depends on the variable self.touch_redirects. - text = page.get(get_redirect = self.touch_redirects) - page.save("Pywikibot touch script") - except pywikibot.NoPage: - pywikibot.output(u"Page %s does not exist?!" % page.title(asLink=True), - level=pywikibot.ERROR) - except pywikibot.IsRedirectPage: - pywikibot.output(u"Page %s is a redirect; skipping." - % page.title(asLink=True), - level=pywikibot.WARNING) - except pywikibot.LockedPage: - pywikibot.output(u"Page %s is locked?!" % page.title(asLink=True), - level=pywikibot.ERROR) - - -def main(*args): - global bot - # Disable cosmetic changes because we don't want to modify any page - # content, so that we don't flood the histories with minor changes. - config.cosmetic_changes = False - #page generator - gen = None - genFactory = pagegenerators.GeneratorFactory() - redirs = False - # If the user chooses to work on a single page, this temporary array is - # used to read the words from the page title. The words will later be - # joined with spaces to retrieve the full title. - pageTitle = [] - for arg in pywikibot.handleArgs(*args): - if genFactory.handleArg(arg): - continue - if arg == '-redir': - redirs = True - else: - pageTitle.append(arg) - - gen = genFactory.getCombinedGenerator() - if not gen: - if pageTitle: - # work on a single page - page = pywikibot.Page(pywikibot.Link(' '.join(pageTitle))) - gen = iter([page]) - else: - pywikibot.showHelp() - return - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = TouchBot(preloadingGen, redirs) - bot.run() - - -if __name__ == "__main__": - try: - main() - finally: - pywikibot.stopme() +#!/usr/bin/python +# -*- coding: utf-8 -*- + +"""This bot goes over multiple pages of a wiki, and edits them without +changing. This is for example used to get category links in templates +working. + +This script understands various command-line arguments: + +&params; + +-redir specifies that the robot should touch redirect pages; + otherwise, they will be skipped. + +All other parameters will be regarded as a page title; in this case, the bot +will only touch a single page. +""" + +__version__='$Id: touch.py,v 1.13 2006/03/01 14:07:06 russblau Exp $' + +import pywikibot +from pywikibot import pagegenerators, catlib, config +import sys + +docuReplacements = {'&params;': pagegenerators.parameterHelp} + + +class TouchBot: + def __init__(self, generator, touch_redirects): + self.generator = generator + self.touch_redirects = touch_redirects + + def run(self): + for page in self.generator: + try: + # get the page, and save it using the unmodified text. + # whether or not getting a redirect throws an exception + # depends on the variable self.touch_redirects. + text = page.get(get_redirect = self.touch_redirects) + page.save("Pywikibot touch script") + except pywikibot.NoPage: + pywikibot.output(u"Page %s does not exist?!" % page.title(asLink=True), + level=pywikibot.ERROR) + except pywikibot.IsRedirectPage: + pywikibot.output(u"Page %s is a redirect; skipping." + % page.title(asLink=True), + level=pywikibot.WARNING) + except pywikibot.LockedPage: + pywikibot.output(u"Page %s is locked?!" % page.title(asLink=True), + level=pywikibot.ERROR) + + +def main(*args): + global bot + # Disable cosmetic changes because we don't want to modify any page + # content, so that we don't flood the histories with minor changes. + config.cosmetic_changes = False + #page generator + gen = None + genFactory = pagegenerators.GeneratorFactory() + redirs = False + # If the user chooses to work on a single page, this temporary array is + # used to read the words from the page title. The words will later be + # joined with spaces to retrieve the full title. + pageTitle = [] + for arg in pywikibot.handleArgs(*args): + if genFactory.handleArg(arg): + continue + if arg == '-redir': + redirs = True + else: + pageTitle.append(arg) + + gen = genFactory.getCombinedGenerator() + if not gen: + if pageTitle: + # work on a single page + page = pywikibot.Page(pywikibot.Link(' '.join(pageTitle))) + gen = iter([page]) + else: + pywikibot.showHelp() + return + preloadingGen = pagegenerators.PreloadingGenerator(gen) + bot = TouchBot(preloadingGen, redirs) + bot.run() + + +if __name__ == "__main__": + try: + main() + finally: + pywikibot.stopme() Property changes on: branches/rewrite/pywikibot/scripts/touch.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/tests/__init__.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/tests/api_tests.py =================================================================== --- branches/rewrite/pywikibot/tests/api_tests.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/tests/api_tests.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,71 +1,71 @@ -import unittest -import pywikibot -import pywikibot.data.api as api - -mysite = pywikibot.Site('en', 'wikipedia') - - -class TestApiFunctions(unittest.TestCase): - - def testObjectCreation(self): - """Test that api.Request() creates an object with desired attributes""" - req = api.Request(site=mysite, foo="", bar="test") - self.assert_(req) - self.assertEqual(req.site, mysite) - self.assert_("foo" in req.params) - self.assertEqual(req["format"], "json") - self.assertEqual(req["bar"], "test") - # test item assignment - req["one"] = "1" - self.assertEqual(req.params['one'], "1") - # test compliance with dict interface - # req.keys() should contain "foo", "bar", "format", "maxlag", "one" - self.assertEqual(len(req.keys()), 5) - self.assert_("test" in req.values()) - self.assert_(all(len(item) == 2 for item in req.items())) - - -class TestPageGenerator(unittest.TestCase): - def setUp(self): - self.gen = api.PageGenerator(site=mysite, - generator="links", - titles="User:R'n'B") - # following test data is copied from an actual api.php response - self.gen.data = { - "query": {"pages": {"296589": {"pageid": 296589, - "ns": 0, - "title": "Broadcaster.com" - }, - "13918157": {"pageid": 13918157, - "ns": 0, - "title": "Broadcaster (definition)" - }, - "156658": {"pageid": 156658, - "ns": 0, - "title": "Wiktionary" - }, - "47757": {"pageid": 47757, - "ns": 4, - "title": "Wikipedia:Disambiguation" - } - } - } - } - - def testGeneratorResults(self): - """Test that PageGenerator yields pages with expected attributes.""" - titles = ["Broadcaster.com", "Broadcaster (definition)", - "Wiktionary", "Wikipedia:Disambiguation"] - results = [p for p in self.gen] - self.assertEqual(len(results), 4) - for page in results: - self.assertEqual(type(page), pywikibot.Page) - self.assertEqual(page.site(), mysite) - self.assert_(page.title() in titles) - - -if __name__ == '__main__': - try: - unittest.main() - except SystemExit: - pass +import unittest +import pywikibot +import pywikibot.data.api as api + +mysite = pywikibot.Site('en', 'wikipedia') + + +class TestApiFunctions(unittest.TestCase): + + def testObjectCreation(self): + """Test that api.Request() creates an object with desired attributes""" + req = api.Request(site=mysite, foo="", bar="test") + self.assert_(req) + self.assertEqual(req.site, mysite) + self.assert_("foo" in req.params) + self.assertEqual(req["format"], "json") + self.assertEqual(req["bar"], "test") + # test item assignment + req["one"] = "1" + self.assertEqual(req.params['one'], "1") + # test compliance with dict interface + # req.keys() should contain "foo", "bar", "format", "maxlag", "one" + self.assertEqual(len(req.keys()), 5) + self.assert_("test" in req.values()) + self.assert_(all(len(item) == 2 for item in req.items())) + + +class TestPageGenerator(unittest.TestCase): + def setUp(self): + self.gen = api.PageGenerator(site=mysite, + generator="links", + titles="User:R'n'B") + # following test data is copied from an actual api.php response + self.gen.data = { + "query": {"pages": {"296589": {"pageid": 296589, + "ns": 0, + "title": "Broadcaster.com" + }, + "13918157": {"pageid": 13918157, + "ns": 0, + "title": "Broadcaster (definition)" + }, + "156658": {"pageid": 156658, + "ns": 0, + "title": "Wiktionary" + }, + "47757": {"pageid": 47757, + "ns": 4, + "title": "Wikipedia:Disambiguation" + } + } + } + } + + def testGeneratorResults(self): + """Test that PageGenerator yields pages with expected attributes.""" + titles = ["Broadcaster.com", "Broadcaster (definition)", + "Wiktionary", "Wikipedia:Disambiguation"] + results = [p for p in self.gen] + self.assertEqual(len(results), 4) + for page in results: + self.assertEqual(type(page), pywikibot.Page) + self.assertEqual(page.site(), mysite) + self.assert_(page.title() in titles) + + +if __name__ == '__main__': + try: + unittest.main() + except SystemExit: + pass Property changes on: branches/rewrite/pywikibot/tests/api_tests.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/tests/page_tests.py =================================================================== --- branches/rewrite/pywikibot/tests/page_tests.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/tests/page_tests.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,281 +1,281 @@ -# -*- coding: utf-8 -*- -""" -Tests for the page module. -""" -# -# (C) Pywikipedia bot team, 2008 -# -# Distributed under the terms of the MIT license. -# -__version__ = '$Id: $' - - -import unittest -import pywikibot -import pywikibot.page - -site = pywikibot.Site() -mainpage = pywikibot.Page(pywikibot.page.Link("Main Page", site)) -maintalk = pywikibot.Page(pywikibot.page.Link("Talk:Main Page", site)) -badpage = pywikibot.Page(pywikibot.page.Link("There is no page with this title", - site)) - -class TestLinkObject(unittest.TestCase): - """Test cases for Link objects""" - - enwiki = pywikibot.Site("en", "wikipedia") - namespaces = {0: [u""], # en.wikipedia.org namespaces for testing - 1: [u"Talk:"], # canonical form first, then others - 2: [u"User:"], # must end with : - 3: [u"User talk:", u"User_talk:"], - 4: [u"Wikipedia:", u"Project:", u"WP:"], - 5: [u"Wikipedia talk:", u"Project talk:", u"Wikipedia_talk:", - u"Project_talk:", u"WT:"], - 6: [u"Image:"], - 7: [u"Image talk:", u"Image_talk:"], - 8: [u"MediaWiki:"], - 9: [u"MediaWiki talk:", u"MediaWiki_talk:"], - 10: [u"Template:"], - 11: [u"Template talk:", u"Template_talk:"], - 12: [u"Help:"], - 13: [u"Help talk:", u"Help_talk:"], - 14: [u"Category:"], - 15: [u"Category talk:", u"Category_talk:"], - 100: [u"Portal:"], - 101: [u"Portal talk:", u"Portal_talk:"], - } - titles = { - # just a bunch of randomly selected titles - # input format : expected output format - u"Cities in Burkina Faso" : u"Cities in Burkina Faso", - u"eastern Sayan" : u"Eastern Sayan", - u"The_Addams_Family_(pinball)" : u"The Addams Family (pinball)", - u"Hispanic (U.S. Census)" : u"Hispanic (U.S. Census)", - u"Stołpce" : u"Stołpce", - u"Nowy_Sącz" : u"Nowy Sącz", - u"battle of Węgierska Górka" : u"Battle of Węgierska Górka", - } - # random bunch of possible section titles - sections = [u"", - u"#Phase_2", - u"#History", - u"#later life", - ] - - def testNamespaces(self): - """Test that Link() normalizes namespace names""" - for num in self.namespaces: - for prefix in self.namespaces[num]: - l = pywikibot.page.Link(prefix+self.titles.keys()[0], - self.enwiki) - self.assertEqual(l.namespace, num) - # namespace prefixes are case-insensitive - m = pywikibot.page.Link(prefix.lower()+self.titles.keys()[1], - self.enwiki) - self.assertEqual(m.namespace, num) - - def testTitles(self): - """Test that Link() normalizes titles""" - for title in self.titles: - for num in (0, 1): - l = pywikibot.page.Link(self.namespaces[num][0]+title) - self.assertEqual(l.title, self.titles[title]) - # prefixing name with ":" shouldn't change result - m = pywikibot.page.Link(":"+self.namespaces[num][0]+title) - self.assertEqual(m.title, self.titles[title]) - - -class TestPageObject(unittest.TestCase): - def testGeneral(self): - self.assertEqual(str(mainpage), "[[%s:%s]]" - % (site.lang, mainpage.title())) - self.assertTrue(mainpage < maintalk) - - def testSite(self): - """Test site() method""" - self.assertEqual(mainpage.site(), site) - self.assertEqual(mainpage.encoding(), site.encoding()) - - def testNamespace(self): - """Test namespace() method""" - self.assertEqual(mainpage.namespace(), 0) - self.assertEqual(maintalk.namespace(), 1) - self.assertEqual(badpage.namespace(), 0) - - def testTitle(self): - """Test title() method options.""" - p1 = pywikibot.Page(site, u"Help:Test page#Testing") - self.assertEqual(p1.title(), - u"Help:Test page#Testing") - self.assertEqual(p1.title(underscore=True), - u"Help:Test_page#Testing") - self.assertEqual(p1.title(withNamespace=False), - u"Test page#Testing") - self.assertEqual(p1.title(withSection=False), - u"Help:Test page") - self.assertEqual(p1.title(withNamespace=False, withSection=False), - u"Test page") - self.assertEqual(p1.title(asUrl=True), - "Help%3ATest_page%23Testing") - self.assertEqual(p1.title(asLink=True), - u"[[Help:Test page#Testing]]") - self.assertEqual(p1.title(asLink=True, forceInterwiki=True), - u"[[en:Help:Test page#Testing]]") - self.assertEqual(p1.title(asLink=True, textlink=True), - p1.title(asLink=True)) - # also test a page with non-ASCII chars and a different namespace - p2 = pywikibot.Page(site, u"Image:Jean-Léon Gérôme 003.jpg") - self.assertEqual(p2.title(), - u"Image:Jean-Léon Gérôme 003.jpg") - self.assertEqual(p2.title(underscore=True), - u"Image:Jean-Léon_Gérôme_003.jpg") - self.assertEqual(p2.title(withNamespace=False), - u"Jean-Léon Gérôme 003.jpg") - self.assertEqual(p2.title(withSection=False), - u"Image:Jean-Léon Gérôme 003.jpg") - self.assertEqual(p2.title(withNamespace=False, withSection=False), - u"Jean-Léon Gérôme 003.jpg") - self.assertEqual(p2.title(asUrl=True), - u"Image%3AJean-L%C3%A9on_G%C3%A9r%C3%B4me_003.jpg") - self.assertEqual(p2.title(asLink=True), - u"[[Image:Jean-Léon Gérôme 003.jpg]]") - self.assertEqual(p2.title(asLink=True, forceInterwiki=True), - u"[[en:Image:Jean-Léon Gérôme 003.jpg]]") - self.assertEqual(p2.title(asLink=True, textlink=True), - u"[[:Image:Jean-Léon Gérôme 003.jpg]]") - self.assertEqual(p2.title(as_filename=True), - u"Image_Jean-Léon_Gérôme_003.jpg") - - def testSection(self): - """Test section() method.""" - # use same pages as in previous test - p1 = pywikibot.Page(site, u"Help:Test page#Testing") - p2 = pywikibot.Page(site, u"Image:Jean-Léon Gérôme 003.jpg") - self.assertEqual(p1.section(), u"Testing") - self.assertEqual(p2.section(), None) - - def testIsTalkPage(self): - """Test isTalkPage() method.""" - p1 = pywikibot.Page(site, u"First page") - p2 = pywikibot.Page(site, u"Talk:First page") - p3 = pywikibot.Page(site, u"User:Second page") - p4 = pywikibot.Page(site, u"User talk:Second page") - self.assertEqual(p1.isTalkPage(), False) - self.assertEqual(p2.isTalkPage(), True) - self.assertEqual(p3.isTalkPage(), False) - self.assertEqual(p4.isTalkPage(), True) - - def testIsCategory(self): - """Test isCategory method.""" - p1 = pywikibot.Page(site, u"First page") - p2 = pywikibot.Page(site, u"Category:Second page") - p3 = pywikibot.Page(site, u"Category talk:Second page") - self.assertEqual(p1.isCategory(), False) - self.assertEqual(p2.isCategory(), True) - self.assertEqual(p3.isCategory(), False) - - def testIsImage(self): - p1 = pywikibot.Page(site, u"First page") - p2 = pywikibot.Page(site, u"Image:Second page") - p3 = pywikibot.Page(site, u"Image talk:Second page") - self.assertEqual(p1.isImage(), False) - self.assertEqual(p2.isImage(), True) - self.assertEqual(p3.isImage(), False) - - def testApiMethods(self): - """Test various methods that rely on API.""" - # since there is no way to predict what data the wiki will return, - # we only check that the returned objects are of correct type. - self.assertTrue(isinstance(mainpage.get(), unicode)) - self.assertTrue(isinstance(maintalk.get(), unicode)) - self.assertRaises(pywikibot.NoPage, badpage.get) - self.assertTrue(isinstance(mainpage.latestRevision(), int)) - self.assertTrue(isinstance(mainpage.userName(), unicode)) - self.assertTrue(isinstance(mainpage.isIpEdit(), bool)) - self.assertTrue(isinstance(mainpage.exists(), bool)) - self.assertTrue(isinstance(mainpage.isRedirectPage(), bool)) - self.assertTrue(isinstance(mainpage.isEmpty(), bool)) - self.assertEqual(mainpage.toggleTalkPage(), maintalk) - self.assertEqual(maintalk.toggleTalkPage(), mainpage) - self.assertTrue(isinstance(mainpage.isDisambig(), bool)) - self.assertTrue(isinstance(mainpage.canBeEdited(), bool)) - self.assertTrue(isinstance(mainpage.botMayEdit(), bool)) - self.assertTrue(isinstance(mainpage.editTime(), unicode)) - self.assertTrue(isinstance(mainpage.previousRevision(), int)) - self.assertTrue(isinstance(mainpage.permalink(), basestring)) - - def testReferences(self): - count = 0 - for p in mainpage.getReferences(): - count += 1 - self.assertTrue(isinstance(p, pywikibot.Page)) - if count >= 10: - break - count = 0 - for p in mainpage.backlinks(): - count += 1 - self.assertTrue(isinstance(p, pywikibot.Page)) - if count >= 10: - break - count = 0 - for p in mainpage.embeddedin(): - count += 1 - self.assertTrue(isinstance(p, pywikibot.Page)) - if count >= 10: - break - - def testLinks(self): - for p in mainpage.linkedPages(): - self.assertTrue(isinstance(p, pywikibot.Page)) - iw = list(mainpage.interwiki(expand=True)) - for p in iw: - self.assertTrue(isinstance(p, pywikibot.Link)) - for p2 in mainpage.interwiki(expand=False): - self.assertTrue(isinstance(p2, pywikibot.Link)) - self.assertTrue(p2 in iw) - for p in mainpage.langlinks(): - self.assertTrue(isinstance(p, pywikibot.Link)) - for p in mainpage.imagelinks(): - self.assertTrue(isinstance(p, pywikibot.ImagePage)) - for p in mainpage.templates(): - self.assertTrue(isinstance(p, pywikibot.Page)) - for t, params in mainpage.templatesWithParams(): - self.assertTrue(isinstance(t, pywikibot.Page)) - self.assertTrue(isinstance(params, list)) - for p in mainpage.categories(): - self.assertTrue(isinstance(p, pywikibot.Category)) - for p in mainpage.extlinks(): - self.assertTrue(isinstance(p, unicode)) - -# methods that still need tests implemented or expanded: - -## def autoFormat(self): -## def isAutoTitle(self): -## def get(self, force=False, get_redirect=False, sysop=False): -## def getOldVersion(self, oldid, force=False, get_redirect=False, -## sysop=False): -## text = property(_textgetter, _textsetter, _cleartext, -## "The edited wikitext (unicode) of this Page") -## def getReferences(self, follow_redirects=True, withTemplateInclusion=True, -## onlyTemplateInclusion=False, redirectsOnly=False, -## namespaces=None): -## def backlinks(self, followRedirects=True, filterRedirects=None, -## namespaces=None): -## def embeddedin(self, filter_redirects=None, namespaces=None): -## def getRedirectTarget(self): -## def getVersionHistory(self, reverseOrder=False, getAll=False, -## revCount=500): -## def getVersionHistoryTable(self, forceReload=False, reverseOrder=False, -## getAll=False, revCount=500): -## def fullVersionHistory(self): -## def contributingUsers(self): - - -if __name__ == '__main__': - try: - try: - unittest.main() - except SystemExit: - pass - finally: - pywikibot.stopme() +# -*- coding: utf-8 -*- +""" +Tests for the page module. +""" +# +# (C) Pywikipedia bot team, 2008 +# +# Distributed under the terms of the MIT license. +# +__version__ = '$Id: $' + + +import unittest +import pywikibot +import pywikibot.page + +site = pywikibot.Site() +mainpage = pywikibot.Page(pywikibot.page.Link("Main Page", site)) +maintalk = pywikibot.Page(pywikibot.page.Link("Talk:Main Page", site)) +badpage = pywikibot.Page(pywikibot.page.Link("There is no page with this title", + site)) + +class TestLinkObject(unittest.TestCase): + """Test cases for Link objects""" + + enwiki = pywikibot.Site("en", "wikipedia") + namespaces = {0: [u""], # en.wikipedia.org namespaces for testing + 1: [u"Talk:"], # canonical form first, then others + 2: [u"User:"], # must end with : + 3: [u"User talk:", u"User_talk:"], + 4: [u"Wikipedia:", u"Project:", u"WP:"], + 5: [u"Wikipedia talk:", u"Project talk:", u"Wikipedia_talk:", + u"Project_talk:", u"WT:"], + 6: [u"Image:"], + 7: [u"Image talk:", u"Image_talk:"], + 8: [u"MediaWiki:"], + 9: [u"MediaWiki talk:", u"MediaWiki_talk:"], + 10: [u"Template:"], + 11: [u"Template talk:", u"Template_talk:"], + 12: [u"Help:"], + 13: [u"Help talk:", u"Help_talk:"], + 14: [u"Category:"], + 15: [u"Category talk:", u"Category_talk:"], + 100: [u"Portal:"], + 101: [u"Portal talk:", u"Portal_talk:"], + } + titles = { + # just a bunch of randomly selected titles + # input format : expected output format + u"Cities in Burkina Faso" : u"Cities in Burkina Faso", + u"eastern Sayan" : u"Eastern Sayan", + u"The_Addams_Family_(pinball)" : u"The Addams Family (pinball)", + u"Hispanic (U.S. Census)" : u"Hispanic (U.S. Census)", + u"Stołpce" : u"Stołpce", + u"Nowy_Sącz" : u"Nowy Sącz", + u"battle of Węgierska Górka" : u"Battle of Węgierska Górka", + } + # random bunch of possible section titles + sections = [u"", + u"#Phase_2", + u"#History", + u"#later life", + ] + + def testNamespaces(self): + """Test that Link() normalizes namespace names""" + for num in self.namespaces: + for prefix in self.namespaces[num]: + l = pywikibot.page.Link(prefix+self.titles.keys()[0], + self.enwiki) + self.assertEqual(l.namespace, num) + # namespace prefixes are case-insensitive + m = pywikibot.page.Link(prefix.lower()+self.titles.keys()[1], + self.enwiki) + self.assertEqual(m.namespace, num) + + def testTitles(self): + """Test that Link() normalizes titles""" + for title in self.titles: + for num in (0, 1): + l = pywikibot.page.Link(self.namespaces[num][0]+title) + self.assertEqual(l.title, self.titles[title]) + # prefixing name with ":" shouldn't change result + m = pywikibot.page.Link(":"+self.namespaces[num][0]+title) + self.assertEqual(m.title, self.titles[title]) + + +class TestPageObject(unittest.TestCase): + def testGeneral(self): + self.assertEqual(str(mainpage), "[[%s:%s]]" + % (site.lang, mainpage.title())) + self.assertTrue(mainpage < maintalk) + + def testSite(self): + """Test site() method""" + self.assertEqual(mainpage.site(), site) + self.assertEqual(mainpage.encoding(), site.encoding()) + + def testNamespace(self): + """Test namespace() method""" + self.assertEqual(mainpage.namespace(), 0) + self.assertEqual(maintalk.namespace(), 1) + self.assertEqual(badpage.namespace(), 0) + + def testTitle(self): + """Test title() method options.""" + p1 = pywikibot.Page(site, u"Help:Test page#Testing") + self.assertEqual(p1.title(), + u"Help:Test page#Testing") + self.assertEqual(p1.title(underscore=True), + u"Help:Test_page#Testing") + self.assertEqual(p1.title(withNamespace=False), + u"Test page#Testing") + self.assertEqual(p1.title(withSection=False), + u"Help:Test page") + self.assertEqual(p1.title(withNamespace=False, withSection=False), + u"Test page") + self.assertEqual(p1.title(asUrl=True), + "Help%3ATest_page%23Testing") + self.assertEqual(p1.title(asLink=True), + u"[[Help:Test page#Testing]]") + self.assertEqual(p1.title(asLink=True, forceInterwiki=True), + u"[[en:Help:Test page#Testing]]") + self.assertEqual(p1.title(asLink=True, textlink=True), + p1.title(asLink=True)) + # also test a page with non-ASCII chars and a different namespace + p2 = pywikibot.Page(site, u"Image:Jean-Léon Gérôme 003.jpg") + self.assertEqual(p2.title(), + u"Image:Jean-Léon Gérôme 003.jpg") + self.assertEqual(p2.title(underscore=True), + u"Image:Jean-Léon_Gérôme_003.jpg") + self.assertEqual(p2.title(withNamespace=False), + u"Jean-Léon Gérôme 003.jpg") + self.assertEqual(p2.title(withSection=False), + u"Image:Jean-Léon Gérôme 003.jpg") + self.assertEqual(p2.title(withNamespace=False, withSection=False), + u"Jean-Léon Gérôme 003.jpg") + self.assertEqual(p2.title(asUrl=True), + u"Image%3AJean-L%C3%A9on_G%C3%A9r%C3%B4me_003.jpg") + self.assertEqual(p2.title(asLink=True), + u"[[Image:Jean-Léon Gérôme 003.jpg]]") + self.assertEqual(p2.title(asLink=True, forceInterwiki=True), + u"[[en:Image:Jean-Léon Gérôme 003.jpg]]") + self.assertEqual(p2.title(asLink=True, textlink=True), + u"[[:Image:Jean-Léon Gérôme 003.jpg]]") + self.assertEqual(p2.title(as_filename=True), + u"Image_Jean-Léon_Gérôme_003.jpg") + + def testSection(self): + """Test section() method.""" + # use same pages as in previous test + p1 = pywikibot.Page(site, u"Help:Test page#Testing") + p2 = pywikibot.Page(site, u"Image:Jean-Léon Gérôme 003.jpg") + self.assertEqual(p1.section(), u"Testing") + self.assertEqual(p2.section(), None) + + def testIsTalkPage(self): + """Test isTalkPage() method.""" + p1 = pywikibot.Page(site, u"First page") + p2 = pywikibot.Page(site, u"Talk:First page") + p3 = pywikibot.Page(site, u"User:Second page") + p4 = pywikibot.Page(site, u"User talk:Second page") + self.assertEqual(p1.isTalkPage(), False) + self.assertEqual(p2.isTalkPage(), True) + self.assertEqual(p3.isTalkPage(), False) + self.assertEqual(p4.isTalkPage(), True) + + def testIsCategory(self): + """Test isCategory method.""" + p1 = pywikibot.Page(site, u"First page") + p2 = pywikibot.Page(site, u"Category:Second page") + p3 = pywikibot.Page(site, u"Category talk:Second page") + self.assertEqual(p1.isCategory(), False) + self.assertEqual(p2.isCategory(), True) + self.assertEqual(p3.isCategory(), False) + + def testIsImage(self): + p1 = pywikibot.Page(site, u"First page") + p2 = pywikibot.Page(site, u"Image:Second page") + p3 = pywikibot.Page(site, u"Image talk:Second page") + self.assertEqual(p1.isImage(), False) + self.assertEqual(p2.isImage(), True) + self.assertEqual(p3.isImage(), False) + + def testApiMethods(self): + """Test various methods that rely on API.""" + # since there is no way to predict what data the wiki will return, + # we only check that the returned objects are of correct type. + self.assertTrue(isinstance(mainpage.get(), unicode)) + self.assertTrue(isinstance(maintalk.get(), unicode)) + self.assertRaises(pywikibot.NoPage, badpage.get) + self.assertTrue(isinstance(mainpage.latestRevision(), int)) + self.assertTrue(isinstance(mainpage.userName(), unicode)) + self.assertTrue(isinstance(mainpage.isIpEdit(), bool)) + self.assertTrue(isinstance(mainpage.exists(), bool)) + self.assertTrue(isinstance(mainpage.isRedirectPage(), bool)) + self.assertTrue(isinstance(mainpage.isEmpty(), bool)) + self.assertEqual(mainpage.toggleTalkPage(), maintalk) + self.assertEqual(maintalk.toggleTalkPage(), mainpage) + self.assertTrue(isinstance(mainpage.isDisambig(), bool)) + self.assertTrue(isinstance(mainpage.canBeEdited(), bool)) + self.assertTrue(isinstance(mainpage.botMayEdit(), bool)) + self.assertTrue(isinstance(mainpage.editTime(), unicode)) + self.assertTrue(isinstance(mainpage.previousRevision(), int)) + self.assertTrue(isinstance(mainpage.permalink(), basestring)) + + def testReferences(self): + count = 0 + for p in mainpage.getReferences(): + count += 1 + self.assertTrue(isinstance(p, pywikibot.Page)) + if count >= 10: + break + count = 0 + for p in mainpage.backlinks(): + count += 1 + self.assertTrue(isinstance(p, pywikibot.Page)) + if count >= 10: + break + count = 0 + for p in mainpage.embeddedin(): + count += 1 + self.assertTrue(isinstance(p, pywikibot.Page)) + if count >= 10: + break + + def testLinks(self): + for p in mainpage.linkedPages(): + self.assertTrue(isinstance(p, pywikibot.Page)) + iw = list(mainpage.interwiki(expand=True)) + for p in iw: + self.assertTrue(isinstance(p, pywikibot.Link)) + for p2 in mainpage.interwiki(expand=False): + self.assertTrue(isinstance(p2, pywikibot.Link)) + self.assertTrue(p2 in iw) + for p in mainpage.langlinks(): + self.assertTrue(isinstance(p, pywikibot.Link)) + for p in mainpage.imagelinks(): + self.assertTrue(isinstance(p, pywikibot.ImagePage)) + for p in mainpage.templates(): + self.assertTrue(isinstance(p, pywikibot.Page)) + for t, params in mainpage.templatesWithParams(): + self.assertTrue(isinstance(t, pywikibot.Page)) + self.assertTrue(isinstance(params, list)) + for p in mainpage.categories(): + self.assertTrue(isinstance(p, pywikibot.Category)) + for p in mainpage.extlinks(): + self.assertTrue(isinstance(p, unicode)) + +# methods that still need tests implemented or expanded: + +## def autoFormat(self): +## def isAutoTitle(self): +## def get(self, force=False, get_redirect=False, sysop=False): +## def getOldVersion(self, oldid, force=False, get_redirect=False, +## sysop=False): +## text = property(_textgetter, _textsetter, _cleartext, +## "The edited wikitext (unicode) of this Page") +## def getReferences(self, follow_redirects=True, withTemplateInclusion=True, +## onlyTemplateInclusion=False, redirectsOnly=False, +## namespaces=None): +## def backlinks(self, followRedirects=True, filterRedirects=None, +## namespaces=None): +## def embeddedin(self, filter_redirects=None, namespaces=None): +## def getRedirectTarget(self): +## def getVersionHistory(self, reverseOrder=False, getAll=False, +## revCount=500): +## def getVersionHistoryTable(self, forceReload=False, reverseOrder=False, +## getAll=False, revCount=500): +## def fullVersionHistory(self): +## def contributingUsers(self): + + +if __name__ == '__main__': + try: + try: + unittest.main() + except SystemExit: + pass + finally: + pywikibot.stopme() Property changes on: branches/rewrite/pywikibot/tests/page_tests.py ___________________________________________________________________ Added: svn:eol-style + native Modified: branches/rewrite/pywikibot/tests/site_tests.py =================================================================== --- branches/rewrite/pywikibot/tests/site_tests.py 2009-03-03 10:01:59 UTC (rev 6476) +++ branches/rewrite/pywikibot/tests/site_tests.py 2009-03-03 10:10:50 UTC (rev 6477) @@ -1,919 +1,919 @@ -# -*- coding: utf-8 -*- -""" -Tests for the site module. -""" -# -# (C) Pywikipedia bot team, 2008 -# -# Distributed under the terms of the MIT license. -# -__version__ = '$Id: $' - - -import unittest -import pywikibot -import warnings - -logger = pywikibot.logging.getLogger("wiki.site.tests") - -mysite = pywikibot.Site() -mainpage = pywikibot.Page(pywikibot.Link("Main Page", mysite)) -imagepage = iter(mainpage.imagelinks()).next() # 1st image on main page - - -class TestSiteObject(unittest.TestCase): - """Test cases for Site methods.""" - - def assertType(self, obj, cls): - """Assert that obj is an instance of type cls""" - return self.assertTrue(isinstance(obj, cls)) - - def testBaseMethods(self): - """Test cases for BaseSite methods""" - - self.assertEqual(mysite.family.name, pywikibot.config2.family) - self.assertEqual(mysite.code, pywikibot.config2.mylang) - self.assertType(mysite.lang, basestring) - self.assertType(mysite == pywikibot.Site("en", "wikipedia"), bool) - self.assertType(mysite.user(), (basestring, type(None))) - self.assertEqual(mysite.sitename(), - "%s:%s" % (pywikibot.config2.family, - pywikibot.config2.mylang)) - self.assertEqual(repr(mysite), - 'Site("%s", "%s")' - % (pywikibot.config2.mylang, pywikibot.config2.family)) - self.assertType(mysite.linktrail(), basestring) - self.assertType(mysite.redirect(default=True), basestring) - self.assertType(mysite.disambcategory(), pywikibot.Category) - self.assertEqual(mysite.linkto("foo"), u"[[Foo]]") - self.assertFalse(mysite.isInterwikiLink("foo")) - self.assertType(mysite.redirectRegex().pattern, basestring) - self.assertType(mysite.category_on_one_line(), bool) - for grp in ("user", "autoconfirmed", "bot", "sysop", "nosuchgroup"): - self.assertType(mysite.has_group(grp), bool) - for rgt in ("read", "edit", "move", "delete", "rollback", "block", - "nosuchright"): - self.assertType(mysite.has_right(rgt), bool) - - def testLanguageMethods(self): - """Test cases for languages() and related methods""" - - langs = mysite.languages() - self.assertType(langs, list) - self.assertTrue(mysite.code in langs) - obs = mysite.family.obsolete - ipf = mysite.interwiki_putfirst() - self.assertType(ipf, list) - self.assertTrue(all(item in langs or item in obs - for item in ipf)) - self.assertTrue(all(item in langs - for item in mysite.validLanguageLinks())) - - def testNamespaceMethods(self): - """Test cases for methods manipulating namespace names""" - - builtins = {'Talk': 1, # these should work in any MW wiki - 'User': 2, - 'User talk': 3, - 'Project': 4, - 'Project talk': 5, - 'Image': 6, - 'Image talk': 7, - 'MediaWiki': 8, - 'MediaWiki talk': 9, - 'Template': 10, - 'Template talk': 11, - 'Help': 12, - 'Help talk': 13, - 'Category': 14, - 'Category talk': 15, - } - self.assertTrue(all(mysite.ns_index(b) == builtins[b] - for b in builtins)) - ns = mysite.namespaces() - self.assertType(ns, dict) - self.assertTrue(all(x in ns for x in xrange(0, 16))) - # built-in namespaces always present - self.assertType(mysite.ns_normalize("project"), basestring) - self.assertTrue(all(isinstance(key, int) - for key in ns)) - self.assertTrue(all(isinstance(val, list) - for val in ns.values())) - self.assertTrue(all(isinstance(name, basestring) - for val in ns.values() - for name in val)) - self.assertTrue(all(isinstance(mysite.namespace(key), basestring) - for key in ns)) - self.assertTrue(all(isinstance(mysite.namespace(key, True), list) - for key in ns)) - self.assertTrue(all(isinstance(item, basestring) - for key in ns - for item in mysite.namespace(key, True))) - - def testApiMethods(self): - """Test generic ApiSite methods""" - - self.assertType(mysite.logged_in(), bool) - self.assertType(mysite.logged_in(True), bool) - self.assertType(mysite.userinfo, dict) - self.assertType(mysite.is_blocked(), bool) - self.assertType(mysite.messages(), bool) - self.assertType(mysite.has_right("edit"), bool) - self.assertFalse(mysite.has_right("nonexistent_right")) - self.assertType(mysite.has_group("bots"), bool) - self.assertFalse(mysite.has_group("nonexistent_group")) - try: - self.assertType(mysite.is_blocked(True), bool) - self.assertType(mysite.has_right("edit", True), bool) - self.assertFalse(mysite.has_right("nonexistent_right", True)) - self.assertType(mysite.has_group("bots", True), bool) - self.assertFalse(mysite.has_group("nonexistent_group", True)) - except pywikibot.NoUsername: - logger.warn( - "Cannot test Site methods for sysop; no sysop account configured.") - for msg in ("1movedto2", "about", "aboutpage", "aboutsite", - "accesskey-n-portal"): - self.assertTrue(mysite.has_mediawiki_message(msg)) - self.assertType(mysite.mediawiki_message(msg), basestring) - self.assertFalse(mysite.has_mediawiki_message("nosuchmessage")) - self.assertRaises(KeyError, mysite.mediawiki_message, "nosuchmessage") - self.assertType(mysite.getcurrenttimestamp(), basestring) - self.assertType(mysite.siteinfo, dict) - self.assertType(mysite.case(), basestring) - ver = mysite.live_version() - self.assertType(ver, tuple) - self.assertTrue(all(isinstance(ver[i], int) for i in (0, 1))) - self.assertType(ver[2], basestring) - - def testPageMethods(self): - """Test ApiSite methods for getting page-specific info""" - - self.assertType(mysite.page_exists(mainpage), bool) - self.assertType(mysite.page_restrictions(mainpage), dict) - self.assertType(mysite.page_can_be_edited(mainpage), bool) - self.assertType(mysite.page_isredirect(mainpage), bool) - if mysite.page_isredirect(mainpage): - self.assertType(mysite.getredirtarget(mainpage), pywikibot.Page) - else: - self.assertRaises(pywikibot.IsNotRedirectPage, - mysite.getredirtarget, mainpage) - a = list(mysite.preloadpages([mainpage])) - self.assertEqual(len(a), int(mysite.page_exists(mainpage))) - if a: - self.assertEqual(a[0], mainpage) - - def testTokens(self): - """Test ability to get page tokens""" - - for ttype in ("edit", "move"): # token types for non-sysops - self.assertType(mysite.token(mainpage, ttype), basestring) - self.assertRaises(KeyError, mysite.token, mainpage, "invalidtype") - - def testPreload(self): - """Test that preloading works""" - - count = 0 - for page in mysite.preloadpages(mysite.pagelinks(mainpage, limit=10)): - self.assertType(page, pywikibot.Page) - self.assertType(page.exists(), bool) - if page.exists(): - self.assertTrue(hasattr(page, "_text")) - count += 1 - if count >= 5: - break - - def testLinkMethods(self): - """Test site methods for getting links to and from a page""" - - backlinks = set(mysite.pagebacklinks(mainpage, namespaces=[0])) - # only non-redirects: - filtered = set(mysite.pagebacklinks(mainpage, namespaces=0, - filterRedirects=False)) - # only redirects: - redirs = set(mysite.pagebacklinks(mainpage, namespaces=0, - filterRedirects=True)) - # including links to redirect pages (but not the redirects): - indirect = set(mysite.pagebacklinks(mainpage, namespaces=[0], - followRedirects=True)) - self.assertEqual(filtered & redirs, set([])) - self.assertEqual(indirect & redirs, set([])) - self.assertTrue(filtered.issubset(indirect)) - self.assertTrue(filtered.issubset(backlinks)) - self.assertTrue(redirs.issubset(backlinks)) - self.assertTrue(backlinks.issubset( - set(mysite.pagebacklinks(mainpage, namespaces=[0, 2])))) - - # pagereferences includes both backlinks and embeddedin - embedded = set(mysite.page_embeddedin(mainpage, namespaces=[0])) - refs = set(mysite.pagereferences(mainpage, namespaces=[0])) - self.assertTrue(backlinks.issubset(refs)) - self.assertTrue(embedded.issubset(refs)) - for bl in backlinks: - self.assertType(bl, pywikibot.Page) - self.assertTrue(bl in refs) - for ei in embedded: - self.assertType(ei, pywikibot.Page) - self.assertTrue(ei in refs) - for ref in refs: - self.assertTrue(ref in backlinks or ref in embedded) - # test embeddedin arguments - self.assertTrue(embedded.issuperset( - set(mysite.page_embeddedin(mainpage, filterRedirects=True, - namespaces=[0])))) - self.assertTrue(embedded.issuperset( - set(mysite.page_embeddedin(mainpage, filterRedirects=False, - namespaces=[0])))) - self.assertTrue(embedded.issubset( - set(mysite.page_embeddedin(mainpage, namespaces=[0, 2])))) - links = set(mysite.pagelinks(mainpage)) - for pl in links: - self.assertType(pl, pywikibot.Page) - # test links arguments - self.assertTrue(links.issuperset( - set(mysite.pagelinks(mainpage, namespaces=[0, 1])))) - for target in mysite.preloadpages( - mysite.pagelinks(mainpage, follow_redirects=True, - limit=5)): - self.assertType(target, pywikibot.Page) - self.assertFalse(target.isRedirectPage()) - # test pagecategories - for cat in mysite.pagecategories(mainpage): - self.assertType(cat, pywikibot.Category) - for cm in mysite.categorymembers(cat): - self.assertType(cat, pywikibot.Page) - # test pageimages - self.assertTrue(all(isinstance(im, pywikibot.ImagePage) - for im in mysite.pageimages(mainpage))) - # test pagetemplates - self.assertTrue(all(isinstance(te, pywikibot.Page) - for te in mysite.pagetemplates(mainpage))) - self.assertTrue(set(mysite.pagetemplates(mainpage)).issuperset( - set(mysite.pagetemplates(mainpage, namespaces=[10])))) - # test pagelanglinks - for ll in mysite.pagelanglinks(mainpage): - self.assertType(ll, pywikibot.Link) - # test page_extlinks - self.assertTrue(all(isinstance(el, basestring) - for el in mysite.page_extlinks(mainpage))) - - def testLoadRevisions(self): - """Test the site.loadrevisions() method""" - - mysite.loadrevisions(mainpage) - self.assertTrue(hasattr(mainpage, "_revid")) - self.assertTrue(hasattr(mainpage, "_revisions")) - self.assertTrue(mainpage._revid in mainpage._revisions) - # TODO test all the optional arguments - - def testAllPages(self): - """Test the site.allpages() method""" - - fwd = list(mysite.allpages(limit=10)) - self.assertTrue(len(fwd) <= 10) - for page in fwd: - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - rev = list(mysite.allpages(reverse=True, start="Aa", limit=12)) - self.assertTrue(len(rev) <= 12) - for page in rev: - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.title() <= "Aa") - for page in mysite.allpages(start="Py", limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.title() >= "Py") - for page in mysite.allpages(prefix="Pre", limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.title().startswith("Pre")) - for page in mysite.allpages(namespace=1, limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 1) - for page in mysite.allpages(filterredir=True, limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.isRedirectPage()) - for page in mysite.allpages(filterredir=False, limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertEqual(page.namespace(), 0) - self.assertFalse(page.isRedirectPage()) -## for page in mysite.allpages(filterlanglinks=True, limit=5): -## self.assertType(page, pywikibot.Page) -## self.assertTrue(mysite.page_exists(page)) -## self.assertEqual(page.namespace(), 0) -## for page in mysite.allpages(filterlanglinks=False, limit=5): -## self.assertType(page, pywikibot.Page) -## self.assertTrue(mysite.page_exists(page)) -## self.assertEqual(page.namespace(), 0) - for page in mysite.allpages(minsize=100, limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertTrue(len(page.text) >= 100) - for page in mysite.allpages(maxsize=200, limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertTrue(len(page.text) <= 200) - for page in mysite.allpages(protect_type="edit", limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertTrue("edit" in page._protection) - for page in mysite.allpages(protect_type="edit", - protect_level="sysop", limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(mysite.page_exists(page)) - self.assertTrue("edit" in page._protection) - self.assertTrue("sysop" in page._protection["edit"]) - - def testAllLinks(self): - """Test the site.alllinks() method""" - - fwd = list(mysite.alllinks(limit=10)) - self.assertTrue(len(fwd) <= 10) - self.assertTrue(all(isinstance(link, pywikibot.Page) for link in fwd)) - uniq = list(mysite.alllinks(limit=10, unique=True)) - self.assertTrue(all(link in uniq for link in fwd)) - for page in mysite.alllinks(start="Link", limit=5): - self.assertType(page, pywikibot.Page) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.title() >= "Link") - for page in mysite.alllinks(prefix="Fix", limit=5): - self.assertType(page, pywikibot.Page) - self.assertEqual(page.namespace(), 0) - self.assertTrue(page.title().startswith("Fix")) - for page in mysite.alllinks(namespace=1, limit=5): - self.assertType(page, pywikibot.Page) - self.assertEqual(page.namespace(), 1) - for page in mysite.alllinks(start="From", namespace=4, fromids=True, - limit=5): - self.assertType(page, pywikibot.Page) - self.assertTrue(page.title(withNamespace=False) >= "From") - self.assertTrue(hasattr(page, "_fromid")) - errgen = mysite.alllinks(unique=True, fromids=True) - self.assertRaises(pywikibot.Error, errgen.next) - - def testAllCategories(self): - """Test the site.allcategories() method""" - - ac = list(mysite.allcategories(limit=10)) - self.assertTrue(len(ac) <= 10) - self.assertTrue(all(isinstance(cat, pywikibot.Category) - for cat in ac)) - for cat in mysite.allcategories(limit=5, start="Abc"): - self.assertType(cat, pywikibot.Category) - self.assertTrue(cat.title(withNamespace=False) >= "Abc") - for cat in mysite.allcategories(limit=5, prefix="Def"): - self.assertType(cat, pywikibot.Category) - self.assertTrue(cat.title(withNamespace=False).startswith("Def")) -## # Bug # 15985 -## for cat in mysite.allcategories(limit=5, start="Hij", reverse=True): -## self.assertType(cat, pywikibot.Category) -## self.assertTrue(cat.title(withNamespace=False) <= "Hij") - - def testAllUsers(self): - """Test the site.allusers() method""" - - au = list(mysite.allusers(limit=10)) - self.assertTrue(len(au) <= 10) - for user in au: - self.assertType(user, dict) - self.assertTrue("name" in user) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) - for user in mysite.allusers(start="B", limit=5): - self.assertType(user, dict) - self.assertTrue("name" in user) - self.assertTrue(user["name"] >= "B") - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) - for user in mysite.allusers(prefix="C", limit=5): - self.assertType(user, dict) - self.assertTrue("name" in user) - self.assertTrue(user["name"].startswith("C")) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) - for user in mysite.allusers(prefix="D", group="sysop", limit=5): - self.assertType(user, dict) - self.assertTrue("name" in user) - self.assertTrue(user["name"].startswith("D")) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) - self.assertTrue("groups" in user and "sysop" in user["groups"]) - - def testAllImages(self): - """Test the site.allimages() method""" - - ai = list(mysite.allimages(limit=10)) - self.assertTrue(len(ai) <= 10) - self.assertTrue(all(isinstance(image, pywikibot.ImagePage) - for image in ai)) - for impage in mysite.allimages(start="Ba", limit=5): - self.assertType(impage, pywikibot.ImagePage) - self.assertTrue(mysite.page_exists(impage)) - self.assertTrue(impage.title(withNamespace=False) >= "Ba") -## # Bug # 15985 -## for impage in mysite.allimages(start="Da", reverse=True, limit=5): -## self.assertType(impage, pywikibot.ImagePage) -## self.assertTrue(mysite.page_exists(impage)) -## self.assertTrue(impage.title() <= "Da") - for impage in mysite.allimages(prefix="Ch", limit=5): - self.assertType(impage, pywikibot.ImagePage) - self.assertTrue(mysite.page_exists(impage)) - self.assertTrue(impage.title(withNamespace=False).startswith("Ch")) - for impage in mysite.allimages(minsize=100, limit=5): - self.assertType(impage, pywikibot.ImagePage) - self.assertTrue(mysite.page_exists(impage)) - self.assertTrue(impage._imageinfo["size"] >= 100) - for impage in mysite.allimages(maxsize=2000, limit=5): - self.assertType(impage, pywikibot.ImagePage) - self.assertTrue(mysite.page_exists(impage)) - self.assertTrue(impage._imageinfo["size"] <= 2000) - - def testBlocks(self): - """Test the site.blocks() method""" - - props = ("id", "by", "timestamp", "expiry", "reason") - bl = list(mysite.blocks(limit=10)) - self.assertTrue(len(bl) <= 10) - for block in bl: - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - # timestamps should be in descending order - timestamps = [block['timestamp'] for block in bl] - for t in xrange(1, len(timestamps)): - self.assertTrue(timestamps[t] <= timestamps[t-1]) - - b2 = list(mysite.blocks(limit=10, reverse=True)) - self.assertTrue(len(b2) <= 10) - for block in b2: - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - # timestamps should be in ascending order - timestamps = [block['timestamp'] for block in b2] - for t in xrange(1, len(timestamps)): - self.assertTrue(timestamps[t] >= timestamps[t-1]) - - for block in mysite.blocks(starttime="2008-07-01T00:00:01Z", limit=5): - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - for block in mysite.blocks(endtime="2008-07-31T23:59:59Z", limit=5): - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - for block in mysite.blocks(starttime="2008-08-02T00:00:01Z", - endtime="2008-08-02T23:59:59Z", - reverse=True, limit=5): - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - for block in mysite.blocks(starttime="2008-08-03T23:59:59Z", - endtime="2008-08-03T00:00:01Z", - limit=5): - self.assertType(block, dict) - for prop in props: - self.assertTrue(prop in block) - # starttime earlier than endtime - self.assertRaises(pywikibot.Error, mysite.blocks, - starttime="2008-08-03T00:00:01Z", - endtime="2008-08-03T23:59:59Z", limit=5) - # reverse: endtime earlier than starttime - self.assertRaises(pywikibot.Error, mysite.blocks, - starttime="2008-08-03T23:59:59Z", - endtime="2008-08-03T00:00:01Z", reverse=True, limit=5) - for block in mysite.blocks(users=mysite.user(), limit=5): - self.assertType(block, dict) - self.assertEqual(block['user'], mysite.user()) - - def testExturlusage(self): - """Test the site.exturlusage() method""" - - url = "www.google.com" - eu = list(mysite.exturlusage(url, limit=10)) - self.assertTrue(len(eu) <= 10) - self.assertTrue(all(isinstance(link, pywikibot.Page) - for link in eu)) - for link in mysite.exturlusage(url, namespaces=[2, 3], limit=5): - self.assertType(link, pywikibot.Page) - self.assertTrue(link.namespace() in (2, 3)) - - def testImageusage(self): - """Test the site.imageusage() method""" - - iu = list(mysite.imageusage(imagepage, limit=10)) - self.assertTrue(len(iu) <= 10) - self.assertTrue(all(isinstance(link, pywikibot.Page) - for link in iu)) - for using in mysite.imageusage(imagepage, namespaces=[3,4], limit=5): - self.assertType(using, pywikibot.Page) - self.assertTrue(imagepage in list(using.imagelinks())) - for using in mysite.imageusage(imagepage, filterredir=True, limit=5): - self.assertType(using, pywikibot.Page) - self.assertTrue(using.isRedirectPage()) - for using in mysite.imageusage(imagepage, filterredir=True, limit=5): - self.assertType(using, pywikibot.Page) - self.assertFalse(using.isRedirectPage()) - - def testLogEvents(self): - """Test the site.logevents() method""" - - le = list(mysite.logevents(limit=10)) - self.assertTrue(len(le) <= 10) - self.assertTrue(all(isinstance(entry, dict) and "type" in entry - for entry in le)) - for typ in ("block", "protect", "rights", "delete", "upload", - "move", "import", "patrol", "merge"): - for entry in mysite.logevents(logtype=typ, limit=3): - self.assertEqual(entry["type"], typ) - for entry in mysite.logevents(page=mainpage, limit=3): - self.assertTrue("title" in entry - and entry["title"] == mainpage.title()) - for entry in mysite.logevents(user=mysite.user(), limit=3): - self.assertTrue("user" in entry - and entry["user"] == mysite.user()) - for entry in mysite.logevents(start="2008-09-01T00:00:01Z", limit=5): - self.assertType(entry, dict) - self.assertTrue(entry['timestamp'] <= "2008-09-01T00:00:01Z") - for entry in mysite.logevents(end="2008-09-02T23:59:59Z", limit=5): - self.assertType(entry, dict) - self.assertTrue(entry['timestamp'] >= "2008-09-02T23:59:59Z") - for entry in mysite.logevents(start="2008-02-02T00:00:01Z", - end="2008-02-02T23:59:59Z", - reverse=True, limit=5): - self.assertType(entry, dict) - self.assertTrue("2008-02-02T00:00:01Z" <= entry['timestamp'] - <= "2008-02-02T23:59:59Z") - for entry in mysite.logevents(start="2008-02-03T23:59:59Z", - end="2008-02-03T00:00:01Z", - limit=5): - self.assertType(entry, dict) - self.assertTrue("2008-02-03T00:00:01Z" <= entry['timestamp'] - <= "2008-02-03T23:59:59Z") - # starttime earlier than endtime - self.assertRaises(pywikibot.Error, mysite.logevents, - start="2008-02-03T00:00:01Z", - end="2008-02-03T23:59:59Z", limit=5) - # reverse: endtime earlier than starttime - self.assertRaises(pywikibot.Error, mysite.logevents, - start="2008-02-03T23:59:59Z", - end="2008-02-03T00:00:01Z", reverse=True, limit=5) - - def testRecentchanges(self): - """Test the site.recentchanges() method""" - - rc = list(mysite.recentchanges(limit=10)) - self.assertTrue(len(rc) <= 10) - self.assertTrue(all(isinstance(change, dict) - for change in rc)) - for change in mysite.recentchanges(start="2008-10-01T01:02:03Z", - limit=5): - self.assertType(change, dict) - self.assertTrue(change['timestamp'] <= "2008-10-01T01:02:03Z") - for change in mysite.recentchanges(end="2008-04-01T02:03:04Z", - limit=5): - self.assertType(change, dict) - self.assertTrue(change['timestamp'] >= "2008-10-01T02:03:04Z") - for change in mysite.recentchanges(start="2008-10-01T03:05:07Z", - limit=5, reverse=True): - self.assertType(change, dict) - self.assertTrue(change['timestamp'] >= "2008-10-01T03:05:07Z") - for change in mysite.recentchanges(end="2008-10-01T04:06:08Z", - limit=5, reverse=True): - self.assertType(change, dict) - self.assertTrue(change['timestamp'] <= "2008-10-01T04:06:08Z") - for change in mysite.recentchanges(start="2008-10-03T11:59:59Z", - end="2008-10-03T00:00:01Z", - limit=5): - self.assertType(change, dict) - self.assertTrue("2008-10-03T00:00:01Z" <= change['timestamp'] - <= "2008-10-03T11:59:59Z") - for change in mysite.recentchanges(start="2008-10-05T06:00:01Z", - end="2008-10-05T23:59:59Z", - reverse=True, limit=5): - self.assertType(change, dict) - self.assertTrue("2008-10-05T06:00:01Z" <= change['timestamp'] - <= "2008-10-05T23:59:59Z") - # start earlier than end - self.assertRaises(pywikibot.Error, mysite.recentchanges, - start="2008-02-03T00:00:01Z", - end="2008-02-03T23:59:59Z", limit=5) - # reverse: end earlier than start - self.assertRaises(pywikibot.Error, mysite.recentchanges, - start="2008-02-03T23:59:59Z", - end="2008-02-03T00:00:01Z", reverse=True, limit=5) - for change in mysite.recentchanges(namespaces=[6,7], limit=5): - self.assertType(change, dict) - self.assertTrue("title" in change and "ns" in change) - title = change['title'] - self.assertTrue(":" in title) - prefix = title[ : title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6,7]) - self.assertTrue(change["ns"] in [6,7]) - for change in mysite.recentchanges(pagelist=[mainpage, imagepage], - limit=5): - self.assertType(change, dict) - self.assertTrue("title" in change) - self.assertTrue(change["title"] in (mainpage.title(), - imagepage.title())) - for typ in ("edit", "new", "log"): - for change in mysite.recentchanges(changetype=typ, limit=5): - self.assertType(change, dict) - self.assertTrue("type" in change) - self.assertEqual(change["type"], typ) - for change in mysite.recentchanges(showMinor=True, limit=5): - self.assertType(change, dict) - self.assertTrue("minor" in change) - for change in mysite.recentchanges(showMinor=False, limit=5): - self.assertType(change, dict) - self.assertTrue("minor" not in change) - for change in mysite.recentchanges(showBot=True, limit=5): - self.assertType(change, dict) - self.assertTrue("bot" in change) - for change in mysite.recentchanges(showBot=False, limit=5): - self.assertType(change, dict) - self.assertTrue("bot" not in change) - for change in mysite.recentchanges(showAnon=True, limit=5): - self.assertType(change, dict) - for change in mysite.recentchanges(showAnon=False, limit=5): - self.assertType(change, dict) - for change in mysite.recentchanges(showRedirects=True, limit=5): - self.assertType(change, dict) - self.assertTrue("redirect" in change) - for change in mysite.recentchanges(showRedirects=False, limit=5): - self.assertType(change, dict) - self.assertTrue("redirect" not in change) - for change in mysite.recentchanges(showPatrolled=True, limit=5): - self.assertType(change, dict) - self.assertTrue("patrolled" in change) - for change in mysite.recentchanges(showPatrolled=False, limit=5): - self.assertType(change, dict) - self.assertTrue("patrolled" not in change) - - def testSearch(self): - """Test the site.search() method""" - - se = list(mysite.search("wiki", limit=10)) - self.assertTrue(len(se) <= 10) - self.assertTrue(all(isinstance(hit, pywikibot.Page) - for hit in se)) - self.assertTrue(all(hit.namespace() == 0 for hit in se)) - for hit in mysite.search("common", namespaces=4, limit=5): - self.assertType(hit, pywikibot.Page) - self.assertEqual(hit.namespace(), 4) - for hit in mysite.search("word", namespaces=[5,6,7], limit=5): - self.assertType(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [5,6,7]) - for hit in mysite.search("another", namespaces="8|9|10", limit=5): - self.assertType(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [8,9,10]) - for hit in mysite.search("wiki", namespaces=0, limit=10, - getredirects=True): - self.assertType(hit, pywikibot.Page) - self.assertEqual(hit.namespace(), 0) - - def testUsercontribs(self): - """Test the site.usercontribs() method""" - - uc = list(mysite.usercontribs(user=mysite.user(), limit=10)) - self.assertTrue(len(uc) <= 10) - self.assertTrue(all(isinstance(contrib, dict) - for contrib in uc)) - self.assertTrue(all("user" in contrib - and contrib["user"] == mysite.user() - for contrib in uc)) - for contrib in mysite.usercontribs(userprefix="John", limit=5): - self.assertType(contrib, dict) - for key in ("user", "title", "ns", "pageid", "revid"): - self.assertTrue(key in contrib) - self.assertTrue(contrib["user"].startswith("John")) - for contrib in mysite.usercontribs(userprefix="Jane", - start="2008-10-06T01:02:03Z", - limit=5): - self.assertTrue(contrib['timestamp'] <= "2008-10-06T01:02:03Z") - for contrib in mysite.usercontribs(userprefix="Jane", - end="2008-10-07T02:03:04Z", - limit=5): - self.assertTrue(contrib['timestamp'] >= "2008-10-07T02:03:04Z") - for contrib in mysite.usercontribs(userprefix="Brion", - start="2008-10-08T03:05:07Z", - limit=5, reverse=True): - self.assertTrue(contrib['timestamp'] >= "2008-10-08T03:05:07Z") - for contrib in mysite.usercontribs(userprefix="Brion", - end="2008-10-09T04:06:08Z", - limit=5, reverse=True): - self.assertTrue(contrib['timestamp'] <= "2008-10-09T04:06:08Z") - for contrib in mysite.usercontribs(userprefix="Tim", - start="2008-10-10T11:59:59Z", - end="2008-10-10T00:00:01Z", - limit=5): - self.assertTrue("2008-10-10T00:00:01Z" <= contrib['timestamp'] - <= "2008-10-10T11:59:59Z") - for contrib in mysite.usercontribs(userprefix="Tim", - start="2008-10-11T06:00:01Z", - end="2008-10-11T23:59:59Z", - reverse=True, limit=5): - self.assertTrue("2008-10-11T06:00:01Z" <= contrib['timestamp'] - <= "2008-10-11T23:59:59Z") - # start earlier than end - self.assertRaises(pywikibot.Error, mysite.usercontribs, - userprefix="Jim", - start="2008-10-03T00:00:01Z", - end="2008-10-03T23:59:59Z", limit=5) - # reverse: end earlier than start - self.assertRaises(pywikibot.Error, mysite.usercontribs, - userprefix="Jim", - start="2008-10-03T23:59:59Z", - end="2008-10-03T00:00:01Z", reverse=True, limit=5) - - for contrib in mysite.usercontribs(user=mysite.user(), - namespaces=14, limit=5): - self.assertType(contrib, dict) - self.assertTrue("title" in contrib) - self.assertTrue(contrib["title"].startswith(mysite.namespace(14))) - for contrib in mysite.usercontribs(user=mysite.user(), - namespaces=[10,11], limit=5): - self.assertType(contrib, dict) - self.assertTrue("title" in contrib) - self.assertTrue(contrib["ns"] in (10, 11)) - for contrib in mysite.usercontribs(user=mysite.user(), - showMinor=True, limit=5): - self.assertType(contrib, dict) - self.assertTrue("minor" in contrib) - for contrib in mysite.usercontribs(user=mysite.user(), - showMinor=False, limit=5): - self.assertType(contrib, dict) - self.assertTrue("minor" not in contrib) - - def testWatchlistrevs(self): - """Test the site.watchlist_revs() method""" - - wl = list(mysite.watchlist_revs(limit=10)) - self.assertTrue(len(wl) <= 10) - self.assertTrue(all(isinstance(rev, dict) - for rev in wl)) - for rev in mysite.watchlist_revs(start="2008-10-11T01:02:03Z", - limit=5): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] <= "2008-10-11T01:02:03Z") - for rev in mysite.watchlist_revs(end="2008-04-01T02:03:04Z", - limit=5): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] >= "2008-10-11T02:03:04Z") - for rev in mysite.watchlist_revs(start="2008-10-11T03:05:07Z", - limit=5, reverse=True): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] >= "2008-10-11T03:05:07Z") - for rev in mysite.watchlist_revs(end="2008-10-11T04:06:08Z", - limit=5, reverse=True): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] <= "2008-10-11T04:06:08Z") - for rev in mysite.watchlist_revs(start="2008-10-13T11:59:59Z", - end="2008-10-13T00:00:01Z", - limit=5): - self.assertType(rev, dict) - self.assertTrue("2008-10-13T00:00:01Z" <= rev['timestamp'] - <= "2008-10-13T11:59:59Z") - for rev in mysite.watchlist_revs(start="2008-10-15T06:00:01Z", - end="2008-10-15T23:59:59Z", - reverse=True, limit=5): - self.assertType(rev, dict) - self.assertTrue("2008-10-15T06:00:01Z" <= rev['timestamp'] - <= "2008-10-15T23:59:59Z") - # start earlier than end - self.assertRaises(pywikibot.Error, mysite.watchlist_revs, - start="2008-09-03T00:00:01Z", - end="2008-09-03T23:59:59Z", limit=5) - # reverse: end earlier than start - self.assertRaises(pywikibot.Error, mysite.watchlist_revs, - start="2008-09-03T23:59:59Z", - end="2008-09-03T00:00:01Z", reverse=True, limit=5) - for rev in mysite.watchlist_revs(namespaces=[6,7], limit=5): - self.assertType(rev, dict) - self.assertTrue("title" in rev and "ns" in rev) - title = rev['title'] - self.assertTrue(":" in title) - prefix = title[ : title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6,7]) - self.assertTrue(rev["ns"] in [6,7]) - for rev in mysite.watchlist_revs(showMinor=True, limit=5): - self.assertType(rev, dict) - self.assertTrue("minor" in rev) - for rev in mysite.watchlist_revs(showMinor=False, limit=5): - self.assertType(rev, dict) - self.assertTrue("minor" not in rev) - for rev in mysite.watchlist_revs(showBot=True, limit=5): - self.assertType(rev, dict) - self.assertTrue("bot" in rev) - for rev in mysite.watchlist_revs(showBot=False, limit=5): - self.assertType(rev, dict) - self.assertTrue("bot" not in rev) - for rev in mysite.watchlist_revs(showAnon=True, limit=5): - self.assertType(rev, dict) - for rev in mysite.watchlist_revs(showAnon=False, limit=5): - self.assertType(rev, dict) - - def testDeletedrevs(self): - """Test the site.deletedrevs() method""" - - if not mysite.logged_in(True): - try: - mysite.login(True) - except pywikibot.NoUsername: - logger.warn( - "Cannot test Site.deleted_revs; no sysop account configured.") - return - dr = list(mysite.deletedrevs(limit=10, page=mainpage)) - self.assertTrue(len(dr) <= 10) - self.assertTrue(all(isinstance(rev, dict) - for rev in dr)) - dr2 = list(mysite.deletedrevs(page=mainpage, limit=10)) - self.assertTrue(len(dr2) <= 10) - self.assertTrue(all(isinstance(rev, dict) - for rev in dr2)) - for rev in mysite.deletedrevs(start="2008-10-11T01:02:03Z", - page=mainpage, limit=5): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] <= "2008-10-11T01:02:03Z") - for rev in mysite.deletedrevs(end="2008-04-01T02:03:04Z", - page=mainpage, limit=5): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] >= "2008-10-11T02:03:04Z") - for rev in mysite.deletedrevs(start="2008-10-11T03:05:07Z", - page=mainpage, limit=5, - reverse=True): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] >= "2008-10-11T03:05:07Z") - for rev in mysite.deletedrevs(end="2008-10-11T04:06:08Z", - page=mainpage, limit=5, - reverse=True): - self.assertType(rev, dict) - self.assertTrue(rev['timestamp'] <= "2008-10-11T04:06:08Z") - for rev in mysite.deletedrevs(start="2008-10-13T11:59:59Z", - end="2008-10-13T00:00:01Z", - page=mainpage, limit=5): - self.assertType(rev, dict) - self.assertTrue("2008-10-13T00:00:01Z" <= rev['timestamp'] - <= "2008-10-13T11:59:59Z") - for rev in mysite.deletedrevs(start="2008-10-15T06:00:01Z", - end="2008-10-15T23:59:59Z", - page=mainpage, reverse=True, - limit=5): - self.assertType(rev, dict) - self.assertTrue("2008-10-15T06:00:01Z" <= rev['timestamp'] - <= "2008-10-15T23:59:59Z") - # start earlier than end - self.assertRaises(pywikibot.Error, mysite.deletedrevs, - page=mainpage, start="2008-09-03T00:00:01Z", - end="2008-09-03T23:59:59Z", limit=5) - # reverse: end earlier than start - self.assertRaises(pywikibot.Error, mysite.deletedrevs, - page=mainpage, start="2008-09-03T23:59:59Z", - end="2008-09-03T00:00:01Z", reverse=True, - limit=5) - - def testUsers(self): - """Test the site.users() method""" - - us = list(mysite.users(mysite.user())) - self.assertEqual(len(us), 1) - self.assertType(us[0], dict) - for user in mysite.users( - ["Jimbo Wales", "Brion VIBBER", "Tim Starling"]): - self.assertType(user, dict) - self.assertTrue(user["name"] - in ["Jimbo Wales", "Brion VIBBER", "Tim Starling"]) - - def testRandompages(self): - """Test the site.randompages() method""" - - rn = list(mysite.randompages(limit=10)) - self.assertTrue(len(rn) <= 10) - self.assertTrue(all(isinstance(a_page, pywikibot.Page) - for a_page in rn)) - self.assertFalse(all(a_page.isRedirectPage() for a_page in rn)) - for rndpage in mysite.randompages(limit=5, redirects=True): - self.assertType(rndpage, pywikibot.Page) - self.assertTrue(rndpage.isRedirectPage()) - for rndpage in mysite.randompages(limit=5, namespaces=[6, 7]): - self.assertType(rndpage, pywikibot.Page) - self.assertTrue(rndpage.namespace() in [6, 7]) - - -if __name__ == '__main__': -# pywikibot.logging.getLogger("").setLevel(pywikibot.logging.DEBUG) - try: - try: - unittest.main() - except SystemExit: - pass - finally: - pywikibot.stopme() +# -*- coding: utf-8 -*- +""" +Tests for the site module. +""" +# +# (C) Pywikipedia bot team, 2008 +# +# Distributed under the terms of the MIT license. +# +__version__ = '$Id: $' + + +import unittest +import pywikibot +import warnings + +logger = pywikibot.logging.getLogger("wiki.site.tests") + +mysite = pywikibot.Site() +mainpage = pywikibot.Page(pywikibot.Link("Main Page", mysite)) +imagepage = iter(mainpage.imagelinks()).next() # 1st image on main page + + +class TestSiteObject(unittest.TestCase): + """Test cases for Site methods.""" + + def assertType(self, obj, cls): + """Assert that obj is an instance of type cls""" + return self.assertTrue(isinstance(obj, cls)) + + def testBaseMethods(self): + """Test cases for BaseSite methods""" + + self.assertEqual(mysite.family.name, pywikibot.config2.family) + self.assertEqual(mysite.code, pywikibot.config2.mylang) + self.assertType(mysite.lang, basestring) + self.assertType(mysite == pywikibot.Site("en", "wikipedia"), bool) + self.assertType(mysite.user(), (basestring, type(None))) + self.assertEqual(mysite.sitename(), + "%s:%s" % (pywikibot.config2.family, + pywikibot.config2.mylang)) + self.assertEqual(repr(mysite), + 'Site("%s", "%s")' + % (pywikibot.config2.mylang, pywikibot.config2.family)) + self.assertType(mysite.linktrail(), basestring) + self.assertType(mysite.redirect(default=True), basestring) + self.assertType(mysite.disambcategory(), pywikibot.Category) + self.assertEqual(mysite.linkto("foo"), u"[[Foo]]") + self.assertFalse(mysite.isInterwikiLink("foo")) + self.assertType(mysite.redirectRegex().pattern, basestring) + self.assertType(mysite.category_on_one_line(), bool) + for grp in ("user", "autoconfirmed", "bot", "sysop", "nosuchgroup"): + self.assertType(mysite.has_group(grp), bool) + for rgt in ("read", "edit", "move", "delete", "rollback", "block", + "nosuchright"): + self.assertType(mysite.has_right(rgt), bool) + + def testLanguageMethods(self): + """Test cases for languages() and related methods""" + + langs = mysite.languages() + self.assertType(langs, list) + self.assertTrue(mysite.code in langs) + obs = mysite.family.obsolete + ipf = mysite.interwiki_putfirst() + self.assertType(ipf, list) + self.assertTrue(all(item in langs or item in obs + for item in ipf)) + self.assertTrue(all(item in langs + for item in mysite.validLanguageLinks())) + + def testNamespaceMethods(self): + """Test cases for methods manipulating namespace names""" + + builtins = {'Talk': 1, # these should work in any MW wiki + 'User': 2, + 'User talk': 3, + 'Project': 4, + 'Project talk': 5, + 'Image': 6, + 'Image talk': 7, + 'MediaWiki': 8, + 'MediaWiki talk': 9, + 'Template': 10, + 'Template talk': 11, + 'Help': 12, + 'Help talk': 13, + 'Category': 14, + 'Category talk': 15, + } + self.assertTrue(all(mysite.ns_index(b) == builtins[b] + for b in builtins)) + ns = mysite.namespaces() + self.assertType(ns, dict) + self.assertTrue(all(x in ns for x in xrange(0, 16))) + # built-in namespaces always present + self.assertType(mysite.ns_normalize("project"), basestring) + self.assertTrue(all(isinstance(key, int) + for key in ns)) + self.assertTrue(all(isinstance(val, list) + for val in ns.values())) + self.assertTrue(all(isinstance(name, basestring) + for val in ns.values() + for name in val)) + self.assertTrue(all(isinstance(mysite.namespace(key), basestring) + for key in ns)) + self.assertTrue(all(isinstance(mysite.namespace(key, True), list) + for key in ns)) + self.assertTrue(all(isinstance(item, basestring) + for key in ns + for item in mysite.namespace(key, True))) + + def testApiMethods(self): + """Test generic ApiSite methods""" + + self.assertType(mysite.logged_in(), bool) + self.assertType(mysite.logged_in(True), bool) + self.assertType(mysite.userinfo, dict) + self.assertType(mysite.is_blocked(), bool) + self.assertType(mysite.messages(), bool) + self.assertType(mysite.has_right("edit"), bool) + self.assertFalse(mysite.has_right("nonexistent_right")) + self.assertType(mysite.has_group("bots"), bool) + self.assertFalse(mysite.has_group("nonexistent_group")) + try: + self.assertType(mysite.is_blocked(True), bool) + self.assertType(mysite.has_right("edit", True), bool) + self.assertFalse(mysite.has_right("nonexistent_right", True)) + self.assertType(mysite.has_group("bots", True), bool) + self.assertFalse(mysite.has_group("nonexistent_group", True)) + except pywikibot.NoUsername: + logger.warn( + "Cannot test Site methods for sysop; no sysop account configured.") + for msg in ("1movedto2", "about", "aboutpage", "aboutsite", + "accesskey-n-portal"): + self.assertTrue(mysite.has_mediawiki_message(msg)) + self.assertType(mysite.mediawiki_message(msg), basestring) + self.assertFalse(mysite.has_mediawiki_message("nosuchmessage")) + self.assertRaises(KeyError, mysite.mediawiki_message, "nosuchmessage") + self.assertType(mysite.getcurrenttimestamp(), basestring) + self.assertType(mysite.siteinfo, dict) + self.assertType(mysite.case(), basestring) + ver = mysite.live_version() + self.assertType(ver, tuple) + self.assertTrue(all(isinstance(ver[i], int) for i in (0, 1))) + self.assertType(ver[2], basestring) + + def testPageMethods(self): + """Test ApiSite methods for getting page-specific info""" + + self.assertType(mysite.page_exists(mainpage), bool) + self.assertType(mysite.page_restrictions(mainpage), dict) + self.assertType(mysite.page_can_be_edited(mainpage), bool) + self.assertType(mysite.page_isredirect(mainpage), bool) + if mysite.page_isredirect(mainpage): + self.assertType(mysite.getredirtarget(mainpage), pywikibot.Page) + else: + self.assertRaises(pywikibot.IsNotRedirectPage, + mysite.getredirtarget, mainpage) + a = list(mysite.preloadpages([mainpage])) + self.assertEqual(len(a), int(mysite.page_exists(mainpage))) + if a: + self.assertEqual(a[0], mainpage) + + def testTokens(self): + """Test ability to get page tokens""" + + for ttype in ("edit", "move"): # token types for non-sysops + self.assertType(mysite.token(mainpage, ttype), basestring) + self.assertRaises(KeyError, mysite.token, mainpage, "invalidtype") + + def testPreload(self): + """Test that preloading works""" + + count = 0 + for page in mysite.preloadpages(mysite.pagelinks(mainpage, limit=10)): + self.assertType(page, pywikibot.Page) + self.assertType(page.exists(), bool) + if page.exists(): + self.assertTrue(hasattr(page, "_text")) + count += 1 + if count >= 5: + break + + def testLinkMethods(self): + """Test site methods for getting links to and from a page""" + + backlinks = set(mysite.pagebacklinks(mainpage, namespaces=[0])) + # only non-redirects: + filtered = set(mysite.pagebacklinks(mainpage, namespaces=0, + filterRedirects=False)) + # only redirects: + redirs = set(mysite.pagebacklinks(mainpage, namespaces=0, + filterRedirects=True)) + # including links to redirect pages (but not the redirects): + indirect = set(mysite.pagebacklinks(mainpage, namespaces=[0], + followRedirects=True)) + self.assertEqual(filtered & redirs, set([])) + self.assertEqual(indirect & redirs, set([])) + self.assertTrue(filtered.issubset(indirect)) + self.assertTrue(filtered.issubset(backlinks)) + self.assertTrue(redirs.issubset(backlinks)) + self.assertTrue(backlinks.issubset( + set(mysite.pagebacklinks(mainpage, namespaces=[0, 2])))) + + # pagereferences includes both backlinks and embeddedin + embedded = set(mysite.page_embeddedin(mainpage, namespaces=[0])) + refs = set(mysite.pagereferences(mainpage, namespaces=[0])) + self.assertTrue(backlinks.issubset(refs)) + self.assertTrue(embedded.issubset(refs)) + for bl in backlinks: + self.assertType(bl, pywikibot.Page) + self.assertTrue(bl in refs) + for ei in embedded: + self.assertType(ei, pywikibot.Page) + self.assertTrue(ei in refs) + for ref in refs: + self.assertTrue(ref in backlinks or ref in embedded) + # test embeddedin arguments + self.assertTrue(embedded.issuperset( + set(mysite.page_embeddedin(mainpage, filterRedirects=True, + namespaces=[0])))) + self.assertTrue(embedded.issuperset( + set(mysite.page_embeddedin(mainpage, filterRedirects=False, + namespaces=[0])))) + self.assertTrue(embedded.issubset( + set(mysite.page_embeddedin(mainpage, namespaces=[0, 2])))) + links = set(mysite.pagelinks(mainpage)) + for pl in links: + self.assertType(pl, pywikibot.Page) + # test links arguments + self.assertTrue(links.issuperset( + set(mysite.pagelinks(mainpage, namespaces=[0, 1])))) + for target in mysite.preloadpages( + mysite.pagelinks(mainpage, follow_redirects=True, + limit=5)): + self.assertType(target, pywikibot.Page) + self.assertFalse(target.isRedirectPage()) + # test pagecategories + for cat in mysite.pagecategories(mainpage): + self.assertType(cat, pywikibot.Category) + for cm in mysite.categorymembers(cat): + self.assertType(cat, pywikibot.Page) + # test pageimages + self.assertTrue(all(isinstance(im, pywikibot.ImagePage) + for im in mysite.pageimages(mainpage))) + # test pagetemplates + self.assertTrue(all(isinstance(te, pywikibot.Page) + for te in mysite.pagetemplates(mainpage))) + self.assertTrue(set(mysite.pagetemplates(mainpage)).issuperset( + set(mysite.pagetemplates(mainpage, namespaces=[10])))) + # test pagelanglinks + for ll in mysite.pagelanglinks(mainpage): + self.assertType(ll, pywikibot.Link) + # test page_extlinks + self.assertTrue(all(isinstance(el, basestring) + for el in mysite.page_extlinks(mainpage))) + + def testLoadRevisions(self): + """Test the site.loadrevisions() method""" + + mysite.loadrevisions(mainpage) + self.assertTrue(hasattr(mainpage, "_revid")) + self.assertTrue(hasattr(mainpage, "_revisions")) + self.assertTrue(mainpage._revid in mainpage._revisions) + # TODO test all the optional arguments + + def testAllPages(self): + """Test the site.allpages() method""" + + fwd = list(mysite.allpages(limit=10)) + self.assertTrue(len(fwd) <= 10) + for page in fwd: + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + rev = list(mysite.allpages(reverse=True, start="Aa", limit=12)) + self.assertTrue(len(rev) <= 12) + for page in rev: + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.title() <= "Aa") + for page in mysite.allpages(start="Py", limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.title() >= "Py") + for page in mysite.allpages(prefix="Pre", limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.title().startswith("Pre")) + for page in mysite.allpages(namespace=1, limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 1) + for page in mysite.allpages(filterredir=True, limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.isRedirectPage()) + for page in mysite.allpages(filterredir=False, limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertEqual(page.namespace(), 0) + self.assertFalse(page.isRedirectPage()) +## for page in mysite.allpages(filterlanglinks=True, limit=5): +## self.assertType(page, pywikibot.Page) +## self.assertTrue(mysite.page_exists(page)) +## self.assertEqual(page.namespace(), 0) +## for page in mysite.allpages(filterlanglinks=False, limit=5): +## self.assertType(page, pywikibot.Page) +## self.assertTrue(mysite.page_exists(page)) +## self.assertEqual(page.namespace(), 0) + for page in mysite.allpages(minsize=100, limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertTrue(len(page.text) >= 100) + for page in mysite.allpages(maxsize=200, limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertTrue(len(page.text) <= 200) + for page in mysite.allpages(protect_type="edit", limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertTrue("edit" in page._protection) + for page in mysite.allpages(protect_type="edit", + protect_level="sysop", limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(mysite.page_exists(page)) + self.assertTrue("edit" in page._protection) + self.assertTrue("sysop" in page._protection["edit"]) + + def testAllLinks(self): + """Test the site.alllinks() method""" + + fwd = list(mysite.alllinks(limit=10)) + self.assertTrue(len(fwd) <= 10) + self.assertTrue(all(isinstance(link, pywikibot.Page) for link in fwd)) + uniq = list(mysite.alllinks(limit=10, unique=True)) + self.assertTrue(all(link in uniq for link in fwd)) + for page in mysite.alllinks(start="Link", limit=5): + self.assertType(page, pywikibot.Page) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.title() >= "Link") + for page in mysite.alllinks(prefix="Fix", limit=5): + self.assertType(page, pywikibot.Page) + self.assertEqual(page.namespace(), 0) + self.assertTrue(page.title().startswith("Fix")) + for page in mysite.alllinks(namespace=1, limit=5): + self.assertType(page, pywikibot.Page) + self.assertEqual(page.namespace(), 1) + for page in mysite.alllinks(start="From", namespace=4, fromids=True, + limit=5): + self.assertType(page, pywikibot.Page) + self.assertTrue(page.title(withNamespace=False) >= "From") + self.assertTrue(hasattr(page, "_fromid")) + errgen = mysite.alllinks(unique=True, fromids=True) + self.assertRaises(pywikibot.Error, errgen.next) + + def testAllCategories(self): + """Test the site.allcategories() method""" + + ac = list(mysite.allcategories(limit=10)) + self.assertTrue(len(ac) <= 10) + self.assertTrue(all(isinstance(cat, pywikibot.Category) + for cat in ac)) + for cat in mysite.allcategories(limit=5, start="Abc"): + self.assertType(cat, pywikibot.Category) + self.assertTrue(cat.title(withNamespace=False) >= "Abc") + for cat in mysite.allcategories(limit=5, prefix="Def"): + self.assertType(cat, pywikibot.Category) + self.assertTrue(cat.title(withNamespace=False).startswith("Def")) +## # Bug # 15985 +## for cat in mysite.allcategories(limit=5, start="Hij", reverse=True): +## self.assertType(cat, pywikibot.Category) +## self.assertTrue(cat.title(withNamespace=False) <= "Hij") + + def testAllUsers(self): + """Test the site.allusers() method""" + + au = list(mysite.allusers(limit=10)) + self.assertTrue(len(au) <= 10) + for user in au: + self.assertType(user, dict) + self.assertTrue("name" in user) + self.assertTrue("editcount" in user) + self.assertTrue("registration" in user) + for user in mysite.allusers(start="B", limit=5): + self.assertType(user, dict) + self.assertTrue("name" in user) + self.assertTrue(user["name"] >= "B") + self.assertTrue("editcount" in user) + self.assertTrue("registration" in user) + for user in mysite.allusers(prefix="C", limit=5): + self.assertType(user, dict) + self.assertTrue("name" in user) + self.assertTrue(user["name"].startswith("C")) + self.assertTrue("editcount" in user) + self.assertTrue("registration" in user) + for user in mysite.allusers(prefix="D", group="sysop", limit=5): + self.assertType(user, dict) + self.assertTrue("name" in user) + self.assertTrue(user["name"].startswith("D")) + self.assertTrue("editcount" in user) + self.assertTrue("registration" in user) + self.assertTrue("groups" in user and "sysop" in user["groups"]) + + def testAllImages(self): + """Test the site.allimages() method""" + + ai = list(mysite.allimages(limit=10)) + self.assertTrue(len(ai) <= 10) + self.assertTrue(all(isinstance(image, pywikibot.ImagePage) + for image in ai)) + for impage in mysite.allimages(start="Ba", limit=5): + self.assertType(impage, pywikibot.ImagePage) + self.assertTrue(mysite.page_exists(impage)) + self.assertTrue(impage.title(withNamespace=False) >= "Ba") +## # Bug # 15985 +## for impage in mysite.allimages(start="Da", reverse=True, limit=5): +## self.assertType(impage, pywikibot.ImagePage) +## self.assertTrue(mysite.page_exists(impage)) +## self.assertTrue(impage.title() <= "Da") + for impage in mysite.allimages(prefix="Ch", limit=5): + self.assertType(impage, pywikibot.ImagePage) + self.assertTrue(mysite.page_exists(impage)) + self.assertTrue(impage.title(withNamespace=False).startswith("Ch")) + for impage in mysite.allimages(minsize=100, limit=5): + self.assertType(impage, pywikibot.ImagePage) + self.assertTrue(mysite.page_exists(impage)) + self.assertTrue(impage._imageinfo["size"] >= 100) + for impage in mysite.allimages(maxsize=2000, limit=5): + self.assertType(impage, pywikibot.ImagePage) + self.assertTrue(mysite.page_exists(impage)) + self.assertTrue(impage._imageinfo["size"] <= 2000) + + def testBlocks(self): + """Test the site.blocks() method""" + + props = ("id", "by", "timestamp", "expiry", "reason") + bl = list(mysite.blocks(limit=10)) + self.assertTrue(len(bl) <= 10) + for block in bl: + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + # timestamps should be in descending order + timestamps = [block['timestamp'] for block in bl] + for t in xrange(1, len(timestamps)): + self.assertTrue(timestamps[t] <= timestamps[t-1]) + + b2 = list(mysite.blocks(limit=10, reverse=True)) + self.assertTrue(len(b2) <= 10) + for block in b2: + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + # timestamps should be in ascending order + timestamps = [block['timestamp'] for block in b2] + for t in xrange(1, len(timestamps)): + self.assertTrue(timestamps[t] >= timestamps[t-1]) + + for block in mysite.blocks(starttime="2008-07-01T00:00:01Z", limit=5): + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + for block in mysite.blocks(endtime="2008-07-31T23:59:59Z", limit=5): + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + for block in mysite.blocks(starttime="2008-08-02T00:00:01Z", + endtime="2008-08-02T23:59:59Z", + reverse=True, limit=5): + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + for block in mysite.blocks(starttime="2008-08-03T23:59:59Z", + endtime="2008-08-03T00:00:01Z", + limit=5): + self.assertType(block, dict) + for prop in props: + self.assertTrue(prop in block) + # starttime earlier than endtime + self.assertRaises(pywikibot.Error, mysite.blocks, + starttime="2008-08-03T00:00:01Z", + endtime="2008-08-03T23:59:59Z", limit=5) + # reverse: endtime earlier than starttime + self.assertRaises(pywikibot.Error, mysite.blocks, + starttime="2008-08-03T23:59:59Z", + endtime="2008-08-03T00:00:01Z", reverse=True, limit=5) + for block in mysite.blocks(users=mysite.user(), limit=5): + self.assertType(block, dict) + self.assertEqual(block['user'], mysite.user()) + + def testExturlusage(self): + """Test the site.exturlusage() method""" + + url = "www.google.com" + eu = list(mysite.exturlusage(url, limit=10)) + self.assertTrue(len(eu) <= 10) + self.assertTrue(all(isinstance(link, pywikibot.Page) + for link in eu)) + for link in mysite.exturlusage(url, namespaces=[2, 3], limit=5): + self.assertType(link, pywikibot.Page) + self.assertTrue(link.namespace() in (2, 3)) + + def testImageusage(self): + """Test the site.imageusage() method""" + + iu = list(mysite.imageusage(imagepage, limit=10)) + self.assertTrue(len(iu) <= 10) + self.assertTrue(all(isinstance(link, pywikibot.Page) + for link in iu)) + for using in mysite.imageusage(imagepage, namespaces=[3,4], limit=5): + self.assertType(using, pywikibot.Page) + self.assertTrue(imagepage in list(using.imagelinks())) + for using in mysite.imageusage(imagepage, filterredir=True, limit=5): + self.assertType(using, pywikibot.Page) + self.assertTrue(using.isRedirectPage()) + for using in mysite.imageusage(imagepage, filterredir=True, limit=5): + self.assertType(using, pywikibot.Page) + self.assertFalse(using.isRedirectPage()) + + def testLogEvents(self): + """Test the site.logevents() method""" + + le = list(mysite.logevents(limit=10)) + self.assertTrue(len(le) <= 10) + self.assertTrue(all(isinstance(entry, dict) and "type" in entry + for entry in le)) + for typ in ("block", "protect", "rights", "delete", "upload", + "move", "import", "patrol", "merge"): + for entry in mysite.logevents(logtype=typ, limit=3): + self.assertEqual(entry["type"], typ) + for entry in mysite.logevents(page=mainpage, limit=3): + self.assertTrue("title" in entry + and entry["title"] == mainpage.title()) + for entry in mysite.logevents(user=mysite.user(), limit=3): + self.assertTrue("user" in entry + and entry["user"] == mysite.user()) + for entry in mysite.logevents(start="2008-09-01T00:00:01Z", limit=5): + self.assertType(entry, dict) + self.assertTrue(entry['timestamp'] <= "2008-09-01T00:00:01Z") + for entry in mysite.logevents(end="2008-09-02T23:59:59Z", limit=5): + self.assertType(entry, dict) + self.assertTrue(entry['timestamp'] >= "2008-09-02T23:59:59Z") + for entry in mysite.logevents(start="2008-02-02T00:00:01Z", + end="2008-02-02T23:59:59Z", + reverse=True, limit=5): + self.assertType(entry, dict) + self.assertTrue("2008-02-02T00:00:01Z" <= entry['timestamp'] + <= "2008-02-02T23:59:59Z") + for entry in mysite.logevents(start="2008-02-03T23:59:59Z", + end="2008-02-03T00:00:01Z", + limit=5): + self.assertType(entry, dict) + self.assertTrue("2008-02-03T00:00:01Z" <= entry['timestamp'] + <= "2008-02-03T23:59:59Z") + # starttime earlier than endtime + self.assertRaises(pywikibot.Error, mysite.logevents, + start="2008-02-03T00:00:01Z", + end="2008-02-03T23:59:59Z", limit=5) + # reverse: endtime earlier than starttime + self.assertRaises(pywikibot.Error, mysite.logevents, + start="2008-02-03T23:59:59Z", + end="2008-02-03T00:00:01Z", reverse=True, limit=5) + + def testRecentchanges(self): + """Test the site.recentchanges() method""" + + rc = list(mysite.recentchanges(limit=10)) + self.assertTrue(len(rc) <= 10) + self.assertTrue(all(isinstance(change, dict) + for change in rc)) + for change in mysite.recentchanges(start="2008-10-01T01:02:03Z", + limit=5): + self.assertType(change, dict) + self.assertTrue(change['timestamp'] <= "2008-10-01T01:02:03Z") + for change in mysite.recentchanges(end="2008-04-01T02:03:04Z", + limit=5): + self.assertType(change, dict) + self.assertTrue(change['timestamp'] >= "2008-10-01T02:03:04Z") + for change in mysite.recentchanges(start="2008-10-01T03:05:07Z", + limit=5, reverse=True): + self.assertType(change, dict) + self.assertTrue(change['timestamp'] >= "2008-10-01T03:05:07Z") + for change in mysite.recentchanges(end="2008-10-01T04:06:08Z", + limit=5, reverse=True): + self.assertType(change, dict) + self.assertTrue(change['timestamp'] <= "2008-10-01T04:06:08Z") + for change in mysite.recentchanges(start="2008-10-03T11:59:59Z", + end="2008-10-03T00:00:01Z", + limit=5): + self.assertType(change, dict) + self.assertTrue("2008-10-03T00:00:01Z" <= change['timestamp'] + <= "2008-10-03T11:59:59Z") + for change in mysite.recentchanges(start="2008-10-05T06:00:01Z", + end="2008-10-05T23:59:59Z", + reverse=True, limit=5): + self.assertType(change, dict) + self.assertTrue("2008-10-05T06:00:01Z" <= change['timestamp'] + <= "2008-10-05T23:59:59Z") + # start earlier than end + self.assertRaises(pywikibot.Error, mysite.recentchanges, + start="2008-02-03T00:00:01Z", + end="2008-02-03T23:59:59Z", limit=5) + # reverse: end earlier than start + self.assertRaises(pywikibot.Error, mysite.recentchanges, + start="2008-02-03T23:59:59Z", + end="2008-02-03T00:00:01Z", reverse=True, limit=5) + for change in mysite.recentchanges(namespaces=[6,7], limit=5): + self.assertType(change, dict) + self.assertTrue("title" in change and "ns" in change) + title = change['title'] + self.assertTrue(":" in title) + prefix = title[ : title.index(":")] + self.assertTrue(mysite.ns_index(prefix) in [6,7]) + self.assertTrue(change["ns"] in [6,7]) + for change in mysite.recentchanges(pagelist=[mainpage, imagepage], + limit=5): + self.assertType(change, dict) + self.assertTrue("title" in change) + self.assertTrue(change["title"] in (mainpage.title(), + imagepage.title())) + for typ in ("edit", "new", "log"): + for change in mysite.recentchanges(changetype=typ, limit=5): + self.assertType(change, dict) + self.assertTrue("type" in change) + self.assertEqual(change["type"], typ) + for change in mysite.recentchanges(showMinor=True, limit=5): + self.assertType(change, dict) + self.assertTrue("minor" in change) + for change in mysite.recentchanges(showMinor=False, limit=5): + self.assertType(change, dict) + self.assertTrue("minor" not in change) + for change in mysite.recentchanges(showBot=True, limit=5): + self.assertType(change, dict) + self.assertTrue("bot" in change) + for change in mysite.recentchanges(showBot=False, limit=5): + self.assertType(change, dict) + self.assertTrue("bot" not in change) + for change in mysite.recentchanges(showAnon=True, limit=5): + self.assertType(change, dict) + for change in mysite.recentchanges(showAnon=False, limit=5): + self.assertType(change, dict) + for change in mysite.recentchanges(showRedirects=True, limit=5): + self.assertType(change, dict) + self.assertTrue("redirect" in change) + for change in mysite.recentchanges(showRedirects=False, limit=5): + self.assertType(change, dict) + self.assertTrue("redirect" not in change) + for change in mysite.recentchanges(showPatrolled=True, limit=5): + self.assertType(change, dict) + self.assertTrue("patrolled" in change) + for change in mysite.recentchanges(showPatrolled=False, limit=5): + self.assertType(change, dict) + self.assertTrue("patrolled" not in change) + + def testSearch(self): + """Test the site.search() method""" + + se = list(mysite.search("wiki", limit=10)) + self.assertTrue(len(se) <= 10) + self.assertTrue(all(isinstance(hit, pywikibot.Page) + for hit in se)) + self.assertTrue(all(hit.namespace() == 0 for hit in se)) + for hit in mysite.search("common", namespaces=4, limit=5): + self.assertType(hit, pywikibot.Page) + self.assertEqual(hit.namespace(), 4) + for hit in mysite.search("word", namespaces=[5,6,7], limit=5): + self.assertType(hit, pywikibot.Page) + self.assertTrue(hit.namespace() in [5,6,7]) + for hit in mysite.search("another", namespaces="8|9|10", limit=5): + self.assertType(hit, pywikibot.Page) + self.assertTrue(hit.namespace() in [8,9,10]) + for hit in mysite.search("wiki", namespaces=0, limit=10, + getredirects=True): + self.assertType(hit, pywikibot.Page) + self.assertEqual(hit.namespace(), 0) + + def testUsercontribs(self): + """Test the site.usercontribs() method""" + + uc = list(mysite.usercontribs(user=mysite.user(), limit=10)) + self.assertTrue(len(uc) <= 10) + self.assertTrue(all(isinstance(contrib, dict) + for contrib in uc)) + self.assertTrue(all("user" in contrib + and contrib["user"] == mysite.user() + for contrib in uc)) + for contrib in mysite.usercontribs(userprefix="John", limit=5): + self.assertType(contrib, dict) + for key in ("user", "title", "ns", "pageid", "revid"): + self.assertTrue(key in contrib) + self.assertTrue(contrib["user"].startswith("John")) + for contrib in mysite.usercontribs(userprefix="Jane", + start="2008-10-06T01:02:03Z", + limit=5): + self.assertTrue(contrib['timestamp'] <= "2008-10-06T01:02:03Z") + for contrib in mysite.usercontribs(userprefix="Jane", + end="2008-10-07T02:03:04Z", + limit=5): + self.assertTrue(contrib['timestamp'] >= "2008-10-07T02:03:04Z") + for contrib in mysite.usercontribs(userprefix="Brion", + start="2008-10-08T03:05:07Z", + limit=5, reverse=True): + self.assertTrue(contrib['timestamp'] >= "2008-10-08T03:05:07Z") + for contrib in mysite.usercontribs(userprefix="Brion", + end="2008-10-09T04:06:08Z", + limit=5, reverse=True): + self.assertTrue(contrib['timestamp'] <= "2008-10-09T04:06:08Z") + for contrib in mysite.usercontribs(userprefix="Tim", + start="2008-10-10T11:59:59Z", + end="2008-10-10T00:00:01Z", + limit=5): + self.assertTrue("2008-10-10T00:00:01Z" <= contrib['timestamp'] + <= "2008-10-10T11:59:59Z") + for contrib in mysite.usercontribs(userprefix="Tim", + start="2008-10-11T06:00:01Z", + end="2008-10-11T23:59:59Z", + reverse=True, limit=5): + self.assertTrue("2008-10-11T06:00:01Z" <= contrib['timestamp'] + <= "2008-10-11T23:59:59Z") + # start earlier than end + self.assertRaises(pywikibot.Error, mysite.usercontribs, + userprefix="Jim", + start="2008-10-03T00:00:01Z", + end="2008-10-03T23:59:59Z", limit=5) + # reverse: end earlier than start + self.assertRaises(pywikibot.Error, mysite.usercontribs, + userprefix="Jim", + start="2008-10-03T23:59:59Z", + end="2008-10-03T00:00:01Z", reverse=True, limit=5) + + for contrib in mysite.usercontribs(user=mysite.user(), + namespaces=14, limit=5): + self.assertType(contrib, dict) + self.assertTrue("title" in contrib) + self.assertTrue(contrib["title"].startswith(mysite.namespace(14))) + for contrib in mysite.usercontribs(user=mysite.user(), + namespaces=[10,11], limit=5): + self.assertType(contrib, dict) + self.assertTrue("title" in contrib) + self.assertTrue(contrib["ns"] in (10, 11)) + for contrib in mysite.usercontribs(user=mysite.user(), + showMinor=True, limit=5): + self.assertType(contrib, dict) + self.assertTrue("minor" in contrib) + for contrib in mysite.usercontribs(user=mysite.user(), + showMinor=False, limit=5): + self.assertType(contrib, dict) + self.assertTrue("minor" not in contrib) + + def testWatchlistrevs(self): + """Test the site.watchlist_revs() method""" + + wl = list(mysite.watchlist_revs(limit=10)) + self.assertTrue(len(wl) <= 10) + self.assertTrue(all(isinstance(rev, dict) + for rev in wl)) + for rev in mysite.watchlist_revs(start="2008-10-11T01:02:03Z", + limit=5): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] <= "2008-10-11T01:02:03Z") + for rev in mysite.watchlist_revs(end="2008-04-01T02:03:04Z", + limit=5): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] >= "2008-10-11T02:03:04Z") + for rev in mysite.watchlist_revs(start="2008-10-11T03:05:07Z", + limit=5, reverse=True): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] >= "2008-10-11T03:05:07Z") + for rev in mysite.watchlist_revs(end="2008-10-11T04:06:08Z", + limit=5, reverse=True): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] <= "2008-10-11T04:06:08Z") + for rev in mysite.watchlist_revs(start="2008-10-13T11:59:59Z", + end="2008-10-13T00:00:01Z", + limit=5): + self.assertType(rev, dict) + self.assertTrue("2008-10-13T00:00:01Z" <= rev['timestamp'] + <= "2008-10-13T11:59:59Z") + for rev in mysite.watchlist_revs(start="2008-10-15T06:00:01Z", + end="2008-10-15T23:59:59Z", + reverse=True, limit=5): + self.assertType(rev, dict) + self.assertTrue("2008-10-15T06:00:01Z" <= rev['timestamp'] + <= "2008-10-15T23:59:59Z") + # start earlier than end + self.assertRaises(pywikibot.Error, mysite.watchlist_revs, + start="2008-09-03T00:00:01Z", + end="2008-09-03T23:59:59Z", limit=5) + # reverse: end earlier than start + self.assertRaises(pywikibot.Error, mysite.watchlist_revs, + start="2008-09-03T23:59:59Z", + end="2008-09-03T00:00:01Z", reverse=True, limit=5) + for rev in mysite.watchlist_revs(namespaces=[6,7], limit=5): + self.assertType(rev, dict) + self.assertTrue("title" in rev and "ns" in rev) + title = rev['title'] + self.assertTrue(":" in title) + prefix = title[ : title.index(":")] + self.assertTrue(mysite.ns_index(prefix) in [6,7]) + self.assertTrue(rev["ns"] in [6,7]) + for rev in mysite.watchlist_revs(showMinor=True, limit=5): + self.assertType(rev, dict) + self.assertTrue("minor" in rev) + for rev in mysite.watchlist_revs(showMinor=False, limit=5): + self.assertType(rev, dict) + self.assertTrue("minor" not in rev) + for rev in mysite.watchlist_revs(showBot=True, limit=5): + self.assertType(rev, dict) + self.assertTrue("bot" in rev) + for rev in mysite.watchlist_revs(showBot=False, limit=5): + self.assertType(rev, dict) + self.assertTrue("bot" not in rev) + for rev in mysite.watchlist_revs(showAnon=True, limit=5): + self.assertType(rev, dict) + for rev in mysite.watchlist_revs(showAnon=False, limit=5): + self.assertType(rev, dict) + + def testDeletedrevs(self): + """Test the site.deletedrevs() method""" + + if not mysite.logged_in(True): + try: + mysite.login(True) + except pywikibot.NoUsername: + logger.warn( + "Cannot test Site.deleted_revs; no sysop account configured.") + return + dr = list(mysite.deletedrevs(limit=10, page=mainpage)) + self.assertTrue(len(dr) <= 10) + self.assertTrue(all(isinstance(rev, dict) + for rev in dr)) + dr2 = list(mysite.deletedrevs(page=mainpage, limit=10)) + self.assertTrue(len(dr2) <= 10) + self.assertTrue(all(isinstance(rev, dict) + for rev in dr2)) + for rev in mysite.deletedrevs(start="2008-10-11T01:02:03Z", + page=mainpage, limit=5): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] <= "2008-10-11T01:02:03Z") + for rev in mysite.deletedrevs(end="2008-04-01T02:03:04Z", + page=mainpage, limit=5): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] >= "2008-10-11T02:03:04Z") + for rev in mysite.deletedrevs(start="2008-10-11T03:05:07Z", + page=mainpage, limit=5, + reverse=True): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] >= "2008-10-11T03:05:07Z") + for rev in mysite.deletedrevs(end="2008-10-11T04:06:08Z", + page=mainpage, limit=5, + reverse=True): + self.assertType(rev, dict) + self.assertTrue(rev['timestamp'] <= "2008-10-11T04:06:08Z") + for rev in mysite.deletedrevs(start="2008-10-13T11:59:59Z", + end="2008-10-13T00:00:01Z", + page=mainpage, limit=5): + self.assertType(rev, dict) + self.assertTrue("2008-10-13T00:00:01Z" <= rev['timestamp'] + <= "2008-10-13T11:59:59Z") + for rev in mysite.deletedrevs(start="2008-10-15T06:00:01Z", + end="2008-10-15T23:59:59Z", + page=mainpage, reverse=True, + limit=5): + self.assertType(rev, dict) + self.assertTrue("2008-10-15T06:00:01Z" <= rev['timestamp'] + <= "2008-10-15T23:59:59Z") + # start earlier than end + self.assertRaises(pywikibot.Error, mysite.deletedrevs, + page=mainpage, start="2008-09-03T00:00:01Z", + end="2008-09-03T23:59:59Z", limit=5) + # reverse: end earlier than start + self.assertRaises(pywikibot.Error, mysite.deletedrevs, + page=mainpage, start="2008-09-03T23:59:59Z", + end="2008-09-03T00:00:01Z", reverse=True, + limit=5) + + def testUsers(self): + """Test the site.users() method""" + + us = list(mysite.users(mysite.user())) + self.assertEqual(len(us), 1) + self.assertType(us[0], dict) + for user in mysite.users( + ["Jimbo Wales", "Brion VIBBER", "Tim Starling"]): + self.assertType(user, dict) + self.assertTrue(user["name"] + in ["Jimbo Wales", "Brion VIBBER", "Tim Starling"]) + + def testRandompages(self): + """Test the site.randompages() method""" + + rn = list(mysite.randompages(limit=10)) + self.assertTrue(len(rn) <= 10) + self.assertTrue(all(isinstance(a_page, pywikibot.Page) + for a_page in rn)) + self.assertFalse(all(a_page.isRedirectPage() for a_page in rn)) + for rndpage in mysite.randompages(limit=5, redirects=True): + self.assertType(rndpage, pywikibot.Page) + self.assertTrue(rndpage.isRedirectPage()) + for rndpage in mysite.randompages(limit=5, namespaces=[6, 7]): + self.assertType(rndpage, pywikibot.Page) + self.assertTrue(rndpage.namespace() in [6, 7]) + + +if __name__ == '__main__': +# pywikibot.logging.getLogger("").setLevel(pywikibot.logging.DEBUG) + try: + try: + unittest.main() + except SystemExit: + pass + finally: + pywikibot.stopme() Property changes on: branches/rewrite/pywikibot/tests/site_tests.py ___________________________________________________________________ Added: svn:eol-style + native Property changes on: branches/rewrite/pywikibot/userinterfaces/__init__.py ___________________________________________________________________ Added: svn:eol-style + native