lists.wikimedia.org
Sign In
Sign Up
Sign In
Sign Up
Manage this list
×
Keyboard Shortcuts
Thread View
j
: Next unread message
k
: Previous unread message
j a
: Jump to all threads
j l
: Jump to MailingList overview
2024
December
November
October
September
August
July
June
May
April
March
February
January
2023
December
November
October
September
August
July
June
May
April
March
February
January
2022
December
November
October
September
August
July
June
May
April
March
February
January
2021
December
November
October
September
August
July
June
May
April
March
February
January
2020
December
November
October
September
August
July
June
May
April
March
February
January
2019
December
November
October
September
August
July
June
May
April
March
February
January
2018
December
November
October
September
August
July
June
May
April
March
February
January
2017
December
November
October
September
August
July
June
May
April
March
February
January
2016
December
November
October
September
August
July
June
May
April
March
February
January
2015
December
November
October
September
August
July
June
May
April
March
February
January
2014
December
November
October
September
August
July
June
May
April
March
February
January
2013
December
November
October
September
August
July
June
May
April
March
February
January
2012
December
November
October
September
August
July
June
May
April
March
February
January
2011
December
November
October
September
August
July
June
May
April
March
February
January
2010
December
November
October
September
August
July
June
May
April
March
February
January
2009
December
November
October
September
August
July
June
May
April
List overview
Download
Pywikipedia-svn
----- 2024 -----
December 2024
November 2024
October 2024
September 2024
August 2024
July 2024
June 2024
May 2024
April 2024
March 2024
February 2024
January 2024
----- 2023 -----
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
----- 2022 -----
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
----- 2021 -----
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
----- 2020 -----
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
----- 2019 -----
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
----- 2018 -----
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
----- 2017 -----
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
----- 2016 -----
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
----- 2015 -----
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
----- 2014 -----
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
----- 2013 -----
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
----- 2012 -----
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
----- 2011 -----
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
----- 2010 -----
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
----- 2009 -----
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
pywikipedia-svn@lists.wikimedia.org
5163 discussions
Start a n
N
ew thread
SVN: [11552] trunk/pywikipedia/families/wikibooks_family.py
by xqt@svn.wikimedia.org
19 May '13
19 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11552
Revision: 11552 Author: xqt Date: 2013-05-19 10:27:43 +0000 (Sun, 19 May 2013) Log Message: ----------- update namespace aliases Modified Paths: -------------- trunk/pywikipedia/families/wikibooks_family.py Modified: trunk/pywikipedia/families/wikibooks_family.py =================================================================== --- trunk/pywikipedia/families/wikibooks_family.py 2013-05-19 09:16:42 UTC (rev 11551) +++ trunk/pywikipedia/families/wikibooks_family.py 2013-05-19 10:27:43 UTC (rev 11552) @@ -25,16 +25,16 @@ for lang in self.languages_by_size]) # Override defaults + self.namespaces[15]['bn'] = [u'বিষয়শ্রেণী আলোচনা'] self.namespaces[14]['bn'] = [u'বিষয়শ্রেণী'] - self.namespaces[15]['bn'] = [u'বিষয়শ্রেণী আলোচনা'] + self.namespaces[3]['ca'] = [u'Usuari Discussió'] + self.namespaces[2]['ca'] = [u'Usuari'] + self.namespaces[3]['cs'] = [u'Diskuse s uživatelem', u'Uživatel diskuse', u'Uživatelka diskuse', u'Diskuse s uživatelkou'] self.namespaces[2]['cs'] = [u'Uživatel', u'Uživatelka'] - self.namespaces[3]['cs'] = [u'Diskuse s uživatelem', u'Uživatel diskuse', u'Uživatelka diskuse', u'Diskuse s uživatelkou'] + self.namespaces[9]['da'] = [u'MediaWiki diskussion', u'MediaWiki-diskussion'] self.namespaces[14]['en'] = [u'Category', u'CAT'] self.namespaces[2]['fr'] = [u'Utilisateur'] self.namespaces[3]['fr'] = [u'Discussion utilisateur', u'Discussion Utilisateur'] - self.namespaces[2]['ca'] = [u'Usuari'] - self.namespaces[3]['ca'] = [u'Usuari Discussió'] - self.namespaces[9]['da'] = [u'MediaWiki diskussion', u'MediaWiki-diskussion'] self.namespaces[13]['da'] = [u'Hjælp diskussion', u'Hjælp-diskussion'] self.namespaces[3]['pl'] = [u'Dyskusja wikipedysty', u'Dyskusja wikipedystki', u'Dyskusja użytkownika', u'Dyskusja użytkowniczki'] self.namespaces[2]['pl'] = [u'Wikipedysta', u'Wikipedystka', u'Użytkownik', u'Użytkowniczka'] @@ -93,7 +93,7 @@ 'kk': [u'Уикикітап', u'Wikibooks'], 'km': u'Wikibooks', 'ko': [u'위키책', u'Wikibooks'], - 'ku': u'Wikibooks', + 'ku': [u'Wîkîpirtûk', u'Wikibooks'], 'ky': u'Wikibooks', 'la': [u'Vicilibri', u'Wikibooks'], 'li': [u'Wikibeuk', u'Wikibooks'], @@ -190,7 +190,7 @@ 'km': [u'ការពិភាក្សាអំពីWikibooks', u'Wikibooks ពិភាក្ស'], 'kn': u'Wikibooks ಚರ್ಚೆ', 'ko': u'위키책토론', - 'ku': [u'Gotûbêja Wîkîpirtûkê', u'Wikibooks nîqaş'], + 'ku': [u'Gotûbêja Wîkîpirtûkê', u'Wîkîpirtûk nîqaş'], 'ky': u'Wikibooks баарлашуу', 'la': u'Disputatio Vicilibrorum', 'lb': u'Wikibooks Diskussioun',
1
0
0
0
SVN: [11551] trunk/pywikipedia
by drtrigon@svn.wikimedia.org
19 May '13
19 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11551
Revision: 11551 Author: drtrigon Date: 2013-05-19 09:16:42 +0000 (Sun, 19 May 2013) Log Message: ----------- improvement; added externals import in order to check and install needed modules Modified Paths: -------------- trunk/pywikipedia/catimages.py trunk/pywikipedia/maintainer.py trunk/pywikipedia/maintcont.py Modified: trunk/pywikipedia/catimages.py =================================================================== --- trunk/pywikipedia/catimages.py 2013-05-18 19:42:20 UTC (rev 11550) +++ trunk/pywikipedia/catimages.py 2013-05-19 09:16:42 UTC (rev 11551) @@ -57,7 +57,7 @@ if not os.path.isabs(scriptdir): scriptdir = os.path.abspath(os.path.join(os.curdir, scriptdir)) -# additional python packages (more exotic and problematic ones) +# additional python packages (non-default but common) try: import numpy as np from scipy import ndimage, fftpack, linalg#, signal @@ -78,6 +78,14 @@ except: # either raise the ImportError later or skip it pass + +# pywikipedia framework python packages +import wikipedia as pywikibot +import pagegenerators, catlib +import checkimages +import externals # allow import from externals + +# additional python packages (more exotic and problematic ones) # modules needing compilation are imported later on request: # (see
https://jira.toolserver.org/browse/TS-1452
) # e.g. opencv, jseg, slic, pydmtx, zbar, (pyml or equivalent) @@ -86,14 +94,8 @@ # TODO: # (pdfminer not used anymore/at the moment...) # python-djvulibre or python-djvu for djvu support - -# pywikipedia framework python packages -import wikipedia as pywikibot -import pagegenerators, catlib -import checkimages -import externals # check for and install needed -externals.check_setup('colormath') # 'externals' modules -externals.check_setup('jseg') # +externals.check_setup('colormath') # check for and install needed +externals.check_setup('jseg') # 'externals' modules externals.check_setup('jseg/jpeg-6b') # #externals.check_setup('_mlpy') # externals.check_setup('_music21') # @@ -106,7 +108,6 @@ from colormath.color_objects import RGBColor from py_w3c.validators.html.validator import HTMLValidator, ValidationFault #from pdfminer import pdfparser, pdfinterp, pdfdevice, converter, cmapdb, layout - #externals.check_setup('_ocropus') locale.setlocale(locale.LC_ALL, '') @@ -3410,6 +3411,7 @@ # self._detect_AudioFeatures_YAAFE() # midi audio feature extraction +# TODO: improve midi/audio stuff ... self._detect_AudioFeatures_MUSIC21() def _existInformation(self, info, ignore = ['Properties', 'ColorAverage']): Modified: trunk/pywikipedia/maintainer.py =================================================================== --- trunk/pywikipedia/maintainer.py 2013-05-18 19:42:20 UTC (rev 11550) +++ trunk/pywikipedia/maintainer.py 2013-05-19 09:16:42 UTC (rev 11551) @@ -12,8 +12,6 @@ # Author: Balasyum #
http://hu.wikipedia.org/wiki/User:Balasyum
-from ircbot import SingleServerIRCBot -from irclib import nm_to_n import random import thread import threading @@ -21,6 +19,10 @@ import rciw import censure import wikipedia as pywikibot +import externals +externals.check_setup('irclib') +from ircbot import SingleServerIRCBot +from irclib import nm_to_n ver = 1 Modified: trunk/pywikipedia/maintcont.py =================================================================== --- trunk/pywikipedia/maintcont.py 2013-05-18 19:42:20 UTC (rev 11550) +++ trunk/pywikipedia/maintcont.py 2013-05-19 09:16:42 UTC (rev 11551) @@ -13,6 +13,8 @@ # Author: Balasyum #
http://hu.wikipedia.org/wiki/User:Balasyum
+import externals +externals.check_setup('irclib') from ircbot import SingleServerIRCBot from irclib import nm_to_n import threading
1
0
0
0
SVN: [11550] trunk/pywikipedia
by drtrigon@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11550
Revision: 11550 Author: drtrigon Date: 2013-05-18 19:42:20 +0000 (Sat, 18 May 2013) Log Message: ----------- improvement; irclib now checked by externals also new feature; midi file support for categorization Modified Paths: -------------- trunk/pywikipedia/articlenos.py trunk/pywikipedia/catimages.py trunk/pywikipedia/externals/__init__.py trunk/pywikipedia/subster.py Modified: trunk/pywikipedia/articlenos.py =================================================================== --- trunk/pywikipedia/articlenos.py 2013-05-18 14:36:57 UTC (rev 11549) +++ trunk/pywikipedia/articlenos.py 2013-05-18 19:42:20 UTC (rev 11550) @@ -14,11 +14,14 @@ # License : LGPL __version__ = '$Id$' +import re + +import wikipedia as pywikibot +import externals +externals.check_setup('irclib') from ircbot import SingleServerIRCBot from irclib import nm_to_n, nm_to_h, irc_lower, ip_numstr_to_quad from irclib import ip_quad_to_numstr -import wikipedia as pywikibot -import re class ArtNoDisp(SingleServerIRCBot): def __init__(self, site, channel, nickname, server, port=6667): Modified: trunk/pywikipedia/catimages.py =================================================================== --- trunk/pywikipedia/catimages.py 2013-05-18 14:36:57 UTC (rev 11549) +++ trunk/pywikipedia/catimages.py 2013-05-18 19:42:20 UTC (rev 11550) @@ -3410,7 +3410,7 @@ # self._detect_AudioFeatures_YAAFE() # midi audio feature extraction -# self._detect_AudioFeatures_MUSIC21() + self._detect_AudioFeatures_MUSIC21() def _existInformation(self, info, ignore = ['Properties', 'ColorAverage']): result = [] Modified: trunk/pywikipedia/externals/__init__.py =================================================================== --- trunk/pywikipedia/externals/__init__.py 2013-05-18 14:36:57 UTC (rev 11549) +++ trunk/pywikipedia/externals/__init__.py 2013-05-18 19:42:20 UTC (rev 11550) @@ -23,7 +23,7 @@ # supports: 0. svn:externals # 1. package management system (yum, apt-get, ...) # 2. download from url (or svn, git repo) -# 3. checkout from mercurial repo ('hg clone ...' since not url available) +# 3. checkout from mercurial repo ('hg clone ...' since url not available) # dependencies: (svn, python) # yum, apt-get or whatever your system uses # mercurial (hg) @@ -66,6 +66,10 @@ #$ diff -Nau TEST_BeautifulSoup.py BeautifulSoup.py > patch-BeautifulSoup 'patch': 'patch-BeautifulSoup',}, {}), # OK + 'irclib': ({'linux-fedora': ['python-irclib'], + 'linux-ubuntu': [''],}, + {}, #
http://python-irclib.sourceforge.net/
+ {}), # OK 'colormath': ({'linux-fedora': [], 'linux-ubuntu': ['python-colormath'],}, { 'url': '
https://github.com/gtaylor/python-colormath/archive/master.zip
', @@ -125,11 +129,11 @@ {}), # OK # TODO: vvv (include) # 'TEST_slic': ({}, +# { 'url': '
http://ivrg.epfl.ch/files/content/sites/ivrg/files/supplementary_material/R…
', +# 'path': 'SLICSuperpixelsAndSupervoxelsCode/SLICSuperpixels',}),# OPEN +# 'TEST_slic': ({}, # { 'url': '
https://github.com/amueller/slic-python/archive/master.zip
', # 'path': 'slic-python-master',}), # OPEN -# 'TEST_slic': ({}, -# { 'url': '
http://ivrg.epfl.ch/files/content/sites/ivrg/files/supplementary_material/R…
', -# 'path': 'SLICSuperpixelsAndSupervoxelsCode/SLICSuperpixels',}),# OPEN # (2 download sources to same dir, compilation) + patch (at least for '__init__.py') needed '_zbar': ({'linux-fedora': ['zbar'], 'linux-ubuntu': ['python-zbar'],}, @@ -149,7 +153,7 @@ # (complex compilation, dependent on '_bob') + patch (at least for '__init__.py') needed } -modules_order = ['crontab', 'odf', 'openpyxl', 'BeautifulSoup.py', +modules_order = ['crontab', 'odf', 'openpyxl', 'BeautifulSoup.py', 'irclib', 'colormath', 'jseg', 'jseg/jpeg-6b', '_mlpy', '_music21', '_ocropus', 'opencv/haarcascades', 'pydmtx', 'py_w3c', '_zbar',] Modified: trunk/pywikipedia/subster.py =================================================================== --- trunk/pywikipedia/subster.py 2013-05-18 14:36:57 UTC (rev 11549) +++ trunk/pywikipedia/subster.py 2013-05-18 19:42:20 UTC (rev 11550) @@ -82,7 +82,6 @@ externals.check_setup('crontab') # externals.check_setup('odf') # externals.check_setup('openpyxl') # -#externals.check_setup('BeautifulSoup.py')# import BeautifulSoup import openpyxl.reader.excel import crontab
1
0
0
0
SVN: [11549] trunk/pywikipedia
by drtrigon@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11549
Revision: 11549 Author: drtrigon Date: 2013-05-18 14:36:57 +0000 (Sat, 18 May 2013) Log Message: ----------- follow-up; adopted docs to changes in r11540, r11541, 11542, 11543 Modified Paths: -------------- trunk/pywikipedia/CONTENTS trunk/pywikipedia/catimages.py trunk/pywikipedia/externals/__init__.py Added Paths: ----------- trunk/pywikipedia/externals/README Modified: trunk/pywikipedia/CONTENTS =================================================================== --- trunk/pywikipedia/CONTENTS 2013-05-18 14:19:35 UTC (rev 11548) +++ trunk/pywikipedia/CONTENTS 2013-05-18 14:36:57 UTC (rev 11549) @@ -30,9 +30,6 @@ === Library routines === apispec.py : Library to handle special pages through API -BeautifulSoup.py : is a Python HTML/XML parser designed for quick - turnaround projects like screen-scraping. See - more:
http://www.crummy.com/software/BeautifulSoup
botlist.py : Allows access to the site's bot user list. catlib.py : Library routines written especially to handle category pages and recurse over category contents. @@ -259,6 +256,16 @@ deadlinks : Contains information retrieved by weblinkchecker.py disambiguations : If you run solve_disambiguation.py with the -primary argument, the bot will save information here +externals : Contains all external software that might be used by + by PyWikipediaBot scripts and libraries. The most + important among them are: + * simplejson; used by query.py and needed for python + release prev. 2.6 + * spelling; dictionaries for spellcheck.py + * BeautifulSoup.py;
http://www.crummy.com/software/BeautifulSoup
+ (and more) + Missing packages are installed or downloaded automa- + tically by externals/__init__.py. families : Contains wiki-specific information like URLs, languages, encodings etc. i18n : Contains i18n translations for bot edit summaries @@ -271,9 +278,6 @@ logs : Contains logfiles. maintenance : contains maintenance scripts for the developing team pywikibot : Contains some libraries and control files -simplejson : A simple, fast, extensible JSON encoder and decoder - used by query.py. Needed for python release prev. 2.6 -spelling : Contains dictionaries for spellcheck.py. test : Some test stuff for the developing team userinterfaces : Contains Tkinter, WxPython, terminal and transliteration interfaces user choose in @@ -296,6 +300,11 @@ * python-celementtree (recommended if you use XML dumps) * python-tkinter (optional, used by some experimental GUI stuff) +All other (more exotic) external software is installed automatically by the +externals module, if missing. This is done by using several methods in following +order; (svn:externals), OS package management system, download and extract from +URL, clone of mercurial repository. + More precise information, and a list of the options that are available for the various programs, can be retrieved by running the bot with the -help parameter, e.g. Modified: trunk/pywikipedia/catimages.py =================================================================== --- trunk/pywikipedia/catimages.py 2013-05-18 14:19:35 UTC (rev 11548) +++ trunk/pywikipedia/catimages.py 2013-05-18 14:36:57 UTC (rev 11549) @@ -107,6 +107,8 @@ from py_w3c.validators.html.validator import HTMLValidator, ValidationFault #from pdfminer import pdfparser, pdfinterp, pdfdevice, converter, cmapdb, layout +#externals.check_setup('_ocropus') + locale.setlocale(locale.LC_ALL, '') Added: trunk/pywikipedia/externals/README =================================================================== --- trunk/pywikipedia/externals/README (rev 0) +++ trunk/pywikipedia/externals/README 2013-05-18 14:36:57 UTC (rev 11549) @@ -0,0 +1,7 @@ +External dependencies for the trunk branch. + +Usually - under normal circumstances - these packages should be installed +separately elsewhere in the OS by the standard package managing system. This +might not be possible for more exotic ones. Therfore several methods are used +in following order; (svn:externals), OS package management system, download and +extract from URL, clone of mercurial repository. Modified: trunk/pywikipedia/externals/__init__.py =================================================================== --- trunk/pywikipedia/externals/__init__.py 2013-05-18 14:19:35 UTC (rev 11548) +++ trunk/pywikipedia/externals/__init__.py 2013-05-18 14:36:57 UTC (rev 11549) @@ -23,21 +23,36 @@ # supports: 0. svn:externals # 1. package management system (yum, apt-get, ...) # 2. download from url (or svn, git repo) -# 3. checkout from mercurial repo ('hg clone ...') +# 3. checkout from mercurial repo ('hg clone ...' since not url available) +# dependencies: (svn, python) +# yum, apt-get or whatever your system uses +# mercurial (hg) +# patch modules_needed = { +# TODO: vvv how to apply patches under windows, e.g. and other OS... ? (needed for BeautifulSoup.py and else) +# 'patch.py': ({}, +# { 'url': '
https://python-patch.googlecode.com/files/patch-1.12.11.py
', +# 'path': '',}), # OK +# { 'url': '
http://google-diff-match-patch.googlecode.com/files/diff_match_patch_201211…
', +# 'path': 'diff_match_patch_20121119/python2',}), # OK +# { 'url': '
http://downloads.sourceforge.net/project/unxutils/unxutils/current/UnxUtils…
', +# 'path': '?',}), # OK 'crontab': ({}, #{ 'url': '
https://github.com/josiahcarlson/parse-crontab/archive/master.zip
', # 'path': 'parse-crontab-master/crontab',}), # OK { 'url': '
https://github.com/josiahcarlson/parse-crontab/archive/1ec538ff67df6a207993…
', - 'path': 'parse-crontab-1ec538ff67df6a207993a6c5b6988f4f628c5776/crontab',}),# OK + 'path': 'parse-crontab-1ec538ff67df6a207993a6c5b6988f4f628c5776/crontab',}, + {}), # OK 'odf': ({}, #{ 'url': '
https://pypi.python.org/packages/source/o/odfpy/odfpy-0.9.6.tar.gz
', # 'path': 'odfpy-0.9.6/odf',}), # OK { 'url': '
https://pypi.python.org/packages/source/o/odfpy/odfpy-0.9.4.tar.gz
', - 'path': 'odfpy-0.9.4/odf',}), # OK + 'path': 'odfpy-0.9.4/odf',}, + {}), # OK 'openpyxl': ({}, { 'url': '
https://bitbucket.org/ericgazoni/openpyxl/get/1.5.6.tar.gz
', - 'path': 'ericgazoni-openpyxl-e5934500ffac/openpyxl',}),# OK + 'path': 'ericgazoni-openpyxl-e5934500ffac/openpyxl',}, + {}), # OK # 'spelling': $ svn propedit svn:externals externals/. # spelling
http://svn.wikimedia.org/svnroot/pywikipedia/trunk/spelling/
# 'simplejson': $ svn propedit svn:externals externals/. @@ -49,32 +64,38 @@ { 'url': '
https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0…
', 'path': 'BeautifulSoup-3.2.0/BeautifulSoup.py', #$ diff -Nau TEST_BeautifulSoup.py BeautifulSoup.py > patch-BeautifulSoup - 'patch': 'patch-BeautifulSoup',}), # OK -# TODO: ^^^ how to apply patches under windows, e.g. ... ? (afterwards remove the file from repo!) + 'patch': 'patch-BeautifulSoup',}, + {}), # OK 'colormath': ({'linux-fedora': [], 'linux-ubuntu': ['python-colormath'],}, { 'url': '
https://github.com/gtaylor/python-colormath/archive/master.zip
', - 'path': 'python-colormath-master/colormath',}), # OK + 'path': 'python-colormath-master/colormath',}, + {}), # OK 'jseg': ({}, { 'url': '
http://vision.ece.ucsb.edu/segmentation/jseg/software/jseg.zip
', 'path': 'jseg', #$ diff -Nau --exclude="*.o" --exclude="*.pyc" --exclude="segdist_cpp*" TEST_jseg/ jseg/ > patch-jseg - 'patch': 'patch-jseg',}), # OK + 'patch': 'patch-jseg',}, + {}), # OK 'jseg/jpeg-6b': ({}, { 'url': '
http://vision.ece.ucsb.edu/segmentation/jseg/software/jpeg-6b.zip
', - 'path': 'jpeg-6b',}), # OK + 'path': 'jpeg-6b',}, + {}), # OK '_mlpy': ({}, { 'url': '
http://downloads.sourceforge.net/project/mlpy/mlpy%203.5.0/mlpy-3.5.0.tar.gz
', - 'path': 'mlpy-3.5.0/mlpy',}), # OK + 'path': 'mlpy-3.5.0/mlpy',}, + {}), # OK '_music21': ({}, { 'url': '
http://music21.googlecode.com/files/music21-1.4.0.tar.gz
', 'path': 'music21-1.4.0', #$ diff -Naur --exclude="*.pyc" TEST__music21/ _music21/ > patch-music21 - 'patch': 'patch-music21',}), # OK -# TODO: vvv (future; enable for and use in 'catimages.py') -# mercurial: $ hg clone -r ocropus-0.6pre3
https://code.google.com/p/ocropus
-# '_ocropus': ({}, -# {}), # OPEN + 'patch': 'patch-music21',}, + {}), # OK +# TODO: vvv (future; enable for and use in 'catimages.py', patch needed) + '_ocropus': ({}, + {}, + { 'url': '
https://code.google.com/p/ocropus
', + 'rev': 'ocropus-0.6',}), # OK # TODO: vvv (further clean-up and unlink - check with 'svn list') # 'opencv': $ svn propedit svn:externals externals/. # opencv
https://svn.toolserver.org/svnroot/drtrigon/externals/opencv
@@ -87,7 +108,8 @@ # createtrainsamples.pl
https://svn.toolserver.org/svnroot/drtrigon/externals/haartraining/createtr…
'opencv/haarcascades': ({}, { 'url': '
https://svn.toolserver.org/svnroot/drtrigon/externals/haarcascades-full.tar…
', - 'path': 'haarcascades',}), # OK + 'path': 'haarcascades',}, + {}), # OK # 'pdfminer' is not used anymore/at the moment... # 'pycolorname': $ svn propset svn:externals 'pycolorname
https://svn.toolserver.org/svnroot/drtrigon/externals/pycolorname
' externals/. 'pydmtx': ({'linux-fedora': ['python-libdmtx'], @@ -95,10 +117,12 @@ { 'url': '
https://github.com/dmtx/dmtx-wrappers/archive/master.zip
', 'path': 'dmtx-wrappers-master/python', #$ diff -Nau --exclude="*.pyc" TEST_pydmtx/ pydmtx/ > patch-pydmtx - 'patch': 'patch-pydmtx',}), # OK + 'patch': 'patch-pydmtx',}, + {}), # OK 'py_w3c': ({}, { 'url': '
https://bitbucket.org/nmb10/py_w3c/downloads/py_w3c-v0.1.0.tar.gz
', - 'path': 'py_w3c-0.1.0/py_w3c',}), # OK + 'path': 'py_w3c-0.1.0/py_w3c',}, + {}), # OK # TODO: vvv (include) # 'TEST_slic': ({}, # { 'url': '
https://github.com/amueller/slic-python/archive/master.zip
', @@ -106,13 +130,14 @@ # 'TEST_slic': ({}, # { 'url': '
http://ivrg.epfl.ch/files/content/sites/ivrg/files/supplementary_material/R…
', # 'path': 'SLICSuperpixelsAndSupervoxelsCode/SLICSuperpixels',}),# OPEN -# (2 download sources to same dir) + patch (at least for '__init__.py') needed +# (2 download sources to same dir, compilation) + patch (at least for '__init__.py') needed '_zbar': ({'linux-fedora': ['zbar'], 'linux-ubuntu': ['python-zbar'],}, { 'url': '
https://pypi.python.org/packages/source/z/zbar/zbar-0.10.tar.bz2
', 'path': 'zbar-0.10', #$ diff -Nau --exclude="*.pyc" TEST__zbar/ _zbar/ > patch-zbar - 'patch': 'patch-zbar',}), # OK + 'patch': 'patch-zbar',}, + {}), # OK # TODO: vvv (include) # 'TEST__bob': ({}, # { 'url': '
https://www.idiap.ch/software/bob/packages/bob-1.1.2.zip
', @@ -124,13 +149,11 @@ # (complex compilation, dependent on '_bob') + patch (at least for '__init__.py') needed } -#modules_order = ['crontab', 'odf', 'openpyxl', -# 'colormath', 'jseg', 'jseg/jpeg-6b', '_mlpy', '_music21', -# '_ocropus', 'opencv', 'opencv/haarcascades', 'pydmtx', -# 'py_w3c', 'slic', '_zbar', '_bob', 'xbob_flandmark',] modules_order = ['crontab', 'odf', 'openpyxl', 'BeautifulSoup.py', 'colormath', 'jseg', 'jseg/jpeg-6b', '_mlpy', '_music21', - 'opencv/haarcascades', 'pydmtx', 'py_w3c', '_zbar',] + '_ocropus', 'opencv/haarcascades', 'pydmtx', 'py_w3c', + '_zbar',] +# OPEN: 'patch.py', 'opencv', 'slic', '_bob', 'xbob_flandmark', import os, sys @@ -250,6 +273,10 @@ else: return False +# TODO: work this out... +def windows_install(dependency_dictionary): + return False + ### END of VisTrails inspired and copied code ### ### ### ### ### ### ### ### @@ -292,6 +319,8 @@ lowlevel_warning(u'Install package "%s" by applying patch to %s.' % (module, os.path.join(path, module))) cmd = 'patch -p0 -d %s < %s' % (path, os.path.join(path, package['patch'])) + #os.chdir(path) + #cmd = 'python patch.py -p0 -- < %s' % os.path.join(path, package['patch']) result = os.system(cmd) lowlevel_warning(u'Package "%s" installed to %s.' @@ -301,9 +330,21 @@ return False def mercurial_repo_install(package, module, path): - pass + if package: + cmd = 'hg clone' + + lowlevel_warning(u'Mercurial clone "%s" from %s' + % (module, package['url'])) + cmd += " -r %s %s %s" % (package['rev'], package['url'], + os.path.join(path, module)) + + result = os.system(cmd) + + return (result == 0) + return False + def check_setup(m): path = os.path.dirname(os.path.abspath(os.path.join(os.curdir, __file__))) mf = os.path.join(path, m) @@ -315,9 +356,9 @@ return # install the missing module - if linux_install(modules_needed[m][0]): + dist = guess_system().split(u'-') + if globals()[dist[0] + '_install'](modules_needed[m][0]): return - # TODO: add 'windows_install()' and more ... if download_install(modules_needed[m][1], m, path): return if mercurial_repo_install(modules_needed[m][2], m, path): @@ -332,3 +373,4 @@ #check_setup_all() +#check_setup('patch.py')
1
0
0
0
SVN: [11548] trunk/pywikipedia/families/wikipedia_family.py
by xqt@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11548
Revision: 11548 Author: xqt Date: 2013-05-18 14:19:35 +0000 (Sat, 18 May 2013) Log Message: ----------- fix for r11547 Modified Paths: -------------- trunk/pywikipedia/families/wikipedia_family.py Modified: trunk/pywikipedia/families/wikipedia_family.py =================================================================== --- trunk/pywikipedia/families/wikipedia_family.py 2013-05-18 09:27:11 UTC (rev 11547) +++ trunk/pywikipedia/families/wikipedia_family.py 2013-05-18 14:19:35 UTC (rev 11548) @@ -445,7 +445,7 @@ 'gl': u'Conversa Wikipedia', 'glk': u'بحث Wikipedia', 'gn': u'Vikipetã myangekõi', - 'got': u'??? Wikipedia ?????????', + 'got': u'𐌸𐌹𐍃 Wikipedia 𐌲𐌰𐍅𐌰𐌿𐍂𐌳𐌾𐌰', 'gu': u'વિકિપીડિયા ચર્ચા', 'gv': u'Resooney Wikipedia', 'ha': u'Wikipedia talk',
1
0
0
0
SVN: [11547] trunk/pywikipedia/families/wikipedia_family.py
by xqt@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11547
Revision: 11547 Author: xqt Date: 2013-05-18 09:27:11 +0000 (Sat, 18 May 2013) Log Message: ----------- update namespace aliases Modified Paths: -------------- trunk/pywikipedia/families/wikipedia_family.py Modified: trunk/pywikipedia/families/wikipedia_family.py =================================================================== --- trunk/pywikipedia/families/wikipedia_family.py 2013-05-18 08:27:44 UTC (rev 11546) +++ trunk/pywikipedia/families/wikipedia_family.py 2013-05-18 09:27:11 UTC (rev 11547) @@ -54,6 +54,8 @@ self.namespaces[13]['de'] = [u'Hilfe Diskussion', u'HD'] self.namespaces[2]['hi'] = [u'सदस्य', u'U', u'स'] self.namespaces[3]['hi'] = [u'सदस्य वार्ता', u'UT', u'सवा'] + self.namespaces[8]['hi'] = [u'मीडियाविकि', u'मी'], + self.namespaces[9]['hi'] = [u'मीडियाविकि वार्ता', u'मीवा'], self.namespaces[10]['hi'] = [u'साँचा', u'सा', u'T'] self.namespaces[11]['hi'] = [u'साँचा वार्ता', u'सावा'] self.namespaces[14]['hi'] = [u'श्रेणी', u'श्र', u'C'] @@ -328,10 +330,10 @@ 'tum': u'Wikipedia', 'tw': u'Wikipedia', 'ty': u'Wikipedia', - 'udm': u'Wikipedia', + 'udm': u'Википедия', 'ug': u'Wikipedia', 'uk': [u'Вікіпедія', u'ВП'], - 'ur': u'منصوبہ', + 'ur': u'ویکیپیڈیا', 'uz': [u'Vikipediya', u'VP'], 've': u'Wikipedia', 'vec': [u'Wikipedia', u'WP'], @@ -443,7 +445,7 @@ 'gl': u'Conversa Wikipedia', 'glk': u'بحث Wikipedia', 'gn': u'Vikipetã myangekõi', - 'got': u'𐌸𐌹𐍃 Wikipedia 𐌲𐌰𐍅𐌰𐌿𐍂𐌳𐌾𐌰', + 'got': u'??? Wikipedia ?????????', 'gu': u'વિકિપીડિયા ચર્ચા', 'gv': u'Resooney Wikipedia', 'ha': u'Wikipedia talk', @@ -610,7 +612,7 @@ 'udm': [u'Википедия сярысь вераськон', u'Wikipedia сярысь вераськон'], 'ug': u'مۇنازىرىسىWikipedia', 'uk': u'Обговорення Вікіпедії', - 'ur': u'تبادلۂ خیال منصوبہ', + 'ur': u'تبادلۂ خیال ویکیپیڈیا', 'uz': [u'Vikipediya munozarasi', u'VM'], 've': u'Wikipedia talk', 'vec': u'Discussion Wikipedia', @@ -1003,10 +1005,12 @@ self.namespaces[446] = { 'en': u'Education Program', + 'mk': u'Education Program', } self.namespaces[447] = { 'en': u'Education Program talk', + 'mk': u'Education Program talk', } self.namespaces[710] = {
1
0
0
0
SVN: [11546] branches/rewrite/pywikibot/families
by xqt@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11546
Revision: 11546 Author: xqt Date: 2013-05-18 08:27:44 +0000 (Sat, 18 May 2013) Log Message: ----------- new wikitionary and vikivoyage site, update from trunk r11544, r11545 Modified Paths: -------------- branches/rewrite/pywikibot/families/wikivoyage_family.py branches/rewrite/pywikibot/families/wiktionary_family.py Modified: branches/rewrite/pywikibot/families/wikivoyage_family.py =================================================================== --- branches/rewrite/pywikibot/families/wikivoyage_family.py 2013-05-18 08:25:54 UTC (rev 11545) +++ branches/rewrite/pywikibot/families/wikivoyage_family.py 2013-05-18 08:27:44 UTC (rev 11546) @@ -12,7 +12,7 @@ self.name = 'wikivoyage' self.languages_by_size = [ 'en', 'de', 'pt', 'nl', 'fr', 'it', 'pl', 'ru', 'sv', 'es', 'ro', - 'uk', 'he', + 'uk', 'he', 'el', ] self.langs = dict([(lang, '%s.wikivoyage.org' % lang) Modified: branches/rewrite/pywikibot/families/wiktionary_family.py =================================================================== --- branches/rewrite/pywikibot/families/wiktionary_family.py 2013-05-18 08:25:54 UTC (rev 11545) +++ branches/rewrite/pywikibot/families/wiktionary_family.py 2013-05-18 08:27:44 UTC (rev 11546) @@ -22,9 +22,9 @@ 'mk', 'be', 'ga', 'nds', 'wo', 'ms', 'ang', 'ky', 'co', 'gn', 'mr', 'csb', 'st', 'ia', 'sd', 'si', 'sh', 'tg', 'sa', 'ug', 'mn', 'kl', 'jbo', 'an', 'ln', 'zu', 'or', 'gu', 'kw', 'rw', 'gv', 'fo', 'qu', - 'ss', 'ie', 'mt', 'om', 'chr', 'roa-rup', 'iu', 'bn', 'pa', 'so', - 'am', 'su', 'za', 'gd', 'mi', 'tpi', 'ne', 'yi', 'ti', 'sg', 'na', - 'dv', 'tn', 'ha', 'ks', 'ay', 'ts', + 'ss', 'vec', 'ie', 'mt', 'om', 'chr', 'roa-rup', 'iu', 'bn', 'pa', + 'so', 'am', 'su', 'za', 'gd', 'mi', 'tpi', 'ne', 'yi', 'ti', 'sg', + 'na', 'dv', 'tn', 'ha', 'ks', 'ay', 'ts', ] self.langs = dict([(lang, '%s.wiktionary.org' % lang)
1
0
0
0
SVN: [11545] trunk/pywikipedia/families/wiktionary_family.py
by xqt@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11545
Revision: 11545 Author: xqt Date: 2013-05-18 08:25:54 +0000 (Sat, 18 May 2013) Log Message: ----------- new wiktionary site vec Modified Paths: -------------- trunk/pywikipedia/families/wiktionary_family.py Modified: trunk/pywikipedia/families/wiktionary_family.py =================================================================== --- trunk/pywikipedia/families/wiktionary_family.py 2013-05-18 08:15:22 UTC (rev 11544) +++ trunk/pywikipedia/families/wiktionary_family.py 2013-05-18 08:25:54 UTC (rev 11545) @@ -22,9 +22,9 @@ 'mk', 'be', 'ga', 'nds', 'wo', 'ms', 'ang', 'ky', 'co', 'gn', 'mr', 'csb', 'st', 'ia', 'sd', 'si', 'sh', 'tg', 'sa', 'ug', 'mn', 'kl', 'jbo', 'an', 'ln', 'zu', 'or', 'gu', 'kw', 'rw', 'gv', 'fo', 'qu', - 'ss', 'ie', 'mt', 'om', 'chr', 'roa-rup', 'iu', 'bn', 'pa', 'so', - 'am', 'su', 'za', 'gd', 'mi', 'tpi', 'ne', 'yi', 'ti', 'sg', 'na', - 'dv', 'tn', 'ha', 'ks', 'ay', 'ts', + 'ss', 'vec', 'ie', 'mt', 'om', 'chr', 'roa-rup', 'iu', 'bn', 'pa', + 'so', 'am', 'su', 'za', 'gd', 'mi', 'tpi', 'ne', 'yi', 'ti', 'sg', + 'na', 'dv', 'tn', 'ha', 'ks', 'ay', 'ts', ] self.langs = dict([(lang, '%s.wiktionary.org' % lang) @@ -195,6 +195,7 @@ 'uk': [u'Вікісловник', u'Wiktionary', u'ВС'], 'ur': [u'وکی لغت', u'Wiktionary'], 'uz': [u'Vikilug‘at', u'Wiktionary'], + 'vec': [u'Wikisionario', u'Wiktionary'], 'vi': u'Wiktionary', 'vo': [u'Vükivödabuk', u'Wiktionary'], 'wa': u'Wiktionary', @@ -354,6 +355,7 @@ 'uk': u'Обговорення Вікісловника', 'ur': u'تبادلۂ خیال وکی لغت', 'uz': u'Vikilug‘at munozarasi', + 'vec': u'Discussion Wikisionario', 'vi': u'Thảo luận Wiktionary', 'vo': [u'Bespik dö Vükivödabuk', u'Wiktionary talk'], 'wa': u'Wiktionary copene',
1
0
0
0
SVN: [11544] trunk/pywikipedia/families/wikivoyage_family.py
by xqt@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11544
Revision: 11544 Author: xqt Date: 2013-05-18 08:15:22 +0000 (Sat, 18 May 2013) Log Message: ----------- new wikivoyage site el Modified Paths: -------------- trunk/pywikipedia/families/wikivoyage_family.py Modified: trunk/pywikipedia/families/wikivoyage_family.py =================================================================== --- trunk/pywikipedia/families/wikivoyage_family.py 2013-05-17 22:26:50 UTC (rev 11543) +++ trunk/pywikipedia/families/wikivoyage_family.py 2013-05-18 08:15:22 UTC (rev 11544) @@ -12,7 +12,7 @@ self.name = 'wikivoyage' self.languages_by_size = [ 'en', 'de', 'pt', 'nl', 'fr', 'it', 'pl', 'ru', 'sv', 'es', 'ro', - 'uk', 'he', + 'uk', 'he', 'el', ] self.langs = dict([(lang, '%s.wikivoyage.org' % lang) @@ -35,6 +35,7 @@ # You only need to enter translations that differ from _default. self.namespaces[4] = { 'de': u'Wikivoyage', + 'el': u'Βικιταξίδια', 'en': u'Wikivoyage', 'es': u'Wikiviajes', 'fr': u'Wikivoyage', @@ -51,6 +52,7 @@ self.namespaces[5] = { 'de': u'Wikivoyage Diskussion', + 'el': u'Συζήτηση Βικιταξίδια', 'en': u'Wikivoyage talk', 'es': u'Wikiviajes discusión', 'fr': u'Discussion Wikivoyage',
1
0
0
0
SVN: [11543] trunk/pywikipedia
by drtrigon@svn.wikimedia.org
18 May '13
18 May '13
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11543
Revision: 11543 Author: drtrigon Date: 2013-05-17 22:26:50 +0000 (Fri, 17 May 2013) Log Message: ----------- improvement; externals support for wikipedia.py early imports Modified Paths: -------------- trunk/pywikipedia/externals/__init__.py trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/externals/__init__.py =================================================================== --- trunk/pywikipedia/externals/__init__.py 2013-05-17 19:58:40 UTC (rev 11542) +++ trunk/pywikipedia/externals/__init__.py 2013-05-17 22:26:50 UTC (rev 11543) @@ -23,7 +23,7 @@ # supports: 0. svn:externals # 1. package management system (yum, apt-get, ...) # 2. download from url (or svn, git repo) -# (?. checkout from mercurial repo 'hg clone ...') +# 3. checkout from mercurial repo ('hg clone ...') modules_needed = { 'crontab': ({}, #{ 'url': '
https://github.com/josiahcarlson/parse-crontab/archive/master.zip
', @@ -40,17 +40,17 @@ 'path': 'ericgazoni-openpyxl-e5934500ffac/openpyxl',}),# OK # 'spelling': $ svn propedit svn:externals externals/. # spelling
http://svn.wikimedia.org/svnroot/pywikipedia/trunk/spelling/
-# TODO: vvv (adopt 'wikipedia.py' in order to be able to download 'simplejson', 'BeautifulSoup.py' before importing!!!) -## 'simplejson': $ svn propedit svn:externals externals/. -## simplejson
http://simplejson.googlecode.com/svn/tags/simplejson-2.1.3/simplejson/
-## 'simplejson': ({'linux-fedora': ['python-simplejson'], -## 'linux-ubuntu': [''],}, -# 'BeautifulSoup.py': ({'linux-fedora': ['python-BeautifulSoup'], +# 'simplejson': $ svn propedit svn:externals externals/. +# simplejson
http://simplejson.googlecode.com/svn/tags/simplejson-2.1.3/simplejson/
+# 'simplejson': ({'linux-fedora': ['python-simplejson'], # 'linux-ubuntu': [''],}, -# { 'url': '
https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0…
', -# 'path': 'BeautifulSoup-3.2.0/BeautifulSoup.py', -# #$ diff -Nau TEST_BeautifulSoup.py BeautifulSoup.py > patch-BeautifulSoup -# 'patch': 'patch-BeautifulSoup',}), # OK + 'BeautifulSoup.py': ({'linux-fedora': ['python-BeautifulSoup'], + 'linux-ubuntu': [''],}, + { 'url': '
https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0…
', + 'path': 'BeautifulSoup-3.2.0/BeautifulSoup.py', + #$ diff -Nau TEST_BeautifulSoup.py BeautifulSoup.py > patch-BeautifulSoup + 'patch': 'patch-BeautifulSoup',}), # OK +# TODO: ^^^ how to apply patches under windows, e.g. ... ? (afterwards remove the file from repo!) 'colormath': ({'linux-fedora': [], 'linux-ubuntu': ['python-colormath'],}, { 'url': '
https://github.com/gtaylor/python-colormath/archive/master.zip
', @@ -128,7 +128,7 @@ # 'colormath', 'jseg', 'jseg/jpeg-6b', '_mlpy', '_music21', # '_ocropus', 'opencv', 'opencv/haarcascades', 'pydmtx', # 'py_w3c', 'slic', '_zbar', '_bob', 'xbob_flandmark',] -modules_order = ['crontab', 'odf', 'openpyxl', #'BeautifulSoup.py', +modules_order = ['crontab', 'odf', 'openpyxl', 'BeautifulSoup.py', 'colormath', 'jseg', 'jseg/jpeg-6b', '_mlpy', '_music21', 'opencv/haarcascades', 'pydmtx', 'py_w3c', '_zbar',] @@ -136,28 +136,65 @@ import os, sys import wikipedia as pywikibot # sets externals path -from pywikibot.comms import http +#from pywikibot.comms import http +# allow imports from externals +sys.path.append(os.path.dirname(os.path.abspath(__file__))) + ### BEGIN of VisTrails inspired and copied code ### ### ### ### ### ### ### ### +def has_logger(): + #return hasattr(sys.modules['wikipedia'], 'logger') + return hasattr(pywikibot, 'logger') + +# TODO: solve properly because this is just a work-a-round, because module +# externals get imported in wikipedia.py before logger is setup properly, which +# should be changed! (meanwhile this is acceptable because code here should be +# executed once only...) +def lowlevel_warning(text): + if has_logger(): + pywikibot.warning(text) + else: + print "WARNING:", text + + def guess_system(): import platform return ("%s-%s" % (platform.system(), platform.dist()[0])).lower() def show_question(which_files): - pywikibot.output("Required package missing") - pywikibot.output("A required package is missing, but externals can" + lowlevel_warning("Required package missing: %s" % which_files) + lowlevel_warning("A required package is missing, but externals can" " automatically install it." " If you say Yes, externals will need administrator" " privileges, and you might be asked for the administrator" " password.") - pywikibot.output("Give externals permission to try to install package?" + lowlevel_warning("Give externals permission to try to install package?" " (y/N)") v = raw_input().upper() return v == 'Y' or v == 'YES' +def python_module_exists(module_name): + """python_module_exists(module_name): Boolean. +Returns if python module of given name can be safely imported.""" + + module_name = module_name.replace(u'.py', u'') + module_name = module_name[1:] if module_name[0] == u'_' else module_name + + try: + sys.modules[module_name] + return True + except KeyError: + pass + try: + __import__(module_name) + return True + except ImportError: + return False + + def linux_ubuntu_install(package_name): cmd = 'apt-get install -y' @@ -169,8 +206,8 @@ raise TypeError("Expected string or list of strings") cmd += ' ' + package - pywikibot.warning("externals wants to install package(s) '%s'" % - package_name) + lowlevel_warning("externals wants to install package(s) '%s'" % + package_name) sucmd = "sudo %s" % cmd result = os.system(sucmd) @@ -188,8 +225,8 @@ raise TypeError("Expected string or list of strings") cmd += ' ' + package - pywikibot.warning("externals wants to install package(s) '%s'" % - package_name) + lowlevel_warning("externals wants to install package(s) '%s'" % + package_name) sucmd = "su -c'%s'" % cmd result = os.system(sucmd) @@ -218,20 +255,24 @@ def download_install(package, module, path): if package: - pywikibot.warning(u'Download package "%s" from %s' - % (module, package['url'])) - import mimetypes#, urllib2 - #response = urllib2.urlopen(package['url']) - response = http.request(pywikibot.getSite(), package['url'], - no_hostname = True, back_response = True)[0] - pywikibot.warning(u'Size of download: %s byte(s)' - % response.headers['Content-Length']) + lowlevel_warning(u'Download package "%s" from %s' + % (module, package['url'])) + import mimetypes, urllib2 + for i in range(3): + response = urllib2.urlopen(package['url']) + #response = http.request(pywikibot.getSite(), package['url'], + # no_hostname = True, back_response = True)[0] + if 'Content-Length' in response.headers: + break + lowlevel_warning(u'Could not retrieve data, re-trying ...') + lowlevel_warning(u'Size of download: %s byte(s)' + % response.headers['Content-Length']) #mime = response.headers['Content-Type'].lower().split('/') mime = mimetypes.guess_type(package['url'], strict=True)[0].lower().split('/') - pywikibot.warning(u'MIME type: %s' % mime) + lowlevel_warning(u'MIME type: %s' % mime) - pywikibot.warning(u'Extract package "%s" to %s' - % (module, os.path.join(path, module))) + lowlevel_warning(u'Extract package "%s" to %s' + % (module, os.path.join(path, module))) if len(mime) > 1: if mime[1] == 'zip': import zipfile, StringIO @@ -248,33 +289,42 @@ result = 0 if 'patch' in package: - pywikibot.warning(u'Install package "%s" by applying patch to %s.' - % (module, os.path.join(path, module))) + lowlevel_warning(u'Install package "%s" by applying patch to %s.' + % (module, os.path.join(path, module))) cmd = 'patch -p0 -d %s < %s' % (path, os.path.join(path, package['patch'])) result = os.system(cmd) - pywikibot.warning(u'Package "%s" installed to %s.' - % (module, os.path.join(path, module))) + lowlevel_warning(u'Package "%s" installed to %s.' + % (module, os.path.join(path, module))) return (result == 0) return False +def mercurial_repo_install(package, module, path): + pass + def check_setup(m): path = os.path.dirname(os.path.abspath(os.path.join(os.curdir, __file__))) mf = os.path.join(path, m) - #__import__(mf) - if not os.path.exists(mf): - # install the missing module - if linux_install(modules_needed[m][0]): - return - if download_install(modules_needed[m][1], m, path): - return -# if svn_repo_install(modules_needed[m][2]): -# return - pywikibot.error(u'Package "%s" could not be found nor installed!' % m) + # search missing module + if python_module_exists(m): + return + if os.path.exists(mf): + return + # install the missing module + if linux_install(modules_needed[m][0]): + return + # TODO: add 'windows_install()' and more ... + if download_install(modules_needed[m][1], m, path): + return + if mercurial_repo_install(modules_needed[m][2], m, path): + return + + pywikibot.error(u'Package "%s" could not be found nor installed!' % m) + def check_setup_all(): #for m in modules_needed: for m in modules_order: Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2013-05-17 19:58:40 UTC (rev 11542) +++ trunk/pywikipedia/wikipedia.py 2013-05-17 22:26:50 UTC (rev 11543) @@ -133,8 +133,6 @@ __version__ = '$Id$' import os, sys -sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), - 'externals')) # allow imports from externals import httplib, socket, urllib, urllib2, cookielib import traceback, pprint import time, threading, Queue @@ -148,6 +146,8 @@ import warnings import unicodedata import xmlreader +import externals # allow imports from externals +externals.check_setup('BeautifulSoup.py') from BeautifulSoup import BeautifulSoup, BeautifulStoneSoup, SoupStrainer import weakref import logging, logging.handlers @@ -159,6 +159,7 @@ #
http://pypi.python.org/pypi/python-json
raise ImportError except ImportError: + externals.check_setup('simplejson') import simplejson as json # Splitting the bot into library parts from pywikibot.support import *
1
0
0
0
← Newer
1
...
21
22
23
24
25
26
27
...
517
Older →
Jump to page:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
Results per page:
10
25
50
100
200