jayvdb created this task.
jayvdb added a subscriber: jayvdb.
jayvdb added a project: Pywikibot-tests.
Restricted Application added subscribers: Aklapper, pywikipedia-bugs.
TASK DESCRIPTION
https://travis-ci.org/wikimedia/pywikibot-core/jobs/60605285
```
LANGUAGE=en
FAMILY=wikipedia
EXTERNALS_HTTPLIB2=1
Python 3.3.5
...
testLoadRevisions_querycontinue (tests.site_tests.TestSiteLoadRevisions)
Test the site.loadrevisions() method with query-continue. ...
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated
```
TASK DETAIL
https://phabricator.wikimedia.org/T97604
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: pywikipedia-bugs, jayvdb, Aklapper
XZise created this task.
XZise added a subscriber: XZise.
XZise added a project: pywikibot-core.
Restricted Application added subscribers: Aklapper, pywikipedia-bugs.
TASK DESCRIPTION
As the overall trend should be towards using AutoFamily I want to list everything which looks unnecessary as it can be fetched via the API here.
The following could be replaced by API calls:
* `namespacesWithSubpage`: This should be already possible via the `Namespace` class, as it's in [[https://www.mediawiki.org/w/api.php?action=query&meta=siteinfo&siprop=names…]] returned as `subpages=""`. Maybe the Namespace class should get properties like `has_subpages` which make it easier to use.
* `linktrails` and `linktrail()`: At least in newer wikis it is reported via the API [[https://www.mediawiki.org/w/api.php?action=query&meta=siteinfo&siprop=gener…]] although I'm not sure how much a “MediaWiki:Linktrail” does change/overwrite it. Main problem there is to parse it into a Python regex (see also [[https://gerrit.wikimedia.org/r/184216/|Gerrit 184216]])
* `known_families` and `get_known_families()`: Could be replaced by using the interwiki map. There is only one usage in the library which could be easily replaced.
* `nocapitalize`: This is namespace specific and already represented in the `Namespace` class (see `Link.parse`). The primary use of it, is when creating a APISite instance that the username is not capitalized. But according to [[https://www.mediawiki.org/wiki/Manual:$wgCapitalLinkOverrides|Manual:$wgCap…]] the User namespace is never affected by that (and thus always False).
* `interwiki_forward` and `interwiki_forwarded_from`: This is can be done via the API to determine to which project `en` for
example redirects (on commons for example to the Wikipedia).
* `obsolete`: This is an odd beast with an ambiguous definition. There is a patch to make it obsolete [[https://gerrit.wikimedia.org/r/187358/|Gerrit 187358]].
* `languages_by_size`: There is a patch, but that only works for some families efficiently. There is also a patch to do that manually which would work on any but is relatively slow as it needs to contact every code.
* `protocol()`: The AutoFamily automatically defines it. Maybe there should be a simpler approach which just reads a `use_https` boolean attribute. So whenever someone needs a normal Family class they can use `use_https = True`. Alternatively `generate_family_file.py` should add that always (and then with the correct defined protocol from the URL) so the user easily sees what needs to be done.
* `ignore_certificate_error()`: Should be similar when a normal Family class is used (boolean attribute and `generate_family_file` does add it correctly set)
* `scriptpath()`: Is in the siteinfo (like the linktrail) but obviously to get to the API that needs to be defined. AutoFamily (with the complete URL) already supply it.
* `versionnumber()` and `version()`: These is already deprecated, and if it needs to be configured, `force_version()` should be used.
* `shared_image_repository()`: There is a patch ([[https://gerrit.wikimedia.org/r/181416/|Gerrit 181416]]) to make it more dynamic, but unfortunately it doesn't work always, so there is still some dynamic configuration needed.
* `shared_data_repository()`: There is already a bug report here (TODO: get ID) and depends on how multiple repositories are represented in the future.
* `server_time()`: Already deprecated with a site method.
There also some configuration variables. These should be moved into config2.py with a “global default” a possibility to overwrite it for each family with a specific setting. One problem could be when they need to be dynamic and executable code.
* `interwiki_attop`
* `interwiki_on_one_line`
* `interwiki_text_separator`
* `category_attop`
* `category_on_one_line`
* `category_text_separator`
* `categories_last`
* `interwiki_putfirst`
* `interwiki_putfirst_doubled`
* `ssl_pathprefix()`: Although it depends on how the siteinfo then changes, it could be retrieved from there (same problem as `scriptpath()`).
* `nicepath()`
* `rcstream_host()`
* `_get_path_regex(self)`: That needs to change especially if a site is accessible via multiple hostnames or it should be never defined.
* `maximum_GET_length()`
* `force_version()`
* `code2encoding()` and `encoding()`: It depends what encoding is meant. The communication with the server on HTTP level? If so shouldn't the server answer accordingly if there is no valid encoding. It could then use that encoding. If it is really required (and not UTF-8) we could still implement it via a configuration variable.
* `post_get_convert()` and `pre_put_convert()`: This should be probably rewritten into a list of converters and then via a configuration some converters could enabled.
Some of the methods are static and don't need to be changed/overwritten and thus don't need to be removed:
* `language_groups`: Although this could be probably statically defined and doesn't change with other families
* `hostname()` and `ssl_hostname()`: Those are set correctly in AutoFamily and the question is, if they need to be overridden in normal Family instances.
* `path()`, `querypath()`, `apipath()`, `nice_get_address()`: Those probably never change and are always relative to `scriptpath()`/`nicepath()`
* `from_url()`
I'm not sure about these however:
* `category_redirect_templates`, `category_redirects()`, `get_cr_templates()`
* `use_hard_category_redirects`
* `disambiguation_templates`, `disambig()`
* `cross_projects`
* `cross_projects_cookies`
* `cross_projects_cookie_username`
* `cross_allowed`
* `disambcatname`
* `ldapDomain`
* `crossnamespace`
* `iw_keys()`: This basically list all codes and the codes of from `interwiki_forward`. Could be probably replaced by a better interwiki map implementation which allows to get the complete mapping (instead of the current way to get only one definition).
* `_addlang()`
* `dbName()`
* `code2encodings()` and `encodings()`: Those two are somewhat strange, because they return by default the same value as the singular variants (not even wrapping them in a list). But even then why does it need to define multiple encodings?
* `isPublic()`
TASK DETAIL
https://phabricator.wikimedia.org/T89451
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise
Cc: pywikipedia-bugs, Aklapper, XZise, jayvdb
jayvdb created this task.
jayvdb claimed this task.
jayvdb added a subscriber: jayvdb.
jayvdb added a project: Pywikibot-tests.
TASK DESCRIPTION
aspects.py provides similar functionality to the package testscenarios. https://pypi.python.org/pypi/testscenarios
Features of aspects that are missing should be added to testscenarios, so that we could adopt it by essentially renaming TestCase.sites to TestCase,scenarios , plus any parts of aspects that may not be able to be easily generalised.
TASK DETAIL
https://phabricator.wikimedia.org/T85899
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: Aklapper, jayvdb, pywikipedia-bugs
jayvdb added a comment.
We can know what each Debian version contains, and I think we should at least determine this for all the major distributions of MediaWiki so that they are properly supported.
wheezy : http://anonscm.debian.org/viewvc/pkg-mediawiki/mediawiki/branches/wheezy/de…
trunk/jessie: http://anonscm.debian.org/viewvc/pkg-mediawiki/mediawiki/trunk/debian/patch…
I dont see any patch which would cause the API to work differently from the released version for pywikibot. A few might make the server work correctly, where the released version doesnt work, however they are not cases where pywikibot does not currently try to workaround a bug in the server.
trunk has two extra patches
allow_pdf.patch
fix_variable_use.patch
Debian at least identifies itself. Fedora also packages MediaWiki, and it doesnt appear that it identifies itself, so we cant know whether its API has backports which cause it to operate differently.
However we should fallback on a sensible value, with a UserWarning to indicate that it isnt a fully supported or understood version. i.e. anything starting with 1.19.20 should be treated as 1.19.20 .
TASK DETAIL
https://phabricator.wikimedia.org/T96813
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: XZise, jayvdb, Aklapper, pywikipedia-bugs
jayvdb created this task.
jayvdb added a subscriber: jayvdb.
jayvdb added a project: pywikibot-core.
Restricted Application added subscribers: Aklapper, pywikipedia-bugs.
TASK DESCRIPTION
https://github.com/GreenSteam/pep257/pull/107 has been merged, but is not yet released on https://pypi.python.org/pypi/pep257
When it is released, and if we can force all our tools to use the new released version...
1. we can remove non-informative docstrings from our .setters ,
2. we should update the property getter docstring to be less 'get' specific.
tools.LazyRegex.flags and tools.LazyRegex.raw are clear examples of a docstring on .setter which is unnecessary
Page.text is a case where we shouldnt change anything, as the have very important information which is different.
I think Bot.current_page and Bot.site could be improved by merging the docstrings.
api's Request.mime setter docstring could be merged into the getter docstring, but I'm not 100% sure about that one.
threadedhttp 's .data look like it shouldnt be changed.
TASK DETAIL
https://phabricator.wikimedia.org/T97382
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: pywikipedia-bugs, jayvdb, Aklapper
Ladsgroup lowered the priority of this task from "Unbreak Now!" to "Low".
Ladsgroup set Security to None.
TASK DETAIL
https://phabricator.wikimedia.org/T70613
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup
Cc: Ladsgroup, gerritbot, jayvdb, Ricordisamoa, Mpaa, pywikipedia-bugs
Ladsgroup added a subscriber: Ladsgroup.
Ladsgroup added a comment.
Since change Ic55cd1a3 is merged, it won't break any test but it still is a valid bug so I refrain from closing this bug but I reduce priority.
TASK DETAIL
https://phabricator.wikimedia.org/T70613
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup
Cc: Ladsgroup, gerritbot, jayvdb, Ricordisamoa, Mpaa, pywikipedia-bugs
Ladsgroup added a subscriber: Ladsgroup.
Ladsgroup added a comment.
It seems it's fixed in the current version in core
TASK DETAIL
https://phabricator.wikimedia.org/T57309
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup
Cc: Ladsgroup, Legoktm, jayvdb, pywikipedia-bugs
XZise added a comment.
Okay in https://phabricator.wikimedia.org/rPWBCd6b6e4b37a33f4aafb418a5d44769e1cf53a… I added that the complete version number must match a certain scheme. You commented there:
> There are also 'official' distro releases with their own generator suffix, which we 'should' support, or at least track which cases currently fail, and have TODO's for any important cases which are not handled correctly.
>
> https://wikiapiary.com/wiki/Generator:Generators/DEBIANhttps://wikiapiary.com/wiki/Generator:Generators/CUSTOM
So it seems this is on of those cases. If there is a sensible scheme behind this we could add it but what if it's 1.19.20 with backported features from newer versions? There is no sensible match then (except for the lowest common denominator). Also the docstring of `force_version` specifically says that it can be used for that:
The site is usually using the version number from the servers'
siteinfo, but if there is a problem with that it's possible to return
a non-empty string here representing another version number.
(I just noticed it should be server's)
So for now I can only suggest to add the capability to return a callable which converts a incompatible version number in a compatible one (so that it's at least somewhat flexible).
TASK DETAIL
https://phabricator.wikimedia.org/T96813
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise
Cc: XZise, jayvdb, Aklapper, pywikipedia-bugs
XZise added a subscriber: XZise.
XZise added a comment.
As a workaround `force_version` could be used. Otherwise please … how do we want to handle that version? Is that 1.19.20 (so 1.19.21 is newer?) and discard the rest? I know that I've worked on changing this and afaik added that it's not allowing such weird versions but mostly because there is no sensible way in interpreting such a version number.
TASK DETAIL
https://phabricator.wikimedia.org/T96813
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise
Cc: XZise, jayvdb, Aklapper, pywikipedia-bugs
jayvdb created this task.
jayvdb added subscribers: Evanontario, jayvdb, pywikipedia-bugs, Jsalsman, Halfak.
jayvdb added a project: pywikibot-core.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
wikiwho currently depends on https://bitbucket.org/halfak/wikimedia-utilities , which is great at xml dump processing , with limited API support.
it would be useful to integrate wikiwho with pywikibot to work on live revisions from the wiki.
TASK DETAIL
https://phabricator.wikimedia.org/T89763
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: Halfak, Jsalsman, jayvdb, Aklapper, Evanontario, pywikipedia-bugs
jayvdb created this task.
jayvdb claimed this task.
jayvdb added subscribers: pywikipedia-bugs, valhallasw, siebrand, Nemo_bis, jayvdb.
jayvdb added projects: Pywikibot-i18n, i18n.
TASK DESCRIPTION
JSON files have been added to pywikibot/i18n, which now has python and JSON files with the same messages. The JSON files are not used yet, as the code changes to enable JSON have exposed packaging problems that are the subject of RFC https://www.mediawiki.org/wiki/Requests_for_comment/pywikibot_2.0_packaging
We need syntax validation of these JSON files for gerrit submissions, as message changes typically need to be approved quickly (and without errors) so they can be merged and the core & compat i18n submodule updated, before the messages can be used in code changes.
TASK DETAIL
https://phabricator.wikimedia.org/T85335
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: Aklapper, valhallasw, siebrand, Nemo_bis, jayvdb, Gryllida, Shizhao, Arrbee, pywikipedia-bugs
jayvdb created this task.
jayvdb claimed this task.
jayvdb added subscribers: jayvdb, Legoktm, hashar.
jayvdb added projects: Continuous-Integration, pywikibot-core.
Restricted Application added subscribers: Aklapper, pywikipedia-bugs.
TASK DESCRIPTION
As jenkins will no longer automatically run rules in tox.ini, we need to hardwire pep8 and pep257 into jenkins, like the mediawiki lint tests.
pep8 runs without arguments, using the config in tox.ini for the exclude and ignore list.
pep257 doesn't have an exclude parameter (yet), but looks like they are interested in that feature:
https://github.com/GreenSteam/pep257/pull/22#issuecomment-70471875
While waiting for feedback on adding exclude to pep257, it would be nice if we didnt need an exclude list .. ;-)
date.py is being ignored by flake8, but is processed by pep257, so I've tackled that.
https://gerrit.wikimedia.org/r/#/c/185815/
pep257 is validating ez_setup.py , which I've tried to fix at the source of the problem:
https://bitbucket.org/pypa/setuptools/pull-request/117/pep8-and-pep257-comp…
pep257 doesnt ignore items with '# noqa' , so it is emitting errors for interwiki.py (easy) and pywikibot/exceptions.py (messy).
Also if the job is hardwired into jenkins, and we can only maintain one match list in tox.ini , we cant reimplement the 'mandatory docstring' list without some serious cleanup first.
TASK DETAIL
https://phabricator.wikimedia.org/T87169
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb
Cc: hashar, Legoktm, jayvdb, Aklapper, greg, pywikipedia-bugs