jayvdb created this task. jayvdb added subscribers: jayvdb, Ricordisamoa, XZise, pywikipedia-bugs. jayvdb added a project: pywikibot-core. Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION mwpfh and parsoid API could be used instead of mwlib to parse the patrol whitelist.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
XZise claimed this task.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
jayvdb added a comment.
An alternative is to use JSON to store the whitelist (https://phabricator.wikimedia.org/T95143), which would be an extension of Load the settings from wiki https://gerrit.wikimedia.org/r/#/c/201446.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, jayvdb Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
XZise added a comment.
Well yesterday I've worked on porting it to `mwparserfromhell` after our Python 3 build failed because of `mwlib`. It's untested but should be pretty usable. But maybe the settings thing is better especially as it will allow patrolling user pages (currently the script would assume that it's a “key”).
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
Ricordisamoa added a comment.
For some time I've been thinking of taking advantage of the Parsoid API for such tasks. That said, mwparserfromhell is still more stable, simpler and does not require server-side code.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Ricordisamoa Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
Ricordisamoa added a project: Pywikibot-Other-scripts. Ricordisamoa set Security to None.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Ricordisamoa Cc: pywikipedia-bugs, XZise, Ricordisamoa, jayvdb, Aklapper
gerritbot added a subscriber: gerritbot. gerritbot added a comment.
Change 202011 had a related patch set uploaded (by XZise): [FIX] patrol: Use mwparserfromhell
https://gerrit.wikimedia.org/r/202011
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, gerritbot Cc: gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
gerritbot added a project: Patch-For-Review.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, gerritbot Cc: gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Krenair added a subscriber: Krenair. Krenair added a comment.
Might also want to consider use of restbase when dealing with wikimedia wikis at least
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Krenair Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
XZise added a comment.
1. How would you use RESTBase https://www.mediawiki.org/wiki/RESTBase (I guess that is what you are talking about?) 2. And would other wikis also need to use RESTBase so that they can use patrol.py?
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Krenair added a comment.
1. The same way you would use Parsoid, but with slightly different URLs. 2. As with a direct Parsoid request, it'd rely on extra server-side code. You'd still need to fall back to something else for wikis which don't run it.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Krenair Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
XZise added a comment.
I think there is a misunderstanding. As I understand it Parsoid is used to convert wikitext into HTML and back. But pywikibot works almost entirely on wikitext so there is no need for Parsoid (or RESTBase) to convert it into HTML. At the moment it is just reading the wikitext (and mwparserfromhell/mwlib are parsing that wikitext) and @jayvdb suggested that someone actually writes JSON content into the page (which would act like wikitext and can be parsed by Python without external libraries). Using HTML would have similar problems like mwpfh/mwlib as we'd need to parse the HTML which could in fact be more problematic.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Ricordisamoa added a comment.
In https://phabricator.wikimedia.org/T95142#1183261, @Krenair wrote:
- The same way you would use Parsoid, but with slightly different URLs.
- As with a direct Parsoid request, it'd rely on extra server-side code. You'd still need to fall back to something else for wikis which don't run it.
Let's not introduce inconsistencies without benefits. We shall use mwparserfromhell.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Ricordisamoa Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Krenair added a comment.
OK. Just making sure that was considered, assuming someone was still thinking about Parsoid :)
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Krenair Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
XZise added a comment.
OH! Did forgot that @Ricordisamoa suggested that hmm. Okay then your comment makes sense.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Ricordisamoa added a comment.
In https://phabricator.wikimedia.org/T95142#1183394, @XZise wrote:
OH! Did forgot that @Ricordisamoa suggested that hmm. Okay then your comment makes sense.
@jayvdb suggested that in the first place.
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Ricordisamoa Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
Ricordisamoa added a comment.
In https://phabricator.wikimedia.org/T95142#1183362, @Krenair wrote:
OK. Just making sure that was considered, assuming someone was still thinking about Parsoid :)
Thanks. I'll consider using it in the future, should the need arise :)
TASK DETAIL https://phabricator.wikimedia.org/T95142
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, Ricordisamoa Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
jayvdb added a comment.
Another reason to remove the dependency on mwlib: it has a nasty list of dependencies, which is both long and has some very restrictive version requirements:
"pyparsing>=1.4.11,<1.6", "timelib>=0.2", "bottle>=0.10", "pyPdf>=1.12", "apipkg>=1.2", "qserve>=0.2.7", "lxml", "py>=1.4", "sqlite3dbm", "simplejson>=2.3", "roman", "gevent", "odfpy>=0.9, <0.10", "Pillow", "setuptools"
https://github.com/pediapress/mwlib/blob/master/setup.py#L44
The worst aspect is that it requires "pyparsing>=1.4.11,<1.6", which prevents using the most recent version of pyparsing, https://pypi.python.org/pypi/pyparsing/2.0.3
The requirement for "odfpy>=0.9, <0.10" is not as bad, as the latest version is https://pypi.python.org/pypi/odfpy/0.9.6
TASK DETAIL https://phabricator.wikimedia.org/T95142
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise, jayvdb Cc: Krenair, gerritbot, XZise, Ricordisamoa, jayvdb, Aklapper, pywikipedia-bugs
pywikipedia-bugs@lists.wikimedia.org