https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
Web browser: --- Bug ID: 55100 Summary: Use API module 'parse' for retrieving interwiki links Product: Pywikibot Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: Unprioritized Component: interwiki.py Assignee: Pywikipedia-bugs@lists.wikimedia.org Reporter: legoktm.wikipedia@gmail.com Classification: Unclassified Mobile Platform: ---
Originally from: http://sourceforge.net/p/pywikipediabot/feature-requests/151/ Reported by: melancholie Created on: 2008-06-13 14:47:11 Subject: Use API module 'parse' for retrieving interwiki links Original description: Currently pages are retrieved in a batch by using Special:Export. Although being fast (as only one request is done), there is a huge data overhead with this method!
Why not use the API with its 'parse' module? Only interwiki links can be fetched with that, reducing traffic (overhead) a lot!
See: http://de.wikipedia.org/w/api.php?action=parse&format=xml&page=Test&...
Outputs could be downloaded in parallel to virtualize a batch (faster).
---- At least make this method optional (config.py) for being able of reducing data traffic, if wanted. API is just more efficient.
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #1 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- - **priority**: 5 --> 7
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #2 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=2089773 Originator: YES
Note: Maybe combine it with 'generator'.
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #3 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- - **summary**: Use API module parse for retrieving interwiki links --> Use API module 'parse' for retrieving interwiki links
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #4 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=2089773 Originator: YES
Important note for getting pages' interwikis in a batch: http://de.wikipedia.org/w/api.php?action=parse&text=%5C%7B%5C%7B:Test%5C...
Either the bot could figure out what interwikis belong together then, or
maybe a marker could placed in between: http://de.wikipedia.org/w/api.php?action=parse&text=%5C%7B%5C%7B:Test%5C...
[[MediaWiki:Iwmarker]] (or 'Llmarker'?) would have to be set up by the MediaWiki developers with [[en:/de:Abuse-save-mark]] as content (but this is potentially misusable).
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #5 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=2089773 Originator: YES
For not being misusable of confusing bots, the yet to be set up MediaWiki message could contain [[foreigncode:{{CURRENTTIMESTAMP}}]] (cache issue?)
(sorry for spamming with this request ;-)
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #6 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=1806226 Originator: NO
Backwards compatibility with non Wikimedia wikis?
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #7 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=2089773 Originator: YES
Backwards compatibility?
That's no reason for not making software more efficient, where possible ;-) That's also why I wrote something about "optional", too. Because for current MediaWiki wikis there is a much more efficient way of retrieving (only) certain contents (langlinks, categories), there should be a method of using that advantage! Will reduce load (bot owner's and server's)...
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #8 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- Logged In: YES user_id=2089773 Originator: YES
See http://meta.wikimedia.org/wiki/Interwiki%5C_bot%5C_access%5C_protocol concerning disambiguations and redirects:
http://de.wikipedia.org/w/api.php?action=parse&format=xml&text=%5C%7...
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #9 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- We are working on a rewrite. The rewrite uses the api as much as possible.
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #10 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- parse mode is deactivated due to overloading the squids. Nothing to do now.
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
--- Comment #11 from Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com --- - **priority**: 7 --> 1
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
Kunal Mehta (Legoktm) legoktm.wikipedia@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://sourceforge.net/p/p | |ywikipediabot/feature-reque | |sts/151
https://bugzilla.wikimedia.org/show_bug.cgi?id=55100
Amir Ladsgroup ladsgroup@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Priority|Unprioritized |Low Status|NEW |RESOLVED CC| |ladsgroup@gmail.com Resolution|--- |FIXED
--- Comment #12 from Amir Ladsgroup ladsgroup@gmail.com --- Fixed in http://www.mediawiki.org/wiki/Special:Code/pywikipedia/11229
pywikipedia-bugs@lists.wikimedia.org