I work on a script that will mass copy pages from one wiki to another. The main purpose is to fetch my favourite templates from huwiki to my own local wiki.
I have found nothing like that in the framework, is it new or I just have not noticed it? In the first case I will publish it.
There are 3 scripts on botwiki written by filnik. Please have a look:
http://botwiki.sno.cc/wiki/Python:Import.py http://botwiki.sno.cc/wiki/Python:Import-en.py http://botwiki.sno.cc/wiki/Python:Import-it.py
I don't know whether you get what you need.
Greetings xqt
----- Original Nachricht ---- Von: Bináris wikiposta@gmail.com An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 09.10.2011 19:21 Betreff: [Pywikipedia-l] Script for mass copy
I work on a script that will mass copy pages from one wiki to another. The main purpose is to fetch my favourite templates from huwiki to my own local wiki.
I have found nothing like that in the framework, is it new or I just have not noticed it? In the first case I will publish it.
-- Bináris
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Thanks! Oh, why are they not in Pywikipedia? However, import.py has one title wired in the code, but will import with page history; my script mass copies pages based on a file, but with no history. Is it useful for Pywiki? The real stuff should be the two together...
2011/10/9 info@gno.de
There are 3 scripts on botwiki written by filnik. Please have a look:
http://botwiki.sno.cc/wiki/Python:Import.py http://botwiki.sno.cc/wiki/Python:Import-en.py http://botwiki.sno.cc/wiki/Python:Import-it.py
I don't know whether you get what you need.
Greetings xqt
----- Original Nachricht ---- Von: Bináris wikiposta@gmail.com An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 09.10.2011 19:21 Betreff: [Pywikipedia-l] Script for mass copy
I work on a script that will mass copy pages from one wiki to another. The main purpose is to fetch my favourite templates from huwiki to my own local wiki.
I have found nothing like that in the framework, is it new or I just have not noticed it? In the first case I will publish it.
-- Bináris
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
2011/10/9 Bináris wikiposta@gmail.com
Thanks! Oh, why are they not in Pywikipedia? However, import.py has one title wired in the code, but will import with page history; my script mass copies pages based on a file, but with no history. Is it useful for Pywiki? The real stuff should be the two together...
I realized how to use them together.
2011/10/9 Merlijn van Deen valhallasw@arctus.nl
Why not use Special:Export and Special:Import?
Hmmm, I don't know... I will find out some good reason. :-)
Thanks for the useful answers!
Hi Merlijn,
your new irc service is really great. Thanks a lot.
I found a bug while reporting code changes at svn. It writes a line like <CIA-16> xqt * http://toolserver.org/~pywikipedia/r9654 /trunk/pywikipedia/families/ (4 files): update namespace aliases lists
but the underlying link end with http://toolserver.org/~pywikipedia/r
and die release number 9654 is highlited in bold.
Greetings xqt
Hi xqt,
On 24 October 2011 06:56, info@gno.de wrote:
I found a bug while reporting code changes at svn. It writes a line like <CIA-16> xqt * http://toolserver.org/~pywikipedia/r9654 /trunk/pywikipedia/families/ (4 files): update namespace aliases lists
but the underlying link end with http://toolserver.org/~pywikipedia/r
and die release number 9654 is highlited in bold.
Thanks! Hm, interesting. There is no such thing as a well-defined url in IRC, so your client is using the bold boundary to end the link (mine doesn't parse links at all, so I can just copy-paste). I'll see if I can think of something practical to work with (as I think having the bold there is a nice thing). The hard thing is debugging it, as every client has a different way of parsing links... ah well.
If you want to play around, you can edit the bot's code yourself at cia.vc( http://cia.vc/account/bots/3251/ )
I still have to play around a bit with the urls (e.g. http://r9654.pywikipedia.org might be nice).
Best, Merlijn
Hi I am still quite new to pywikipediabot framework, so excuse me if my issue is trivial. I am writing some code for maintenance of pages in Creator namespace on Commons and I run into a weird problem with Page.templatesWithParams function. The function is apparently not able to handle template parameters, like "{{{1|}}}". For example I run the following code:
#!/usr/bin/python # -*- coding: utf-8 -*-
import sys, string sys.path.append("C:/Programs/pywikipedia/") import wikipedia
if __name__ == "__main__": site = wikipedia.getSite(u'commons', u'commons') page = wikipedia.Page(site, u'Creator:Ansel Adams') wikipedia.output(u'Original page text:\n'+page.get()+u'\n\nTemplates detected by templatesWithParams:') for template in page.templatesWithParams(): wikipedia.output(u'template[0]="%s"' % template[0])
thistxt = string.replace(page.get(), '{{{1|}}}', '') wikipedia.output(u'\nModified page text:\n'+thistxt+u'\n\nTemplates detected by templatesWithParams:') for template in page.templatesWithParams(thistxt): wikipedia.output(u'template[0]="%s"' % template[0])
and got the following output: Original page text: {{Creator | Image = Ansel_Adams_and_camera.jpg | ImgWidth = 120 | Name = Ansel Adams | Alternative names = Ansel Easton Adams | Sortkey = Adams, Ansel | Nationality = US | Gender = male | Occupation = photographer | Description = | Birthdate = 1902-02-20 | Birthloc = {{San Francisco}} | Deathdate = 1984-04-22 | Deathloc = Monterey | Workperiod = | Workloc = American West | Homecat = Ansel Adams | Option = {{{1|}}} | Authority = {{Authority control|PND=11881303X|LCCN=n/79/56359|VIAF=61625857|ARC=1332556|bare=1}} }}
Templates detected by templatesWithParams: template[0]="San Francisco" template[0]="1" template[0]="Authority control"
Modified page text: {{Creator | Image = Ansel_Adams_and_camera.jpg | ImgWidth = 120 | Name = Ansel Adams | Alternative names = Ansel Easton Adams | Sortkey = Adams, Ansel | Nationality = US | Gender = male | Occupation = photographer | Description = | Birthdate = 1902-02-20 | Birthloc = {{San Francisco}} | Deathdate = 1984-04-22 | Deathloc = Monterey | Workperiod = | Workloc = American West | Homecat = Ansel Adams | Option = | Authority = {{Authority control|PND=11881303X|LCCN=n/79/56359|VIAF=61625857|ARC=1332556|bare=1}} }}
Templates detected by templatesWithParams: template[0]="San Francisco" template[0]="Authority control" template[0]="Creator"
As one can see in the first case the function was not able to find "Creator" template, and was only able to do so after deletion of "{{{1|}}}".
Is this a feature or a bug? Or in other words, am I misusing the function somehow and there is some way to get it to work correctly, or is it a bug that should be reported somewhere, so it can be fixed?
Jarek T. (user:jarekt)
Its better to make a bug request at sourceForge [1]. I guess it is not easy to fix so its better to use the bug tracker.
Greetings xqt
[1] https://sourceforge.net/tracker/?group_id=93107&atid=603138
----- Original Nachricht ---- Von: "Tuszynski, Jaroslaw W." JAROSLAW.W.TUSZYNSKI@saic.com An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 24.10.2011 18:37 Betreff: [Pywikipedia-l] Problems with Page.templatesWithParams function
Hi I am still quite new to pywikipediabot framework, so excuse me if my issue is trivial. I am writing some code for maintenance of pages in Creator namespace on Commons and I run into a weird problem with Page.templatesWithParams function. The function is apparently not able to handle template parameters, like "{{{1|}}}". For example I run the following code:
#!/usr/bin/python # -*- coding: utf-8 -*-
import sys, string sys.path.append("C:/Programs/pywikipedia/") import wikipedia
if __name__ == "__main__": site = wikipedia.getSite(u'commons', u'commons') page = wikipedia.Page(site, u'Creator:Ansel Adams') wikipedia.output(u'Original page text:\n'+page.get()+u'\n\nTemplates detected by templatesWithParams:') for template in page.templatesWithParams(): wikipedia.output(u'template[0]="%s"' % template[0])
thistxt = string.replace(page.get(), '{{{1|}}}', '') wikipedia.output(u'\nModified page text:\n'+thistxt+u'\n\nTemplates
detected by templatesWithParams:') for template in page.templatesWithParams(thistxt): wikipedia.output(u'template[0]="%s"' % template[0])
and got the following output: Original page text: {{Creator | Image = Ansel_Adams_and_camera.jpg | ImgWidth = 120 | Name = Ansel Adams | Alternative names = Ansel Easton Adams | Sortkey = Adams, Ansel | Nationality = US | Gender = male | Occupation = photographer | Description = | Birthdate = 1902-02-20 | Birthloc = {{San Francisco}} | Deathdate = 1984-04-22 | Deathloc = Monterey | Workperiod = | Workloc = American West | Homecat = Ansel Adams | Option = {{{1|}}} | Authority = {{Authority control|PND=11881303X|LCCN=n/79/56359|VIAF=61625857|ARC=1332556|bare=1}} }}
Templates detected by templatesWithParams: template[0]="San Francisco" template[0]="1" template[0]="Authority control"
Modified page text: {{Creator | Image = Ansel_Adams_and_camera.jpg | ImgWidth = 120 | Name = Ansel Adams | Alternative names = Ansel Easton Adams | Sortkey = Adams, Ansel | Nationality = US | Gender = male | Occupation = photographer | Description = | Birthdate = 1902-02-20 | Birthloc = {{San Francisco}} | Deathdate = 1984-04-22 | Deathloc = Monterey | Workperiod = | Workloc = American West | Homecat = Ansel Adams | Option = | Authority = {{Authority control|PND=11881303X|LCCN=n/79/56359|VIAF=61625857|ARC=1332556|bare=1}} }}
Templates detected by templatesWithParams: template[0]="San Francisco" template[0]="Authority control" template[0]="Creator"
As one can see in the first case the function was not able to find "Creator" template, and was only able to do so after deletion of "{{{1|}}}".
Is this a feature or a bug? Or in other words, am I misusing the function somehow and there is some way to get it to work correctly, or is it a bug that should be reported somewhere, so it can be fixed?
Jarek T. (user:jarekt)
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
There is an old and serious problem with embedded templates that has not been fixed. Please have a look at this:
http://sourceforge.net/tracker/index.php?func=detail&aid=3158761&gro...
Here I wrote a solution that worked for me in the given case, but it was not applied to Pywiki beacuse somebody showed an example where it causes another problem. It is not easy at all to catch embedded templates with regular expresses. Try my solution, but this is just an idea and not sure to work for you. You may want to adjust the regex to your needs (but don't forget you have changed it!).
Bináris, I am not sure what you mean by "embedded templates", in case of pages from Commons creator namespace they are technically all templates and take template parameters. I suspect not many people in other wikis were applying bots to process templates and that is why not many people run into this issue.
In my case I know now what is causing the problem and can strip page text from app the parameters like "{{{1|}}}" before applying Page.templatesWithParams function . In my case it is easy since, text in creator namespace follows very specific pattern and there is not much variation. However this might be a serious issue for others.
Jarek T.
(user:Jarekt)
From: pywikipedia-l-bounces@lists.wikimedia.org [mailto:pywikipedia-l-bounces@lists.wikimedia.org] On Behalf Of Bináris Sent: Monday, October 24, 2011 2:59 PM To: Pywikipedia discussion list Subject: Re: [Pywikipedia-l] Problems with Page.templatesWithParams function
There is an old and serious problem with embedded templates that has not been fixed. Please have a look at this:
http://sourceforge.net/tracker/index.php?func=detail&aid=3158761&gro...
Here I wrote a solution that worked for me in the given case, but it was not applied to Pywiki beacuse somebody showed an example where it causes another problem. It is not easy at all to catch embedded templates with regular expresses. Try my solution, but this is just an idea and not sure to work for you. You may want to adjust the regex to your needs (but don't forget you have changed it!).
On 9 October 2011 19:21, Bináris wikiposta@gmail.com wrote:
I work on a script that will mass copy pages from one wiki to another. The main purpose is to fetch my favourite templates from huwiki to my own local wiki.
Why not use Special:Export and Special:Import?
Best, Merlijn
pywikipedia-l@lists.wikimedia.org