As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested. It went very well thanks Legoktm (a BIG THANK YOU) but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies.
Maarten (multichill) has suggested we change setup.py and generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example: When someone doesn't install i18n submodule. Codes breaks (error that says "import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code.
Best
Hi Amir,
On 14 August 2013 16:22, Amir Ladsgroup ladsgroup@gmail.com wrote:
As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested.
Cool!
but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies.
This surprises me, as this was not an issue during the Amsterdam hackathon - and we used rewrite! However, we used a nightly instead of git, which might explain at least part of the difference. What was the audience (how experienced with wikipedia as writer / AWB / ..., running windows or linux, etc), and how did they install and configure pwb?
Maarten (multichill) has suggested we change setup.py and
generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
When someone doesn't install i18n submodule. Codes breaks (error that says
"import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code.
It's impossible to run a bot without i18n submodule, because *all* translations are in the translation file, also the English version! The code only knows the key (e.g. 'commons-file-moved', not '[[:File:%(localfile)s|File]] moved to [[:commons:File:%(commonsfile)s|commons]].').
Best, Merlijn
I feel a big need of a detailed help page to run /shared/pywikipedia/compat into Labs, I painfully run rewrite but there are too many new issues to manage (labs vs toolserver, git vs svn, rewrite vs trunk, recent changes .... all together, is far above my limited skill, it has been an endless nightmare)
Alex brollo
2013/8/14 Merlijn van Deen valhallasw@arctus.nl
Hi Amir,
On 14 August 2013 16:22, Amir Ladsgroup ladsgroup@gmail.com wrote:
As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested.
Cool!
but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies.
This surprises me, as this was not an issue during the Amsterdam hackathon
- and we used rewrite! However, we used a nightly instead of git, which
might explain at least part of the difference. What was the audience (how experienced with wikipedia as writer / AWB / ..., running windows or linux, etc), and how did they install and configure pwb?
Maarten (multichill) has suggested we change setup.py and
generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
When someone doesn't install i18n submodule. Codes breaks (error that says
"import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code.
It's impossible to run a bot without i18n submodule, because *all* translations are in the translation file, also the English version! The code only knows the key (e.g. 'commons-file-moved', not '[[:File:%(localfile)s|File]] moved to [[:commons:File:%(commonsfile)s|commons]].').
Best, Merlijn
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 15.08.2013 01:48, Alex Brollo wrote:
I feel a big need of a detailed help page to run /shared/pywikipedia/compat into Labs, I painfully run rewrite but there are too many new issues to manage (labs vs toolserver, git vs svn, rewrite vs trunk, recent changes .... all together, is far above my limited skill, it has been an endless nightmare)
Alex brollo
Yes that's a problem. In fact that are numerous problems: 1.) If you start considering Labs you could be suspected not to support TS anymore but Labs... ;) 2.) I am looking into this since quite some time, check e.g. [1] but first needed some changes on the framework code (they are mostly finished now) and also more progress on the Labs side (which should be ok now too - had no time to check) 3.) I am up to start setting up more bots on Labs, BUT I have a huge lack in time because of; Wikidata startup (not completely finished yet), SVN->GIT Migration (still an issue - at least for me), Labs, ...
[1] https://wikitech.wikimedia.org/wiki/DrTrigonBot
I am trying hard to dig myself out of all the work in order to focus on Labs... any help and thoughts are greatly appreciated and needed.
I will help as much as I can. Just tell me
On Sat, Aug 17, 2013 at 2:40 PM, Dr. Trigon dr.trigon@surfeu.ch wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 15.08.2013 01:48, Alex Brollo wrote:
I feel a big need of a detailed help page to run /shared/pywikipedia/compat into Labs, I painfully run rewrite but there are too many new issues to manage (labs vs toolserver, git vs svn, rewrite vs trunk, recent changes .... all together, is far above my limited skill, it has been an endless nightmare)
Alex brollo
Yes that's a problem. In fact that are numerous problems: 1.) If you start considering Labs you could be suspected not to support TS anymore but Labs... ;) 2.) I am looking into this since quite some time, check e.g. [1] but first needed some changes on the framework code (they are mostly finished now) and also more progress on the Labs side (which should be ok now too - had no time to check) 3.) I am up to start setting up more bots on Labs, BUT I have a huge lack in time because of; Wikidata startup (not completely finished yet), SVN->GIT Migration (still an issue - at least for me), Labs, ...
[1] https://wikitech.wikimedia.org/wiki/DrTrigonBot
I am trying hard to dig myself out of all the work in order to focus on Labs... any help and thoughts are greatly appreciated and needed. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlIPTCYACgkQAXWvBxzBrDConQCfaNhV6Q8+A4unmibfrjfheOJh AVQAn3GHVdtTI7Fw2SV/GEyvmEA6em7Z =vH5X -----END PGP SIGNATURE-----
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I started setting up a copy of core on Labs 'tools' project (the 'bots' project which I initially used is obsolete now). You can see my first pre-mature manual/how to on [1]. This manual needs to be further improved - anybody is free to help - but e.g. the compat part which was mentioned explicitely is hard to handle before I get the new additional repos requested on this list... ;) But in fact the procedure described there for compat *should* work already for most of the bots, so please test it and give feedback to us.
[1] https://wikitech.wikimedia.org/wiki/DrTrigonBot#Setup_pywikibot_on_Labs
Greetings DrTrigon
On 17.08.2013 12:15, Amir Ladsgroup wrote:
I will help as much as I can. Just tell me
On Sat, Aug 17, 2013 at 2:40 PM, Dr. Trigon <dr.trigon@surfeu.ch mailto:dr.trigon@surfeu.ch> wrote:
On 15.08.2013 01:48, Alex Brollo wrote:
I feel a big need of a detailed help page to run /shared/pywikipedia/compat into Labs, I painfully run rewrite but there are too many new issues to manage (labs vs toolserver, git vs svn, rewrite vs trunk, recent changes .... all together, is far above my limited skill, it has been an endless nightmare)
On Wed, Aug 14, 2013 at 11:40 PM, Merlijn van Deen valhallasw@arctus.nlwrote:
Hi Amir,
On 14 August 2013 16:22, Amir Ladsgroup ladsgroup@gmail.com wrote:
As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested.
Cool!
but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies.
This surprises me, as this was not an issue during the Amsterdam hackathon
- and we used rewrite! However, we used a nightly instead of git, which
might explain at least part of the difference. What was the audience (how experienced with wikipedia as writer / AWB / ..., running windows or linux, etc), and how did they install and configure pwb?
They were from different kind of expertise from new to wikipedia to
someone who ran bots in toolserver. The biggest issue was we couldn't get reach to user-cofnig.py (sometimes people try to install several clones). I really prefer the old school in this field. We can store user-config in the clone (core) folder instead of .pywikibot
Maarten (multichill) has suggested we change setup.py and
generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
Nightly needs some fixations I don't Legoktm did it or not but we have to check it
When someone doesn't install i18n submodule. Codes breaks (error that
says "import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code.
It's impossible to run a bot without i18n submodule, because *all* translations are in the translation file, also the English version! The code only knows the key (e.g. 'commons-file-moved', not '[[:File:%(localfile)s|File]] moved to [[:commons:File:%(commonsfile)s|commons]].').
Yeah. You're right
Best, Merlijn
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Best
Alex has a point. We need to write a detailed help about the PWB rewrite branch
Best
On Thu, Aug 15, 2013 at 12:31 PM, Amir Ladsgroup ladsgroup@gmail.comwrote:
On Wed, Aug 14, 2013 at 11:40 PM, Merlijn van Deen valhallasw@arctus.nlwrote:
Hi Amir,
On 14 August 2013 16:22, Amir Ladsgroup ladsgroup@gmail.com wrote:
As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested.
Cool!
but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies.
This surprises me, as this was not an issue during the Amsterdam hackathon - and we used rewrite! However, we used a nightly instead of git, which might explain at least part of the difference. What was the audience (how experienced with wikipedia as writer / AWB / ..., running windows or linux, etc), and how did they install and configure pwb?
They were from different kind of expertise from new to wikipedia to
someone who ran bots in toolserver. The biggest issue was we couldn't get reach to user-cofnig.py (sometimes people try to install several clones). I really prefer the old school in this field. We can store user-config in the clone (core) folder instead of .pywikibot
Maarten (multichill) has suggested we change setup.py and
generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
Nightly needs some fixations I don't Legoktm did it or not but we have to check it
When someone doesn't install i18n submodule. Codes breaks (error that
says "import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code.
It's impossible to run a bot without i18n submodule, because *all* translations are in the translation file, also the English version! The code only knows the key (e.g. 'commons-file-moved', not '[[:File:%(localfile)s|File]] moved to [[:commons:File:%(commonsfile)s|commons]].').
Yeah. You're right
Best, Merlijn
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Best
Amir
On 15 August 2013 10:01, Amir Ladsgroup ladsgroup@gmail.com wrote:
The biggest issue was we couldn't get reach to user-cofnig.py (sometimes people try to install several clones).
I'm not sure what you mean by that.
I really prefer the old school in this field. We can store user-config in the clone (core) folder instead of .pywikibot
This is already possible, by using the pwb.py wrapper. However, this is not a feasable option for a shared library (e.g. when it is installed centrally), which is why ~/.pywikibot is used (by default) in that case.
Nightly needs some fixations I don't Legoktm did it or not but we have to check it
What are the problems?
Merlijn
On Thu, Aug 15, 2013 at 1:19 PM, Merlijn van Deen valhallasw@arctus.nlwrote:
On 15 August 2013 10:01, Amir Ladsgroup ladsgroup@gmail.com wrote:
The biggest issue was we couldn't get reach to user-cofnig.py (sometimes people try to install several clones).
I'm not sure what you mean by that.
So many times people want to change the user-config file and so many times they want to run several bots (in different usernames) or maybe they want to just change the file. storing a central user-config in a hidden place is not a good thing
I really prefer the old school in this field. We can store user-config in the clone (core) folder instead of .pywikibot
This is already possible, by using the pwb.py wrapper. However, this is not a feasable option for a shared library (e.g. when it is installed centrally), which is why ~/.pywikibot is used (by default) in that case.
I tested changing directory but it's really hard for newbies (that's what I'm talking about) I suggest that we change the default and make it like the trunk
Nightly needs some fixations I don't Legoktm did it or not but we have
to check it
What are the problems?
I downloaded the core in .tar but It was compat (or vice verse, I can't
recall correctly)
Merlijn
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Best
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Maarten (multichill) has suggested we change setup.py and generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Of course that is what I am working on since some time. Please give compat (trunk) a try, download it and then try to set it up and start a bot. IF there is an unsatisfied dependency the framework should complain and ask you whether you want to install it. This is still somehow pre-mature thus I need as much feedback as possible in order to adopt it to your needs - for me it works quite well installing about 10 (part of them quite big packages) external deps without an issue.
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
Good point; try also to use the nigtlies.
In fact this is something we have to discuss; what external libraries are *NEEDED*? E.g. in trunk articleno.py, subster_irc.py and in rewrite pywikibot.ircbot.py need "python-irclib". Or as another example, catimages.py needs a lot of externals like "opencv" and others. I tried to implement a "install-when-used" policy in order not to download hughe (~100MB) packages without needing them at all. But we REALLY SHOULD have some automated system in order to make it easier for beginners (and others) to install all the deps needed (when needed).
On Sat, Aug 17, 2013 at 2:33 PM, Dr. Trigon dr.trigon@surfeu.ch wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Maarten (multichill) has suggested we change setup.py and generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example:
This is certainly an option. Maybe Dr.Trigon can suggest something in this direction - the current method of downloading dependencies/externals when needed is reasonable, I think.
Of course that is what I am working on since some time. Please give compat (trunk) a try, download it and then try to set it up and start a bot. IF there is an unsatisfied dependency the framework should complain and ask you whether you want to install it. This is still somehow pre-mature thus I need as much feedback as possible in order to adopt it to your needs - for me it works quite well installing about 10 (part of them quite big packages) external deps without an issue.
As a feedback, It was very hard for newbies we have to work on it
Alternatively, I'd like to suggest nightlies as main distribution method. At least the core nightly is completely self-contained: it has translations *and* httplib2 (the only required external library). For most people, that would be the easiest way of installing pywikibot.
Good point; try also to use the nigtlies.
Let's work on it!
In fact this is something we have to discuss; what external libraries are *NEEDED*? E.g. in trunk articleno.py, subster_irc.py and in rewrite pywikibot.ircbot.py need "python-irclib". Or as another example, catimages.py needs a lot of externals like "opencv" and others. I tried to implement a "install-when-used" policy in order not to download hughe (~100MB) packages without needing them at all. But we REALLY SHOULD have some automated system in order to make it easier for beginners (and others) to install all the deps needed (when needed).
Can you give me a list of ALL of externals and which codes these externals are used?
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlIPSngACgkQAXWvBxzBrDC3JgCfUrLySkceNX976r8ci38saU7b Oh4An29o0r7huEvNE/q9ucDaBPOMrx9d =IRTR -----END PGP SIGNATURE-----
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Thanks for your feedback!
As a feedback, It was very hard for newbies we have to work on it
What do you mean?
In fact this is something we have to discuss; what external libraries are *NEEDED*? E.g. in trunk articleno.py, subster_irc.py and in rewrite pywikibot.ircbot.py http://pywikibot.ircbot.py need "python-irclib". Or as another example, catimages.py needs a lot of externals like "opencv" and others. I tried to implement a "install-when-used" policy in order not to download hughe (~100MB) packages without needing them at all. But we REALLY SHOULD have some automated system in order to make it easier for beginners (and others) to install all the deps needed (when needed).
Can you give me a list of ALL of externals and which codes these externals are used?
A list of all externals handeled automatic now can be found at [1]. In order to find the codes the best is to use "grep".
[1] https://git.wikimedia.org/blob/pywikibot%2Fcompat/1ef435be9b8feabbaae7159ea2...
Greetings DrTrigon