As we discussed several times some bot operators do not like having .exe files executed without explicit permission to do it.
In my opinion it is not a good solution blocking the bot now if the permission is denied because the bot worked 10 years ago without this permission and patching beautifullsoup. That library was included to our code since the first steps of pwb. Where are the problems to include them or what are the licenses problems? Maybe you are right and we did wrong in past. And what are the special necessity for that patch? Externals can be included without running patch.exe. But I do not understand them. I understood that patch as a sample for that technoloy which should be used when needed. Now I only see the exception raising while starting any script of the framework (which was few days ago and not some months behind. Sorry I am not up to day while testing ;) Ok this behaviour seems a good way to let the trunk/compat branch die in honest. In this respect I am with you :D
Best
xqt
----- Original Nachricht ----
Von: "Dr. Trigon" <dr.trigon(a)surfeu.ch>
An: pywikipedia-l(a)lists.wikimedia.org
Datum: 01.08.2013 13:29
Betreff: Re: [Pywikipedia-l] Question about externals and patches
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> > I don't like any installation process including rewrite branch/core
> > and I am using pwb.py to have a directory based environment. Maybe
> > for some reasons a installation is necessary but it should be very
> > rare! There is no reason for patching beautifullsoup because we
> > could either use the external library (as something like external)
> > as it is or include it to the framework as it was previously. This
> > is a central library for the compat branch and you cannot run it
> > whithout that stuff. Missing this library stops running the whole
> > framework if some users did not want to install any .exe file.
> > Other patches only depends few scripts or not even a script is
> > affected.
>
> I don't like installation either!! This is not about installing the
> framework itself, but the dependencies needed! That's a big difference!
> Nobody forces you to use the externals/__init__.py you CAN do this all
> manually. But I for my part am very unhappy, if I have to install
> several packages and search them on the web, find the correct version,
> try to compile it and all the stuff - just to TEST, whether this
> pywikipedia framework could be useful to me. It is a good thing if it
> is able to do this by itself - still if you do it by yourself in
> advance there is no need for externals to jump in. That's supposed to
> be more beginner-friendly (it is a one-time help to start using the
> framework).
>
> It is a bad idea to include code like beautifulsoup.py from another
> source. That makes confusion (who has written the code originally), is
> problematic with licenses and gives issues with merging new versions.
> (cosmetic changes were applied to our version and so it was not clear
> anymore how to merge this with the one upstream) - so we should NOT
> include other code into our repos! AND on linux systems there is
> beautifulsoup already included into the distro repo - so it makes no
> sense to include it with pywikipedia. (similarly for simplejson,
> setuptools, httplib2, ...)
>
> > And please check whether a patch is necessary for other parts too
> > and keep the code simple. I guess there could be other ways which
> > have the same effect without running any .exe file which some
> > people don't like to. E.g. for music21, pydmtx, zbar I don't
> > understand why a patch is required (and for what scripts they are
> > needed at all).
>
> In fact we discussed about all the stuff already in the past [1] (May
> 2013 till now - search for 'externals' and 'patch') where I explained
> all that in detail. (I kept it as simple as possible - I explained
> already several options - The patches are reduced to a minimum -
> Forking those packages just creates additional workload - Some of
> those packages are huge - ...)
>
> [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/
>
> I already mentioned - several times - if you have a better solution (I
> searched one for months now!) then PLEASE tell me! I do not know any
> other solution to include all my code and scripts here! If you want me
> to remove them again, fine - I will do this - but it would have been
> nice to tell me this before I invested weeks of coding into that stuff!
> And as you should have been seeing e.g. in the migration to git
> submodules - externals/__init__.py was in fact an easy, simple and
> fast solution to work-a-round that stuff. Although we migrated to git,
> it is still as bad as svn - regarding externals or submodules neither
> (svn, git) has a good solution! But PLEASE tell me a better one if you
> know it!!
>
> One last way out that I can think of is, we make it non-automatic; we
> could do it together with the creation of user-config.py as one
> explicit step the user has to do manually in the beginning (instead of
> jumping in automatically) - what about that?
>
> Greetings and all the best!
> DrTrigon
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.13 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlH6RqkACgkQAXWvBxzBrDAAdQCfS6/6eAJUEPlK+FRMfAfFdXHf
> kgcAoKNsoJaLyK7h05BNWaWG6P3gxGaE
> =PuAP
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Pywikipedia-l mailing list
> Pywikipedia-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
>
In updating the documentation for the Mac install (as well as installing it myself for the first time) I encountered the following SVN error:
> svn: warning: Error handling externals definition for 'pywikipedia/externals/pycolorname':
> svn: warning: OPTIONS of 'https://svn.toolserver.org/svnroot/drtrigon/externals/pycolorname':
> Could not read status line: connection was closed by server (https://svn.toolserver.org)
I assume this to be a transient SVN error -- changes being made, in progress, etc. ???
T.T.F.N.
William H. Magill
# iMac11,3 Core i7 [2.93GHz - 8 GB 1067MHz] OS X 10.8.4
# Macmini6,1 Intel Core i5 [2.5 Ghz - 4GB 1600MHz] OS X 10.8.3
magill(a)icloud.com
magill(a)mac.com
whmagill(a)gmail.com
----- Original Nachricht ----
Von: "Dr. Trigon" <dr.trigon(a)surfeu.ch>
An: pywikipedia-l(a)lists.wikimedia.org
Datum: 01.08.2013 11:03
Betreff: Re: [Pywikipedia-l] Question about externals and patches
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 01.08.2013 05:37, info(a)gno.de wrote:
> > There is a remaining bug. Try the following e.g. from idle:
> >
> >>>> import Wikipedia
> >
> > you are asked to install patch.exe. If you decline this request you
> > are asked a second time. If declined a second time the script
> > raises a fatal error while importing beautifullsoup. Beautifullsoup
> > is a central part if the screen scraping compat framework and we
> > should not prohibit using the framework for a cosmetic patch.
> > Please give us back that part as e.g. externals or a library
> > script. Anyway it is not a good idea to ask for installation
> > (twice) and fall into a fatal error if you answer is NO. There
> > should be alternatives to patching the working copy (e.g. nightly
> > dump etc.)
>
> Thank you very much for this info! I will look at this as soon as
> possible!
>
> The question is how to handle the case you descibed? This questions
> have to be regarded as final part of the install/setup process
> (checkout, clone or download), so how to handle a sequence of N(o)
> answers? In fact this means that the user does not want to finish the
> install/setup process - this is somehow a paradox... Shall I simply
> output a message:
>
> "Install process could not be finished, framework NOT ready for use!"
>
> Please contribute your ideas! externals/__init__.py is supposed to
> help us, not to bother us... :) By the way the concept is not new, is
> was inspired by how "VisTrails" does this.
>
> Thanks and Greetings
> DrTrigon
>
I don't like any installation process including rewrite branch/core and I am using pwb.py to have a directory based environment. Maybe for some reasons a installation is necessary but it should be very rare! There is no reason for patching beautifullsoup because we could either use the external library (as something like external) as it is or include it to the framework as it was previously. This is a central library for the compat branch and you cannot run it whithout that stuff. Missing this library stops running the whole framework if some users did not want to install any .exe file. Other patches only depends few scripts or not even a script is affected.
And please check whether a patch is necessary for other parts too and keep the code simple. I guess there could be other ways which have the same effect without running any .exe file which some people don't like to. E.g. for music21, pydmtx, zbar I don't understand why a patch is required (and for what scripts they are needed at all).
best
xqt
I had no doubts ;)
----- Ursprüngliche Nachricht -----
Von: Bináris
Gesendet: 27.07.2013 17:16
An: Pywikipedia discussion list
Betreff: Re: [Pywikipedia-l] Additional parameter in a replace function
It works! Thank you.
2013/7/22 <info(a)gno.de>
----- Original Nachricht ----
Von: Bináris <wikiposta(a)gmail.com>
An: Pywikipedia discussion list <pywikipedia-l(a)lists.wikimedia.org>
Datum: 21.07.2013 18:14
Betreff: [Pywikipedia-l] Additional parameter in a replace function
> Hi,
>
> I use functions in my fixes.py as written in
> https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:Bin%C3%A1ris/Fixes_and_functio
> ns_HOWTO
> .
>
> Now, my idea is to use an additional parameter. (In case nobody breaks the
> framework. :-))
> So my fix has something like this in replacements:
>
> (ur'someregex', MyFunc),
> Then MyFunc takes the match object as a parameter and is executed. I have
> to write
> def MyFunc(match):
> etc.
>
> Now I want to write def MyFunc(match, mode), but I can't pass mode to
> MyFunc anyway.
> I guessed the line in question is #195 in textlib.py:
> replacement = new(match)
>
> Here I stopped. How could I pass additional optional parameters?
Just an idea, I haven't investigated in it but you could try it out:
you may use a class instance inside fixes.py and pass the option through the constructor. Your MyFunc should be an instance method.
e.g.
class myClass(object):
def __init__(self, option):
self.option = option
def myFunc(self, match):
result = u''
# change the result depending of match and self.option
# ...
return result
thisClass = myClass(option='submit anything as option')
and play with replacements using thisClass.myFunc
Best
xqt
--
Bináris
Sounds there is an illegal character in the page name. Could you try to normalize the page name e.g. search for it at gl-wikt and fetch the displayed title by hand.
xqt
----- Ursprüngliche Nachricht -----
Von: Adrián Chaves Fernández
Gesendet: 28.07.2013 09:30
An: Bináris
Cc: Pywikipedia discussion list
Betreff: Re: [Pywikipedia-l][Rewrite] Cannot download http://gl.wiktionary.org/wiki/𐌰𐌽𐌳𐌰𐌿𐍂𐌰
Thanks for the suggestion. I get the same result, though.
I should probably mention that it works with asian characters, and that I’m running this on an utf8-enabled GNU/Linux terminal.
O Domingo, 28 de Xullo de 2013 09:11:14 Bináris escribiu:
> Just a try: what happens for
> pywikibot.output(page.get())
> instead of print?
>
>
> 2013/7/28 Adrián Chaves Fernández <adriyetichaves(a)gmail.com>
>
> > I have written the following script:
> >
> >
> > # -*- coding: utf-8 -*-
> >
> > import pywikibot
> >
> > site = pywikibot.Site("gl", "wiktionary")
> > page = pywikibot.Page(site, u"𐌰𐌽𐌳𐌰𐌿𐍂𐌰")
> > print page.get()
> >
> >
> > It fails with the following output:
> >
> >
> > [gallaecio@afonso fontes]$ python2 test.py
> > Traceback (most recent call last):
> > File "test.py", line 7, in <module>
> > print page.get()
> > File "/usr/lib/python2.7/site-packages/pywikibot/__init__.py", line 249,
> > in wrapper
> > return method(*__args, **__kw)
> > File "/usr/lib/python2.7/site-packages/pywikibot/__init__.py", line 249,
> > in wrapper
> > return method(*__args, **__kw)
> > File "/usr/lib/python2.7/site-packages/pywikibot/page.py", line 278, in
> > get
> > self._getInternals(sysop)
> > File "/usr/lib/python2.7/site-packages/pywikibot/page.py", line 302, in
> > _getInternals
> > self.site.loadrevisions(self, getText=True, sysop=sysop)
> > File "/usr/lib/python2.7/site-packages/pywikibot/page.py", line 96, in
> > site
> > return self._link.site
> > File "/usr/lib/python2.7/site-packages/pywikibot/page.py", line 3080, in
> > site
> > self.parse()
> > File "/usr/lib/python2.7/site-packages/pywikibot/page.py", line 3037, in
> > parse
> > u"contains illegal char(s) '%s'" % m.group(0))
> > pywikibot.exceptions.InvalidTitle: contains illegal char(s) '𐌰'
> >
> >
> >
> >
> > _______________________________________________
> > Pywikipedia-l mailing list
> > Pywikipedia-l(a)lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
> >
>
>
>
>
_______________________________________________
Pywikipedia-l mailing list
Pywikipedia-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Could you please five us the full traceback and your python release.
xqt
----- Ursprüngliche Nachricht -----
Von: Adam Waldstein
Gesendet: 01.08.2013 02:20
An: pywikipedia-l(a)lists.wikimedia.org
Betreff: Re: [Pywikipedia-l] Setup of pywikipedia for private mediawiki
I am also trying to create a family but get a different error. It comes up
with:
Traceback (most recent call last):
File "generate_family_file.py", line 22, in ?
from urlparse import urlparse, urljoin, ParseResult
I don't know what is wrong with line 22. I tried the trick you suggested above
of removing the user-config file first, but it still comes up with the same
error.
Any suggestions would be more then appreciated.
Adam
_______________________________________________
Pywikipedia-l mailing list
Pywikipedia-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l