Hi,
with the recent switch to HTTPS by default [0] I'm wondering if it still makes sense to keep httplib2 (the upstream version, not the one maintained at [1]) as a dependency in setup.py?
According to setuptools' documentation [2] the effect of listing httplib2 (or any other dependency) in install_requires is that
When your project is installed, either by using EasyInstall, setup.py install, or setup.py develop, all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed.
For PWB this means that installing the project via `setup.py install` in an environment where [1] is not already installed, setuptools will download httplib2 from PyPI. This is not a good idea because all it will do is give people an httplib2 that can't verify Wikimedias certificate. AFAIK the only way to recognize this is during the install step - and let's be honest - nooes reading *all* of the output ;)
I did try adding depency links as described by [2]:
dependency_links=[ 'git+https://git.wikimedia.org/git/pywikibot/externals/httplib2.git#egg=httplib2-...' ],
but surprisingly, this only works with pip but not setuptools itself ('git+https' is an unknown url type to setuptools).
So it seems to me that specifying httlib2 as a dependency in setup.py does more harm than good and it should be removed - [3] already lists the submodule as a dependency, although it could be made more clear that the git submodule differs from upstream.
As I'm not really that familiar with the python packaging ecosystem, I'm not sure if there's a way to make (for example) the dependency links stuff work or if the dependency is worth keeping for some other reason.
If there are no objections/other ideas to make this work, I'd submit a patch removing the dependency from setup.py reusing most of this mail as the commit message.
[0] https://git.wikimedia.org/commit/pywikibot%2Fcore.git/6bb502983afc93b4222d3c... [1] https://git.wikimedia.org/summary/?r=pywikibot/externals/httplib2.git [2] http://pythonhosted.org/setuptools/setuptools.html#declaring-dependencies [3] https://www.mediawiki.org/wiki/Manual:Pywikipediabot/Installation#Manual_.28...
The main thing against your proposition is compatibility: 1-PWB is designed to work in ALL wikis and so many of them are not WMF-related and even don't have any SSL connection ability 2-Iran and Chinese people are excluded for SSL and They can't run bots anymore if we use just SSL
Best
On Wed, Sep 4, 2013 at 6:59 PM, Wieland Hoffmann themineo@gmail.com wrote:
Hi,
with the recent switch to HTTPS by default [0] I'm wondering if it still makes sense to keep httplib2 (the upstream version, not the one maintained at [1]) as a dependency in setup.py?
According to setuptools' documentation [2] the effect of listing httplib2 (or any other dependency) in install_requires is that
When your project is installed, either by using EasyInstall, setup.py install, or setup.py develop, all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed.
For PWB this means that installing the project via `setup.py install` in an environment where [1] is not already installed, setuptools will download httplib2 from PyPI. This is not a good idea because all it will do is give people an httplib2 that can't verify Wikimedias certificate. AFAIK the only way to recognize this is during the install step - and let's be honest - nooes reading *all* of the output ;)
I did try adding depency links as described by [2]:
dependency_links=[ 'git+ https://git.wikimedia.org/git/pywikibot/externals/httplib2.git#egg=httplib2-... ' ],
but surprisingly, this only works with pip but not setuptools itself ('git+https' is an unknown url type to setuptools).
So it seems to me that specifying httlib2 as a dependency in setup.py does more harm than good and it should be removed - [3] already lists the submodule as a dependency, although it could be made more clear that the git submodule differs from upstream.
As I'm not really that familiar with the python packaging ecosystem, I'm not sure if there's a way to make (for example) the dependency links stuff work or if the dependency is worth keeping for some other reason.
If there are no objections/other ideas to make this work, I'd submit a patch removing the dependency from setup.py reusing most of this mail as the commit message.
[0] https://git.wikimedia.org/commit/pywikibot%2Fcore.git/6bb502983afc93b4222d3c... [1] https://git.wikimedia.org/summary/?r=pywikibot/externals/httplib2.git [2] http://pythonhosted.org/setuptools/setuptools.html#declaring-dependencies [3] https://www.mediawiki.org/wiki/Manual:Pywikipediabot/Installation#Manual_.28...
-- Wieland
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Hallo Amir Ladsgroup:
The main thing against your proposition is compatibility: 1-PWB is designed to work in ALL wikis and so many of them are not WMF-related and even don't have any SSL connection ability 2-Iran and Chinese people are excluded for SSL and They can't run bots anymore if we use just SSL
I'm not sure why you're telling me this, my question/suggestion had nothing to with with en- or disabling SSL for any number of people - in fact, that decision seems to have been made already by other people.
Hi Wieland,
On 4 September 2013 16:29, Wieland Hoffmann themineo@gmail.com wrote:
with the recent switch to HTTPS by default [0] I'm wondering if it still makes sense to keep httplib2 (the upstream version, not the one maintained at [1]) as a dependency in setup.py?
Yes, you are completely right. It makes much more sense to refer to our version instead.
I did try adding depency links as described by [2]:
dependency_links=[ 'git+ https://git.wikimedia.org/git/pywikibot/externals/httplib2.git#egg=httplib2-... ' ],
I realised today there was a simpler solution than getting setuptools/pip to understand git: git.wikimedia.org has a .tar.gz download option. As such, I have submitted
https://gerrit.wikimedia.org/r/83370
which suggests https://git.wikimedia.org/zip/?r=pywikibot/externals/httplib2.git&format... as download link.
Thank you for bringing this to our attention!
Merlijn
And that was my bad So I'm sorry and I merged the patch right away :)
Best
On Sun, Sep 8, 2013 at 8:36 PM, Merlijn van Deen valhallasw@arctus.nlwrote:
Hi Wieland,
On 4 September 2013 16:29, Wieland Hoffmann themineo@gmail.com wrote:
with the recent switch to HTTPS by default [0] I'm wondering if it still makes sense to keep httplib2 (the upstream version, not the one maintained at [1]) as a dependency in setup.py?
Yes, you are completely right. It makes much more sense to refer to our version instead.
I did try adding depency links as described by [2]:
dependency_links=[ 'git+ https://git.wikimedia.org/git/pywikibot/externals/httplib2.git#egg=httplib2-... ' ],
I realised today there was a simpler solution than getting setuptools/pip to understand git: git.wikimedia.org has a .tar.gz download option. As such, I have submitted
https://gerrit.wikimedia.org/r/83370
which suggests https://git.wikimedia.org/zip/?r=pywikibot/externals/httplib2.git&format... as download link.
Thank you for bringing this to our attention!
Merlijn
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Hallo Merlijn van Deen:
I did try adding depency links as described by [2]:
dependency_links=[ 'git+ https://git.wikimedia.org/git/pywikibot/externals/httplib2.git#egg=httplib2-... ' ],
I realised today there was a simpler solution than getting setuptools/pip to understand git: git.wikimedia.org has a .tar.gz download option. As such, I have submitted
Awesome, one less thing on my todo list without me doing anything :-)
which suggests https://git.wikimedia.org/zip/?r=pywikibot/externals/httplib2.git&format... as download link.
Ah, that was way too obvious for me to find.
pywikipedia-l@lists.wikimedia.org