Switch to HTTPS, implications for Pywikibot? - Pywikipedia-l - lists.wikimedia.org

List overview All Threads
Download

Switch to HTTPS, implications for Pywikibot?

Re: [Pywikipedia-l] Git error...

rollback the edits made by bot

Morten Wang

20 Aug 2013 20 Aug '13

3:57 a.m.

I noticed this message from a thread on wikitech-l and wikibots-l: http://lists.wikimedia.org/pipermail/wikibots-l/2013-August/000353.html Does this have any particular implications for pywikibot, or is it already using HTTPS? Cheers, Morten

Attachments:

attachment.htm (text/html — 426 bytes)

Reply

Show replies by date

Amir Ladsgroup

20 Aug 20 Aug

8:57 a.m.

We just need to do some setups. It's not a big deal. I'll put it in my TODO list On Tue, Aug 20, 2013 at 8:27 AM, Morten Wang <nettrom(a)gmail.com> wrote:

I noticed this message from a thread on wikitech-l and wikibots-l: http://lists.wikimedia.org/pipermail/wikibots-l/2013-August/000353.html Does this have any particular implications for pywikibot, or is it already using HTTPS? Cheers, Morten _______________________________________________ Pywikipedia-l mailing list Pywikipedia-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

-- Amir

Reply

Amir Ladsgroup

1:40 p.m.

I made two changes and this changes has to be merged ASAP https://gerrit.wikimedia.org/r/80003 https://gerrit.wikimedia.org/r/80006 after that bot operators just need to update On Tue, Aug 20, 2013 at 1:27 PM, Amir Ladsgroup <ladsgroup(a)gmail.com> wrote:

We just need to do some setups. It's not a big deal. I'll put it in my TODO list On Tue, Aug 20, 2013 at 8:27 AM, Morten Wang <nettrom(a)gmail.com> wrote:

I noticed this message from a thread on wikitech-l and wikibots-l: http://lists.wikimedia.org/pipermail/wikibots-l/2013-August/000353.html Does this have any particular implications for pywikibot, or is it already using HTTPS? Cheers, Morten _______________________________________________ Pywikipedia-l mailing list Pywikipedia-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

-- Amir

-- Amir

Reply

Merlijn van Deen

9:46 p.m.

On 20 August 2013 15:40, Amir Ladsgroup <ladsgroup(a)gmail.com> wrote:

after that bot operators just need to update

OK, this seems to be giving some problems. If you get ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed you need to update your httplib2 certificates: see http://stackoverflow.com/questions/13321302/python-foursquare-ssl3-certific… for instructions. I'm not sure how to fix this for us - should we just bundle wget's cacerts.pam? How would this work on windows? See http://pwnetics.wordpress.com/2012/02/06/ssl-certificate-verification-and-h… for a possible implementation. Merlijn

Reply

Merlijn van Deen

22 Aug 22 Aug

10:12 a.m.

I have submitted two revert patches, as the WMF is not switching until next week. This gives us some time to fix the certificate issue. Please check and +2. https://gerrit.wikimedia.org/r/80350 https://gerrit.wikimedia.org/r/80351 Merlijn On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

On 20 August 2013 15:40, Amir Ladsgroup <ladsgroup(a)gmail.com> wrote:

after that bot operators just need to update

OK, this seems to be giving some problems. If you get ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed you need to update your httplib2 certificates: see http://stackoverflow.com/questions/13321302/python-foursquare-ssl3-certific… for instructions. I'm not sure how to fix this for us - should we just bundle wget's cacerts.pam? How would this work on windows? See http://pwnetics.wordpress.com/2012/02/06/ssl-certificate-verification-and-h… for a possible implementation. Merlijn

Reply

Amir Ladsgroup

11:13 a.m.

Done On 8/22/13, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

I have submitted two revert patches, as the WMF is not switching until next week. This gives us some time to fix the certificate issue. Please check and +2. https://gerrit.wikimedia.org/r/80350 https://gerrit.wikimedia.org/r/80351 Merlijn On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

On 20 August 2013 15:40, Amir Ladsgroup <ladsgroup(a)gmail.com> wrote:

after that bot operators just need to update

OK, this seems to be giving some problems. If you get ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed you need to update your httplib2 certificates: see http://stackoverflow.com/questions/13321302/python-foursquare-ssl3-certific… for instructions. I'm not sure how to fix this for us - should we just bundle wget's cacerts.pam? How would this work on windows? See http://pwnetics.wordpress.com/2012/02/06/ssl-certificate-verification-and-h… for a possible implementation. Merlijn

-- Amir

Reply

Merlijn van Deen

24 Aug 24 Aug

7:28 p.m.

On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Small update on this: - compat doesn't do certificate checking at all, so the only thing https brings us is end-to-end encryption - but no authentication. - for core, there are some issues with the httplib2-as-externals that are easiest to fix by maintaining a port in git. This also means it's relatively easy to add WMF-relevant certificates to the cacerts.txt. I've requested a new repository in gerrit for this. Merlijn

Reply

Amir Ladsgroup

25 Aug 25 Aug

9:24 a.m.

Thank you :) On 8/24/13, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Small update on this: - compat doesn't do certificate checking at all, so the only thing https brings us is end-to-end encryption - but no authentication. - for core, there are some issues with the httplib2-as-externals that are easiest to fix by maintaining a port in git. This also means it's relatively easy to add WMF-relevant certificates to the cacerts.txt. I've requested a new repository in gerrit for this. Merlijn

-- Amir

Reply

Merlijn van Deen

29 Aug 29 Aug

8:10 a.m.

We have switched to HTTPS again! If you get the following error message: On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

the easiest option is to install httplib2 - either in a virtualenv or globally - from the externals/ directory. This directory contains an updated cacerts.txt, which included the root certificate required for editing on WMF servers. To do this: cd externals/httplib2 python setup.py install If you use pwb.py, you should not get an error message at all. If you /do/ get one, please let me know. Best, Merlijn

Reply

Morten Wang

3:08 p.m.

I updated SuggestBot's pywikibot version yesterday and everything seems to be working just fine. Thanks to everyone who helped out getting this taken care of, much appreciated! Cheers, Morten On 29 August 2013 03:10, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

We have switched to HTTPS again! If you get the following error message: On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

the easiest option is to install httplib2 - either in a virtualenv or globally - from the externals/ directory. This directory contains an updated cacerts.txt, which included the root certificate required for editing on WMF servers. To do this: cd externals/httplib2 python setup.py install If you use pwb.py, you should not get an error message at all. If you /do/ get one, please let me know. Best, Merlijn _______________________________________________ Pywikipedia-l mailing list Pywikipedia-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

Reply

Cheng Yu Tin

3 Sep 3 Sep

10:23 a.m.

I still get the error. Microsoft Windows XP [版本 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. D:\pywikipedia>pwb version Pywikibot: [https] r/pywikibot/core (r2012, 15287da, 2013/09/03, 07:07:35, ok) Release version: 2.0b1 Python: 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] unicode test: ok Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound m ethod ConnectionPool.__del__ of {}> ignored It seems that pwb.py cannot import httplib2 from externals directory. And after installed the httplib2, the error appears. I then try to use the pip one, same happened. And the error is a little different than the one said, it's {FatalServerError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SER VER_CERTIFICATE:certificate verify failed}, instead of SSLHandshakeError. The whole log is on http://pastebin.com/3JGLcJxZ . Yours, JustinC 於 29/8/2013 16:10, Merlijn van Deen 提到:

We have switched to HTTPS again! If you get the following error message: On 20 August 2013 23:46, Merlijn van Deen <valhallasw(a)arctus.nl <mailto:valhallasw@arctus.nl>> wrote: ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot\comms\http.py", line 116, in request raise request.data SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed the easiest option is to install httplib2 - either in a virtualenv or globally - from the externals/ directory. This directory contains an updated cacerts.txt, which included the root certificate required for editing on WMF servers. To do this: cd externals/httplib2 python setup.py install If you use pwb.py, you should not get an error message at all. If you /do/ get one, please let me know. Best, Merlijn _______________________________________________ Pywikipedia-l mailing list Pywikipedia-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

Reply

Merlijn van Deen

10:34 a.m.

On 3 September 2013 12:23, Cheng Yu Tin <justincheng12345(a)gmail.com> wrote:

It seems that pwb.py cannot import httplib2 from externals directory.

Possibly your httplib2 still refers to the old repository, which does not include fixes for this. Run git submodule sync git submodule update to re-sync to the new repository. It should be OK then.

And after installed the httplib2, the error appears. I then try to use the pip one, same happened. And the error is a little different than the one said, it's {FatalServerError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SER VER_CERTIFICATE:certificate verify failed}, instead of SSLHandshakeError.

Yes, this is correct. It's the same error, but a recent code addition made sure the code stops instead of retrying into infinity. You could either update the certificate manually ( http://stackoverflow.com/questions/13321302/python-foursquare-ssl3-certific…), but I would suggest first trying the sync-and-update trick. Merlijn

Reply

Antoine Musso

22 Aug 22 Aug

2:06 p.m.

Le 20/08/13 05:57, Morten Wang a écrit :

I noticed this message from a thread on wikitech-l and wikibots-l: http://lists.wikimedia.org/pipermail/wikibots-l/2013-August/000353.html Does this have any particular implications for pywikibot, or is it already using HTTPS?

You might want to pipeline requests to avoid having to renegotiate the certificate each time you want to do an action. -- Antoine "hashar" Musso

Reply

Merlijn van Deen

3:23 p.m.

On 22 August 2013 16:06, Antoine Musso <hashar+wmf(a)free.fr> wrote:

You might want to pipeline requests to avoid having to renegotiate the certificate each time you want to do an action.

Very good point. core keeps connections alive (and hangs when I kill the connection... that's not supposed to happen [1]). Compat creates a new connection per request. It might be possible to implement this in the U2RedirectHandler, but we could also consider this 'wontfix, upgrade to core'. [1] https://sourceforge.net/p/pywikipediabot/bugs/1657/ Merlijn

Reply

Amir Ladsgroup

4:31 p.m.

I disagree about won't fix option. I don't want to start a "compat vs. core" discussion but compat is not "the old version" of core and for me upgrading from compat is nonsense. compat is for compatibility and for some wikis and some people using core is impossible so we have to save compatibility Best On 8/22/13, Merlijn van Deen <valhallasw(a)arctus.nl> wrote:

On 22 August 2013 16:06, Antoine Musso <hashar+wmf(a)free.fr> wrote:

You might want to pipeline requests to avoid having to renegotiate the certificate each time you want to do an action.

Very good point. core keeps connections alive (and hangs when I kill the connection... that's not supposed to happen [1]). Compat creates a new connection per request. It might be possible to implement this in the U2RedirectHandler, but we could also consider this 'wontfix, upgrade to core'. [1] https://sourceforge.net/p/pywikipediabot/bugs/1657/ Merlijn

-- Amir

Reply

3899

days inactive

3913

days old

pywikipedia-l@lists.wikimedia.org

Manage subscription

14 comments

5 participants

tags (0)

participants (5)

Amir Ladsgroup
Antoine Musso
Cheng Yu Tin
Merlijn van Deen
Morten Wang