I'm running MediaWiki 1.29.1 on FreeBSD 10.3-RELEASE running mysql 5.6.38, php 7.1.10 and python 2.7.14. The wiki is working fine.
I installed pywikibot using git clone. The ID is $Id: 03e22e8ea7bf53dc7c9d47b6d7ad98c3c7afabb4.
When I try to generate the family file, I get this error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wiki.vvfh.org', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
It's strange, because the certs verify from the commandline and SSLLabs says the site rates an A and the entire chain is found. I'm not sure why python is failing the verification test.
I've been googling and trying to find a way to disable ssl verification, but I haven't found it. The software is installed on the server the wiki runs on, and I'm running it from the commandline, so not verifying SSL isn't a big issue. (Only I have access to the server.)
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
It seems requests is having trouble connecting to the website. See if connecting using requests directly from the interpreter prompt works or not:
import requests requests.get('https://wiki.vvfh.org/')
<Response [200]>
I would also try installing/upgrading requests[security]: $pip install -U requests[security]
On Wed, Nov 15, 2017 at 6:58 AM, Paul Schmehl pschmehl@tx.rr.com wrote:
I'm running MediaWiki 1.29.1 on FreeBSD 10.3-RELEASE running mysql 5.6.38, php 7.1.10 and python 2.7.14. The wiki is working fine.
I installed pywikibot using git clone. The ID is $Id: 03e22e8ea7bf53dc7c9d47b6d7ad98c3c7afabb4.
When I try to generate the family file, I get this error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wiki .vvfh.org', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
It's strange, because the certs verify from the commandline and SSLLabs says the site rates an A and the entire chain is found. I'm not sure why python is failing the verification test.
I've been googling and trying to find a way to disable ssl verification, but I haven't found it. The software is installed on the server the wiki runs on, and I'm running it from the commandline, so not verifying SSL isn't a big issue. (Only I have access to the server.)
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Thanks.
I ran requests from the the interpreter prompt and got the same error.
# python Python 2.7.14 (default, Oct 23 2017, 13:50:22) [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information.
import requests requests.get('https://wiki.vvfh.org')
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 72, in get return request('get', url, params=params, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 508, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 618, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 506, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='wiki.vvfh.org', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
I ran pip install -U requests[security] as you suggested, and got the same result.
Then I installed python3.5 and pip3 and tried again. Same result.
pywikibot.exceptions.FatalServerError: HTTPSConnectionPool(host='wiki.vvfh.org', port=443): Max retries exceeded with url: /w/api.php (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
I even added this to user-config.py, but still got the same result.
def ignore_certificate_error(self, code): return True
You will also notice that the url is /w/api.php. This is despite defining scriptpath in the family file.
def scriptpath(self, code): return '/'
--On November 15, 2017 at 8:04:30 AM +0330 Dalba dalba.wiki@gmail.com wrote:
It seems requests is having trouble connecting to the website. See if connecting using requests directly from the interpreter prompt works or not:
import requests requests.get('https://wiki.vvfh.org/')
<Response [200]>
I would also try installing/upgrading requests[security]: $pip install -U requests[security]
On Wed, Nov 15, 2017 at 6:58 AM, Paul Schmehl pschmehl@tx.rr.com wrote:
I'm running MediaWiki 1.29.1 on FreeBSD 10.3-RELEASE running mysql 5.6.38, php 7.1.10 and python 2.7.14. The wiki is working fine.
I installed pywikibot using git clone. The ID is $Id: 03e22e8ea7bf53dc7c9d47b6d7ad98c3c7afabb4.
When I try to generate the family file, I get this error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wiki.vvfh.org', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
It's strange, because the certs verify from the commandline and SSLLabs says the site rates an A and the entire chain is found. I'm not sure why python is failing the verification test.
I've been googling and trying to find a way to disable ssl verification, but I haven't found it. The software is installed on the server the wiki runs on, and I'm running it from the commandline, so not verifying SSL isn't a big issue. (Only I have access to the server.)
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
I even added this to user-config.py, but still got the same result.
def ignore_certificate_error(self, code): return True
I think ignore_certificate_error should be defined inside the family file and has no effect in user-config.py. As a workaround you can modify the ignore_certificate_error in pywikibot/family.py to make it return True. Then generate your family file using the generator script (hopefully there won't be any error now). After that you can revert the change in family.py and define ignore_certificate_error in the newly generated file.
I screwed up. I meant to type family file and typed user-config.py instead.
I changed the ignore_certificate_error return value to true in pywikibot/family.py, but it still chokes on ssl verify. That surprised me. It's like the code is being ignored.
--On November 15, 2017 at 10:19:26 AM +0330 Dalba dalba.wiki@gmail.com wrote:
I even added this to user-config.py, but still got the same result.
def ignore_certificate_error(self, code): return True
I think ignore_certificate_error should be defined inside the family file and has no effect in user-config.py. As a workaround you can modify the ignore_certificate_error in pywikibot/family.py to make it return True. Then generate your family file using the generator script (hopefully there won't be any error now). After that you can revert the change in family.py and define ignore_certificate_error in the newly generated file.
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
OK, we probably should add an option to generate_family_file.py for ignoring SSL errors. But for now try using this pregenerated file: https://phabricator.wikimedia.org/F10802604 (it should be placed at pywikibot/families directory)
On Wed, Nov 15, 2017 at 10:52 AM, Paul Schmehl pschmehl@tx.rr.com wrote:
I screwed up. I meant to type family file and typed user-config.py instead.
I changed the ignore_certificate_error return value to true in pywikibot/family.py, but it still chokes on ssl verify. That surprised me. It's like the code is being ignored.
--On November 15, 2017 at 10:19:26 AM +0330 Dalba dalba.wiki@gmail.com wrote:
I even added this to user-config.py, but still got the same result.
def ignore_certificate_error(self, code): return True
I think ignore_certificate_error should be defined inside the family file and has no effect in user-config.py. As a workaround you can modify the ignore_certificate_error in pywikibot/family.py to make it return True. Then generate your family file using the generator script (hopefully there won't be any error now). After that you can revert the change in family.py and define ignore_certificate_error in the newly generated file.
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Thank you for that. That got me a step farther. I was able to auto-generate the user-config.py file.
When I try to login, I get a 404 error. (python pwb.py login)
I'll post the log message (I used debug) in a second, but here's the url that pywikibot is using:
If you load that in your browser, you'll see that it displays json data, so I don't understand why I'm getting the 404 message.
Here's the log entry:
2017-11-15 10:35:23 bot.py, 977 in handle_args: DEBUG handle_args() completed. 2017-11-15 10:35:23 __init__.py, 1252 in Site: DEBUG Instantiated APISite object 'vvfh:en' 2017-11-15 10:35:23 site.py, 1614 in _get_general: DEBUG general siteinfo not loaded yet. 2017-11-15 10:35:23 site.py, 1625 in _get_general: DEBUG Load siteinfo properties 'namespaces', 'namespacealiases' along with 'general' 2017-11-15 10:35:23 throttle.py, 83 in checkMultiplicity: DEBUG Checking multiplicity: pid = False 2017-11-15 10:35:23 throttle.py, 140 in checkMultiplicity: VERBOSE Found 1 vvfh:en processes running, including this one. 2017-11-15 10:35:23 api.py, 1948 in submit: DEBUG API request to vvfh:en (uses get: False): Headers: {'Content-Type': 'application/x-www-form-urlencoded'} URI: '/api.php' Body: 'maxlag=5&meta=siteinfo%7Cuserinfo&uiprop=blockinfo%7Chasmsg&continue=&format=json&action=query&siprop=namespaces%7Cnamespacealiases%7Cgeneral' 2017-11-15 10:35:23 /usr/local/lib/python3.5/site-packages/urllib3/connectionpool.py, 858 in connectionpool: WARNING /usr/local/lib/python3.5/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings 2017-11-15 10:35:23 http.py, 409 in error_handling_callback: WARNING Http response status 404 2017-11-15 10:35:23 threadedhttp.py, 139 in encoding: VERBOSE Http response doesn't contain a charset. 2017-11-15 10:35:23 api.py, 1981 in submit: DEBUG API response received from vvfh:en: <!doctype html> <!--[if lt IE 7]> <html lang="en-us" class="no-js ie6"> <![endif]--> <!--[if IE 7]> <html lang="en-us" class="no-js ie7"> <![endif]--> <!--[if IE 8]> <html lang="en-us" class="no-js ie8"> <![endif]--> <!--[if IE 9]> <html lang="en-us" class="no-js ie9"> <![endif]--> <!--[if gt IE 9]><!--> <html lang="en-us" class="no-js"> <!--<![endif]--> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=Edge;chrome=1" /> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="copyright" content="Copyright 2012 Ubiquiti Networks, Inc. All rights reserved.">
<title>EdgeOS - 404 Error</title>
<link rel="shortcut icon" href="/media/global/favicon.png" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/reset.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/fonts/open-sans/stylesheet.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/core.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/edge.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/fileuploader.min.css" type="text/css" media="screen" />
<script type="text/javascript" src="/lib/1434480045/js/libs/modernizr.custom.js"></script> </head> <body>
<!-- Start: Page Content --> <div id="PageContent">
<!-- Start: Header --> <div id="PageHeader" class="page-header"> <div class="ir edgeos-header-logo">EdgeOS</div> </div> <!-- End: Header -->
<!-- Start: Main Panel --> <div id="MainPanel" class="main-panel">
<!-- Start: Content --> <div class="content-outer-container"> <div class="content-inner-container"> <div class="content"> <div id="Error" class="main-section"> <h1 class="section-title">Error: Not Found</h1>
<div class="section-container tall basic">
<div id="StatusCode">
404 </div>
</div> </div> </div> </div> </div> </div> </div> <!-- End: Page Content --> </body> </html> 2017-11-15 10:35:23 api.py, 1991 in submit: WARNING Non-JSON response received from server vvfh:en; the server may be down. 2017-11-15 10:35:23 api.py, 2207 in wait: WARNING Waiting 5 seconds before retrying.
I don't understand why I'm getting a 404 when the same query, loaded in a browser, returns the json-formatted data requested.
Do I need to be using a -p and typing in the password?
--On November 15, 2017 at 11:53:55 AM +0330 Dalba dalba.wiki@gmail.com wrote:
OK, we probably should add an option to generate_family_file.py for ignoring SSL errors. But for now try using this pregenerated file: https://phabricator.wikimedia.org/F10802604 (it should be placed at pywikibot/families directory)
On Wed, Nov 15, 2017 at 10:52 AM, Paul Schmehl pschmehl@tx.rr.com wrote:
I screwed up. I meant to type family file and typed user-config.py instead.
I changed the ignore_certificate_error return value to true in pywikibot/family.py, but it still chokes on ssl verify. That surprised me. It's like the code is being ignored.
--On November 15, 2017 at 10:19:26 AM +0330 Dalba dalba.wiki@gmail.com wrote:
I even added this to user-config.py, but still got the same result.
def ignore_certificate_error(self, code): return True
I think ignore_certificate_error should be defined inside the family file and has no effect in user-config.py. As a workaround you can modify the ignore_certificate_error in pywikibot/family.py to make it return True. Then generate your family file using the generator script (hopefully there won't be any error now). After that you can revert the change in family.py and define ignore_certificate_error in the newly generated file.
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
"The man who never looks into a newspaper is better informed than he who reads them, inasmuch as he who knows nothing is nearer the truth than he whose mind is filled with falsehoods and errors." - Thomas Jefferson
Paul Schmehl (pschmehl@tx.rr.com) Independent Researcher
2017-11-16 8:26 GMT+02:00 Paul Schmehl pschmehl@tx.rr.com:
Thank you for that. That got me a step farther. I was able to auto-generate the user-config.py file.
When I try to login, I get a 404 error. (python pwb.py login)
I'll post the log message (I used debug) in a second, but here's the url that pywikibot is using:
If you load that in your browser, you'll see that it displays json data, so I don't understand why I'm getting the 404 message.
Here's the log entry:
2017-11-15 10:35:23 bot.py, 977 in handle_args: DEBUG handle_args() completed. 2017-11-15 10:35:23 __init__.py, 1252 in Site: DEBUG Instantiated APISite object 'vvfh:en' 2017-11-15 10:35:23 site.py, 1614 in _get_general: DEBUG general siteinfo not loaded yet. 2017-11-15 10:35:23 site.py, 1625 in _get_general: DEBUG Load siteinfo properties 'namespaces', 'namespacealiases' along with 'general' 2017-11-15 10:35:23 throttle.py, 83 in checkMultiplicity: DEBUG Checking multiplicity: pid = False 2017-11-15 10:35:23 throttle.py, 140 in checkMultiplicity: VERBOSE Found 1 vvfh:en processes running, including this one. 2017-11-15 10:35:23 api.py, 1948 in submit: DEBUG API request to vvfh:en (uses get: False): Headers: {'Content-Type': 'application/x-www-form-urlencoded'} URI: '/api.php' Body: 'maxlag=5&meta=siteinfo%7Cuserinfo&uiprop=blockinfo%7Chasmsg&continue=&format=json&action=query&siprop=namespaces%7Cnamespacealiases%7Cgeneral' 2017-11-15 10:35:23 /usr/local/lib/python3.5/site-packages/urllib3/connectionpool.py, 858 in connectionpool: WARNING /usr/local/lib/python3.5/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings 2017-11-15 10:35:23 http.py, 409 in error_handling_callback: WARNING Http response status 404 2017-11-15 10:35:23 threadedhttp.py, 139 in encoding: VERBOSE Http response doesn't contain a charset. 2017-11-15 10:35:23 api.py, 1981 in submit: DEBUG API response received from vvfh:en:
<!doctype html>
<!--[if lt IE 7]> <html lang="en-us" class="no-js ie6"> <![endif]-->
<!--[if IE 7]> <html lang="en-us" class="no-js ie7"> <![endif]-->
<!--[if IE 8]> <html lang="en-us" class="no-js ie8"> <![endif]-->
<!--[if IE 9]> <html lang="en-us" class="no-js ie9"> <![endif]-->
<!--[if gt IE 9]><!--> <html lang="en-us" class="no-js"> <!--<![endif]-->
<head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=Edge;chrome=1"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="copyright" content="Copyright 2012 Ubiquiti Networks, Inc. All rights reserved.">
<title>EdgeOS - 404 Error</title> <link rel="shortcut icon" href="/media/global/favicon.png" /> <link rel="stylesheet"href="/lib/1434480045/stylesheets/css/reset.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/fonts/open-sans/stylesheet.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/core.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/edge.min.css" type="text/css" media="screen" /> <link rel="stylesheet" href="/lib/1434480045/stylesheets/css/fileuploader.min.css" type="text/css" media="screen" />
<script type="text/javascript"src="/lib/1434480045/js/libs/modernizr.custom.js"></script> </head> <body>
<!-- Start: Page Content --> <div id="PageContent"> <!-- Start: Header --> <div id="PageHeader" class="page-header"> <div class="iredgeos-header-logo">EdgeOS</div> </div> <!-- End: Header -->
<!-- Start: Main Panel --> <div id="MainPanel" class="main-panel"> <!-- Start: Content --> <div class="content-outer-container"> <div class="content-inner-container"> <div class="content"> <div id="Error"class="main-section"> <h1 class="section-title">Error: Not Found</h1>
<divclass="section-container tall basic">
<div id="StatusCode">
404 </div>
</div> </div> </div> </div> </div> </div> </div> <!-- End: Page Content --> </body> </html>
That HTML looks suspicious, since it seems like an Ubiquity router error. Are you sure you're actually getting to the server and not being blocked/filtered by your WiFi?
Also 404 errors usually leave log messages on the server, you might want to checke those as well for any clues.
Best regards, Strainu