On Fri, May 29, 2015 at 2:09 AM, Brian Wolff bawolff@gmail.com wrote:
Shouldnt such [pywikibot] tests be run against beta wiki not testwiki?
Primarily this hasnt happened because pywikibot doesnt have a family file for the beta wiki, but that is our issue, so I've done a simple test run on beta wiki
four initial beta wiki problems:
- no https
(not nice - that means test accounts must be created and accessed using passwords that are sent in essentially cleartext - so sharing passwords with the same account name on the real wikis is a security risk)
- invalid siteinfo interwiki map; includes entries to wikis that do not exist. That means pywikibot cant parse wikitext links, as it cant distinguish between iwprefix: and namespace: (pywikibot team might be able to reduce the impact of this wiki config problem)
- paramInfo failure that breaks pywikibot's dynamic api detection; pywikibot batch loads the paraminfo for all modules, for example issuing the following API as part of the initialisation, and it is a 503 Service Unavailable http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
A little digging shows that the problematic module is fancycaptchareload
http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
"Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes.
If you report this error to the Wikimedia System Administrators, please include the details below. Request: GET http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=..., from 127.0.0.1 via deployment-cache-text02 deployment-cache-text02 ([127.0.0.1]:3128), Varnish XID 855090368 Forwarded for: [your IP], 127.0.0.1 Error: 503, Service Unavailable at Fri, 29 May 2015 07:57:09 GMT "
Could someone investigate this / fix that module?
Pywikibot devs could also help reduce the impact of this type of problem in the future, by falling back from batch mode fetch to loading individual modules in order to skip buggy modules.
- no SUL with the real wikis
(probably the best choice given no https on the beta cluster, but it complicates adding beta wiki to our existing Travis-CI test matrix which includes real wikis)
actual results at https://travis-ci.org/jayvdb/pywikibot-core/builds/64532033
-- John Vandenberg
On Fri, May 29, 2015 at 3:14 PM, John Mark Vandenberg jayvdb@gmail.com wrote:
..
- paramInfo failure that breaks pywikibot's dynamic api detection;
pywikibot batch loads the paraminfo for all modules, for example issuing the following API as part of the initialisation, and it is a 503 Service Unavailable http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
A little digging shows that the problematic module is fancycaptchareload
http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
"Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes.
If you report this error to the Wikimedia System Administrators, please include the details below. Request: GET http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=..., from 127.0.0.1 via deployment-cache-text02 deployment-cache-text02 ([127.0.0.1]:3128), Varnish XID 855090368 Forwarded for: [your IP], 127.0.0.1 Error: 503, Service Unavailable at Fri, 29 May 2015 07:57:09 GMT "
Could someone investigate this / fix that module?
After pywikibot devs cleared some other test breakages (our problems) we see this problem is also affecting the test wikis, which I assume means this bug will be affecting real wikis soon, so I've created an Unbreak Now! ticket for it and cc'd @greg.
https://phabricator.wikimedia.org/T100775
On Fri, May 29, 2015 at 7:01 AM, John Mark Vandenberg jayvdb@gmail.com wrote:
On Fri, May 29, 2015 at 3:14 PM, John Mark Vandenberg jayvdb@gmail.com wrote:
..
- paramInfo failure that breaks pywikibot's dynamic api detection;
pywikibot batch loads the paraminfo for all modules, for example issuing the following API as part of the initialisation, and it is a 503 Service Unavailable
http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
A little digging shows that the problematic module is fancycaptchareload
http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=...
"Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes.
If you report this error to the Wikimedia System Administrators, please include the details below. Request: GET
http://en.wikipedia.beta.wmflabs.org/w/api.php?action=paraminfo&modules=... ,
from 127.0.0.1 via deployment-cache-text02 deployment-cache-text02 ([127.0.0.1]:3128), Varnish XID 855090368 Forwarded for: [your IP], 127.0.0.1 Error: 503, Service Unavailable at Fri, 29 May 2015 07:57:09 GMT "
Could someone investigate this / fix that module?
After pywikibot devs cleared some other test breakages (our problems) we see this problem is also affecting the test wikis, which I assume means this bug will be affecting real wikis soon, so I've created an Unbreak Now! ticket for it and cc'd @greg.
https://gerrit.wikimedia.org/r/#/c/214610/
I'll very likely go ahead and backport it to 1.26wmf8 as soon as it's merged and I confirm Beta is fixed.
On 29 May 2015 at 09:14, John Mark Vandenberg jayvdb@gmail.com wrote:
- no https
(not nice - that means test accounts must be created and accessed using passwords that are sent in essentially cleartext - so sharing passwords with the same account name on the real wikis is a security risk)
It's risky anyway, do you know who has access to the beta cluster? It's not considered secure and you do not need any NDA or anything to get access - if you are using a real password on beta, change it. It's in labs. https://phabricator.wikimedia.org/T50501 is about HTTPS on beta.
- no SUL with the real wikis
(probably the best choice given no https on the beta cluster, but it complicates adding beta wiki to our existing Travis-CI test matrix which includes real wikis)
Beta cluster will never get access to CentralAuth passwords in production. Maybe via OpenID or something, but not proper SUL with production wikis.
On May 29, 2015 09:10, "Alex Monk" krenair@gmail.com wrote:
On 29 May 2015 at 09:14, John Mark Vandenberg jayvdb@gmail.com wrote: It's risky anyway, do you know who has access to the beta cluster? It's
not
considered secure and you do not need any NDA or anything to get access - if you are using a real password on beta, change it. It's in labs.
+1. we don't really need HTTPS on beta to secure beta. Rogue sysops and other vandals aren't the end of the world. rather we need HTTPS so that we can do tests with HTTPS.
Don't share passwords between beta and any other site unless you also don't care about the other site. (e.g. you could share if both passwords are so weak as abcde12345)
-Jeremy
On Fri, May 29, 2015 at 3:14 PM, John Mark Vandenberg jayvdb@gmail.com wrote:
On Fri, May 29, 2015 at 2:09 AM, Brian Wolff bawolff@gmail.com wrote:
Shouldnt such [pywikibot] tests be run against beta wiki not testwiki?
Primarily this hasnt happened because pywikibot doesnt have a family file for the beta wiki, but that is our issue, so I've done a simple test run on beta wiki
I've create a task for this little project:
https://phabricator.wikimedia.org/T100796
wikitech-l@lists.wikimedia.org