Hi mobile folk,
After 48 patches merged, I have updated all the browser tests in the MobileFrontend repo to conform to RSpec3 syntax. Along the way I did a few other things:
* removed every sleep() statement except one necessary to get around a bug in Chrome * consolidated the lines within each step in each test to be as succinct as possible * handled and removed a number of FIXME comments * removed a significant amount of dead/unused/irrelevant code * made the Feature description of each test step consistent with what each step actually accomplishes * removed all the instances where "Then" steps were re-used as Given or When (they are conceptually different)
On the style front:
* all the steps are now in alphabetical order according to Given/When/Then, and all the GWT specifications in the .feature files conform to their corresponding implementations in steps files. * in the Features, every Then step contains the word "should", in the steps files, every Then step contains an RSpec assertion * no Given or When steps contain either the word "should" or an RSpec assertion
This all should make working in the browser test repo significantly easier and more straightforward, as well as making far better use of the most modern implementation of RSpec.
For my next trick I am going to make the browser test repo conform as closely as possible to rubocop style rules, but the heavy lifting with regard to technical debt in the browser test repos is mostly handled.
Let me know if you have any questions or if you would like a tour...
-Chris
Thanks for all the work Chris.
Not long ago I followed all instructions here https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Running_tes... but I can't get all the tests to pass so I'm not sure if there is something else to configure or enable, so I figured I'd ask before starting to look into the individual tests.
Here you can see vagrant roles, final output of the tests and env variables https://gist.github.com/joakin/eae3b086dd5a0c498669
I'm running the tests with phantomjs by default for convenience and speed, but I've tried Firefox and I've gotten similar results.
Anything I've missed that I should do? I'd really love to get them green to rely on them more often and avoid regressions.
On Thu, Dec 4, 2014 at 11:43 PM, Chris McMahon cmcmahon@wikimedia.org wrote:
Hi mobile folk,
After 48 patches merged, I have updated all the browser tests in the MobileFrontend repo to conform to RSpec3 syntax. Along the way I did a few other things:
- removed every sleep() statement except one necessary to get around a bug
in Chrome
- consolidated the lines within each step in each test to be as succinct
as possible
- handled and removed a number of FIXME comments
- removed a significant amount of dead/unused/irrelevant code
- made the Feature description of each test step consistent with what each
step actually accomplishes
- removed all the instances where "Then" steps were re-used as Given or
When (they are conceptually different)
On the style front:
- all the steps are now in alphabetical order according to
Given/When/Then, and all the GWT specifications in the .feature files conform to their corresponding implementations in steps files.
- in the Features, every Then step contains the word "should", in the
steps files, every Then step contains an RSpec assertion
- no Given or When steps contain either the word "should" or an RSpec
assertion
This all should make working in the browser test repo significantly easier and more straightforward, as well as making far better use of the most modern implementation of RSpec.
For my next trick I am going to make the browser test repo conform as closely as possible to rubocop style rules, but the heavy lifting with regard to technical debt in the browser test repos is mostly handled.
Let me know if you have any questions or if you would like a tour...
-Chris
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On Fri, Dec 5, 2014 at 2:23 AM, Joaquin Oltra Hernandez < jhernandez@wikimedia.org> wrote:
Here you can see vagrant roles, final output of the tests and env variables https://gist.github.com/joakin/eae3b086dd5a0c498669
I'm running the tests with phantomjs by default for convenience and speed, but I've tried Firefox and I've gotten similar results.
Anything I've missed that I should do? I'd really love to get them green to rely on them more often and avoid regressions.
Sure, I see a few issues right off, even without looking at how these failed.
* Uploads has been disabled in the MobileFrontend for a long time. That feature may or may not be re-enabled in the future. That test is simply not valid, but I have kept it in the repo at the request of the Mobile team. You can see in the tags in special_upload.feature and uploads_copyvio.feature that those tests have no target environments and do not run in the build.
For the rest, these tests have been designed to work primarily in beta labs. If you set your env vars to
export MEDIAWIKI_URL=http://en.m.wikipedia.beta.wmflabs.org/wiki/
export MEDIAWIKI_API_URL=http://en.wikipedia.beta.wmflabs.org/w/api.php
export MEDIAWIKI_USER=Selenium_user
export MEDIAWIKI_PASSWORD=(password)
export BROWSER=firefox or equivalent, I bet the suite would pass (except for the uploads tests).
* Do you have the API URL set correctly? That would be a source of failure. * Do you have VisualEditor in place? If not, the VE tests will fail. * The language tests rely on having the page available in another language wiki, and the db records to show it. * Do you have Nearby enabled? Search? etc?
Each test failure will display a detailed message describing what condition the test expected and what condition violated that expectation. I think if you look at the detailed failure message it will tell you exactly why the test failed in your environment.
-Chris
On Fri, Dec 5, 2014 at 8:17 AM, Chris McMahon cmcmahon@wikimedia.org wrote:
On Fri, Dec 5, 2014 at 2:23 AM, Joaquin Oltra Hernandez < jhernandez@wikimedia.org> wrote:
Here you can see vagrant roles, final output of the tests and env variables https://gist.github.com/joakin/eae3b086dd5a0c498669
I was thinking some more about this. Try running the tests in Firefox, with enviroment variables set properly, using these tags:
bundle exec cucumber --tags @firefox --tags @en.m.wikipedia.beta.wmflabs.org
and see what the results are. If you have any failures let us know.
-Chris