Hi Arthur,

I think the only good way to move forward is to schedule regular pairing sessions consisting of a person familiar with browser automation (Chris or me) and somebody that wants to learn more about it.

I pair with Niklas and Amir (added to cc) from language team twice weekly. Each pairing session lasts about two hours. (To make it explicit, I pair with them separately, not all three of us together in one session.) In just a week or two we made all language Jenkins jobs[1][2][3] almost rock solid. (We are still not there, but we are really really close.) We pair for 4 weeks or so and they have seriously improved their browser automation skills.

A pairing session consists of reviewing code that is waiting to get merged, checking if any of the Jenkins jobs failed (and fixing them if needed), refactoring old tests and writing new ones.

If you (yes, you!) want to pair with me (or Chris), let us know. Since I am in a time zone 9 hours ahead of you, if you want to pair with me, a good time would be as early in your day as possible. Since Chris is in your time zone, it would probably be easier for you to arrange pairing sessions with him, if he is available.

Another option would be to organize a workshop(s) that more people could attend, but I vote for pairing.

Željko

1: https://wmf.ci.cloudbees.com/view/r-tr/

2: https://wmf.ci.cloudbees.com/view/r-tw/

3: https://wmf.ci.cloudbees.com/view/r-uls/

On Thu, Nov 7, 2013 at 11:10 PM, Arthur Richards <arichards@wikimedia.org> wrote:

The 'Feature: Footer links resolve' test fails because the Terms of Use link on enwiki on betalabs points to m.wikimediafoundation.org - which does not exist. The link is correct in production. So either there is a configuration issue on betalabs, or there's been a code change that makes the footer link point to m.wikimediafoundation.org, which we should fix ASAP before it finds its way to enwiki.

The 'Feature: Footer links resolve' test fails, but when I follow the scenario manually, I see the tutorial. Does the test need updating?

Same thing with the 'Create Article.Create Article' test.

I fear the false positives/broken tests make it easy to not see real (or potentially real) issues and as people regularly see the test failure emails, they get ignored. What needs to be done to clean this up?

On Thu, Nov 7, 2013 at 2:49 PM, <jenkins-no-reply@cloudbees.com> wrote:

See <https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/142/changes>

Changes:

[cmcmahon] fix notification overlay automation test

[maxsem.wiki] Bug 56704: Allow mobileview api to handle section ranges

[jrobson] Make debug=true work again

[l10n-bot] Localisation updates from http://translatewiki.net.

------------------------------------------
Started by timer
Building remotely on s-c60f2019 in workspace <https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/ws/>
Cloning the remote Git repository
Cloning repository https://gerrit.wikimedia.org/r/mediawiki/extensions/MobileFrontend
git --version
git version 1.7.11.7
Checking out Revision 6babf4376db1bed8005169d496f6d94424855f9a (origin/master)
[MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox] $ /bin/sh -xe /tmp/hudson151044989842042353.sh
+ export MEDIAWIKI_USER=Selenium_user
+ MEDIAWIKI_USER=Selenium_user
+ export MEDIAWIKI_PASSWORD_VARIABLE=MEDIAWIKI_PASSWORD_SELENIUM_USER_WMFLABS_ORG
+ MEDIAWIKI_PASSWORD_VARIABLE=MEDIAWIKI_PASSWORD_SELENIUM_USER_WMFLABS_ORG
+ export BROWSER_LABEL=firefox
+ BROWSER_LABEL=firefox
+ export MEDIAWIKI_URL=http://en.m.wikipedia.beta.wmflabs.org/wiki/
+ MEDIAWIKI_URL=http://en.m.wikipedia.beta.wmflabs.org/wiki/
+ curl -s -o use-ruby https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/ruby/use-ruby
+ RUBY_VERSION=2.0.0-p247
+ source ./use-ruby
++ [[ -z 2.0.0-p247 ]]
++ ruby187='1.8.7-*'
++ [[ 2.0.0-p247 == 1.8.7-* ]]
++ BASE=/scratch/jenkins/addons
++ mkdir -p /scratch/jenkins/addons
++ '[' '!' -f /scratch/jenkins/addons/use.sh ']'
++ . /scratch/jenkins/addons/use.sh
++ mkdir -p /scratch/jenkins/addons/ruby
+++ addon_architecture
+++ uname -m
++ ruby_name=ruby-2.0.0-p247-x86_64
++ '[' '!' -f /scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64.tar.bz2 ']'
++ '[' '!' -f /scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin/ruby ']'
++ export RUBY_HOME=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64
++ RUBY_HOME=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64
++ export PATH=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin:/opt/jdk/latest/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/opt/maven/latest/bin:/opt/ant/latest/bin:/opt/android/android-sdk-linux/tools:/opt/gradle/latest/bin:/home/jenkins/bin:/opt/jdk/jdk.latest/bin
++ PATH=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin:/opt/jdk/latest/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/opt/maven/latest/bin:/opt/ant/latest/bin:/opt/android/android-sdk-linux/tools:/opt/gradle/latest/bin:/home/jenkins/bin:/opt/jdk/jdk.latest/bin
++ unset GEM_PATH
++ unset GEM_HOME
++ unset BUNDLE_PATH
+ gem install bundler --no-ri --no-rdoc
Successfully installed bundler-1.3.5
1 gem installed
+ '[' -d tests/browser/ ']'
+ cd tests/browser/
+ bundle install
Using builder (3.2.2)
Using ffi (1.9.3)
Using childprocess (0.3.9)
Using diff-lcs (1.2.4)
Using multi_json (1.8.2)
Using gherkin (2.12.2)
Using multi_test (0.0.2)
Using cucumber (1.3.8)
Using i18n (0.6.5)
Using faker (1.2.0)
Using yml_reader (0.2)
Using data_magic (0.16.1)
Using json (1.8.1)
Using net-http-persistent (2.9)
Using page_navigation (0.9)
Using rubyzip (1.0.0)
Using websocket (1.0.7)
Using selenium-webdriver (2.37.0)
Using watir-webdriver (0.6.4)
Using page-object (0.9.3)
Using mime-types (2.0)
Using rest-client (1.6.7)
Using rspec-expectations (2.14.3)
Using syntax (1.0.0)
Using mediawiki-selenium (0.1.12)
Using bundler (1.3.5)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
+ bundle exec cucumber --verbose --profile ci
Code:
* features/support/env.rb
* features/support/hooks.rb
* features/support/modules/url_module.rb
* features/support/pages/article_page.rb
* features/support/pages/beta_page.rb
* features/support/pages/create_article_page.rb
* features/support/pages/create_user_page.rb
* features/support/pages/edit_page.rb
* features/support/pages/home_page.rb
* features/support/pages/language_page.rb
* features/support/pages/login_page.rb
* features/support/pages/notification_page.rb
* features/support/pages/random_page.rb
* features/support/pages/search_page.rb
* features/support/pages/uploads_page.rb
* features/step_definitions/common_steps.rb
* features/step_definitions/create_article_steps.rb
* features/step_definitions/external_links_steps.rb
* features/step_definitions/footer_steps.rb
* features/step_definitions/header_logged_in_steps.rb
* features/step_definitions/header_steps.rb
* features/step_definitions/image_resolves_steps.rb
* features/step_definitions/language_steps.rb
* features/step_definitions/login_required_uploads_steps.rb
* features/step_definitions/login_required_watchlist_steps.rb
* features/step_definitions/menu_opens_page_steps.rb
* features/step_definitions/notification_steps.rb
* features/step_definitions/random_ua_steps.rb
* features/step_definitions/search_steps.rb
* features/step_definitions/uploads_steps.rb
* features/step_definitions/watchlist_steps.rb

Features:
* features/create_article.feature
* features/external_links.feature
* features/footer.feature
* features/footer_logged_in.feature
* features/header.feature
* features/header_logged_in.feature
* features/image_resolves.feature
* features/language.feature
* features/language_logged_in.feature
* features/login_required_uploads.feature
* features/login_required_watchlist.feature
* features/menu_opens_page.feature
* features/notification.feature
* features/random_ua.feature
* features/search.feature
* features/uploads.feature
* features/watchlist.feature
Parsing feature files took 0m0.050s

Build step 'Execute shell' marked build as failure
Recording test results

--
Arthur Richards
Software Engineer, Mobile
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687