[QA] [WikimediaMobile] Qualifiers for selecting test articles for vagrant role

[Jon Robson]

On Wed, Jul 22, 2015 at 9:58 AM, Dan Duvall <dduvall at wikimedia.org> wrote:
> I like this idea for development and manual testing, but I'm not sure it's
> appropriate for automated testing.
>
Agreed. I was thinking more for manual testing.

> Generally in automated testing, each test case (unit test or integration
> scenario) should be able to set up or guarantee its preconditions whenever
> possible—and otherwise communicate its assumptions—which in the context of
> MediaWiki includes user accounts, user settings, or article content that the
> test case will manipulate or otherwise depend on. Putting this initial
> content in MW-Vagrant, which is orthogonal to MW test suites, creates the
> need for more coupling between test code and test environments and promotes
> more non-deterministic test behavior, something we've been trying very hard
> to reduce.[1][2]
>
> Also, for development content, we might want to figure out a better place
> for it than in the MW-Vagrant repo itself. Article dumps are likely to build
> up fast, and we risk bloating the project repo itself with big blobs.
>
> [1]
> https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Environment_abstraction_layer
> [2]
> https://doc.wikimedia.org/rubygems/mediawiki-selenium/index.html#User_Factory
>
>
> On Tue, Jul 21, 2015 at 10:25 AM, Jon Robson <jdlrobson at gmail.com> wrote:
>>
>> On 20 Jul 2015 5:56 pm, "Greg Grossmeier" <greg at wikimedia.org> wrote:
>>>
>>> Given the topic, let's keep the QA list in the loop on this so the
>>> MW-Vagrant maintainers can participate/see.
>>
>> Great :)
>>>
>>>
>>> Also, it looks like the original bug (reported in the MW-Vagrant
>>> project) covers this specific request from Reading, no? Essentially,
>>>
>>> let's see how far we can get with a general "MW-Vagrant (WMF?) testing
>>> data import" instead of a vertical specific "reading-web test data set".
>>> If what the Reading team needs is way too much for this then we can
>>> break it out, otherwise it seems like a needless distinction.
>>
>>
>>
>> It does yup. I've already tagged the bug with it. I'm hoping by tackling
>> this we can come up with a common solution. The way I imagine this working
>> in future is we have various vagrant roles for stock data e.g.
>> reading-web-stock-data, editing-web-stock-data, sad-web-stock-data
>> There would also be non team specific stock data that might be a sub role
>> of this, for example, the reading web team commonly has to setup the
>> wikidata role and manually create articles in the wikidata instance and
>> local instance that are tied to each other - this takes a ridiculous amount
>> of time and is one I'm keen to automate, given that we are leaning more
>> heavily on wikidata descriptions and other data in there.
>>
>> Rob - I've setup https://www.mediawiki.org/wiki/Reading/QA/Sample_articles
>> as a place we can start to collect and think about these pages.
>>
>>>
>>> Greg
>>>
>>> PS: https://wikitech.wikimedia.org/wiki/Labs_labs_labs
>>>
>>> <quote name="Rob Moen" date="2015-07-20" time="17:11:07 -0700">
>>> > Historically developers have had to setup their own content in
>>> > mediawiki
>>> > and in mediawiki-vagrant.  While this can be done with a simple import,
>>> > getting everyone on the same page is apparently not as easy.  This is
>>> > generally problematic as we would like to test code locally and
>>> > remotely
>>> > with the same content for various reasons.
>>> >
>>> > Slightly more frustrating, there are pages titled "0.4425590476103759"
>>> > on
>>> > beta labs.  While trying to sign off on a feature, there is usually a
>>> > struggle when trying to find an article with suitable content.  AFAIK
>>> > this
>>> > won't change beta labs but would provide a nice standard for our
>>> > content on
>>> > test wikis.
>>> >
>>> > We aim to better things by creating a vagrant role for importing a set
>>> > of
>>> > articles for testing purposes.  For more information please see related
>>> > phabricator tasks [1] and [2].
>>> >
>>> > In hopes of making this a nice collection of articles that multiple
>>> > teams
>>> > would use, we would like to get input from our designers and devs on
>>> > what
>>> > types of articles should be in this import.  What qualities should
>>> > these
>>> > articles contain?
>>> >
>>> > 1: https://phabricator.wikimedia.org/T104561
>>> > 2: https://phabricator.wikimedia.org/T62116
>>> >
>>> >
>>> >
>>> > --
>>> > Rob Moen
>>> > Wikimedia Foundation
>>> > rmoen at wikimedia.org
>>>
>>> > _______________________________________________
>>> > Mobile-l mailing list
>>> > Mobile-l at lists.wikimedia.org
>>> > https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>
>>>
>>> --
>>> | Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
>>> | identi.ca: @greg                A18D 1138 8E47 FAC8 1C7D |
>>>
>>> _______________________________________________
>>> QA mailing list
>>> QA at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/qa
>>
>>
>> _______________________________________________
>> Mobile-l mailing list
>> Mobile-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>
>
>
> --
> Dan Duvall
> Automation Engineer
> Wikimedia Foundation



-- 
Jon Robson
* http://jonrobson.me.uk
* https://www.facebook.com/jonrobson
* @rakugojon