Dan, Jon,

I got caught up in meetings yesterday – you'll see this email a lot during Q4 ;) – so I delayed sending this email, so forgive the repetitions of some of Dan's points/questions:

Here are a few ways I can think of:
  • include feedback on browser tests – or lack thereof – during code review
  • make browser test failures even more visible than they currently are – but maybe not the success reports, eh?
  • can these reports be made to point at a bunch of candidate changes that may have broken 'em?
  • hold a browser-test-athon with the team and any volunteers at the {Lyon,Wikimania} hackathon
  • make it trivial to run 'em, if it isn't already
From what little experience I have of trying to establish team practices, I'd say that it's best to advocate for <practice> and demonstrate its value*, rather than criticise. I'd love to see you funnel your passion for browser testing into a talk or series of talks for the mobile team – the org, maybe? – or maybe you've got some recommended reading or talks you'd like to share that'll inspire.

–Sam

* If you'd like to hear my opinions about browser testing, then insert one beer and wind me up a little


On Mon, Mar 30, 2015 at 8:47 PM, Dan Duvall <dduvall@wikimedia.org> wrote:
https://phabricator.wikimedia.org/T94472

On Mon, Mar 30, 2015 at 12:39 PM, Dan Duvall <dduvall@wikimedia.org> wrote:
> On Mon, Mar 30, 2015 at 10:30 AM, Jon Robson <jdlrobson@gmail.com> wrote:
>> It really saddens me how very few engineers seem to care about browser
>> tests. Our browser tests are failing all over the place. I just saw this bug
>> [1] which has been sitting around for ages and denying us green tests in
>> Echo one of our most important features.
>>
>> How can we change this anti-pattern?
>
> That's exactly what I'd like to explore with you and other like minds.
>
>> Dan Duval, would it make sense to do a survey as you did with Vagrant to
>> understand how our developers think of these? Such as who owns them... who
>> is responsible for a test failing... who writes them... who doesn't
>> understand them.. why they don't understand them etc...?
>
> Great idea! I suspect that the number of false positives in a given
> repo's test suite is inversely related to the number of developers on
> the team actually writing tests, and the affordance by managers to do
> so. If you're not regularly writing tests, you're probably not going
> to feel comfortable troubleshooting and refactoring someone else's. If
> TDD isn't factored in to your team's velocity, you may feel like the
> investment in writing tests (or learning to write them) isn't worth it
> or comes at the risk of missing deadlines.
>
> A survey could definitely help us to verify (or disprove) these relationships.
>
> Some other questions I can think of:
>
>  - How valuable are unit tests to the health/quality of a software project?
>  - How valuable are browser tests to the health/quality of a software project?
>  - How much experience do you have with TDD?
>  - Would you like more time to learn or practice TDD?
>  - How often do you write tests when developing a new feature?
>    - What kinds of test? (% of unit test vs. browser test)
>  - How often do you write tests to verify a bugfix?
>    - What kinds of test? (% of unit test vs. browser test)
>  - When would you typically write a unit test? (before implementation,
> after implementation, when stuff breaks)
>  - When would you typically write a browser test? (during conception,
> before implementation, after implementation, when stuff breaks)
>  - What are the largest barriers to writing/running unit tests? (test
> framework, documentation/examples, execution time, CI, structure of my
> code, structure of code I depend on)
>  - What are the largest barriers to writing/running browser tests?
> (test framework, documentation/examples, execution time, CI)
>  - What are the largest barriers to debugging test failure? (test
> framework, confusing errors/stack traces, documentation/examples,
> debugging tools)
>
> I'll create a Phab task to track it. :)
>
> --
> Dan Duvall
> Automation Engineer
> Wikimedia Foundation



--
Dan Duvall
Automation Engineer
Wikimedia Foundation

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l