[QA] bugzilla and needs-test

[Chris McMahon]

A couple of thoughts on this:

* This might be a topic better suited for the "teampractices" mail list
* Julisz Gonera is giving a presentation on Test Driven Development later
today that will touch on some of this.
* We have here conflated whole worlds of dev processes into a couple of
Bugzilla tags without any context.

We can write a UI/browser regression test at any point before or after the
feature being tested is available.

But that "need-unittest" for example is kind of odd.  Unit tests should
ideally be written by the developers working on the code at the time that
the code is being written.  Writing unit tests after the fact is not a very
good practice.   People who actually attempt to write unit tests after the
fact will usually end up having to spend extensive time understanding large
chunks of code in order to refactor/rewrite them from scratch in order to
make the code testable.   Given that the cultures of neither PHP nor
Javascript have historically had much support for testable code, this can
be an enormous challenge.  A tag "need-unittest" should be renamed "code
needs refactoring/rewrite"-- it's the same thing.  One does not simply walk
into unit testing.

I think this is a "teampractices" discussion because we should have some
sort of common understanding of what constitutes a "unit test" and an
"integration test", and how they are implemented-- not to mention
performance test, accessibility test, A/B test, etc. etc. that in my
experience are often all conflated into one fuzzy concept of "oh, it's
testing, it's all about the same thing, right?"

-Chris



On Wed, Jan 15, 2014 at 3:25 PM, Greg Grossmeier <greg at wikimedia.org> wrote:

> Observation 1:
> We have test-writing related keywords in Bugzilla:
> * need-integration-test
> * need-parsertest
> * need-unittest
>
> Obvservation 2:
> Without hard data to back up my claim, I feel like a large number of
> bugs reported are fixed but do not also include a test to the
> functionality. I might just be missing them. I don't think I am (at
> least always); I think they just aren't created. That's kind of the
> standard way of developing, right? ;)
>
> Suggestion:
> We use some kind of way of tracking this. Maybe not every bug fix needs
> an associated test, but I'd like to believe it is more than the
> ridiculously low percentage number that we are doing now. We might not
> use all of those current keywords (maybe just "needs-test" and let the
> dev team/QA person on point for that team figure it out). We can figure
> that part out later.
>
> Thoughts?
> Is this too cumbersome? Does anyone have experience with trying to
> accomplish something similar with a different group of engineers (both
> "dev" and "QA", where/if those are separate roles). We don't really have
> the bandwidth to start this in earnest right now, but should we start
> mulling it over?
>
> Greg
>
> --
> | Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
> | identi.ca: @greg                A18D 1138 8E47 FAC8 1C7D |
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140116/a0f6d0b2/attachment.html>