On 04/06/13 00:20, Jeroen De Dauw wrote:
Hey,
He claims that just because you have guard rails you shouldn't stop paying
attention and expect them to save you.
Being someone who practices TDD, I fully agree with that. It is not because
you wrote tests first and got a 100% coverage that you proved your code to
be correct and bug free. That tests can prevent all bugs or that they
prevent the need to to think about what you are doing are two common
arguments made against a mythical strawman version of TDD. They are both
clearly fallacious and not the goal of TDD or writing tests in general. [0]
Indeed, TDD doesn't produce bug-free code. In fact, according to a
literature review published in the book "Making Software" (Ed. Andy
Oram & Greg Wilson), TDD does not even significantly *reduce* the
post-release defect density in controlled trials, let alone eliminate
bugs altogether.
"There is some evidence to suggest that TDD improves external quality.
Although the outcomes of controlled experiments are mostly
inconclusive, industrial use and pilot studies strongly favor TDD.
However, the supporting evidence from industrial use and controlled
experiments disappears after filtering out the less rigorous studies
(i.e., L0 and L1 trials). Furthermore, the evidence from pilot studies
and controlled experiments is contradictory once L0 and L1 trials are
filtered out."
The review was even more negative about the effect of TDD on productivity:
"The available evidence from the trials suggests that TDD does not
have a consistent effect on productivity. The evidence from controlled
experiments suggests an improvement in productivity when TDD is used.
However, the pilot studies provide mixed evidence, some in favor of
and others against TDD. In the industrial studies, the evidence
suggests that TDD yields worse productivity. Even when considering
only the more rigorous studies (L2 and L3), the evidence is equally
split for and against a positive effect on productivity."
And also in the discussion section:
"One basic fact on which almost everyone agrees is that TDD is
difficult to learn. It involves a steep learning curve that requires
skill, maturity, and time, particularly when developers are entrenched
in the code-then-test paradigm."
Because of this, I can be fairly confident in recommending that my
team avoids the use of TDD.
-- Tim Starling