On 04/06/13 00:20, Jeroen De Dauw wrote:
Hey,
He claims that just because you have guard rails you shouldn't stop paying
attention and expect them to save you.
Being someone who practices TDD, I fully agree with that. It is not because you wrote tests first and got a 100% coverage that you proved your code to be correct and bug free. That tests can prevent all bugs or that they prevent the need to to think about what you are doing are two common arguments made against a mythical strawman version of TDD. They are both clearly fallacious and not the goal of TDD or writing tests in general. [0]
Indeed, TDD doesn't produce bug-free code. In fact, according to a literature review published in the book "Making Software" (Ed. Andy Oram & Greg Wilson), TDD does not even significantly *reduce* the post-release defect density in controlled trials, let alone eliminate bugs altogether.
"There is some evidence to suggest that TDD improves external quality. Although the outcomes of controlled experiments are mostly inconclusive, industrial use and pilot studies strongly favor TDD. However, the supporting evidence from industrial use and controlled experiments disappears after filtering out the less rigorous studies (i.e., L0 and L1 trials). Furthermore, the evidence from pilot studies and controlled experiments is contradictory once L0 and L1 trials are filtered out."
The review was even more negative about the effect of TDD on productivity:
"The available evidence from the trials suggests that TDD does not have a consistent effect on productivity. The evidence from controlled experiments suggests an improvement in productivity when TDD is used. However, the pilot studies provide mixed evidence, some in favor of and others against TDD. In the industrial studies, the evidence suggests that TDD yields worse productivity. Even when considering only the more rigorous studies (L2 and L3), the evidence is equally split for and against a positive effect on productivity."
And also in the discussion section:
"One basic fact on which almost everyone agrees is that TDD is difficult to learn. It involves a steep learning curve that requires skill, maturity, and time, particularly when developers are entrenched in the code-then-test paradigm."
Because of this, I can be fairly confident in recommending that my team avoids the use of TDD.
-- Tim Starling