Thanks for your thoughtful reply, Tim! Am 03.06.2013 07:35, schrieb Tim Starling: I think testability is important, but I think it's not the only (or even main) reason to support the principles from that post. I think these principles are also important for maintainability and extensibility. Essentially, they enforce modularization of code in a way that makes all parts as independent of each other as possible. This means they can also be understood by themselves, and can easily be replaced. That's why we should have unit tests *and* integration tests. I agree though that it's not necessary or helpful to enforce the maximum possible breakdown of the code. However, I feel that the current code is way to the monolithic end of the spectrum - we could and should do a lot better. If there are indeed such detrimental effects. I see two main inconveniences: * More classes/files. This is, in my opinion, mostly a question of using the proper tools. * Working with "passive" objects, e.g. $chargeProcessor->process( $card ) instead of $card->charge(). This means additional code for injecting the processor, and more code for calling the logic. That is inconvenient, but not detrimental, IMHO: it makes responsibilities clearer and allows for easy substitution of logic. yay :) Yes, absolutely: value objects can be constructed directly. I'd even go so far as to say that it's ok, at least at first, to construct controller objects directly, using servies injected into the local scope (though it would be better to have a factory for the controllers). To me, 9 is pretty essential, since without that principle, value objects will soon cease to be thus, and will again grow into the monsters we see in the code base now. Item 2 is less essential, though still important, I think; basically, it requires every component (class) to make explicit which other component it relies on for collaboration. Only then, it can easily be isolated and "transplanted" - that is, re-used in a different context (like testing). Yes, I agree. Separation of concerns can be driven to the atomic level, and at some point becomes more of a pain than an aid. But we definitely should split more than we do now. -- daniel

Hey, He claims that just because you have guard rails you shouldn't stop paying Being someone who practices TDD, I fully agree with that. It is not because you wrote tests first and got a 100% coverage that you proved your code to be correct and bug free. That tests can prevent all bugs or that they prevent the need to to think about what you are doing are two common arguments made against a mythical strawman version of TDD. They are both clearly fallacious and not the goal of TDD or writing tests in general. [0] 4. Don't write automated tests at all and do lots of code reviews and Absolutist statements are typically wrong. There are almost always cases in which some practice is not applicable. However I strongly disagree with your recommendation of not writing tests and automating them. I disagree even stronger with the notion that manual testing is generally something you want to do. I've seen many experts in the field of software design recommend strongly against manual testing, and am currently seeing the same theme being pretty prevalent here at the International PHP Conference I'm currently attending. So my question is not "how do we write code that is maximally This contains the suggestion that testable code inherently is badly designed. That is certainly not the case. Good design and testability go hand in hand. One of the selling points of testing is that it strongly encourages you to create well designed software. There are other advantages to writing tests as well. Just out of the top of my head: * Regression detection * Replaces manual testing with automated testing, saves lots of time, esp in projects with multiple devs. Manual testing tends to be incomplete and skipped as well, so the number of bugs caught is much lower. And it does not scale. At all. * Documentation so formal it can be executed and is never out of date * Perhaps the most important: removes the fear of change. One can refactor code to clean up some mess without having to fear one broke existing behavior. Tests are a great counter to code rot. Without tests, your code quality is likely to decline. I'm sure I missed some other good points. Lots of great literature on the subject out there btw. [0] http://codemanship.co.uk/parlezuml/blog/?postid=1170 Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. ~=[,,_,,]:3 --

On Mon, Jun 3, 2013 at 10:20 AM, Jeroen De Dauw <jeroendedauw(a)gmail.com>wrote;wrote: I think not having automated tests is right in some situations but I certainly wouldn't recommend it. Manual testing sucks and having nice tests with Selenium or some such tool is way better in most situations but there are totally times where a good code review and manual verification are perfect. I'm thinking of temporary solutions or styling issues are difficult to verify with automated tests. I'm certainly no expert and I'd _love_ to learn more about things that help in the situations where I feel like manual testing is best. I'd love nothing more than to be wrong. IMHO you can design code so that it is both easy to understand and easy to test but there is a real temptation to sacrifice comprehenability for testability. Mostly I see this in components being split into incomprehensibly small chunks and then tested via an intricate mock waltz. I'm not saying this happens all the time, only that this happens and we need to be vigilent. The guidelines in the article help prevent such craziness. This is perfect! If you think of your tests as formal verification documents then you are in good shape because this implies that the tests are readable. If I had my druthers I'd like all software to be designed in such a way that it can be tested automatically with informative tests that read like documentation. We'd all like that. To me it looks like there are three problems: 1. How do you keep out tests that are incomprehensible as documentation? 2. What do you do with components for which no unit test can be written that could serve as documentation? 3. What do you do when the formal documentation will become out of date so fast that it feels like a waste of time to write it? I really only have a good answer for #2 and that is to test components together like the DB and Repository or the server side application and the browser. 1 troubles me quite a bit because I've found those tests to be genuinely hurtful in that they give you the sense that you are acomplishing something when you aren't. Nik

Am 03.06.2013 18:48, schrieb Chris Steipp: I agree with the sentiment, but disagree with the metric used. Currently, we have relatively few components, which have very complex internal information flows, and quite complex dependency networks (or quite simple: everything depends on everything). I'm advocating a system of many more components with several dependencies each, but with simple internal information flow and a clear hierarchy of dependency. So, which one is simpler to hold in your head? Well, it's simpler to remember fewer components. But not fully understanding their internal information flow (EditPage, anyone) or how they interact and depend on each other is what is really hurting security (and overall code quality). So, I'd argue that even if you have to remember 15 (well named) classes instead of 5, you are still better off if these 15 classes only depend on a total of 5000 lines of code, as opposed to 50k or more with the current system. tl;dr: number of lines is a better metric for the impact of dependencies than the number of classes is. Big, multi purpose classes and context objects (and global state) keep the number of classes low, but cause dependency on a huge number of LoC. -- daniel

On 04/06/13 00:20, Jeroen De Dauw wrote: Indeed, TDD doesn't produce bug-free code. In fact, according to a literature review published in the book "Making Software" (Ed. Andy Oram & Greg Wilson), TDD does not even significantly *reduce* the post-release defect density in controlled trials, let alone eliminate bugs altogether. "There is some evidence to suggest that TDD improves external quality. Although the outcomes of controlled experiments are mostly inconclusive, industrial use and pilot studies strongly favor TDD. However, the supporting evidence from industrial use and controlled experiments disappears after filtering out the less rigorous studies (i.e., L0 and L1 trials). Furthermore, the evidence from pilot studies and controlled experiments is contradictory once L0 and L1 trials are filtered out." The review was even more negative about the effect of TDD on productivity: "The available evidence from the trials suggests that TDD does not have a consistent effect on productivity. The evidence from controlled experiments suggests an improvement in productivity when TDD is used. However, the pilot studies provide mixed evidence, some in favor of and others against TDD. In the industrial studies, the evidence suggests that TDD yields worse productivity. Even when considering only the more rigorous studies (L2 and L3), the evidence is equally split for and against a positive effect on productivity." And also in the discussion section: "One basic fact on which almost everyone agrees is that TDD is difficult to learn. It involves a steep learning curve that requires skill, maturity, and time, particularly when developers are entrenched in the code-then-test paradigm." Because of this, I can be fairly confident in recommending that my team avoids the use of TDD. -- Tim Starling

As the dumb ass trying to merge a lot of the code last year at Wikidata I would say stop bitching about whether to make tests or not. Any tests are better than no tests, without tests merging code is pure gambling. Yes you can create a small piece of code and be fairly sure that your own code works when you merge it in, but no you can not be sure that it will continue to work when a bunch of other nerds are wild-guessing what your code is doing and then merge their own code in. It is inly one sure method to make the code keep on working and that is by adding tests. Those tests are not primarily for you, unless you do TDD, they are for everyone else. If you want some real comparison on what can be achieved with proper testing, then take a look at Coverity scan: 2012 Open Source Report [1], they are tracking 254 active projects. You must check what the projects are doing to make the code bugfree. (Amanda uses a automatic gatekeeper, and _they_have_extensive_tests_) You should not make tests that is really testing internals of an unit, you should make tests that operates on the API of the unit. You should be able to change the internals of an unit without changing the tests. If you must change the tests because of internal changes in an unit, then there is probably something wrong with the test. Security issues in an implementation is most of the time not found by unit tests, because the unit tests should test the API and not the implementation. An other thing is that if the unit test in fact tests the API the necessary seems are probably in place to do source code fault injection, and that is a very efficient tool to find security issues. Tak a look at "Software Security Assessment Tools Review" [2] for a list of proper methods for security assessment. Also tests are not a replacement for proper code review, but it will speed up the process in those cases where bugs are detected during the testing process. In some cases you can infact use the time spent on merges and the frequency whereby you break the build as a measure of how good test coverage you have. Less coverage gives higher break frequency. TDD is nice, but often it turns into testing internals of some idea the developer has, which is wrong. It also have a strange effect to create lock-in on certain solutions which can be a problem. That is you chose implementation after how easy it is to test, not how efficient the implementation is. The same thing can be said about other formal processes so I'm not sure it is a valid point. It is said that those familiar with TDD writes the tests and implementation in close to the same time as the implementation alone. Rumors has it that this comes from the fact that the prewritten tests makes the implementation more testable, and that this is an easier way than adding tests to an easier implementation later. I don't know. Code with some tests are sometimes compared to code with no tests at all, and then there are made claims that the code without tests are somehow similarly bugfree to some degree. This argument usually doesn't hold, as "some tests" isn't good enough, you need real numbers for the coverage before you can make valid claims and if the coverage is low it can be very difficult to say anything about the real effects. If you really want to make an effort to get bugfree code you need a lot of tests, above 50% and probably more like 80%. Not only do you need a lot of tests, you need a lot of people to fix those bugs, because lots of tests will find more bugs to fix. It is like writing a document with no spell checker vs using a spell checker. [1] http://www.coverity.com/library/pdf/coverity-scan-2011-open-source-integrit… [2] http://samate.nist.gov/docs/NAVSEA-Tools-Paper-2009-03-02.pdf

Hey, My own experience is that "test coverage" is a poor evaluation metric Test coverage is a metric to see how much of your code is executed by your tests. From this alone you cannot say if some code is good or bad. You can have bad code with 100% coverage, and good code without any coverage. You are first stating it is a poor metric to measure quality and then proceed to make the claim that more coverage implies bad code. Aside from contradicting yourself, this is pure nonsense. Perhaps you just expressed yourself badly, as test coverage does not "produce" code to begin with. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. ~=[,,_,,]:3 --

On 06/04/2013 12:57 PM, Nikolas Everett wrote: Probably better expressed than I did. My point is: clearly test coverage doesn't /produce/ bad code -- but writing *for* test coverage does. Or at least, I've observed a strong correlation between mandated test coverage metrics and code with atrocious factorization and poor conceptual coherency. Tests are good. Unit testing has valuable uses in a number of cases. Trying to universally shoehorn either into the development process is rarely useful, and often disastrous. (Often, for instance, coherency of the code is sacrificed atop the altar of separation of concerns for a vacuous gain in "testability" at the expense of clarity). -- Marc

Test coverage is not a quality metric, it is a quantity metric. That is it says something about the amount of tests. The coverage can say something about the overall code given that the code under test infact reflect the remaining code. As the code under test usually are better than the remaining code the overall code is usually worse than the predicted number of faults in the remaining code. My 0.05€ is to enforce testing wherever possible, possibly by using automatic gatekeepers, and also add in other quality and security metrics for the reports. Err, automated reports, that is another problem... On Tue, Jun 4, 2013 at 6:36 PM, Jeroen De Dauw <jeroendedauw(a)gmail.com> wrote: