<div dir="ltr">Hi <span style="font-family:arial,sans-serif;font-size:13px">Nik,</span><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">apologies for the late reply, I was on vacation. Comments are inline.<br>
</font><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 21, 2013 at 8:18 PM, Nikolas Everett <span dir="ltr"><<a href="mailto:neverett@wikimedia.org" target="_blank">neverett@wikimedia.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>1. What do you do with developers that think it is beneath them to write integration tests? Unit tests?<br></div></div></div></div></blockquote><div><br></div><div>I would also ask why would a developer think that. All code is created equal, right? Writing test code should be not different than writing production code. Or am I wrong?</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div></div><div>2. Where should you store those tests? (We've mostly settled on next to the production code which seems like the right answer to me for lots of reasons.)<br></div></div></div></div></blockquote><div>
<br></div><div>+1</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">
<div class="gmail_quote"><div>
</div><div>2. What code/features/changes should have unit tests? Integration tests? What is the difference, any way?<br></div></div></div></div></blockquote><div><br></div><div>Looks like you have two questions with number 2. :)</div>
<div><br></div><div>As a rule of thumb, code that is more likely to break, that is hard to understand, that is written recently... should have more tests than other code.</div><div><br></div><div>I would say that most (if not all code) should have unit tests, and things that are hard or impossible to test with an unit test should be tested on a higher level.</div>
<div><br></div><div>The difference?</div><div><br></div><div>"A test is not a unit test if:</div><br>- It talks to the database<br>- It communicates across the network<br>- It touches the file system<br>- It can't run at the same time as any of your other unit tests<br>
- You have to do special things to your environment (such as editing config files) to run it."[1]<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>3. How do you make sure you stick to #3?<br></div></div></div></div></blockquote><div><br></div><div>Well, this is #3, but I guess you were asking about previous question. I am not sure what to do.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>4. Where does documentation live? Do you build documentation from tests? Do you build tests from documentation? (Which one of these do we do or is it even listed?)<br>
</div></div></div></div></blockquote><div><br></div><div>I have been struggling with this myself. Feature files are one form of documentation.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
</div><div>5. If integration testing is really just another speciality like databases what does this mean for people that have spent years doing one or the other?<br></div></div></div></div></blockquote><div><br></div><div>
I am not sure what you ask here.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">
<div class="gmail_extra"><div class="gmail_quote"><div></div><div>6. What do you do when your organization doesn't jib with this reality? (WMF has a QA team when it might make more sense to matrix integration test specialists into teams to teach them how to write integration tests, for example.)<br>
</div></div></div></div></blockquote><div><br></div><div>We are moving towards integrating QA team into other teams. The problem is that there is just a few of us and a lot of teams.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
</div><div>7. Holy cow how does this all line up with volunteers who only want to do one thing or the other and we really should take any help we can get.<br></div></div></div></div></blockquote><div><br></div><div>I do not think it will be trivial, but I am pretty sure we can handle volunteers that just want to test or just want to code. :)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div></div><div>8. Who pays attention to build failures and what do they do about them?<br></div></div></div></div></blockquote><div><br></div><div>A lot of people actually. Mobile team, language team, QA team... When a build fails, we fix it.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>
</div><div>9. What system, exactly, are those build failures testing?<br></div></div></div></div></blockquote><div><br></div><div>A build can fail for several reasons. We had an entire workshop about it[2].</div><div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>Now that I think about it, who is coming to the architecture summit? These kinds of questions would be pretty interesting to talk about and might deserve a (late) RFC.<br></div></div></div></div>
</blockquote></div><br></div></div><div class="gmail_extra">I am not sure if anybody from QA team is coming.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Željko</div><div class="gmail_extra">--</div><div class="gmail_extra">
1: <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=126923">http://www.artima.com/weblogs/viewpost.jsp?thread=126923</a></div><div class="gmail_extra">2: <a href="https://www.youtube.com/watch?v=fs2VmlQ_h7Q">https://www.youtube.com/watch?v=fs2VmlQ_h7Q</a></div>
</div>