[QA] Fwd: SonarQube and Puppet

[Guillaume Lederrey]

Message below cross posted from ops at lists.wikimedia.org.

Seems that the discussion might be interesting to QA team as well.

---------- Forwarded message ----------
From: Guillaume Lederrey <glederrey at wikimedia.org>
Date: Fri, Feb 5, 2016 at 10:43 AM
Subject: SonarQube and Puppet
To: ops at lists.wikimedia.org
Cc: David Racodon <david.racodon at gmail.com>


Hello all !

Since I'm fairly new here, I still have a few idea coming from my former
life. Time to expose some of them before I forget them...

While trying to familiarize myself a bit with our Puppet code base, I did
run a SonarQube analysis on it. And I remembered having a few discussion
about SonarQube during my interview process. So, short presentation:

SonarQube is an amazing project to manage code quality. It supports a long
list of languages, from Java to PHP, from Cobol to ABAP. And of course
Puppet [1] (even if that support is still a bit young).

First things first, how to try it? Of course, docker [2] is our friend
(tested myself with v4.5.6). Or David Racodon has a simple package [3] to
test the puppet support.


**Why do we need SonarQube, we already have puppet-lint, rspec-puppet,
cucumber-puppet, ...**

1) SonarQube rules go a bit further than puppet-lint. For example the
DuplicateHashKeys rule [4] has no equivalent in puppet-lint and a few
violations on our code base that are clear indication of a problem. Note
that all puppet-lint rules have been re implemented in the SonarQube
plugin. Rules about code complexity, code duplication and quite a few other
metrics are also available.

2) Holistic view of code quality (yes, I know, big words). SonarQube web
interface provides a good way to compare quality of projects, to dig into
specific issues, keep track of evolution over time. Much richer than a
build time check that either pass or fail.

3) Actually help you improve. A binary check like puppet-lint (or other
similar tools) gives you a very simple feedback, you're good or you're not.
Reality is usually more complex. We have existing code base which have a
history. We might not want to fix all issues right now (after all, our
current code is in production, so it is probably mostly good enough) but we
want to improve on the long term. We want to introduce new checks, higher
quality standards, but not stop everything while we are improving our
standards. SonarQube gives us "quality gates", where we define rules about
what is good enough. And those rules can be differential. For example:
"quality gate passes if the commit does not introduce any new issue (I
don't care about existing issues)".


**Do we need SonarQube at WMF**

Honestly, I don't know enough about how we manage Puppet (or other code) to
have an opinion on this (yet). I have not seen anything scary in my code
analysis. You tell me...


**Disclaimer**

I have worked with David (the author of the Puppet plugin for SonarQube)
for some time. He has convinced me, perverted me and all those things about
code quality. Beside being a Nice Guy (tm) he is Pretty Smart (c) and knows
SonarQube fairly well. He might be available for a chat if anyone is
interested.


**Note on testing on Docker**

The Docker image provided by SonarQube only contains a minimal set of
plugins. To add Puppet support, go to the web interface (
https://localhost:9000, user: admin, pwd: admin) look for the update center
and add the Puppet plugin. Restart required. You'll need to install
sonar-runner [5] locally.



[1] https://github.com/iwarapter/sonar-puppet
[2] https://hub.docker.com/_/sonarqube/
[3] https://github.com/racodond/package-test-sonarqube-puppet
[4]
https://github.com/iwarapter/sonar-puppet/blob/master/puppet-checks/src/main/resources/org/sonar/l10n/pp/rules/puppet/DuplicatedHashKeys.html
[5]
http://central.maven.org/maven2/org/codehaus/sonar/runner/sonar-runner-dist/2.4/sonar-runner-dist-2.4.zip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/qa/attachments/20160205/4f630120/attachment.html>