Why Jenkins does not detect Failures
Jenkins, the brilliant Continuous Integration build server, has a bit of a problem with the Maven surefire jUnit test plugin. Last sunday, I discovered that our Jenkins build server suddenly started ignoring test failures. While the logfile clearly states that the Unittests contain failures, Jenkins marks the builds as "stable".
After some digging around, I found that even though Jenkins explicitly tells you in the logfile that it will fail the build, it will not do so if the Surefire XML reports are not generated. In our case, somebody in the team decided that the generation of the XML Surefire reports was taking too long and had disabled them in the Maven pom.xml.
In order to solve this, I re-enabled the XML reports and voila, Jenkins happily started reporting errors again. Here is the correct Surefire plugin configuration for you to use in your maven pom.xml file:
[sourcecode language="xml" toolbar="false" wraplines="false" padlinenumbers="true" gutter="false" autolinks="false" highlight="10"]
<!-- Please note that Jenkins needs Surefire
XML reports in order for detection to work.
Keep this property set to false. -->
<disableXmlReport>false</disableXmlReport>
</configuration>