Dirk Sturzebecher
added a comment - 22/Nov/05 4:25 PM one more detail, if I include the plugin in the build section (as in the documentation) it is not triggered. I have included it in the reporting section (which kind of makes sense)...

Brett Porter
added a comment - 25/Nov/05 5:45 PM if we omit the test running, then just running "mvn site" will fail.
I'm not sure what failures are encountered other than being annoying
Anyway, this is a core Maven issue, and we are looking to improve it in Maven 2.1/2.2

Dirk Sturzebecher
added a comment - 27/Nov/05 2:42 PM do not run the tests unnecessarily again, this can only cause trouble!
the best would be if the tests would be considered as a dependencies, so not run if there is something in the surefire dir, run if it is empty (or something of this kind)

Grégory Joseph
added a comment - 13/Jan/06 6:39 PM For some reason, when executing the test phase, the settings of the surefire plugin seem to not be taken into account: my tests are not forked, thus failing.

Grégory Joseph
added a comment - 13/Jan/06 8:12 PM Hmm, nevermind that - my tests are failing because they're run twice - if I remove the clover report plugin, which also runs the tests, then they don't fail ..

Just tried the same again with maven 2.0.2 and set my surefire to fork. Works fine now. That is the result looks good on first inspection, but the console shows errors as below. First one looks like a missing file, second one seams a problem with the local. My environment is a german local. Seams to be te typical difference between english numbers (2,000.) and german numbers (2.000,)

That would be great if you could provide a mean to prevent from having the surefire-report plugin kicking off a test phase when triggered. Some boolean like 'invokeTest' in the <configuration> section of the plugin would be just fine.

(And btw, Brett, i've been looking for you solution but was not able to find any <skip> tag in either the Maven 1.1 or Maven 2 POM XML schema.)

Pascal Magnard
added a comment - 12/May/06 2:52 AM I've got the same issue as Dirk. I added the surefire-report plugin declaration to the reporting section of my pom and each time this plugin is executed, the test phase is invoked. The expected behavior, according to the documentation, would be the surefire plugin to invoke the test phase ( http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html ) but not the surefire-report plugin ( http://maven.apache.org/plugins/maven-surefire-report-plugin/report-mojo.html ).
Having the tests ran twice when invoking mvn clean install site is more than annoying for the project i'm working on because the compilation and the testing are performed twice and they unfortunatly take a big while.
That would be great if you could provide a mean to prevent from having the surefire-report plugin kicking off a test phase when triggered. Some boolean like 'invokeTest' in the <configuration> section of the plugin would be just fine.
(And btw, Brett, i've been looking for you solution but was not able to find any <skip> tag in either the Maven 1.1 or Maven 2 POM XML schema.)

I have the same issue. I am using "clean install site" and for some reason, the surefire test is executed four times. This is more than annoying. It considerably slows down the build process because I have some DAO tests against real database.

I hope Maven can remove all duplicated goals within a single maven command so that people can safely execute something like "mvn clean compile test package install deploy site site-deploy".

Kenny Cheang
added a comment - 07/Jul/06 9:11 PM I have the same issue. I am using "clean install site" and for some reason, the surefire test is executed four times. This is more than annoying. It considerably slows down the build process because I have some DAO tests against real database.
I hope Maven can remove all duplicated goals within a single maven command so that people can safely execute something like "mvn clean compile test package install deploy site site-deploy".

I my eyes, running the test during site generation is no good idea as it modifies content outside the "/target/site" folder. If I run "mvn clean package site" there should be no problem with having no test results. If I run "mvn site" with no test results available, well for me it's ok to have no (or an empty) report in this case. If others like to have the tests done during site generation, I'd see this as an optional configuration as it modifies content outside the "/target/site" folder and is a time issue (especially for larger projects).

According to Kenny's problem, I assume he is using the surefire, surefire-report and clover plugin (like me). In this case I can explain why the tests are run four times. If you enable the debug output, you will see that clover does something like running a second lifecycle with goal install. So the first time the test are executed, it is done with instrumented code. The second time (in the original) lifecycle the test are run with non-instrumented code. To be honest, from some point of view this makes sense as instrumentation does not influence your tests (especially when making performance tests), but I would like to decide this myself. The second two executions are simple to explain. This is because surefire-report reruns the test (including all prior phases) and this includes instrumentation too.

Martin Brunninger
added a comment - 22/Aug/06 5:22 AM I encountered this too.
I my eyes, running the test during site generation is no good idea as it modifies content outside the "/target/site" folder. If I run "mvn clean package site" there should be no problem with having no test results. If I run "mvn site" with no test results available, well for me it's ok to have no (or an empty) report in this case. If others like to have the tests done during site generation, I'd see this as an optional configuration as it modifies content outside the "/target/site" folder and is a time issue (especially for larger projects).
According to Kenny's problem, I assume he is using the surefire, surefire-report and clover plugin (like me). In this case I can explain why the tests are run four times. If you enable the debug output, you will see that clover does something like running a second lifecycle with goal install. So the first time the test are executed, it is done with instrumented code. The second time (in the original) lifecycle the test are run with non-instrumented code. To be honest, from some point of view this makes sense as instrumentation does not influence your tests (especially when making performance tests), but I would like to decide this myself. The second two executions are simple to explain. This is because surefire-report reruns the test (including all prior phases) and this includes instrumentation too.
Hope this helps.

This is a big issue for us. We are migrating a large ant based project to Maven and have 1200 unit tests. If a test fails the only way to find which test has failed is to run the surefire-report:report goal this then re-runs all the tests, which takes a long time. Need a way to get this report plugin to only rerun tests if explicityly asked to. It should just compile a report on the results that are already there, if there are no individual test reports then exit without error. Also is there a way to get the surefire plugin to only create reports for tests that have failed.

Peter Anning
added a comment - 03/Sep/06 6:18 PM This is a big issue for us. We are migrating a large ant based project to Maven and have 1200 unit tests. If a test fails the only way to find which test has failed is to run the surefire-report:report goal this then re-runs all the tests, which takes a long time. Need a way to get this report plugin to only rerun tests if explicityly asked to. It should just compile a report on the results that are already there, if there are no individual test reports then exit without error. Also is there a way to get the surefire plugin to only create reports for tests that have failed.

The workaround I've been using is the problem Peter Anning has above is the following. First I run a target that invokes the tests, like:

mvn test OR mvn install

Then, if the tests fail, I run:

mvn -Dmaven.test.skip=true surefire-report:report

OR

mvn -Dmaven.test.skip=true site

Either of these commands runs the surefire report goal without running the tests again. Obviously, this is just a workaround, and it doesn't work well for continuous integration systems that need to run everything automagically, but it has helped me a lot.

Sasha A
added a comment - 01/Nov/06 11:05 AM The workaround I've been using is the problem Peter Anning has above is the following. First I run a target that invokes the tests, like:
mvn test OR mvn install
Then, if the tests fail, I run:
mvn -Dmaven.test.skip=true surefire-report:report
OR
mvn -Dmaven.test.skip=true site
Either of these commands runs the surefire report goal without running the tests again. Obviously, this is just a workaround, and it doesn't work well for continuous integration systems that need to run everything automagically, but it has helped me a lot.

After adding the following code to my pom.xml, doing a "mvn install site" only runs the tests once. It is a good workaround if you're using a continuous integration system or can't be bothered to type two things.

aaron pieper
added a comment - 28/Nov/06 3:46 PM Ignore my previously suggested workaround. "mvn install site" was only running the tests once for me because surefire reports were not being included.
I'm unsure why tests were being run twice since surefire was not in use, but the workaround I suggested does not work.

Barrie Treloar
added a comment - 11/Dec/06 9:08 PM Patch as requested by someone to create a "report-only" goal.
Includes update to index.apt to reference the goal.
Based on subversion revision 485987.
Workaround until someone sorts out reactor/forked lifecycle issues.

Barrie Treloar
added a comment - 15/Feb/07 9:29 PM I don't know if a snapshot build has been created that includes this patch yet.
The error message you list would indicate this.
Your alternative is to create a locally patched version of the plugin.
That's what I have done and it works fine.

That would work for me only. However, I have a group of developers that are either Maven skeptics or unfamiliar with Maven. I am attempting to provide them a working and acceptable alternative to our current ant build. I don't want to place a burden of asking them to checkout and patch a plugin. That would not reflect well on Maven. Is is possible to get a more recent SNAPSHOT deployed?

Brandon Goodin
added a comment - 16/Feb/07 12:20 AM That would work for me only. However, I have a group of developers that are either Maven skeptics or unfamiliar with Maven. I am attempting to provide them a working and acceptable alternative to our current ant build. I don't want to place a burden of asking them to checkout and patch a plugin. That would not reflect well on Maven. Is is possible to get a more recent SNAPSHOT deployed?

Barrie Treloar
added a comment - 17/Feb/07 6:44 PM Have a read of http://docs.codehaus.org/display/MAVENUSER/Patching+Maven+Plugins
and http://docs.codehaus.org/display/MAVENUSER/Using+Maven+in+a+corporate+environment .
It is my opinion that your build should never depend upon snapshot versions.
And including a snapshot repository can also mean pulling in more snapshot plugins than intended.

Thanks for the feedback. The website (http://maven.apache.org/plugins/maven-surefire-report-plugin/report-mojo.html) is presenting the report-only functionality as though it is available. If you do not intend for people to use it you might want to roll your site documentation back to what is actually available. What is the timeline for a new release? This functionality is very much wanted and being that the project I am working with is an Apache project I'd prefer not to go setting up custom repos to serve up jars for our build. I'd much rather use yours.

Brandon Goodin
added a comment - 17/Feb/07 8:03 PM Thanks for the feedback. The website ( http://maven.apache.org/plugins/maven-surefire-report-plugin/report-mojo.html ) is presenting the report-only functionality as though it is available. If you do not intend for people to use it you might want to roll your site documentation back to what is actually available. What is the timeline for a new release? This functionality is very much wanted and being that the project I am working with is an Apache project I'd prefer not to go setting up custom repos to serve up jars for our build. I'd much rather use yours.

Unfortunately, I don't believe we can mark this resolved as surefire-report:report-only is a workaround.

More thought was meant to be given to this and lifecycles for Maven 2.1.

For instance, I want the assembly plugin to use the target/site files so I can grab the html from converted apts.
To do this, I'd like to bind site to the default lifecyle, which will cause surefire-report to run (with all the re-running of tests that are listed in this defect)

Barrie Treloar
added a comment - 20/Feb/07 2:16 PM Unfortunately, I don't believe we can mark this resolved as surefire-report:report-only is a workaround.
More thought was meant to be given to this and lifecycles for Maven 2.1.
For instance, I want the assembly plugin to use the target/site files so I can grab the html from converted apts.
To do this, I'd like to bind site to the default lifecyle, which will cause surefire-report to run (with all the re-running of tests that are listed in this defect)

Graham Lea
added a comment - 20/Feb/07 8:08 PM Regardless of whether this is the best solution or not, it contains the functionality that most people want.
Please commit it to the Maven repository so we can use it.

I'm attempting to use report-only but I cannot seem to get maven to only execute "report-only" and not "report". It still executes both report-only and report hence my tests are still being executed again when "report" runs. Any ideas?

Mike Youngstrom
added a comment - 15/May/07 1:08 PM I'm attempting to use report-only but I cannot seem to get maven to only execute "report-only" and not "report". It still executes both report-only and report hence my tests are still being executed again when "report" runs. Any ideas?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<goals>
<goal>report-only</goal>
</goals>
</execution>
</executions>
</plugin>

If this isn't available with v2.3, then there's a documentation problem, as the "report-only" goal is advertised as available now and available since 2.3. See the following URLs, which as of this posting say 'report-only' is available.

Nathan Beyer (Cerner)
added a comment - 18/Jun/07 2:27 PM If this isn't available with v2.3, then there's a documentation problem, as the "report-only" goal is advertised as available now and available since 2.3. See the following URLs, which as of this posting say 'report-only' is available.
http://maven.apache.org/plugins/maven-surefire-report-plugin/
http://maven.apache.org/plugins/maven-surefire-report-plugin/report-only-mojo.html

I'm using Maven 2.0.7 and surefire-report-plugin:2.3. From my experience, "report-only" already works in this version. Mike's problem with the execution of both "report" and "report-only" should be solved by configuring surefire-report under the <reporting> section like this:

Benjamin Bentmann
added a comment - 11/Aug/07 7:50 AM I'm using Maven 2.0.7 and surefire-report-plugin:2.3. From my experience, "report-only" already works in this version. Mike's problem with the execution of both "report" and "report-only" should be solved by configuring surefire-report under the <reporting> section like this:
<reporting>
...
<plugin>
<artifactId> maven-surefire-report-plugin </artifactId>
...
<reportSets>
<reportSet>
<reports>
<report> report-only </report>
</reports>
</reportSet>
</reportSets>
</plugin>
...
</reporting>
This seems to be in conformance with http://maven.apache.org/pom.html#Reporting , one cannot configure a reporting plugin using the <build> section.
If I type "mvn site", the tests are not run. However, "report-only" still seems to fork a new lifecycle build. This is not really an issue, but seems unnecessary, at least to me as a Maven novice.

Damien Lecan
added a comment - 23/Aug/07 5:14 AM Benjamin's example doesn't not work for me. html files for site are not generated.
I work with surefire 2.3 and if I try to do mvn surefire-report:report-only I get this error :
[INFO] Searching repository for plugin with prefix: 'surefire-report'.
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Required goal not found: surefire-report:report-only
Are you sure this should work with Surefire 2.3 ?

surefire and surefire-report are separate JARs, are you sure you use v2.3 of surefire-report? I just retried your example command-line and had no problems. Could you append the switch "-X" to enables Maven's debug output and verify it really states something like
Configuring mojo 'org.apache.maven.plugins:maven-surefire-report-plugin:2.3:report-only'
If you still have an older version, appending the switch "-U" should tell Maven to search for updates for the plugins (used to execute the current goal).

Benjamin Bentmann
added a comment - 23/Aug/07 5:44 AM surefire and surefire-report are separate JARs, are you sure you use v2.3 of surefire-report? I just retried your example command-line and had no problems. Could you append the switch "-X" to enables Maven's debug output and verify it really states something like
Configuring mojo 'org.apache.maven.plugins:maven-surefire-report-plugin:2.3:report-only'
If you still have an older version, appending the switch "-U" should tell Maven to search for updates for the plugins (used to execute the current goal).

Benjamin Bentmann
added a comment - 23/Aug/07 8:07 AM Just to make sure I have the proper version 2.3, I re-downloaded http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-surefire-report-plugin/2.3/maven-surefire-report-plugin-2.3.jar . The plugin.xml contained in this JAR defines a <mojo> with <goal>report-only</goal> that is mapped to a class named SurefireReportOnlyMojo which is also contained in the JAR. Based on these practical observations, I would consider the "report-only" goal to be released

The goal "report-only" actually exists in version 2.3, but when you launch a "mvn install site", you can see that the tests get executed twice. Which is a bug.
This bug is fixed in SVN. So if you use version 2.3.1-SNAPSHOT, this works perfectly.

Fabrice Bellingard
added a comment - 24/Aug/07 4:07 AM The goal "report-only" actually exists in version 2.3, but when you launch a "mvn install site", you can see that the tests get executed twice. Which is a bug.
This bug is fixed in SVN. So if you use version 2.3.1-SNAPSHOT, this works perfectly.

Luc, report-only is not an option/parameter but a plugin goal and these can in general not be directly activated via a property from the CLI. You can however add a profile to your POM that runs report-only and have this profile be activated via a system property.

Benjamin Bentmann
added a comment - 06/Nov/08 10:32 AM Luc, report-only is not an option/parameter but a plugin goal and these can in general not be directly activated via a property from the CLI. You can however add a profile to your POM that runs report-only and have this profile be activated via a system property.

i added the report-only option to ALL report entries in my multi module project. still some project rerun
the complete project ( 20 modules , 3 levels of grandparent/parent/child)
this is killing my builds and memory

i also added this for the javadoc plugin because it was doing the same thing

i basicly want to do "mvn clean deploy site-deploy" every night so we have new deployed artifacts and update project site after
every (nightly) build cycle.

for now , the site-deploy stuff is getting to complex because of all this issues

Luc Willems
added a comment - 07/Nov/08 3:07 PM indead , i was going to fast here
i added the report-only option to ALL report entries in my multi module project. still some project rerun
the complete project ( 20 modules , 3 levels of grandparent/parent/child)
this is killing my builds and memory
i also added this for the javadoc plugin because it was doing the same thing
i basicly want to do "mvn clean deploy site-deploy" every night so we have new deployed artifacts and update project site after
every (nightly) build cycle.
for now , the site-deploy stuff is getting to complex because of all this issues

Luc, what you describe (forking the entire multi-module build by a single module) looks like unintended usage of aggregator goals. These are goals whose doc pages say "Executes as an aggregator plugin.".

The goal report-only from the latest Surefire Report Plugin 2.4.3 does not work as as aggregator and hence should not cause you issues. As for the Javadoc Plugin, its latest release 2.5 includes those problematic aggregator goals. Unless you explicitly configured which report goals to execute, all reports from a plugin are executed. So be sure to setup the Javadoc Plugin as outlined in Selective Javadocs Reports and leave the *-aggregate goals out.

Benjamin Bentmann
added a comment - 07/Nov/08 3:22 PM Luc, what you describe (forking the entire multi-module build by a single module) looks like unintended usage of aggregator goals. These are goals whose doc pages say "Executes as an aggregator plugin.".
The goal report-only from the latest Surefire Report Plugin 2.4.3 does not work as as aggregator and hence should not cause you issues. As for the Javadoc Plugin, its latest release 2.5 includes those problematic aggregator goals. Unless you explicitly configured which report goals to execute, all reports from a plugin are executed. So be sure to setup the Javadoc Plugin as outlined in Selective Javadocs Reports and leave the *-aggregate goals out.

Kristian Rosenvold
added a comment - 22/Jan/11 11:05 AM Fixed with IT in r1062201.
2.7.2 will be staged for testing in the next 24 hours12 or so, so if you're one of the numerous watchers of this issue you can contribute by testing the staged release.