Description

Scenario:

dependency:copy-dependencies is used to copy a dependency artifact that is part of the same multi-module build.

The compile phase is executed, but not the package phase.

An example of this scenario is using maven-eclipse-plugin to import a Maven project with generated test (re)sources. In this case, one would execute "mvn generate-test-resources eclipse:eclipse" to make sure that the generated (re)sources are imported into the workspace (by default, maven-eclipse-plugin executes generate-sources and generate-resources, but not generate-test-sources and generate-test-resources).

Execute "mvn generate-test-resources" from the root project -> fails (because the compile phase is executed, but not the package phase)

Execute "mvn generate-test-resources" in project2 -> success (because the dependency is not part of the same build)

Root cause analysis: In the scenario described above (compile phase executed, package phase not executed), Artifact#getFile() points to the target/classes directory instead of the output artifact. dependency:copy-dependencies doesn't detect this situation and blindly attempts to execute the copy operation. This fails with the error message shown above. Note that even if the operation didn't fail, it would produce an unexpected result.

Proposed fix (see attached patch): Change maven-dependency-plugin to detect this situation and let it replace the original Artifact object by a new one resolved from the repository (which would then refer to the artifact generated by a previous build, exactly as in the mvn generate-resources case).

Activity

There is a similar issue in maven-war-plugin: MWAR-192. One of the proposed fixes for that issue is to build the artifact on-the-fly from the files in the target/classes folder of the dependency project (see the patch attached to MWAR-192). Something similar could also be implemented for the issue described here, but this would only work for JAR artifacts, not for other packagings (I originally encountered the issue in Axis2 with dependencies on MAR files and in another project with OSGi bundles produced by maven-bundle-plugin from Apache Felix).

Andreas Veithen
added a comment - 18/Apr/10 8:14 AM There is a similar issue in maven-war-plugin: MWAR-192 . One of the proposed fixes for that issue is to build the artifact on-the-fly from the files in the target/classes folder of the dependency project (see the patch attached to MWAR-192 ). Something similar could also be implemented for the issue described here, but this would only work for JAR artifacts, not for other packagings (I originally encountered the issue in Axis2 with dependencies on MAR files and in another project with OSGi bundles produced by maven-bundle-plugin from Apache Felix).

This is a problem with Maven Core, not the dependency plugin. It will only happen if you are doing just mvn compile or maybe mvn test on the multimodule build, because those modules haven't been packaged yet. When this happens, the core hands the plugin a reference to the classes folder, but we expect a jar. Instead of running mvn compile/test just always use mvn install and you'll be fine. Alternatively, bind the dependency plugin to the package phase and it won't run unless you do at least mvn package in which case all of the modules will be jar'd already.

The only workaround we could do in the plugin is to cause the dependency plugin to detect that it has a folder and either:
1) ignore this dependency
2) go and attempt to create a jar from the classes

Neither of these is a correct fix because in 1, the resulting folder/archive would be incomplete and 2 we have no way of reliably constructing the jar exactly as it would be done in its own pom.

brianfox brianfox
added a comment - 10/Aug/10 9:18 AM This is a problem with Maven Core, not the dependency plugin. It will only happen if you are doing just mvn compile or maybe mvn test on the multimodule build, because those modules haven't been packaged yet. When this happens, the core hands the plugin a reference to the classes folder, but we expect a jar. Instead of running mvn compile/test just always use mvn install and you'll be fine. Alternatively, bind the dependency plugin to the package phase and it won't run unless you do at least mvn package in which case all of the modules will be jar'd already.
The only workaround we could do in the plugin is to cause the dependency plugin to detect that it has a folder and either:
1) ignore this dependency
2) go and attempt to create a jar from the classes
Neither of these is a correct fix because in 1, the resulting folder/archive would be incomplete and 2 we have no way of reliably constructing the jar exactly as it would be done in its own pom.

Sorry Brian, but I have to reopen the issue because your analysis is incomplete:

1. It's not a problem in Maven Core. Maven Core works as designed and the dependency plugin has to take into account how Maven works. The problem is that the dependency plugin makes the assumption that Artifact#getFile() always refers to a plain file and not to a directory. That assumption is wrong, and the minimum would be to add a check and fail the build with a meaningful error message if the artifact refers to a directory.

2. Your enumeration of the possible solutions/workarounds suggests that you neither read the full description of the issue, nor did you have a look at the patch. What I suggest is to "replace the original Artifact object by a new one resolved from the repository (which would then refer to the artifact generated by a previous build, exactly as in the mvn generate-resources case)" if the original Artifact refers to a directory.

BTW, I used that approach successfully in a plugin I wrote specifically for Axis2, which does something quite similar to the dependency plugin, but for a highly specialized use case (namely generating an Axis2 repository based on the project dependencies of type aar and mar). See [1].

Andreas Veithen
added a comment - 10/Aug/10 1:10 PM Sorry Brian, but I have to reopen the issue because your analysis is incomplete:
1. It's not a problem in Maven Core. Maven Core works as designed and the dependency plugin has to take into account how Maven works. The problem is that the dependency plugin makes the assumption that Artifact#getFile() always refers to a plain file and not to a directory. That assumption is wrong, and the minimum would be to add a check and fail the build with a meaningful error message if the artifact refers to a directory.
2. Your enumeration of the possible solutions/workarounds suggests that you neither read the full description of the issue, nor did you have a look at the patch. What I suggest is to "replace the original Artifact object by a new one resolved from the repository (which would then refer to the artifact generated by a previous build, exactly as in the mvn generate-resources case)" if the original Artifact refers to a directory.
BTW, I used that approach successfully in a plugin I wrote specifically for Axis2, which does something quite similar to the dependency plugin, but for a highly specialized use case (namely generating an Axis2 repository based on the project dependencies of type aar and mar). See [1] .
[1] https://svn.apache.org/repos/asf/axis/axis2/java/core/trunk/modules/tool/axis2-repo-maven-plugin/src/main/java/org/apache/axis2/maven2/repo/AbstractCreateRepositoryMojo.java

I have also seen this issue manifest itself when trying to import an Existing Maven Project in to the Eclipse workspace using M2Eclipse 0.10.2 where the pom of the project that is being imported has a execution of maven-dependency-plugin:copy-dependencies at phase process-resources. (M2Eclipse has been configured with goal of process-resources on project import).

The execution of the goal works as expected when executed from command line (using Maven 2.2.1 and 3.0-beta-2), but when the goal is executed in Eclipse (and M2Eclipse), it fails execution at the first dependency that is resolved within the workspace.

The real break point is in the class CopyDependenciesMojo copyArtifact( Artifact artifact, boolean removeVersion ) method. Specifically, the copyFile( artifact.getFile(), destFile ) method fails. It fails because the destFileName is not correct.

A few lines above that copyFile method call, the following code is executed:

When artifact is resolved in M2Eclipse via workspace resolution, the artifact is resolved to a project folder that ends in "target/classes". It does not resolve to a file name like the other dependencies. DependencyUtil.getFormattedFileName( artifact, removeVersion ) then executes. If the "removeVersion" flag is false, then this method simply determines the file name by the following means:

So the filename is simply determined as "all characters to the right of the last occurance of separatorChar." In the case of a M2Eclipse workspace resolved project, this becomes "classes".

I agree with Andreas' proposed solution here. In the case of this specific use case, it is probably best to retrieve the last built artifact verses trying to copy a directory folder. You may not get the "latest and greatest" from your active development project that is resolved in the Eclipse workspace, but you would get at least some artifact to copy over from the local repository.

I tried Andreas' patch attached to this defect but it did not help in my case. I do not know why it was not working in my case as he set it up.

What I did for a workaround was make a modification to the CopyDependenciesMojo copyArtifact(Artifact, boolean) method. I added an "if" statement just before the call to the copyFile( artifact.getFile(), destFile ) method. This "if" statement simply looks to see if the artifact.getFile().getAbsolutePath() ends with the artifact.getArtifactHandler().getExtension(). If it fails this test, as this use case would, it simply logs a warning that the artifact is a directory and bypasses the copyFile call. Here is the code excerpt:

// Only copy the file if it is a real file and not a directory
if (artifact.getFile().getAbsolutePath().endsWith(artifact.getArtifactHandler().getExtension()))
{
copyFile( artifact.getFile(), destFile );

// Copy POM if asked
if ( isCopyPom() )
{

This workaround may not be the correct approach as it assumes that there is no use case where an artifact could fail this check and still be a file, but it does serve my needs for now. The "if" statement may however be a possible approach to identify when this siutation arises. I don't currently have the time to implement the proposed solution that Andreas suggested ...but I might tackle it in the future.

John Daniel
added a comment - 13/Aug/10 3:55 PM - edited I have also seen this issue manifest itself when trying to import an Existing Maven Project in to the Eclipse workspace using M2Eclipse 0.10.2 where the pom of the project that is being imported has a execution of maven-dependency-plugin:copy-dependencies at phase process-resources. (M2Eclipse has been configured with goal of process-resources on project import).
The execution of the goal works as expected when executed from command line (using Maven 2.2.1 and 3.0-beta-2), but when the goal is executed in Eclipse (and M2Eclipse), it fails execution at the first dependency that is resolved within the workspace.
The real break point is in the class CopyDependenciesMojo copyArtifact( Artifact artifact, boolean removeVersion ) method. Specifically, the copyFile( artifact.getFile(), destFile ) method fails. It fails because the destFileName is not correct.
A few lines above that copyFile method call, the following code is executed:
String destFileName = DependencyUtil.getFormattedFileName( artifact, removeVersion );
When artifact is resolved in M2Eclipse via workspace resolution, the artifact is resolved to a project folder that ends in "target/classes". It does not resolve to a file name like the other dependencies. DependencyUtil.getFormattedFileName( artifact, removeVersion ) then executes. If the "removeVersion" flag is false, then this method simply determines the file name by the following means:
destFileName = artifact.getFile().getName();
The File.getName() method looks like the following:
public String getName()
{
int index = path.lastIndexOf(separatorChar);
if (index < prefixLength) return path.substring(prefixLength);
return path.substring(index + 1);
}
So the filename is simply determined as "all characters to the right of the last occurance of separatorChar." In the case of a M2Eclipse workspace resolved project, this becomes "classes".
I agree with Andreas' proposed solution here. In the case of this specific use case, it is probably best to retrieve the last built artifact verses trying to copy a directory folder. You may not get the "latest and greatest" from your active development project that is resolved in the Eclipse workspace, but you would get at least some artifact to copy over from the local repository.
I tried Andreas' patch attached to this defect but it did not help in my case. I do not know why it was not working in my case as he set it up.
What I did for a workaround was make a modification to the CopyDependenciesMojo copyArtifact(Artifact, boolean) method. I added an "if" statement just before the call to the copyFile( artifact.getFile(), destFile ) method. This "if" statement simply looks to see if the artifact.getFile().getAbsolutePath() ends with the artifact.getArtifactHandler().getExtension(). If it fails this test, as this use case would, it simply logs a warning that the artifact is a directory and bypasses the copyFile call. Here is the code excerpt:
// Only copy the file if it is a real file and not a directory
if (artifact.getFile().getAbsolutePath().endsWith(artifact.getArtifactHandler().getExtension()))
{
copyFile( artifact.getFile(), destFile );
// Copy POM if asked
if ( isCopyPom() )
{
This workaround may not be the correct approach as it assumes that there is no use case where an artifact could fail this check and still be a file, but it does serve my needs for now. The "if" statement may however be a possible approach to identify when this siutation arises. I don't currently have the time to implement the proposed solution that Andreas suggested ...but I might tackle it in the future.
Anyway, I hope this helps.
Cheers!
John

Pierre-Arnaud Marcelot
added a comment - 18/Mar/11 9:39 AM Anything new on that issue?
I think we got some kind of related issue when building site in one of our projects at the Apache Directory project.
http://directory.markmail.org/thread/6m66jithffwj7kja

Any news on this issue? I run into this issue when trying to use the Sonar plugin for certain projects. Sonar executes a second build cycle after the main cycle, which presumably excludes the package goal, triggering this error.

aaron pieper
added a comment - 04/Oct/11 8:17 AM Any news on this issue? I run into this issue when trying to use the Sonar plugin for certain projects. Sonar executes a second build cycle after the main cycle, which presumably excludes the package goal, triggering this error.

1. Project A is compiled and published using classifier C
2. Project B declares project A as a dependency but neglectes to specify its classifier
3. maven-dependency-plugin attempts to copy the dependency from the "classes" directory instead of complaining that the dependency does not exist.

This could be another instance of what Andreas was describing in that maven-dependency-plugin sees that project A was compiled but not published. The proposed patch looks like it catches this use-case and throws the expected error (dependency not found).

Gili
added a comment - 24/Oct/11 10:52 AM I've also ran into this problem in the following case:
1. Project A is compiled and published using classifier C
2. Project B declares project A as a dependency but neglectes to specify its classifier
3. maven-dependency-plugin attempts to copy the dependency from the "classes" directory instead of complaining that the dependency does not exist.
This could be another instance of what Andreas was describing in that maven-dependency-plugin sees that project A was compiled but not published. The proposed patch looks like it catches this use-case and throws the expected error (dependency not found).

I had another look at the issue and it turns out that my proposed change no longer works on Maven 3 because of some subtle changes in the behavior of ArtifactResolver. I don't see any other way to get around this.

In my particular case (importing a complex multi-module build into Eclipse) I now recommend users to do this with "mvn -DskipTests=true install eclipse:eclipse" instead of "mvn generate-test-resources eclipse:eclipse". The drawback is that it increases the risk of getting into a chicken-and-egg problem where one wants to refresh the Eclipse project to investigate a build failure but it is not possible to do that because that build failure causes the Maven command used to import the sources into Eclipse to fail...

Andreas Veithen
added a comment - 01/Nov/11 8:34 AM I had another look at the issue and it turns out that my proposed change no longer works on Maven 3 because of some subtle changes in the behavior of ArtifactResolver. I don't see any other way to get around this.
In my particular case (importing a complex multi-module build into Eclipse) I now recommend users to do this with "mvn -DskipTests=true install eclipse:eclipse" instead of "mvn generate-test-resources eclipse:eclipse". The drawback is that it increases the risk of getting into a chicken-and-egg problem where one wants to refresh the Eclipse project to investigate a build failure but it is not possible to do that because that build failure causes the Maven command used to import the sources into Eclipse to fail...

Mike Carr
added a comment - 22/Feb/12 1:33 PM - edited I am still having the same issue with 2.4, I noticed that this error only occurs when I run from m2eclipse when the dependency projects are in the same workspace as the main project.
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.4:copy (copy-snap-bundles) on project felix-server: Error copying artifact from C:\p4\dev\server\XXXX\target\classes to C:\p4\XXX\server\XXXX\target\bundles\XXX-1.2.00.1000-SNAPSHOT.jar

I'm watching this bug because of the Cobertura plugin. We updated Sonar and our builds started failing. I noticed that this error was happening during code coverage analysis (in our WAR module). Switching to JaCoCo worked around the problem.

Dustin Parker
added a comment - 15/May/12 11:29 AM I'm watching this bug because of the Cobertura plugin. We updated Sonar and our builds started failing. I noticed that this error was happening during code coverage analysis (in our WAR module). Switching to JaCoCo worked around the problem.

Scenario:
* dependency:copy-dependencies is used to copy a dependency artifact that is part of the same multi-module build.
* The compile phase is executed, but not the package phase.

An example of this scenario is using maven-eclipse-plugin to import a Maven project with generated test (re)sources. In this case, one would execute "mvn generate-test-resources eclipse:eclipse" to make sure that the generated (re)sources are imported into the workspace (by default, maven-eclipse-plugin executes generate-sources and generate-resources, but not generate-test-sources and generate-test-resources).

Steps to reproduce:
* Unpack the attached test project and build the entire project once with "mvn install".
* Execute "mvn generate-resources" from the root project -> success (because the compile phase is not executed)
* Execute "mvn package" from the root project -> success (because the package phase is executed)
* Execute "mvn generate-test-resources" from the root project -> fails (because the compile phase is executed, but not the package phase)
* Execute "mvn generate-test-resources" in project2 -> success (because the dependency is not part of the same build)

Root cause analysis: In the scenario described above (compile phase executed, package phase not executed), Artifact#getFile() points to the target/classes directory instead of the output artifact. dependency:copy-dependencies doesn't detect this situation and blindly attempts to execute the copy operation. This fails with the error message shown above. Note that even if the operation didn't fail, it would produce an unexpected result.

Proposed fix (see attached patch): Change maven-dependency-plugin to detect this situation and let it replace the original Artifact object by a new one resolved from the repository (which would then refer to the artifact generated by a previous build, exactly as in the mvn generate-resources case).

Scenario:
* dependency:copy-dependencies is used to copy a dependency artifact that is part of the same multi-module build.
* The compile phase is executed, but not the package phase.

An example of this scenario is using maven-eclipse-plugin to import a Maven project with generated test (re)sources. In this case, one would execute "mvn generate-test-resources eclipse:eclipse" to make sure that the generated (re)sources are imported into the workspace (by default, maven-eclipse-plugin executes generate-sources and generate-resources, but not generate-test-sources and generate-test-resources).

Steps to reproduce:
* Unpack the attached test project and build the entire project once with "mvn install".
* Execute "mvn generate-resources" from the root project -> success (because the compile phase is not executed)
* Execute "mvn package" from the root project -> success (because the package phase is executed)
* Execute "mvn generate-test-resources" from the root project -> fails (because the compile phase is executed, but not the package phase)
* Execute "mvn generate-test-resources" in project2 -> success (because the dependency is not part of the same build)

Root cause analysis: In the scenario described above (compile phase executed, package phase not executed), Artifact#getFile() points to the target/classes directory instead of the output artifact. dependency:copy-dependencies doesn't detect this situation and blindly attempts to execute the copy operation. This fails with the error message shown above. Note that even if the operation didn't fail, it would produce an unexpected result.

Proposed fix (see attached patch): Change maven-dependency-plugin to detect this situation and let it replace the original Artifact object by a new one resolved from the repository (which would then refer to the artifact generated by a previous build, exactly as in the mvn generate-resources case).

Ian Brandt
added a comment - 01/Aug/12 4:14 PM - edited Andreas,
Would you be able to elaborate at all on your comment:
...my proposed change no longer works on Maven 3 because of some subtle changes in the behavior of ArtifactResolver.
I'm going to take a stab at implementing a fix for this, so any details you could provide would be helpful.

Ian Brandt
added a comment - 02/Aug/12 3:56 PM @Andreas: No worries. I'm going to work on some tests to better understand how the resolver behaves in various reactor scenarios. I'll see if I can't rediscover the issue you encountered.