[NEXUS-757] Offering to work on Maven reverse dependency view

[NEXUS-757] Offering to work on Maven reverse dependency view

This post has NOT been accepted by the mailing list yet.

Hello,

I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..

i understand that this functionality has been talked about on this forum before (i did a quick search but couldn't find the threads), so i was hoping to start a new thread and hear some thoughts on how i might go about it.. i took a look at Archiva to see how they were pulling this off and it looks like they are indexing all of their artifacts in a Derby db, so it's not much more complicated than a db query.. i didn't see any indexing going on in Nexus that covered dependencies, so i'm guessing that it's not going to be as trivial as exposing a new search criteria..

if i had to start now, i'd probably tackle this by using an in-memory DB to store store the dependency relationship between artifacts and let a background thread index all of the artifacts in the repo and populate the db.. i don't know enough about Lucene to know if it makes more sense to just make it part of that index, but for small-ish repos, it seems like an in-memory DB would suffice..

as far as exposing the data, a nice dependency tree in a new tab of the artifact detail view seems natural, especially if we can flip between upstream and downstream dependencies.

any thoughts would be much appreciated, and any hints on where to get started in the code would be nice, as well.. haven't had any experience messing with the Nexus code thus far..

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

Hello,

I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..

i understand that this functionality has been talked about on
this forum before (i did a quick search but couldn't find the threads),
so i was hoping to start a new thread and hear some thoughts on how i
might go about it.. i took a look at Archiva to see how they were
pulling this off and it looks like they are indexing all of their
artifacts in a Derby db, so it's not much more complicated than a db
query.. i didn't see any indexing going on in Nexus that covered
dependencies, so i'm guessing that it's not going to be as trivial as
exposing a new search criteria..

if i had to start now, i'd probably tackle this by using an
in-memory DB to store store the dependency relationship between
artifacts and let a background thread index all of the artifacts in the
repo and populate the db.. i don't know enough about Lucene to know if
it makes more sense to just make it part of that index, but for
small-ish repos, it seems like an in-memory DB would suffice..

as far as exposing the data, a nice dependency tree in a new tab
of the artifact detail view seems natural, especially if we can flip
between upstream and downstream dependencies.

any thoughts would be much appreciated, and any hints on where
to get started in the code would be nice, as well.. haven't had any
experience messing with the Nexus code thus far..

I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..

i understand that this functionality has been talked about on
this forum before (i did a quick search but couldn't find the threads),
so i was hoping to start a new thread and hear some thoughts on how i
might go about it.. i took a look at Archiva to see how they were
pulling this off and it looks like they are indexing all of their
artifacts in a Derby db, so it's not much more complicated than a db
query.. i didn't see any indexing going on in Nexus that covered
dependencies, so i'm guessing that it's not going to be as trivial as
exposing a new search criteria..

if i had to start now, i'd probably tackle this by using an
in-memory DB to store store the dependency relationship between
artifacts and let a background thread index all of the artifacts in the
repo and populate the db.. i don't know enough about Lucene to know if
it makes more sense to just make it part of that index, but for
small-ish repos, it seems like an in-memory DB would suffice..

as far as exposing the data, a nice dependency tree in a new tab
of the artifact detail view seems natural, especially if we can flip
between upstream and downstream dependencies.

any thoughts would be much appreciated, and any hints on where
to get started in the code would be nice, as well.. haven't had any
experience messing with the Nexus code thus far..

I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..

i understand that this functionality has been talked about on
this forum before (i did a quick search but couldn't find the threads),
so i was hoping to start a new thread and hear some thoughts on how i
might go about it.. i took a look at Archiva to see how they were
pulling this off and it looks like they are indexing all of their
artifacts in a Derby db, so it's not much more complicated than a db
query.. i didn't see any indexing going on in Nexus that covered
dependencies, so i'm guessing that it's not going to be as trivial as
exposing a new search criteria..

if i had to start now, i'd probably tackle this by using an
in-memory DB to store store the dependency relationship between
artifacts and let a background thread index all of the artifacts in the
repo and populate the db.. i don't know enough about Lucene to know if
it makes more sense to just make it part of that index, but for
small-ish repos, it seems like an in-memory DB would suffice..

as far as exposing the data, a nice dependency tree in a new tab
of the artifact detail view seems natural, especially if we can flip
between upstream and downstream dependencies.

any thoughts would be much appreciated, and any hints on where
to get started in the code would be nice, as well.. haven't had any
experience messing with the Nexus code thus far..

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..

i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?

I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..

i understand that this functionality has been talked about on
this forum before (i did a quick search but couldn't find the threads),
so i was hoping to start a new thread and hear some thoughts on how i
might go about it.. i took a look at Archiva to see how they were
pulling this off and it looks like they are indexing all of their
artifacts in a Derby db, so it's not much more complicated than a db
query.. i didn't see any indexing going on in Nexus that covered
dependencies, so i'm guessing that it's not going to be as trivial as
exposing a new search criteria..

if i had to start now, i'd probably tackle this by using an
in-memory DB to store store the dependency relationship between
artifacts and let a background thread index all of the artifacts in the
repo and populate the db.. i don't know enough about Lucene to know if
it makes more sense to just make it part of that index, but for
small-ish repos, it seems like an in-memory DB would suffice..

as far as exposing the data, a nice dependency tree in a new tab
of the artifact detail view seems natural, especially if we can flip
between upstream and downstream dependencies.

any thoughts would be much appreciated, and any hints on where
to get started in the code would be nice, as well.. haven't had any
experience messing with the Nexus code thus far..

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,
but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,
but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,
but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,
but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

As for the test dependency of Maven 2.0.9, that is used to test maven (as a client) interacting with Nexus.

As for building a pom, it is never going to be 100% because different profiles could be triggered (which can bring in other dependencies). Using the defaults (or close too) it should be accurate enough for most uses.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,
but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>
> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>
> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>
>
> thanks,
> Saleem.
>
> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:
>
>
> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>
> VELO
>
> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:
> Hello,
>
>
> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>
>
> i understand that this functionality has been talked about on
> this forum before (i did a quick search but couldn't find the threads),
> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were
> pulling this off and it looks like they are indexing all of their
> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered
> dependencies, so i'm guessing that it's not going to be as trivial as
> exposing a new search criteria..
>
>
> if i had to start now, i'd probably tackle this by using an
> in-memory DB to store store the dependency relationship between
> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if
> it makes more sense to just make it part of that index, but for
> small-ish repos, it seems like an in-memory DB would suffice..
>
>
> as far as exposing the data, a nice dependency tree in a new tab
> of the artifact detail view seems natural, especially if we can flip
> between upstream and downstream dependencies.
>
>
> any thoughts would be much appreciated, and any hints on where
> to get started in the code would be nice, as well.. haven't had any
> experience messing with the Nexus code thus far..
>
>
> thanks,
>
> Saleem.
>
>
>
>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

in case anyone is interested, i've pushed some changes up to the github repo that more or less makes this functional.. the UI bits seem to be working, and i've implemented a scheduled task to handle populating the reverse dependency mappings for a system startup & an event that updates the mappings for new artifacts coming into the system.. there are still a few things to do, but it seems to be working so far..

if anyone is so inclined, i'd love for someone to code review what's out there.. i've tried to borrow as much as i could from other plugins and examples, but i'm sure there's a better way to do some of this, and perhaps i'm missing a few things that are important and never knew about..

after fixing a couple of minor things (refreshing the view when the selected artifact changes & adding scope information so that the view can be filtered), i'm going to add in the Aether-based dependency resolution logic.. from what i can tell, i should be able to construct a Maven environment that allows the dependencies to be resolved according to the repositories defined in Nexus, but i'm pretty sure i'm going to need to setup a "local repository" and any resolved artifacts are going to get copied there.. i could clean this up afterwards to save on disk space, but it feels a little weird to have to do that.. if anyone has any ideas that could avoid that ugliness, please let me know..

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..Saleem.

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

took another few steps forward this week by implementing the Aether dependency resolution and fixing a few bugs..

one in particular might be worth noting in case this is something that can be addressed in future versions of Nexus.. essentially i ran into classloader issues with my plugin because of a conflict with the plexus container.. v1.9.1 of Nexus comes with 1.8.1 of some aether jars.. however, aether-connector-wagon:1.8.1 brings in plexus-container-default:1.5.5, which conflicts with sisu-inject-plexus, which these rest of Nexus uses.. i tried to upgrade to 1.12 of aether, which also uses sisu-inject-plexus, but that obviously caused problems with the provided versions of aether.. i assume these will line up again at some point in a future version of Nexus, but i wanted to point it out just in case someone needed to pay attention to it..

also, i've currently got a scheduled task define to get the reverse dependency mappings started and an event handler to update the mappings whenever an artifact is uploaded or deleted, but it's gotten really annoying to have to manually run the task every time i start up Nexus.. is there a recommended way or kicking off some processing when the Nexus server starts up?

in case anyone is interested, i've pushed some changes up to the github repo that more or less makes this functional.. the UI bits seem to be working, and i've implemented a scheduled task to handle populating the reverse dependency mappings for a system startup & an event that updates the mappings for new artifacts coming into the system.. there are still a few things to do, but it seems to be working so far..

if anyone is so inclined, i'd love for someone to code review what's out there.. i've tried to borrow as much as i could from other plugins and examples, but i'm sure there's a better way to do some of this, and perhaps i'm missing a few things that are important and never knew about..

after fixing a couple of minor things (refreshing the view when the selected artifact changes & adding scope information so that the view can be filtered), i'm going to add in the Aether-based dependency resolution logic.. from what i can tell, i should be able to construct a Maven environment that allows the dependencies to be resolved according to the repositories defined in Nexus, but i'm pretty sure i'm going to need to setup a "local repository" and any resolved artifacts are going to get copied there.. i could clean this up afterwards to save on disk space, but it feels a little weird to have to do that.. if anyone has any ideas that could avoid that ugliness, please let me know..

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..Saleem.

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

took another few steps forward this week by implementing the Aether dependency resolution and fixing a few bugs..

one in particular might be worth noting in case this is something that can be addressed in future versions of Nexus.. essentially i ran into classloader issues with my plugin because of a conflict with the plexus container.. v1.9.1 of Nexus comes with 1.8.1 of some aether jars.. however, aether-connector-wagon:1.8.1 brings in plexus-container-default:1.5.5, which conflicts with sisu-inject-plexus, which these rest of Nexus uses.. i tried to upgrade to 1.12 of aether, which also uses sisu-inject-plexus, but that obviously caused problems with the provided versions of aether.. i assume these will line up again at some point in a future version of Nexus, but i wanted to point it out just in case someone needed to pay attention to it..

also, i've currently got a scheduled task define to get the reverse dependency mappings started and an event handler to update the mappings whenever an artifact is uploaded or deleted, but it's gotten really annoying to have to manually run the task every time i start up Nexus.. is there a recommended way or kicking off some processing when the Nexus server starts up?

Why do you need to do this every time nexus starts up ? You could also catch the event when a pom is deployed and calculate the dep tree. The task would still be need to back fill a repository.

in case anyone is interested, i've pushed some changes up to the github repo that more or less makes this functional.. the UI bits seem to be working, and i've implemented a scheduled task to handle populating the reverse dependency mappings for a system startup & an event that updates the mappings for new artifacts coming into the system.. there are still a few things to do, but it seems to be working so far..

if anyone is so inclined, i'd love for someone to code review what's out there.. i've tried to borrow as much as i could from other plugins and examples, but i'm sure there's a better way to do some of this, and perhaps i'm missing a few things that are important and never knew about..

after fixing a couple of minor things (refreshing the view when the selected artifact changes & adding scope information so that the view can be filtered), i'm going to add in the Aether-based dependency resolution logic.. from what i can tell, i should be able to construct a Maven environment that allows the dependencies to be resolved according to the repositories defined in Nexus, but i'm pretty sure i'm going to need to setup a "local repository" and any resolved artifacts are going to get copied there.. i could clean this up afterwards to save on disk space, but it feels a little weird to have to do that.. if anyone has any ideas that could avoid that ugliness, please let me know..

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..Saleem.

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

Because the data is currently being stored in memory, whenever Nexus goes down, all of the artifact usage information is lost, so it needs to get repopulated.. i added an Initializable interface to my ScheduledTask, and that seems to work as long as an instance of that task is defined (even if it's triggered manually), but it would be nice if i didn't have to create an instance of the task first.. and i do have an event picking up new artifacts, it's just the need to recalculate the usage data that's driving me to need something that runs at server start time..

btw, "reverse dependencies" got to sounding a little clumsy, and not entirely accurate if i can start picking up references to plugins, as well, so i've renamed to "nexus-artifact-usage-plugin" in case anyone was watching the GitHub repo: https://github.com/saleemshafi/nexus-artifact-usage-plugin.

took another few steps forward this week by implementing the Aether dependency resolution and fixing a few bugs..

one in particular might be worth noting in case this is something that can be addressed in future versions of Nexus.. essentially i ran into classloader issues with my plugin because of a conflict with the plexus container.. v1.9.1 of Nexus comes with 1.8.1 of some aether jars.. however, aether-connector-wagon:1.8.1 brings in plexus-container-default:1.5.5, which conflicts with sisu-inject-plexus, which these rest of Nexus uses.. i tried to upgrade to 1.12 of aether, which also uses sisu-inject-plexus, but that obviously caused problems with the provided versions of aether.. i assume these will line up again at some point in a future version of Nexus, but i wanted to point it out just in case someone needed to pay attention to it..

also, i've currently got a scheduled task define to get the reverse dependency mappings started and an event handler to update the mappings whenever an artifact is uploaded or deleted, but it's gotten really annoying to have to manually run the task every time i start up Nexus.. is there a recommended way or kicking off some processing when the Nexus server starts up?

Why do you need to do this every time nexus starts up ? You could also catch the event when a pom is deployed and calculate the dep tree. The task would still be need to back fill a repository.

in case anyone is interested, i've pushed some changes up to the github repo that more or less makes this functional.. the UI bits seem to be working, and i've implemented a scheduled task to handle populating the reverse dependency mappings for a system startup & an event that updates the mappings for new artifacts coming into the system.. there are still a few things to do, but it seems to be working so far..

if anyone is so inclined, i'd love for someone to code review what's out there.. i've tried to borrow as much as i could from other plugins and examples, but i'm sure there's a better way to do some of this, and perhaps i'm missing a few things that are important and never knew about..

after fixing a couple of minor things (refreshing the view when the selected artifact changes & adding scope information so that the view can be filtered), i'm going to add in the Aether-based dependency resolution logic.. from what i can tell, i should be able to construct a Maven environment that allows the dependencies to be resolved according to the repositories defined in Nexus, but i'm pretty sure i'm going to need to setup a "local repository" and any resolved artifacts are going to get copied there.. i could clean this up afterwards to save on disk space, but it feels a little weird to have to do that.. if anyone has any ideas that could avoid that ugliness, please let me know..

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..Saleem.

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>

Re: [NEXUS-757] Offering to work on Maven reverse dependency view

Because the data is currently being stored in memory, whenever Nexus goes down, all of the artifact usage information is lost, so it needs to get repopulated.. i added an Initializable interface to my ScheduledTask, and that seems to work as long as an instance of that task is defined (even if it's triggered manually), but it would be nice if i didn't have to create an instance of the task first.. and i do have an event picking up new artifacts, it's just the need to recalculate the usage data that's driving me to need something that runs at server start time..

I see, so you should be able to persist the store to get around this.

btw, "reverse dependencies" got to sounding a little clumsy, and not entirely accurate if i can start picking up references to plugins, as well, so i've renamed to "nexus-artifact-usage-plugin" in case anyone was watching the GitHub repo: https://github.com/saleemshafi/nexus-artifact-usage-plugin.

took another few steps forward this week by implementing the Aether dependency resolution and fixing a few bugs..

one in particular might be worth noting in case this is something that can be addressed in future versions of Nexus.. essentially i ran into classloader issues with my plugin because of a conflict with the plexus container.. v1.9.1 of Nexus comes with 1.8.1 of some aether jars.. however, aether-connector-wagon:1.8.1 brings in plexus-container-default:1.5.5, which conflicts with sisu-inject-plexus, which these rest of Nexus uses.. i tried to upgrade to 1.12 of aether, which also uses sisu-inject-plexus, but that obviously caused problems with the provided versions of aether.. i assume these will line up again at some point in a future version of Nexus, but i wanted to point it out just in case someone needed to pay attention to it..

also, i've currently got a scheduled task define to get the reverse dependency mappings started and an event handler to update the mappings whenever an artifact is uploaded or deleted, but it's gotten really annoying to have to manually run the task every time i start up Nexus.. is there a recommended way or kicking off some processing when the Nexus server starts up?

Why do you need to do this every time nexus starts up ? You could also catch the event when a pom is deployed and calculate the dep tree. The task would still be need to back fill a repository.

in case anyone is interested, i've pushed some changes up to the github repo that more or less makes this functional.. the UI bits seem to be working, and i've implemented a scheduled task to handle populating the reverse dependency mappings for a system startup & an event that updates the mappings for new artifacts coming into the system.. there are still a few things to do, but it seems to be working so far..

if anyone is so inclined, i'd love for someone to code review what's out there.. i've tried to borrow as much as i could from other plugins and examples, but i'm sure there's a better way to do some of this, and perhaps i'm missing a few things that are important and never knew about..

after fixing a couple of minor things (refreshing the view when the selected artifact changes & adding scope information so that the view can be filtered), i'm going to add in the Aether-based dependency resolution logic.. from what i can tell, i should be able to construct a Maven environment that allows the dependencies to be resolved according to the repositories defined in Nexus, but i'm pretty sure i'm going to need to setup a "local repository" and any resolved artifacts are going to get copied there.. i could clean this up afterwards to save on disk space, but it feels a little weird to have to do that.. if anyone has any ideas that could avoid that ugliness, please let me know..

agreed, Andreas.. i'm expecting to try to get a super-set of the dependencies and would definitely rather err on the side of having false positives vs. missing something.. to that point, it'll be interesting to see if Aether, as Brian recommended, will allow me to get information about dependencies that are only relevant when a particular profile is active.. i'd like to be able to get all of the dependencies regardless of profile, but i'm not sure how that would work..

in any case, i'll do what i can and if anyone has any thoughts on the best way to get the maximum data for the dependencies, i'll do what i can to incorporate it.. the filter idea sounds interesting, and i'll try to collect the scope information, but i'm not sure if i'll get around to implementing the filter right away or not.. thanks for the suggestion, though..Saleem.

I really appreciate your effort here. When you write about the scope of the dependencies, I would suggest to go for the “maximum scope” of dependencies: When in doubt if some maven project really has a dependency to something else, better include it as a “false positive” than to remove it:

- It’s very painful if you miss some projects, because this means that the problem of the missed project might only hit you a couple of builds / weeks / releases later.

- If people get annoyed by too many false positives, it might be possible - depending on your approach – to register additional filters to your algorithm.

When thinking about the dependency definitions of the Maven pom, will you allow to filter for the different “scopes” like “compile”, “test”, “provided” and so on? That could be useful – for example test-dependencies are not so relevant for productive scenarios and so on.

alright, i've managed to make some progress, at least on the UI portion, so thanks for everyone's help with that..

the trouble i'm running into now is in processing the artifacts in the repository and determining their dependencies.. considering the surprising, but now understandable, fact that Nexus doesn't seem to make much use of Maven itself, perhaps this is a question better addressed to a Maven forum, but does anyone have any advice on how to process a pom file to get the artifact's dependencies?

i got as far as reading the POM and converting it to a MavenProject instance (btw, i'm using v2.0.9 of maven-project since that seems to be what's included (transitively) for the Nexus test cases), but getting a resolved list of artifacts is another matter.. i can pull a list of dependencies from the Model, but that only tells me what's actually written in the pom, it doesn't resolve property references or inherit version numbers and scope from the parent, etc..

i've been trying to hack my way through the code, but it's been slow going, especially since i'm not familiar enough with Plexus to understand exactly how everything's getting wired together.. i would appreciate any suggestions you all might have..

actually, i think what i'm looking for is a description of sonatype-all.js.. i see that it's using a rather old version of ExtJS, but is there some documentation around the Sonatype object and how i might go about hooking into the artifact detail tabs.. if there isn't anything written up about it, i can probably hack my through, i was just hoping there might be shortcut..

Youcan use GWT to write the UI. Right now Im away from real computer,but if you wanna go that path lemme know.

On Monday, May 16, 2011, Saleem Shafi <[hidden email]> wrote:
> thanks for tips.. i think the limitation of only displaying uses of an artifact that are also stored locally is fine.. the use case i'm trying to support is one where you are deploying your own artifacts into Nexus and want to understand which of them are dependent on a particular artifact.. i think it's actually better that it only picks up the locally stored artifacts that depend on the selected artifact..
>> i'm brand new to this, so i expect it will be slow going, but i've started a github project here: https://github.com/saleemshafi/nexus-reverse-dependency-plugin in case anyone else wants to join in..
>> i was also hoping to show the dependencies as a tab on the artifact detail view, i.e. next to the "Maven Information" and "Artifact Information" tabs that show up when you click on an artifact in the Repository view.. thanks Brian for the link to the reference on building a Nexus plugin.. any chance there's some information somewhere about the JavaScript that's available to me to augment the UI?
>>> thanks,> Saleem.>> On Mon, May 9, 2011 at 11:16 AM, Marvin Froeder <[hidden email]> wrote:>>> FWIW, Archiva only do that for locally stored artifacts (either hosted or proxied artifacts).
>> VELO>> On Wed, May 4, 2011 at 5:08 PM, Saleem Shafi <[hidden email]> wrote:> Hello,>>> I've been hoping to get a "reverse dependency" view in Nexus for a while, and i'm happy to see that it's on the roadmap (https://issues.sonatype.org/browse/NEXUS-757), but since it hasn't been implemented yet, i thought maybe i could try to help out..
>>> i understand that this functionality has been talked about on> this forum before (i did a quick search but couldn't find the threads),> so i was hoping to start a new thread and hear some thoughts on how i
> might go about it.. i took a look at Archiva to see how they were> pulling this off and it looks like they are indexing all of their> artifacts in a Derby db, so it's not much more complicated than a db
> query.. i didn't see any indexing going on in Nexus that covered> dependencies, so i'm guessing that it's not going to be as trivial as> exposing a new search criteria..>>
> if i had to start now, i'd probably tackle this by using an> in-memory DB to store store the dependency relationship between> artifacts and let a background thread index all of the artifacts in the
> repo and populate the db.. i don't know enough about Lucene to know if> it makes more sense to just make it part of that index, but for> small-ish repos, it seems like an in-memory DB would suffice..
>>> as far as exposing the data, a nice dependency tree in a new tab> of the artifact detail view seems natural, especially if we can flip> between upstream and downstream dependencies.
>>> any thoughts would be much appreciated, and any hints on where> to get started in the code would be nice, as well.. haven't had any> experience messing with the Nexus code thus far..
>>> thanks,>> Saleem.>>>>