Hello Björn,
I've analyzed this and discovered some very important points to make sure
everyone understands, so I hope my information below is very clear. Please
don't hesitate to ask any questions for clarification. The items fall into 2
categories.

Advertising

First item:
========
The first item for discussion is a known weakness of the tool shed upload
process (or, to be more correct, the process of adding a new changeset revision
to a repository). When you commit a changeset to a repository that has
previous revisions, the tool shed repository metadata process is executed and
the contents of the new changeset are analyzed to determine if a new metadata
record should be created for the changeset or if the previous metadata record
should be moved up in the repository changelog chain (this is a very complex
analysis).
Until about 1 year ago, I reset all metadata on the entire repository
chjangelog when a new changeset was committed. This process worked very well,
and metadata was cleanly set for the entire changelog. However, this process
began to take significant periods of time as soon as repositories became larger
and the changelogs became longer (a good example is
http://testtoolshed.g2.bx.psu.edu/view/mzytnicki/s_mart).
To improve performance when adding new changesets to repositories, the metadata
analysis process was enhanced to inspect only those changesets that go back to
the previous revision with associated metadata. This process works very well
in most cases. However, there are unusual cases where the process breaks, and
your confab repository fell into this category. The issue arises when the
contents of a specific change set do not properly move the previous metadata
revision forward in the changelog, but instead create a new, additional
metadata revision.
When I initially inspected your confab repository had metadata associated with
the following revisions:
9:dd3ee8e742dc
8:113e876c2ec6
7:9e38b8bd4cdb
6:7593411dcd5a
5:aac0c82ac354
4:6c8f72ee4a51
3:09acaeb233d1
2:e7bb18ef7f54
1:49274c60f392
0:ea7816847e5e
I inspected the main contents (tools and dependencies) of these revisions and
noticed that some did not seem to have any differences (contents of the
containers had the same labels between revisions - I did not inspect the
changelog itself though). Because of this I decided to reset all metadata on
the repository (available in the Repository Actions menu), and now metadata is
associated with only the following revisions:
9:dd3ee8e742dc
6:7593411dcd5a
5:aac0c82ac354
3:09acaeb233d1
2:e7bb18ef7f54
1:49274c60f392
0:ea7816847e5e
This is a very complex issue to handle, as there are many things to consider.
Obviously automatically inspecting the entire changelog when a new changeset is
added will not work. I have added the ability for a repository owner to reset
all metadata on the repository, but there is nothing to alert them to initiate
this process (and I'm not sure how / when to do so). Also, resetting all
metadata on a repository eliminates the tool test results associated with each
of the original metadata revisions.
Second item:
==========
It seems there is some confusion about the changeset revision that is defined
in dependency definitions. This is a fairly complex subject, and it seems I
may have muddied the waters a bit yesterday. If so, I apologize, but I think
Peter's last response to my example clarifies the issue.
Looking at the changelog of your confab repository, I see the following. You
have unnecessarily changed the changeset_revision setting in your tool
dependency definitions, and this has resulted in a new additional metadata
record being associated with changeset 9:dd3ee8e742dc of your confab
repository. This changeset revision setting should not have been changed - see
my discussion below.
Repository 'confab'
===============
Changeset 9e38b8bd4cdb
modified:
tool_dependencies.xml
tool_dependencies.xml
--- a/tool_dependencies.xmlTue May 14 04:12:06 2013 -0400
+++ b/tool_dependencies.xmlTue May 14 04:17:08 2013 -0400
@@ -1,13 +1,13 @@
<tool_dependency>
<package name="eigen" version="2.0.17">
- <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b"
prior_installation_required="True" />
+ <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0"
prior_installation_required="True" />
</package>
<package name="confab" version="1.0.1">
<install version="1.0">
<actions>
<!-- populate the environment variables from the dependend
repos -->
<action type="set_environment_for_install">
- <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b">
+ <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0">
<package name="eigen" version="2.0.17" />
</repository>
</action>
Looking at the changelog of your package_eigen_2_0 repository, I see the
following:
Repository 'package_eigen_2_0'
==========================
Changesets
Repository metadata is associated with this change set.
Description: Uploaded
Commit: 2:09eb05087cd0
Parent: 1:294a30630e0b
Commited by: bgruening
Pushed: ~ 11 hours ago
Description: Uploaded
Commit: 1:294a30630e0b
Parent: 0:5c44ce7e3f7b
Commited by: bgruening
Pushed: ~ 11 hours ago
Description: Uploaded
Commit: 0:5c44ce7e3f7b
Parent: None
Commited by: bgruening
Pushed: ~ 11 hours ago
Notice that metadata is associated with only changeset revision 2:09eb05087cd0.
What this implies is that any existing dependency definition that defines the
changeset_revision value as one of 5c44ce7e3f7b, 294a30630e0b or 09eb05087cd0
will all get the same changeset revision installed, and that installed revision
will be 09eb05087cd0. This is a very important point, so please let me know if
the reasons for this are not clear.
Yesterday we had a discussion about the "minimum" setting for a changeset
revision. Here is Peter's feedback at the end of that discussion.
====================
Perhaps this is clearer?:
changset revision Installable revision
-----------------------------------------------------------------------------
0: sjekvub not any more, superseded by r1
1: jjtofvp not any more, superseded by r2
2: htocegy yes, final revision with this metadata
-----------------------------------------------------------------------------
3: jswofpt not any more, superseded by r4
4: jaqvkrc yes, final revision with this metadata
====================
Do you understand why you should not have changed the value of the
changeset_revision setting in the tool dependency definition your confab
repository? It is because this concept of "minimum" setting comes into play
here. When the value of the changeset_revision setting is changed, it will
always generate a new metadata revision, which is not what you wanted in this
case.
- <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b"
prior_installation_required="True" />
+ <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0"
prior_installation_required="True" />
- <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b">
+ <repository toolshed="http://testtoolshed.g2.bx.psu.edu/&quot;
name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0">
Greg Von Kuster
On May 14, 2013, at 3:59 AM, Björn Grüning wrote:
> Hi Greg,
>
> during the addition of the next dependency (libeigen) I found an other
> bug related to repository updates. Please see revision 6c8f72ee4a51 of
> confab in the testtoolshed. I installed that and want to update to
> revision aac0c82ac354 with new dependency. I saw that a new metadata is
> associated, but I'm not able to update:
>
> 'The installed repository named 'confab' is current, there are no
> updates available.'
>
> Cheers,
> Bjoern
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/