Re: [Plplot-devel] make-cvs-tarball.sh is working now

* Alan W. Irwin <irwin@...> [2005-02-26 13:06]:
> Rafael, did you miss my suggestion to simply use cvs tag -c tagname?
No, I did not miss it. I simply did not understand it. With the comments
below I am even more confused:
> The -c option checks whether there have been any changes between the local
> version and repo and aborts if there is a problem with that. This is
> probably never going to happen (since somebody has to commit right at the
> instant when you are running your script in order to trigger such a
> circumstance), but if you have a decent message with the abort, the user of
> your script will know just to re-run it at a more opportune time. This
> result falls in parallel with what should happen if the examples don't build
> and run. In that case, Tom should also wait for a more opportune time to
> make a development release.
>
> In sum, I believe using the -c option for the cvs tag command inside
> scripts/make-cvs-tarball.sh is a somewhat cleaner solution that will allow
> proper named tags (v5_5_0, etc.) to mark in cvs our development release
> versions.
I do not understand what you are proposing. From what the CVS documentation
tell me about cvs tag -c, I completely fail to see how it can help. Please,
suggest a concrete patch against the make-cvs-tarball.sh script implementing
your idea. In Linux parlance, please "put up or shut up" :-)
--
Rafael

Thread view

With the recent changes made by Alan and I to fonts/Makefile.am,
include/Makefile.am and bindings/python/gcw/Makefile.am, the
make-cvs-tarball.sh script is working again.
Tom, you are able now to start your new RM duties. Please notice that each
time the script is run, it creates a tag (cvs-tarball-<date>-<time>) in the
CVS repository. These tags must be deleted by hand in order to avoid
clobbering the repository with useless tags.
--
Rafael

On 2005-02-26 21:18+0100 Rafael Laboissiere wrote:
> With the recent changes made by Alan and I to fonts/Makefile.am,
> include/Makefile.am and bindings/python/gcw/Makefile.am, the
> make-cvs-tarball.sh script is working again.
>
> Tom, you are able now to start your new RM duties. Please notice that each
> time the script is run, it creates a tag (cvs-tarball-<date>-<time>) in the
> CVS repository. These tags must be deleted by hand in order to avoid
> clobbering the repository with useless tags.
Rafael, did you miss my suggestion to simply use cvs tag -c tagname?
The -c option checks whether there have been any changes between the local
version and repo and aborts if there is a problem with that. This is
probably never going to happen (since somebody has to commit right at the
instant when you are running your script in order to trigger such a
circumstance), but if you have a decent message with the abort, the user of
your script will know just to re-run it at a more opportune time. This
result falls in parallel with what should happen if the examples don't build
and run. In that case, Tom should also wait for a more opportune time to
make a development release.
In sum, I believe using the -c option for the cvs tag command inside
scripts/make-cvs-tarball.sh is a somewhat cleaner solution that will allow
proper named tags (v5_5_0, etc.) to mark in cvs our development release
versions.
Alan
__________________________
Alan W. Irwin
email: irwin@...
phone: 250-727-2902
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the
Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________

* Alan W. Irwin <irwin@...> [2005-02-26 13:06]:
> Rafael, did you miss my suggestion to simply use cvs tag -c tagname?
No, I did not miss it. I simply did not understand it. With the comments
below I am even more confused:
> The -c option checks whether there have been any changes between the local
> version and repo and aborts if there is a problem with that. This is
> probably never going to happen (since somebody has to commit right at the
> instant when you are running your script in order to trigger such a
> circumstance), but if you have a decent message with the abort, the user of
> your script will know just to re-run it at a more opportune time. This
> result falls in parallel with what should happen if the examples don't build
> and run. In that case, Tom should also wait for a more opportune time to
> make a development release.
>
> In sum, I believe using the -c option for the cvs tag command inside
> scripts/make-cvs-tarball.sh is a somewhat cleaner solution that will allow
> proper named tags (v5_5_0, etc.) to mark in cvs our development release
> versions.
I do not understand what you are proposing. From what the CVS documentation
tell me about cvs tag -c, I completely fail to see how it can help. Please,
suggest a concrete patch against the make-cvs-tarball.sh script implementing
your idea. In Linux parlance, please "put up or shut up" :-)
--
Rafael

On 2005-02-26 22:26+0100 Rafael Laboissiere wrote:
> I do not understand what you are proposing.
That's okay; I didn't understand exactly what you were trying to accomplish
with your change to make-cvs-tarball.sh. :-)
Let's start over. What we need is something really simple. Instead of
manually tagging for the development release before running
scripts/make-cvs-tarball.sh, optionally do it as the first cvs command that
is executed by the script, and then retrieve that version with the cvs
export command with appropriate tag.
One minor issue is the bypass_rtag=yes option is programmed, but it doesn't
seem to be documented in the usage information.
Another minor issue is your current script doesn't cvs export using the tag
generated by that option. Thus,, there could be a synchronization problem
if somebody cvs committed between the cvs rtag command and the cvs export
command. I agree that scenario is unlikely, but the script should be
tightened up nevertheless to avoid that issue.
The most important issue is the name of the tags. We need symbolic tags of
the form v5_5_0, v5_5_1, ... coincident with the release of version 5.5.0,
5.5.1, etc. Instead, as the result of your change I see a whole series of
cvs tag names like cvs-tarball_2005-02-26-19-40-17 and
cvs-tarball_2005-02-26-13-38-29 which have nothing to do with version
numbers. Could you please not use those tag names and instead derive the
name from the release version in configure.ac? So right now, the release
version is 5.3.1.cvs.20040822 in configure.ac, and that would need to be
translated to "v5_3_1_cvs_20040822", but in normal use Tom would have set the
release version in configure.ac to 5.5.0 which would be translated to
"v5_5_0".
Of course if you use symbolic tags based on the release version in
configure.ac, then a final issue is there would have to be some logic to
delete the tag first if that tag already existed. Something like if the cvs
rtag command fails with a particular tag (which it will do if that tag has
already been used), remove that tag, and try the cvs rtag command a second
time. This would allow the tarball generation script to be run more than
once with bypass_rtag=yes when the version number in configure.ac is not
changed.
(This scenario is exactly similar to the manual method; typically you think
you have everything perfect for the release, you manually tag, you generate
the tarball, and then some last-minute issue raises its head, you make the
required change, delete the tag, and tag again with the same tag before
making the tarball. It would be wonderful to get rid of all these
complications by doing the tag removal [whenever it is required] properly
and automatically in a script.)
Alan
__________________________
Alan W. Irwin
email: irwin@...
phone: 250-727-2902
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the
Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________

* Alan W. Irwin <irwin@...> [2005-02-26 14:54]:
> On 2005-02-26 22:26+0100 Rafael Laboissiere wrote:
>
> >I do not understand what you are proposing.
>
> That's okay; I didn't understand exactly what you were trying to accomplish
> with your change to make-cvs-tarball.sh. :-)
That explains the mutual confusion.
> Let's start over.
Good idea.
> What we need is something really simple.
Agreed.
> Instead of manually tagging for the development release before running
> scripts/make-cvs-tarball.sh, optionally do it as the first cvs command that
> is executed by the script, and then retrieve that version with the cvs
> export command with appropriate tag.
This is exactly what the script does (or, at least, what it is expected to
do).
> One minor issue is the bypass_rtag=yes option is programmed, but it doesn't
> seem to be documented in the usage information.
I will eventually document this option. Actually, I added it during my
preliminary tests and was not yet sure whether I would keep it.
> Another minor issue is your current script doesn't cvs export using the tag
> generated by that option.
Are you sure? The version currently in CVS does exactly the opposite of
what you are saying. It cvs rtags the repository and then uses that very
same tag in the cvs export command. This is the intended behavior, but it
there may be a bug that causes what you saying.
> Thus,, there could be a synchronization problem if somebody cvs committed
> between the cvs rtag command and the cvs export command. I agree that
> scenario is unlikely, but the script should be tightened up nevertheless to
> avoid that issue.
See above.
> The most important issue is the name of the tags. We need symbolic tags of
> the form v5_5_0, v5_5_1, ... coincident with the release of version 5.5.0,
> 5.5.1, etc.
I fully agree.
> Instead, as the result of your change I see a whole series of cvs tag names
> like cvs-tarball_2005-02-26-19-40-17 and cvs-tarball_2005-02-26-13-38-29
> which have nothing to do with version numbers.
As I told already twice, these tags are temporary and should be deleted.
> Could you please not use those tag names and instead derive the name from
> the release version in configure.ac? So right now, the release version is 5.3.1.cvs.20040822 in
> configure.ac, and that would need to be translated to
> "v5_3_1_cvs_20040822", but in normal use Tom would have set the release
> version in configure.ac to 5.5.0 which would be translated to "v5_5_0".
I am not sure this is a good idea. In the process of generating the
tarball, the RM will need to run the script perhaps several times until he
gets it right. This means that tags like v5_0_0 may be created several
times. Besides the fact that there is a potential conflict situation (see
your comment below), I dislike this for two reasons: first, tags like v5_5_0
are really important and should be created/deleted like crazy in the
repository. Second, one of these tags can be left for a long time unnoticed
in the repository even if they are not final.
Doing what you propose would be too confusing and error prone. On the other
hand, having those cvs-tarball-<data>-<time> tags (which are unique) would
be okay, because we know that we can delete them safely. Besides, the the
date/time stamp helps us to know exactly when they where generated.
> Of course if you use symbolic tags based on the release version in
> configure.ac, then a final issue is there would have to be some logic to
> delete the tag first if that tag already existed. Something like if the cvs
> rtag command fails with a particular tag (which it will do if that tag has
> already been used), remove that tag, and try the cvs rtag command a second
> time.
No, no, no!!! That could result in real disasters! Imagine that we
released version 5.5.0 some weeks ago. Today, Joe Developer (not
necessarily the RM) decides to try make-cvs-tarball.sh (modified according
to your proposal above). He types:
path/to/make-cvs-tarball.sh -v 5.5.0
and then: BOOM!!! the precious tage v5_5_0 is deleted forever and there is
no way to recover it.
(I know, the chances of this happens is probably below 0.001%, but I am not
going to take the risk.)
> (This scenario is exactly similar to the manual method; typically you think
> you have everything perfect for the release, you manually tag, you generate
> the tarball, and then some last-minute issue raises its head, you make the
> required change, delete the tag, and tag again with the same tag before
> making the tarball. It would be wonderful to get rid of all these
> complications by doing the tag removal [whenever it is required] properly
> and automatically in a script.)
Deleting tags in CVS is a quite delicate issue, the CVS manual warns loudly
about it. I would never, never, NEVER manipulate important tags like v5_5_0
automatically in a script. Period.
With the current version of make-cvs-tarball.sh, the release precess is
actually quite simple:
0) Change the release number in AC_INIT in configure.ac
1) Generate the tarball
2) Test it
3) Are you happy with it?
NO:
Make changes in CVS and go back to 1)
YES:
4) Release it
5) Tag the cvs repository with:
cvs rtag -r cvs-tarball-<date>-<time> vN_M_P
(where cvs-tarball-<date>-<time> was obtained in step 1)
Notice that in the algorithm above, there is no need for deleting the
temporary tags constantly. We could even keep them in CVS, they are just
harmless junk. At any rate, I will write a trivial script that deletes them
automatically. It is very easy to get all the cvs-tarball-* tags in the
repository:
cvs -f log README | grep cvs-tarball_....-..-..-..-..-..:
Right now, we have cvs-tarball_2005-02-26-19-40-17, which resulted from my
tests some hours ago. I will soon write the promised delete-cvs-tarball-tags
script.
--
Rafael

On the issue of deletion of cvs tags, I agree it's best to be safe. There
really is no problem of proliferating tags.. cvs can handle it. At DejaNews,
one release system in particular was reknown for creating a tag any time you
sneezed, practically. So we wound up with *lots* of tags, perhaps thousands.
I was a little worried about it impacting performance at some point, but I
don't think we ever hit it.
From a UI perspective, it can be a bit daunting to wade through all those
tags whenever you do a cvs log. So don't! I use:
$ cat ~/.cvsrc
log -N
checkout -P
update -P -d
which sets 'cvs log' to not print any tag info. If I ever do want to look at
the tag info, I type
$ cvs -f log <file> | less
which makes it convenient to go hunting for a particular tag (add 'grep' or
'grep -v' at will). If we do have tags that are more "temporary" than others,
as long as they are named according to a unique convention, we could always
go clean them up later.
--
Maurice LeBrun mjl@...

* mjl@... <mjl@...> [2005-02-26 18:24]:
> On the issue of deletion of cvs tags, I agree it's best to be safe. There
> really is no problem of proliferating tags.. cvs can handle it. At DejaNews,
> one release system in particular was reknown for creating a tag any time you
> sneezed, practically. So we wound up with *lots* of tags, perhaps thousands.
> I was a little worried about it impacting performance at some point, but I
> don't think we ever hit it.
>
> >From a UI perspective, it can be a bit daunting to wade through all those
> tags whenever you do a cvs log. So don't! I use:
>
> $ cat ~/.cvsrc
> log -N
> checkout -P
> update -P -d
>
> which sets 'cvs log' to not print any tag info.
I am happy to agree with you in every and all points above.
> If I ever do want to look at the tag info, I type
>
> $ cvs -f log <file> | less
>
> which makes it convenient to go hunting for a particular tag (add 'grep' or
> 'grep -v' at will).
>
> If we do have tags that are more "temporary" than others, as long as they
> are named according to a unique convention, we could always go clean them
> up later.
Bingo! This is exactly what the recently committed script
rm-cvs-tarball-tags.pl does.
--
Rafael

On 2005-02-27 01:11+0100 Rafael Laboissiere wrote:
> * Alan W. Irwin <irwin@...> [2005-02-26 14:54]:
>> Another minor issue is your current script doesn't cvs export using the tag
>> generated by that option.
>
> Are you sure? The version currently in CVS does exactly the opposite of
> what you are saying. It cvs rtags the repository and then uses that very
> same tag in the cvs export command. This is the intended behavior, but it
> there may be a bug that causes what you saying.
Part of the problem is I read your script in a hurry, and assumed BRANCH
meant something else. Now that I understand your use of that variable, it
looks like your script will do exactly what you said it will do.
> No, no, no!!!
Your "no" and exclamation buttons are both stuck. :-)
You do make a good argument (although it could have been done in a sentence
or so) that somebody could come along and screw it up afterward if my
suggestion were adopted. So forget my suggestion.
>
> 0) Change the release number in AC_INIT in configure.ac
> 1) Generate the tarball
> 2) Test it
> 3) Are you happy with it?
> NO:
> Make changes in CVS and go back to 1)
> YES:
> 4) Release it
> 5) Tag the cvs repository with:
> cvs rtag -r cvs-tarball-<date>-<time> vN_M_P
> (where cvs-tarball-<date>-<time> was obtained in step 1)
Thanks for summarizing exactly how you think your script should be used.
I was unaware of that -r option for cvs rtag until just now, but I agree
that should work fine so long as the RM is careful to get the right
date-time AND the right vN_M_P. My experience (and I am sure you can
confirm this as well) is the RM is completely tired at exactly this point,
and it is extremely easy to make tag mistakes. Even Linus has done this
with the kernel. Thus, to give the RM all the
help possible at this critical point, how about outputting a help message
from your script as follows if the option to cvs rtag is exercised:
"If completely satisfied with TARBALLNAME, then after the file release of
that tarball you will probably want to tag the corresponding cvs version
with a more release-specific symbolic tagname as follows:
cvs rtag -r cvs-tarball-<date>-<time> vN_M_P_suffix"
TARBALLNAME, <date>-<time>, and vN_M_P_suffix would have the correct values
of course which is the whole point of this suggested help message to guide
the RM. (I have included _suffix here to be general since sometimes we do
have a suffix in our version number in configure.ac even when we are
making a tarball release. RC1, RC2, final, etc.)
Alan
__________________________
Alan W. Irwin
email: irwin@...
phone: 250-727-2902
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the
Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________

* Alan W. Irwin <irwin@...> [2005-02-26 21:45]:
> Part of the problem is I read your script in a hurry, and assumed BRANCH
> meant something else. Now that I understand your use of that variable, it
> looks like your script will do exactly what you said it will do.
No problem. I agree that the script is convoluted and I am not surprised
you misunderstood how it works.
> You do make a good argument (although it could have been done in a sentence
> or so) that somebody could come along and screw it up afterward if my
> suggestion were adopted. So forget my suggestion.
Maurice has also made good points about not manipulating important tags in
an automatic way and about temporary tags being harmless. I think that the
vN_M_P tags must be only created by hand by the RM.
> >0) Change the release number in AC_INIT in configure.ac
> >1) Generate the tarball
> >2) Test it
> >3) Are you happy with it?
> > NO:
> > Make changes in CVS and go back to 1)
> > YES:
> > 4) Release it
> > 5) Tag the cvs repository with:
> > cvs rtag -r cvs-tarball-<date>-<time> vN_M_P
> > (where cvs-tarball-<date>-<time> was obtained in step 1)
>
> Thanks for summarizing exactly how you think your script should be used.
I put an improved version of these instructions in www/README. I am not
sure it is the ideal place for them. Please, feel free to move it around
and fix the text (in particular my quick-&-dirty English).
> I was unaware of that -r option for cvs rtag until just now, but I agree
> that should work fine so long as the RM is careful to get the right
> date-time AND the right vN_M_P. My experience (and I am sure you can
> confirm this as well) is the RM is completely tired at exactly this point,
> and it is extremely easy to make tag mistakes. Even Linus has done this
> with the kernel. Thus, to give the RM all the
> help possible at this critical point, how about outputting a help message
> from your script as follows if the option to cvs rtag is exercised:
>
> "If completely satisfied with TARBALLNAME, then after the file release of
> that tarball you will probably want to tag the corresponding cvs version
> with a more release-specific symbolic tagname as follows:
>
> cvs rtag -r cvs-tarball-<date>-<time> vN_M_P_suffix"
>
> TARBALLNAME, <date>-<time>, and vN_M_P_suffix would have the correct values
> of course which is the whole point of this suggested help message to guide
> the RM. (I have included _suffix here to be general since sometimes we do
> have a suffix in our version number in configure.ac even when we are
> making a tarball release. RC1, RC2, final, etc.)
This is a good idea and I will think about it. Although, the script already
outputs the tag used, but before running the cvs export command. I usually
generate the tarball with a command like this:
path/to/make-cvs-tarball.sh 2>&1 | tee build.log
such that I see the progress on the screen and both stdout and stderr are
stored in build.log. I can later read in the file which cvs-tarball tag was
generated.
Tom, I hope you are taking notes from all this discussion!
--
Rafael