Just a heads-up for anyone experiencing build failures like me.
Recently the forge macros were ported to use distprefix on f30. An
unrelated change followed and a new build was pushed to rawhide (maybe
without realising that it would also push the forge macro changes).
This change was introduced with redhat-rpm-config-121-1.fc30.
It looks like the forge macros weren't tested to be backwards
compatible, because packages that build successfully now fail to build
on rawhide (see, for example, build [0]).
The reason: The %{gosource} macro now expands to a different string on
rawhide. So, sources uploaded on <f30 won't be found by koji's
buildSRPMfromSCM on rawhide, and vice versa - see the failed SRPM
build [1].
The difference is the inclusion (or lack thereof) of a leading "v"
from the tag in the file name. Up to f29, it will include the leading
"v" in the Source tag (and file download, and source file entry, and
lookaside cache file), and on rawhide, it won't.
!!! This will break koji builds of all packages that use the forge macros
!!! (with a tag, not a commit) in rawhide.
!!! Koschei builds are already starting to fail because of this change.
Since backporting the change won't work due to distprefix not being
available, I think the leading "v" in the version part of the Source
tag has to be reintroduced.
But I won't upload two differently named source archives to the
lookaside cache just for building the same version of a package on
different releases of fedora.
I reported this issue at [2].
Fabio
[0]: https://koji.fedoraproject.org/koji/taskinfo?taskID=30225034
[1]: https://kojipkgs.fedoraproject.org//work/tasks/5035/30225035/build.log
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1639025

It looks like the forge macros weren't tested to be backwards
compatible, because packages that build successfully now fail to build
on rawhide (see, for example, build [0]).

Fabio,
You can’t assume the archive name returned by forge macros will be
stable over time.
First, because GitHub, GitLab, etc are not themselves stable over time.
They change their URLs and the files those URL generate regularly.
Sometimes they change due to issues filled by Fedora people (as happened
to GitLab recently after Neal Gompa pointed problems in their old URLs).
Second, because at any time they allow access to the same files through
multiple URLs, and the one Fedora chooses to use may change over time,
due to better understanding of the (undocumented) way forge work, or the
old URL not working for some projects, and so on.
The forge macros abstract all this complexity away, so you do not need
to read the wiki and rewrite your spec file every time Fedora a hosting
service changes its rules or Fedora changes its preferred URL pattern.
The drawback being, the resulting archive filename *may* change from the
one in the lookaside cache over time. You can not both adapt
transparently to changes, and not-change filenames impacted by those
changes.
In the specific case you hit the old code required that you set a tag
and the new code will work even without a vversion tag, and recognizes
that the tag you set is really a version tag. So, net win for everyone
except packagers that had to workaround via a tag the fact the older
code was not smart enough.
The macro still computes for your spec a correct archive URL and a
correct archive name. Archive name which BTW is better than the previous
one since it removes the v GitHub loves to inject in random places, and
matches the archive name one would get today in clicking on GitHub’s
download link (no idea if the old URL you used before still works and
even if it did how long GitHub will keep it working now it switched to
newer better API for its own service).
Also, the next version of the forge macro with is being proposed in
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
will give you more flexibility and isolate archive-specific metadata
block so you can build a download for a tag unrelated to the package
version if that's what you want.
So, the problem you hit is annoying but it's only a spectool away. And
we should probably sync older Fedora releases to the same level of forge
macros once the new redhat-rpm-config gets some use in devel.
Now, if you want to avoid this kind of problem in the future:
— lobby hosting services to commit to and document a stable set of APIs
to retrieve project archives (not just an URL set in stone, an URL that
produces the same filename with the same topdir inside). That's
basically what Neal did for Gitlab and why I have good hopes that GitLab
archive and url naming will be stable now
— lobby git upstream to create a first-class release tag, with the same
syntax for every hosting service and project, so I can kill the huge
pile of heuristics that tries to guess what a specific project will use.
An heuristic as everyone knows is something that works perfectly most of
the time and fails miserably the rest of the time. And it is definitely
*not* stable over time.
Alternatively, you can hardcode an url and archive name in your spec
files, the result will be stable within the spec, but probably not work
anymore whenever you need to bump the project version or commit.
Regards,
--
Nicolas Mailhot

Le dimanche 14 octobre 2018 à 15:37 +0200, Fabio Valentini a écrit :
> It looks like the forge macros weren't tested to be backwards
> compatible, because packages that build successfully now fail to
> build
> on rawhide (see, for example, build [0]).
Fabio,
You can’t assume the archive name returned by forge macros will be
stable over time.

And, BTW, I did lobby the lookaside maintainer last year to stop
considering a source unit is a checksummed archive, but to consider two
archives which content checksums the same as identical (checksum content
not enveloppe, since enveloppes are not stable in our brand new web2
world).
That went nowhere but feel free to explain him the problem better than
me.
--
Nicolas Mailhot

Le dimanche 14 octobre 2018 à 15:37 +0200, Fabio Valentini a écrit :
>
> It looks like the forge macros weren't tested to be backwards
> compatible, because packages that build successfully now fail to build
> on rawhide (see, for example, build [0]).
Fabio,
You can’t assume the archive name returned by forge macros will be
stable over time.

Why not? I would surely hope that the file names are stable.
Otherwise packages might just randomly break in rawhide. Oh, wait ...

First, because GitHub, GitLab, etc are not themselves stable over
time.
They change their URLs and the files those URL generate regularly.
Sometimes they change due to issues filled by Fedora people (as happened
to GitLab recently after Neal Gompa pointed problems in their old URLs).

You know that GitHub has supported the same thing for a long time?
The URL
"https://github.com/project/repo/archive/$REF/whatever-I-feel-like-1.0.tar.gz"
will produce an archive of ref $REF (be it a tag, commit, or branch)
with the file name "whatever-I-feel-like-1.0.tar.gz". The forge macros
just don't make use of that feature, but use the old "#anchor hack",
which not even the SourceURL packaging guidelines mention anymore.

Second, because at any time they allow access to the same files
through
multiple URLs, and the one Fedora chooses to use may change over time,
due to better understanding of the (undocumented) way forge work, or the
old URL not working for some projects, and so on.
The forge macros abstract all this complexity away, so you do not need
to read the wiki and rewrite your spec file every time Fedora a hosting
service changes its rules or Fedora changes its preferred URL pattern.
The drawback being, the resulting archive filename *may* change from the
one in the lookaside cache over time. You can not both adapt
transparently to changes, and not-change filenames impacted by those
changes.

Of course you can - if you map "unstable URLs" to the expected file names.
I thought that's what the "abstraction" is for.

In the specific case you hit the old code required that you set a
tag
and the new code will work even without a vversion tag, and recognizes
that the tag you set is really a version tag. So, net win for everyone
except packagers that had to workaround via a tag the fact the older
code was not smart enough.
The macro still computes for your spec a correct archive URL and a
correct archive name. Archive name which BTW is better than the previous
one since it removes the v GitHub loves to inject in random places, and
matches the archive name one would get today in clicking on GitHub’s
download link (no idea if the old URL you used before still works and
even if it did how long GitHub will keep it working now it switched to
newer better API for its own service).

The URL used for the download is _exactly_ the same as before this
change, just the "#repo-vVersion.tar.gz" anchor is different now than
it was before - so that's entirely in macros.forge's hands and has
nothing to do with GitHub.

Also, the next version of the forge macro with is being proposed in
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
will give you more flexibility and isolate archive-specific metadata
block so you can build a download for a tag unrelated to the package
version if that's what you want.
So, the problem you hit is annoying but it's only a spectool away. And
we should probably sync older Fedora releases to the same level of forge
macros once the new redhat-rpm-config gets some use in devel.

Now, if you want to avoid this kind of problem in the future:
— lobby hosting services to commit to and document a stable set of APIs
to retrieve project archives (not just an URL set in stone, an URL that
produces the same filename with the same topdir inside). That's
basically what Neal did for Gitlab and why I have good hopes that GitLab
archive and url naming will be stable now

You know that github supports the same thing already, right? (Except
the "v" that's inserted before tags that don't match SemVer.)

— lobby git upstream to create a first-class release tag, with the
same
syntax for every hosting service and project, so I can kill the huge
pile of heuristics that tries to guess what a specific project will use.
An heuristic as everyone knows is something that works perfectly most of
the time and fails miserably the rest of the time. And it is definitely
*not* stable over time.

Sure, upstream projects can always do stupid things ... but I don't
think that "stupid" should be the base-line here.

Alternatively, you can hardcode an url and archive name in your spec
files, the result will be stable within the spec, but probably not work
anymore whenever you need to bump the project version or commit.

I've been doing exactly that for all my ~50 github based projects for
two years now, following this scheme:
for stable releases:
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{version}/%{name}-%{version}.tar.gz
- %{autosetup}
for snapshots (setting commit myself and computing shortcommit automatically):
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{commit}/%{name}-%{shortcommit}.tar.gz
- %autosetup -n %{name}-%{commit}
And this never broke. Not once.
Which is why the SourceURL packaging guidelines recommend using
exactly these schemes.
(They already do so for github for a long time, and I updated them
some time ago for the new and better URLs that gitlab supports now.)
But I've been trying to tell you this when the original forge macros
were introduced, and that has led nowhere either.
So, I'll probably just gradually move back to using the Source URLs
for github/gitlab which are _actually documented_ in the Guidelines
and produce _stable_ results.
Fabio
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Actually, they do, that's the exact change you're complaining about
here.
And whatever-I-feel-like-1.0.tar.gz is useless because the topdir inside
whatever-I-feel-like-1.0.tar.gz is definitively not whatever-I-feel-
like-1.0, so you need to compute the actual topdir GitHub will generate
and while you're at it the filename better match it or much %setup
hilarity and packager unhappiness will ensue.
And besides the code handle other things that GiHub that have changed
behaviour over time.

but use the old "#anchor hack",
which not even the SourceURL packaging guidelines mention anymore.

When the code was written it matched the guidelines whenever they
produced working URLs (some guidelines didn’t). The guidelines and the
code have been improved and fixed since. One reason the guidelines have
been fixed is that the forge macro showed they were erroneous.

You know that github supports the same thing already, right? (Except
the "v" that's inserted before tags that don't match SemVer.)

Ha ha ha. “except” v. And v does not have any simple rule, it is GiHub
heuristic pile land, you have projects with some of their releases that
uses v and others not, and GitHub can not even make up its mind in how
to use it consistently for the same project release (it's in the url but
not in the archive). I so wish GitHub would just give up on the whole v
thing. Or do it all the time. But not sometimes yes, other times no.
Since Google uses v in googlecode, and other hosting services do not,
and GitHub tries to mirror everyone, their setup is doomed to fail as
long as they do not standardise release tags git-upstream-side. By they
still try to heuristic the problem away.

> — lobby git upstream to create a first-class release tag, with
the
> same
> syntax for every hosting service and project, so I can kill the huge
> pile of heuristics that tries to guess what a specific project will
> use.
> An heuristic as everyone knows is something that works perfectly
> most of
> the time and fails miserably the rest of the time. And it is
> definitely
> *not* stable over time.
Sure, upstream projects can always do stupid things ... but I don't
think that "stupid" should be the base-line here.
> Alternatively, you can hardcode an url and archive name in your spec
> files, the result will be stable within the spec, but probably not
> work
> anymore whenever you need to bump the project version or commit.
I've been doing exactly that for all my ~50 github based projects for
two years now, following this scheme:
for stable releases:
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{version}/%{name}-%{version}.tar.gz
- %{autosetup}
for snapshots (setting commit myself and computing shortcommit
automatically):
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{commit}/%{name}-%{shortcommit}.tar.gz
- %autosetup -n %{name}-%{commit}
And this never broke. Not once.
Which is why the SourceURL packaging guidelines recommend using
exactly these schemes.
(They already do so for github for a long time, and I updated them
some time ago for the new and better URLs that gitlab supports now.)
But I've been trying to tell you this when the original forge macros
were introduced, and that has led nowhere either.
So, I'll probably just gradually move back to using the Source URLs
for github/gitlab which are _actually documented_ in the Guidelines
and produce _stable_ results.
Fabio
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Le dimanche 14 octobre 2018 à 18:56 +0200, Fabio Valentini a écrit :
>
> You know that GitHub has supported the same thing for a long time?
> The URL "
> https://github.com/project/repo/archive/$REF/whatever-I-feel-like-1.0.tar.gz
> "
> will produce an archive of ref $REF (be it a tag, commit, or branch)
> with the file name "whatever-I-feel-like-1.0.tar.gz". The forge macros
> just don't make use of that feature,
Actually, they do, that's the exact change you're complaining about
here.
And whatever-I-feel-like-1.0.tar.gz is useless because the topdir inside
whatever-I-feel-like-1.0.tar.gz is definitively not whatever-I-feel-
like-1.0, so you need to compute the actual topdir GitHub will generate
and while you're at it the filename better match it or much %setup
hilarity and packager unhappiness will ensue.
And besides the code handle other things that GiHub that have changed
behaviour over time.
> but use the old "#anchor hack",
> which not even the SourceURL packaging guidelines mention anymore.
When the code was written it matched the guidelines whenever they
produced working URLs (some guidelines didn’t). The guidelines and the
code have been improved and fixed since. One reason the guidelines have
been fixed is that the forge macro showed they were erroneous.
> You know that github supports the same thing already, right? (Except
> the "v" that's inserted before tags that don't match SemVer.)
Ha ha ha. “except” v. And v does not have any simple rule, it is GiHub
heuristic pile land, you have projects with some of their releases that
uses v and others not, and GitHub can not even make up its mind in how
to use it consistently for the same project release (it's in the url but
not in the archive). I so wish GitHub would just give up on the whole v
thing. Or do it all the time. But not sometimes yes, other times no.
Since Google uses v in googlecode, and other hosting services do not,
and GitHub tries to mirror everyone, their setup is doomed to fail as
long as they do not standardise release tags git-upstream-side. By they
still try to heuristic the problem away.

Because right now, package builds prepared on fedora 27-29 will
result
in failing koji builds for rawhide - and nobody should have to install
rawhide to be able to build packages.

Well basically you can
1. use a rawhide vm on container of whatever to prepare your rawhide
srpms (but, as you noted, not cheap to setup)
2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory
that works – not tested)
3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide
behaviour (only takes changing dist definition in
/etc/macros.d/macros.dist to %{?distprefix}.fcxx
4. use a local backport of the code. You basicaly just need to insert
the following at line 251 or 252 of the macros.forge rawhide file
-- Workaround releases where distprefix is not used by default
local dist = rpm.expand("%{?dist}")
local edistprefix = rpm.expand(distprefix)
if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~=
edistprefix) then
rpmmacros.explicitset("dist", "%{?distprefix}" .. dist,verbose)
end
5. ask to accelerate the backport to stable. I can prepare the backport
PR, but applying the PR is out of my hands
6. ask redhat-rpm-config maintainers to process
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
since I have a backport already prepared and tested for this one
7. all of the above
I don’t know which solution best matches your workflow
And normally, you do stuff in and for rawhide first, and then backport,
but this is kind of a special case since the rawhide bit does not
usually extend to the srpm part, and Go packaging in Fedora is so
immature every release is pretty much a devel release from Go's POW, so
I understand where you're coming from.
Regards,
--
Nicolas Mailhot

From: "Nicolas Mailhot"
<nicolas.mailhot(a)laposte.net&gt;
To: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
Cc: "Fabio Valentini" <nicolas.mailhot(a)laposte.net&gt;
Sent: Monday, October 15, 2018 11:56:45 AM
Subject: Re: PSA: builds using forge macros with tags broken on rawhide
Le 2018-10-15 09:09, Fabio Valentini a écrit :
> Because right now, package builds prepared on fedora 27-29 will result
> in failing koji builds for rawhide - and nobody should have to install
> rawhide to be able to build packages.
Well basically you can
1. use a rawhide vm on container of whatever to prepare your rawhide
srpms (but, as you noted, not cheap to setup)
2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory
that works – not tested)
3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide
behaviour (only takes changing dist definition in
/etc/macros.d/macros.dist to %{?distprefix}.fcxx
4. use a local backport of the code. You basicaly just need to insert
the following at line 251 or 252 of the macros.forge rawhide file
-- Workaround releases where distprefix is not used by default
local dist = rpm.expand("%{?dist}")
local edistprefix = rpm.expand(distprefix)
if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~=
edistprefix) then
rpmmacros.explicitset("dist", "%{?distprefix}" .. dist,verbose)
end
5. ask to accelerate the backport to stable. I can prepare the backport
PR, but applying the PR is out of my hands
6. ask redhat-rpm-config maintainers to process
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
since I have a backport already prepared and tested for this one
7. all of the above

And what about backing up the breaking change in Rawhide? At least until there is a
backward compatible way of doing that(or it is backported in to the stable releases)?
To be honest we should not introduce deliberately backwards incompatible changes without
at least publicly announcing them way ahead(and in general avoid them). This creates
really bad experience for all and waste lot of time of the packagers.
JC
>
> I don’t know which solution best matches your workflow
>
> And normally, you do stuff in and for rawhide first, and then backport,
> but this is kind of a special case since the rawhide bit does not
> usually extend to the srpm part, and Go packaging in Fedora is so
> immature every release is pretty much a devel release from Go's POW, so
> I understand where you're coming from.
>
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>

And what about backing up the breaking change in Rawhide? At least
until there is a backward compatible way of doing that(or it is
backported in to the stable releases)?

We're talking about redhat-rpm-config there. No one wants to backport
redhat-rpm-config changes to stable directly, without some proof period
in devel first (and that decision is pretty much in FPC’s hands anyway).
The “fastest” path I see is several people telling FPC the change is ok
now and the backport should not be delayed any more. And maybe we are
all being overly cautious and the caution delay does more harm than good
.
I will prepare a backport PR.
Whether it's applied or not (and when) is something else. The change
that hit devel recently was queued in the review pipe two months ago.
And there's also PR35, that will add further changes, and has been
cooking quite a lot of time already. Though it will be mostly new
features, now that the forge URL fixes have hit devel.
Regards,
--
Nicolas Mailhot

Le vendredi 19 octobre 2018 à 04:58 -0400, Jakub Cajka a écrit :
> And what about backing up the breaking change in Rawhide? At least
> until there is a backward compatible way of doing that(or it is
> backported in to the stable releases)?
We're talking about redhat-rpm-config there. No one wants to backport
redhat-rpm-config changes to stable directly, without some proof
period
in devel first (and that decision is pretty much in FPC’s hands
anyway).
The “fastest” path I see is several people telling FPC the change is
ok
now and the backport should not be delayed any more. And maybe we are
all being overly cautious and the caution delay does more harm than
good
I will prepare a backport PR.

----- Original Message -----
> From: "Nicolas Mailhot" <nicolas.mailhot(a)laposte.net&gt;
> To: "Development discussions related to Fedora" <
devel(a)lists.fedoraproject.org&gt;
> Cc: "Fabio Valentini" <nicolas.mailhot(a)laposte.net&gt;
> Sent: Monday, October 15, 2018 11:56:45 AM
> Subject: Re: PSA: builds using forge macros with tags broken on rawhide
>
> Le 2018-10-15 09:09, Fabio Valentini a écrit :
>
> > Because right now, package builds prepared on fedora 27-29 will result
> > in failing koji builds for rawhide - and nobody should have to install
> > rawhide to be able to build packages.
>
> Well basically you can
> 1. use a rawhide vm on container of whatever to prepare your rawhide
> srpms (but, as you noted, not cheap to setup)
> 2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory
> that works – not tested)
> 3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide
> behaviour (only takes changing dist definition in
> /etc/macros.d/macros.dist to %{?distprefix}.fcxx
> 4. use a local backport of the code. You basicaly just need to insert
> the following at line 251 or 252 of the macros.forge rawhide file
> -- Workaround releases where distprefix is not used by default
> local dist = rpm.expand("%{?dist}")
> local edistprefix = rpm.expand(distprefix)
> if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~=
> edistprefix) then
> rpmmacros.explicitset("dist", "%{?distprefix}" ..
dist,verbose)
> end
> 5. ask to accelerate the backport to stable. I can prepare the backport
> PR, but applying the PR is out of my hands
> 6. ask redhat-rpm-config maintainers to process
> https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
> since I have a backport already prepared and tested for this one
> 7. all of the above
And what about backing up the breaking change in Rawhide? At least until
there is a backward compatible way of doing that(or it is backported in to
the stable releases)?
To be honest we should not introduce deliberately backwards incompatible
changes without at least publicly announcing them way ahead(and in general
avoid them). This creates really bad experience for all and waste lot of
time of the packagers.

By the way, with the latest update to redhat-rpm-config in rawhide
(122-1.fc30), *all* packages using the new go macros are now broken. With
that, I now have about 40 broken golang packages.
I have neither the time nor the patience to fix them.
Fabio

Yeah all "gosetup -q" (which is gofed default) are broken
because of
your commit:

Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).
And I doubt -q is your problem, since (*precisely for backwards
compatible reasons) it is still accepted by forgesetup (even though it's
ignored, because it’s the default behaviour now).
Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
was already its default behaviour. So you're forcing somewhere a -q that
I don't think was ever needed. I will add the -q to the macro definition
if that makes you feel better.
So, no biggie. Easy to fix. That's why such changes hit -devel before
anyone dreams of queuing them to stable.
What package exactly installs your gosetup macro in /usr/lib/rpm so I
can see what other things it tries to do? I see no macro file in the
Fedora gofed package manifests.
If you could PR your changes to the official Fedora packages that
coordinate Fedora go macros (the go-macros repository on github, that
will be rehosted in pagure as soon as the @rh contributors ok the move)
instead of doing it in other places, that would be a tad easier to
coordinate.

Please revert this, we should be able to use whatever flag supported
by %setup and
%autosetup, not a small subset. How do you even pass the basic -n
flags now?

So basically, you have a macro, which sole purpose is to pass
precomputed -n values to *setup, and you want to use it with manual -n
flags. Why? Could you tell us what exactly is the point of
1. wrapping a Fedora macro in another name just because you do not like
the official macro name in Fedora
2. overriding the only thing this macro does over autosetup
3. and then complaining all the argument passing to autosetup does not
work as you wish it does
So all this complexity, because what you really want is to use autosetup
directly. Then just do. What's the problem exactly with typing autosetup
in your spec? No one stops you from doing it.
One reason I removed the -n in forgeautosetup and forgesetup precisely
to stop packagers confusing themselves the way you are confusing
yourself.
The other being -n is incompatible with the processing of multiple
archives that many people asked me to add to the macros. Which is
finally implemented after months of work. And which just hit rawhide
after more than a month of review.
So, do you have actual spec files in Fedora with this use of the redhat-
rpm-config macros? If that is the case, I can put those flags in the
macros so you can continue shooting yourself in the foot using undefined
known-broken use patterns. If not, I'd rather remove the possibility
altogether before someone harms himself.

Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
:
> Yeah all "gosetup -q" (which is gofed default) are broken because of
> your commit:
Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually
added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).
And I doubt -q is your problem, since (*precisely for backwards
compatible reasons) it is still accepted by forgesetup (even though
it's
ignored, because it’s the default behaviour now).
Oh, I see, I forgot to add a phantom -q to forgeautosetup as it
already
was already its default behaviour. So you're forcing somewhere a -q
that
I don't think was ever needed. I will add the -q to the macro
definition
if that makes you feel better.
So, no biggie. Easy to fix. That's why such changes hit -devel before
anyone dreams of queuing them to stable.

Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
:
> Yeah all "gosetup -q" (which is gofed default) are broken because of
> your commit:
Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually
added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).

However, you *can* PR your own definition of gosetup to the Fedora go
macros package if you absolutely want it to exist. I will be happy to
support the PR so this macro has a maintainer in Fedora.
I'm not the Fedora maintainer of this package (even though, at this
point, I wrote most of it in line numbers). But I will be happy to
support your request.
Regards,
--
Nicolas Mailhot

From: "Nicolas Mailhot"
<nicolas.mailhot(a)laposte.net&gt;
To: "Robert-André Mauchin" <nicolas.mailhot(a)laposte.net&gt;,
"Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
Cc: "Nicolas Mailhot" <nicolas.mailhot(a)laposte.net&gt;
Sent: Monday, October 22, 2018 10:00:40 AM
Subject: Re: PSA: builds using forge macros with tags broken on rawhide
Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit :
> Yeah all "gosetup -q" (which is gofed default) are broken because of
> your commit:
Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).
And I doubt -q is your problem, since (*precisely for backwards
compatible reasons) it is still accepted by forgesetup (even though it's
ignored, because it’s the default behaviour now).
Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
was already its default behaviour. So you're forcing somewhere a -q that
I don't think was ever needed. I will add the -q to the macro definition
if that makes you feel better.
So, no biggie. Easy to fix. That's why such changes hit -devel before
anyone dreams of queuing them to stable.

I think that it would be reasonable to test the changes against the Fedora package base
before even pushing the change in to the rawhide. This kind of unnecessary breakage is
easily avoidable if you would spent sometime on testing this by scratch rebuilding the
Fedora packages locally, prior pushing this.
Could we avoid any such kind of regressions in the future?
JC
>
> What package exactly installs your gosetup macro in /usr/lib/rpm so I
> can see what other things it tries to do? I see no macro file in the
> Fedora gofed package manifests.
>
> If you could PR your changes to the official Fedora packages that
> coordinate Fedora go macros (the go-macros repository on github, that
> will be rehosted in pagure as soon as the @rh contributors ok the move)
> instead of doing it in other places, that would be a tad easier to
> coordinate.
>
> > Please revert this, we should be able to use whatever flag supported
> > by %setup and
> > %autosetup, not a small subset. How do you even pass the basic -n
> > flags now?
>
> So basically, you have a macro, which sole purpose is to pass
> precomputed -n values to *setup, and you want to use it with manual -n
> flags. Why? Could you tell us what exactly is the point of
>
> 1. wrapping a Fedora macro in another name just because you do not like
> the official macro name in Fedora
> 2. overriding the only thing this macro does over autosetup
> 3. and then complaining all the argument passing to autosetup does not
> work as you wish it does
>
> So all this complexity, because what you really want is to use autosetup
> directly. Then just do. What's the problem exactly with typing autosetup
> in your spec? No one stops you from doing it.
>
> One reason I removed the -n in forgeautosetup and forgesetup precisely
> to stop packagers confusing themselves the way you are confusing
> yourself.
>
> The other being -n is incompatible with the processing of multiple
> archives that many people asked me to add to the macros. Which is
> finally implemented after months of work. And which just hit rawhide
> after more than a month of review.
>
> So, do you have actual spec files in Fedora with this use of the redhat-
> rpm-config macros? If that is the case, I can put those flags in the
> macros so you can continue shooting yourself in the foot using undefined
> known-broken use patterns. If not, I'd rather remove the possibility
> altogether before someone harms himself.
>
> > With you mods, we have to edit hundreds of specs to remove
> > the -q flags.
>
> And that concretely, if why pre-generating specs instead of making the
> effort to include default processing in the macro code Fedora ships is
> wrong.
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>

I think that it would be reasonable to test the changes against the
Fedora package base before even pushing the change in to the rawhide.
This kind of unnecessary breakage is easily avoidable if you would
spent sometime on testing this by scratch rebuilding the Fedora
packages locally, prior pushing this.

There's certainly many things that could be done better if someone took
the time to do it. But, everyone's time is limited including mine. I
already rebuild hundreds of Go packages regularly to QA the changes. It
takes days. Those 40 packages were not part of the set.
So, if we want more things to be done with the existing manpower, we
need to fix all the thousands of little things that waste the available
contributor time:
* rehost things cleanly on pagure
* clean up the maze of repurposed Go infra specs so any change or fix
can be QAed and pushed quickly
* contribute the fixes we want or need to the Fedora packages that ship
the corresponding code, instead of 'saving' time by doing messy
ill-thought workarounds in a private spec file generator
And then when things are clearly in one place this place can be used to
sync the reviews, QA runs and Koji/copr mass rebuilds. Which is clearly
not the case today.
Call that purism if you like
--
Nicolas Mailhot

From: "Nicolas Mailhot"
<nicolas.mailhot(a)laposte.net&gt;
To: "Jakub Cajka" <jcajka(a)redhat.com&gt;
Cc: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org&gt;
Sent: Monday, October 22, 2018 1:55:02 PM
Subject: Re: PSA: builds using forge macros with tags broken on rawhide
Le 2018-10-22 13:09, Jakub Cajka a écrit :
> I think that it would be reasonable to test the changes against the
> Fedora package base before even pushing the change in to the rawhide.
> This kind of unnecessary breakage is easily avoidable if you would
> spent sometime on testing this by scratch rebuilding the Fedora
> packages locally, prior pushing this.
There's certainly many things that could be done better if someone took
the time to do it. But, everyone's time is limited including mine. I
already rebuild hundreds of Go packages regularly to QA the changes. It
takes days. Those 40 packages were not part of the set.

In my experience this doesn't require that much time as this is fully scriptable from
getting all the dependent(Go based packages) and rebuilding them locally(even using
something dumb as mockchain, it takes under 1 day to complete for Go, without needing any
babysitting(in VM on i7/ssd laptop)). One day(of computation power) seems like reasonable
trade-off for not wasting time of significant number of (Go) packagers by breaking macros.
Or AFAIK it should be possible to do such change in the koji side tag(ideally with formal
change proposal) not affecting the main tag.

So, if we want more things to be done with the existing manpower, we
need to fix all the thousands of little things that waste the available
contributor time:
* rehost things cleanly on pagure
* clean up the maze of repurposed Go infra specs so any change or fix
can be QAed and pushed quickly
* contribute the fixes we want or need to the Fedora packages that ship
the corresponding code, instead of 'saving' time by doing messy
ill-thought workarounds in a private spec file generator

I'm sure that we all want to advance Fedora, but could you be more specific? What
needs to be done in your opinion? Would you mind opening issues in Go SIG for those
specific changes(or better PRs in respective packages)?
JC
>
> And then when things are clearly in one place this place can be used to
> sync the reviews, QA runs and Koji/copr mass rebuilds. Which is clearly
> not the case today.
>
> Call that purism if you like
>
> --
> Nicolas Mailhot
>

Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
:
> Yeah all "gosetup -q" (which is gofed default) are broken because of
> your commit:
Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).

Yet, you were aware of that. And, it was me who told you I am fine with
most of the go macros you implemented, including their names. We both
spent many hours improving the implementation, extending golist binary I
provided. And the %gosetup was the only macro I wanted to keep so all
the regular go macros are in %goVERB form. We have %gometa, %gosetup,
%gobuild, %goinstall, %gotest on purpose so they reflect:
- meta: after some effort even meta can be interpreted as a verb, i.e.
meta the macros
- setup: extract the tarball and prepare working dir
- build: set gopath, build go binaries
- install: install resources under proper directories
- test: test installed go packages
Do you ignore that fact on purpose? Since after discussion we had so
many times you still seem to put more emphasis on your own and only your
opinion. I though we made a compromise and that we agreed on that. I.e.
I will respect all the macro names you choose up to the only one and you
will respect the gosetup one. After having merged your changes you are
sending me a message that even if we come to a common agreement you
still don't care as long as you reach your own goals. I would really
like to think I am wrong in this opinion. Though, you actions say otherwise.

And I doubt -q is your problem, since (*precisely for backwards
compatible reasons) it is still accepted by forgesetup (even though it's
ignored, because it’s the default behaviour now).
Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
was already its default behaviour. So you're forcing somewhere a -q that
I don't think was ever needed. I will add the -q to the macro definition
if that makes you feel better.

You were the one that implemented the forge macros (pardon me if I am
wrong, I am unsure if it was all of it of most of it). And I love those
macros since they save so much time and make the spec file a lot
simpler. So its natural to be protective if anyone wants to change them
they way that does not align with your world view. Yet, go macros
started to rely on the forge macros and the change you made is not
backward compatible (given we started using the macros in older versions
of Fedora as well). So please, make the %forgeautosetup compatible with
the %gosetup again.
Once the %gosetup macro is functional again, we can discuss its removal
in favor of forgesetup as part of global announcement so all go
packagers have time to migrate to new macros.
For future reference, please don't change the forge macros in any way
that makes the go macros incompatible or broken until we have official
Go packaging guidelines and set of macros the go-sig community agrees on.
Thank you Nicolas

So, no biggie. Easy to fix. That's why such changes hit -devel
before
anyone dreams of queuing them to stable.
What package exactly installs your gosetup macro in /usr/lib/rpm so I
can see what other things it tries to do? I see no macro file in the
Fedora gofed package manifests.
If you could PR your changes to the official Fedora packages that
coordinate Fedora go macros (the go-macros repository on github, that
will be rehosted in pagure as soon as the @rh contributors ok the move)
instead of doing it in other places, that would be a tad easier to
coordinate.
> Please revert this, we should be able to use whatever flag supported
> by %setup and
> %autosetup, not a small subset. How do you even pass the basic -n
> flags now?
So basically, you have a macro, which sole purpose is to pass
precomputed -n values to *setup, and you want to use it with manual -n
flags. Why? Could you tell us what exactly is the point of
1. wrapping a Fedora macro in another name just because you do not like
the official macro name in Fedora
2. overriding the only thing this macro does over autosetup
3. and then complaining all the argument passing to autosetup does not
work as you wish it does
So all this complexity, because what you really want is to use autosetup
directly. Then just do. What's the problem exactly with typing autosetup
in your spec? No one stops you from doing it.
One reason I removed the -n in forgeautosetup and forgesetup precisely
to stop packagers confusing themselves the way you are confusing
yourself.
The other being -n is incompatible with the processing of multiple
archives that many people asked me to add to the macros. Which is
finally implemented after months of work. And which just hit rawhide
after more than a month of review.
So, do you have actual spec files in Fedora with this use of the redhat-
rpm-config macros? If that is the case, I can put those flags in the
macros so you can continue shooting yourself in the foot using undefined
known-broken use patterns. If not, I'd rather remove the possibility
altogether before someone harms himself.
> With you mods, we have to edit hundreds of specs to remove
> the -q flags.
And that concretely, if why pre-generating specs instead of making the
effort to include default processing in the macro code Fedora ships is
wrong.
Regards,

So please, make the %forgeautosetup
compatible with the %gosetup again.
Once the %gosetup macro is functional again, we can discuss its
removal in favor of forgesetup as part of global announcement so all
go packagers have time to migrate to new macros.
For future reference, please don't change the forge macros in any way
that makes the go macros incompatible or broken until we have official
Go packaging guidelines and set of macros the go-sig community agrees
on.
Thank you Nicolas
> So, no biggie. Easy to fix. That's why such changes hit -devel before
> anyone dreams of queuing them to stable.
>
> What package exactly installs your gosetup macro in /usr/lib/rpm so I
> can see what other things it tries to do? I see no macro file in the
> Fedora gofed package manifests.
>
> If you could PR your changes to the official Fedora packages that
> coordinate Fedora go macros (the go-macros repository on github, that
> will be rehosted in pagure as soon as the @rh contributors ok the
> move)
> instead of doing it in other places, that would be a tad easier to
> coordinate.
>
>> Please revert this, we should be able to use whatever flag supported
>> by %setup and
>> %autosetup, not a small subset. How do you even pass the basic -n
>> flags now?
> So basically, you have a macro, which sole purpose is to pass
> precomputed -n values to *setup, and you want to use it with manual -n
> flags. Why? Could you tell us what exactly is the point of
>
> 1. wrapping a Fedora macro in another name just because you do not
> like
> the official macro name in Fedora
> 2. overriding the only thing this macro does over autosetup
> 3. and then complaining all the argument passing to autosetup does not
> work as you wish it does
>
> So all this complexity, because what you really want is to use
> autosetup
> directly. Then just do. What's the problem exactly with typing
> autosetup
> in your spec? No one stops you from doing it.
>
> One reason I removed the -n in forgeautosetup and forgesetup precisely
> to stop packagers confusing themselves the way you are confusing
> yourself.
>
> The other being -n is incompatible with the processing of multiple
> archives that many people asked me to add to the macros. Which is
> finally implemented after months of work. And which just hit rawhide
> after more than a month of review.
>
> So, do you have actual spec files in Fedora with this use of the
> redhat-
> rpm-config macros? If that is the case, I can put those flags in the
> macros so you can continue shooting yourself in the foot using
> undefined
> known-broken use patterns. If not, I'd rather remove the possibility
> altogether before someone harms himself.
>
>> With you mods, we have to edit hundreds of specs to remove
>> the -q flags.
> And that concretely, if why pre-generating specs instead of making the
> effort to include default processing in the macro code Fedora ships is
> wrong.
>
> Regards,
>
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Jan,
I do not recall this all the same way, but no matter, that was months
ago, a long development tunnel away.
The thing which is failing today is a brittle and dangerous rpm macro
aliasing code that was added against my best technical advice. Sound
advice as we see now. And I had honestly forgotten about it. It serves
absolutely no technical purpose, it is buried inside an unrelated macro,
and it is not documented anywhere¹.
All the blame games in the world won't change where the technical
failure is.
So what do you want me to do now? Fix this aliasing code? That requires
rewriting it. The way it was written it can not keep working reliably
with or without redhat-rpm-config changes.
Rewrite it just so gosetup calls can be removed from Fedora specs? I'd
rather write the script to remove the problem calls directly.
Rewrite it so removal can be postponed indefinitely via endless vetos by
you or Jakub? That makes me the effective maintainer of code I have no
use for and disagree with.
I'm the one doing the work here. I'd appreciate if it was taken in
account sometimes.
Regards,
¹ Neither on https://fedoraproject.org/wiki/More_Go_packaging, nor on
https://fedoraproject.org/wiki/PackagingDrafts/Go, nor in the macro
files themselves.
--
Nicolas Mailhot

Anyway, since no one answers when I list the possible fixes and ask to
choose between them, I fixed the Fedora spec files myself.
Feel free to reintroduce %gosetup calls if you really want them, once
the %gosetup implementation has been fixed.
--
Nicolas Mailhot

Anyway, since no one answers when I list the possible fixes and ask
to
choose between them, I fixed the Fedora spec files myself.
Feel free to reintroduce %gosetup calls if you really want them, once
the %gosetup implementation has been fixed.
--
Nicolas Mailhot

Thanks for doing but you messed up the bumping of all the %changelog entries.

On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
> Anyway, since no one answers when I list the possible fixes and ask to
> choose between them, I fixed the Fedora spec files myself.
>
> Feel free to reintroduce %gosetup calls if you really want them, once
> the %gosetup implementation has been fixed.
Thanks for doing but you messed up the bumping of all the %changelog
entries.

On mardi 23 octobre 2018 23:47:05 CEST you wrote:
> On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
> > Anyway, since no one answers when I list the possible fixes and
> > ask to
> > choose between them, I fixed the Fedora spec files myself.
> >
> > Feel free to reintroduce %gosetup calls if you really want them,
> > once
> > the %gosetup implementation has been fixed.
>
> Thanks for doing but you messed up the bumping of all the %changelog
> entries.
Used this Fish script to fix my SPEC:

This does not need any fixing, it may not be the changelog format you're
used to, but it's one of the changelog styles rpm understands, which is
used by packages in Fedora, and which was documented at one point in
guidelines (no idea where the corresponding paragraph is today with the
doc.fedoraproject changes)
Regards,
--
Nicolas Mailhot

Le mercredi 24 octobre 2018 à 00:52 +0200, Robert-André Mauchin a
écrit :
> On mardi 23 octobre 2018 23:47:05 CEST you wrote:
> > On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
> > > Anyway, since no one answers when I list the possible fixes and
> > > ask to
> > > choose between them, I fixed the Fedora spec files myself.
> > >
> > > Feel free to reintroduce %gosetup calls if you really want them,
> > > once
> > > the %gosetup implementation has been fixed.
> >
> > Thanks for doing but you messed up the bumping of all the
> > %changelog
> > entries.
>
> Used this Fish script to fix my SPEC:
>
This does not need any fixing, it may not be the changelog format
you're
used to, but it's one of the changelog styles rpm understands, which
is
used by packages in Fedora, and which was documented at one point in
guidelines (no idea where the corresponding paragraph is today with
the
doc.fedoraproject changes)

Concretely the
* <date> <name> <mail>
- version-release
- changes
format is a tad more flexible and allows publishing several fixes in a
row without duplicating date lines
* <date> <name> <mail>
- version-release
- changes
- version-release
-
changes
and so on
And, our infra does not care if a changelog uses mixed style or not,
since mass rebuilds will drop changelog entries in a single format in
all Fedora specs, so you have many mixed style changelogs in the
distribution today.
(of course it's your specs, you can rewrite changelog entries any way
that makes you feel better)
Regards,
--
Nicolas Mailhot

> This does not need any fixing,
it may not be the changelog format
> you're
> used to, but it's one of the changelog styles rpm understands, which
> is
> used by packages in Fedora, and which was documented at one point in
> guidelines (no idea where the corresponding paragraph is today with
> the
> doc.fedoraproject changes)
Concretely the
* <date> <name> <mail

-
version-release
- changes
- version-release
-
changes
and so on
And, our infra does not care if a changelog uses mixed style or not,
since mass rebuilds will drop changelog entries in a single format in
all Fedora specs, so you have many mixed style changelogs in the
distribution today.
(of course it's your specs, you can rewrite changelog entries any way
that makes you feel better)
Regards,
--
Nicolas Mailhot

I'm talking about dropping the git hash from the release part of the entry.

I'm talking about dropping the git hash from the release part of
the
entry.

Ah, interesting, that's a side-effect of distprefix being pulled in by
dist in now, and mass rebuild changelog scripts typically null dist in
release entries… So, probably not possible any longer to do mass
rebuilds without full dist in changelogs (unless you want to enter
hack-land).
Not sure if going full dist is a problem, mass rebuilds are pretty much
release specific anyway.
I had missed this part, sorry about that.
However, I'm pretty sure mass improving changelog entries is not in the
scope of
https://fedoraproject.org/wiki/Who_is_allowed_to_modify_which_packages
Fixing failing builds clearly was.
Regards,
--
Nicolas Mailhot

On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
> Anyway, since no one answers when I list the possible fixes and ask
> to
> choose between them, I fixed the Fedora spec files myself.
>
> Feel free to reintroduce %gosetup calls if you really want them,
> once
> the %gosetup implementation has been fixed.
>
Thanks for doing but you messed up the bumping of all the %changelog
entries.