ConfigurationOf vs. upgradeSeasideImage

ConfigurationOf vs. upgradeSeasideImage

I'm working on upgrading a system from 3.2.15 to 3.3.5 and hit a snag. I'm not sure if this is a bug in upgradeSeasideImage or just an odd quirk of our setup.

When upgradeSeasideImage tries to auto-detect ConfigurationOfX classes to remove, it is apparently deciding to include the base `ConfigurationOf` class from Metacello in its list of removals. The script ends up loading it fresh, removing it, and then bombing when one of the packages triggers `ConfigurationOf class>>ensureMetacello`.

I worked around this by building my #BootstrapExistingConfigurationList that omitted ConfigurationOf, and then the upgradeSeasideImage script completed without errors.

Here are some highlights from the upgradeTo3x.log leading up to the failure:

Automatically generating list of loaded configuration classes to be removed:

Again, this might be just a quirk of my setup… but it seems (in my limited understanding) that the upgrade wouldn't need to remove ConfigurationOf after having just loaded it.

In case anybody else needs the workaround, I just copied the auto-generation code from the log, hacked it as follows, and used it to set #BootstrapExistingConfigurationList explicitly before running `upgradeSeasideImage`:

Re: ConfigurationOf vs. upgradeSeasideImage

Ken,

I suggest that you take a look at the GsDevKit_home upgradeStone
script[1] for inspiration. That sounds like a familiar problem and
I'm pretty sure that I've addressed that issue plus others in the
upgradeStone script.

I'm working on upgrading a system from 3.2.15 to 3.3.5 and hit a
snag. I'm not sure if this is a bug in upgradeSeasideImage or just
an odd quirk of our setup.

When upgradeSeasideImage tries to auto-detect
ConfigurationOfX classes to remove, it is apparently deciding to
include the base `ConfigurationOf` class from Metacello in its
list of removals. The script ends up loading it fresh, removing
it, and then bombing when one of the packages triggers
`ConfigurationOf class>>ensureMetacello`.

I worked around this by building my
#BootstrapExistingConfigurationList that omitted
ConfigurationOf, and then the upgradeSeasideImage script
completed without errors.

Here are some highlights from the upgradeTo3x.log
leading up to the failure:

Automatically generating list of loaded
configuration classes to be removed:

Again, this might be just a quirk of my setup… but
it seems (in my limited understanding) that the upgrade wouldn't
need to remove ConfigurationOf after having just loaded it.

In case anybody else needs the workaround, I just
copied the auto-generation code from the log, hacked it as
follows, and used it to set #BootstrapExistingConfigurationList
explicitly before running `upgradeSeasideImage`:

Re: ConfigurationOf vs. upgradeSeasideImage

Thanks, Dale. It turns out I was premature in declaring victory yesterday, and the "needs recompile" error is still happening even with my triumphal "fix".

Should I be using the upgradeStone script from GsDevKit rather than the upgradeSeasideImage script in the official GS 64-bit distribution? I'm not using GsDevKit / tODE on this project yet, but migration to tODE is a near-term goal.

I suggest that you take a look at the GsDevKit_home upgradeStone
script[1] for inspiration. That sounds like a familiar problem and
I'm pretty sure that I've addressed that issue plus others in the
upgradeStone script.

I'm working on upgrading a system from 3.2.15 to 3.3.5 and hit a
snag. I'm not sure if this is a bug in upgradeSeasideImage or just
an odd quirk of our setup.

When upgradeSeasideImage tries to auto-detect
ConfigurationOfX classes to remove, it is apparently deciding to
include the base `ConfigurationOf` class from Metacello in its
list of removals. The script ends up loading it fresh, removing
it, and then bombing when one of the packages triggers
`ConfigurationOf class>>ensureMetacello`.

I worked around this by building my
#BootstrapExistingConfigurationList that omitted
ConfigurationOf, and then the upgradeSeasideImage script
completed without errors.

Here are some highlights from the upgradeTo3x.log
leading up to the failure:

Automatically generating list of loaded
configuration classes to be removed:

Again, this might be just a quirk of my setup… but
it seems (in my limited understanding) that the upgrade wouldn't
need to remove ConfigurationOf after having just loaded it.

In case anybody else needs the workaround, I just
copied the auto-generation code from the log, hacked it as
follows, and used it to set #BootstrapExistingConfigurationList
explicitly before running `upgradeSeasideImage`:

Re: ConfigurationOf vs. upgradeSeasideImage

Yeah, I suggest that you take the topaz scripts from the
upgradeStone script and add them to your own scripts ... there are
also some version dependent steps that is done using tODE scripts
... what versions are involved in your upgrade ... I can decode
the tODE scripts and let you know the additional steps that might
be needed.

Dale

On 02/08/2018 10:30 AM, Ken Treis
wrote:

Thanks, Dale. It turns out I was premature in declaring victory
yesterday, and the "needs recompile" error is still happening even
with my triumphal "fix".

Should I be using the upgradeStone script from
GsDevKit rather than the upgradeSeasideImage script in the
official GS 64-bit distribution? I'm not using GsDevKit / tODE
on this project yet, but migration to tODE is a near-term goal.

I suggest that you take a look at the
GsDevKit_home upgradeStone script[1] for inspiration.
That sounds like a familiar problem and I'm pretty
sure that I've addressed that issue plus others in the
upgradeStone script.

I'm working on upgrading a system from 3.2.15 to 3.3.5
and hit a snag. I'm not sure if this is a bug in
upgradeSeasideImage or just an odd quirk of our setup.

When upgradeSeasideImage tries to
auto-detect ConfigurationOfX classes to remove, it
is apparently deciding to include the base
`ConfigurationOf` class from Metacello in its list
of removals. The script ends up loading it fresh,
removing it, and then bombing when one of the
packages triggers `ConfigurationOf
class>>ensureMetacello`.

I worked around this by building my
#BootstrapExistingConfigurationList that omitted
ConfigurationOf, and then the upgradeSeasideImage
script completed without errors.

Here are some highlights from the
upgradeTo3x.log leading up to the failure:

Automatically generating list of
loaded configuration classes to be removed:

Again, this might be just a quirk of my
setup… but it seems (in my limited understanding)
that the upgrade wouldn't need to remove
ConfigurationOf after having just loaded it.

In case anybody else needs the
workaround, I just copied the auto-generation code
from the log, hacked it as follows, and used it to
set #BootstrapExistingConfigurationList explicitly
before running `upgradeSeasideImage`:

Re: ConfigurationOf vs. upgradeSeasideImage

Dale,

I'm stumped.

I'm trying to upgrade from 3.2.15 (non-tODE but with some MetacelloPreview stuff loaded) to 3.3.6. I get through the regular `upgradeImage` step without any problems. But I'm still getting the same error during a GLASS bootstrap when it processes the BootstrapApplicationLoadSpecs:

In topaz, I can run `ConfigurationOf ensureMetacello` and it returns immediately. This makes sense, since it just got loaded fresh from Metacello-Base-dkh.103.mcz earlier in the bootstrap process.

But it seems that ConfigurationOfGsSqueakCommon — which it just fetched — somehow ends up referring to an obsolete version of the ConfigurationOf class. This is weird on multiple levels:

* ConfigurationOf has only one entry in its classHistory.

* Before the attempted load, ConfigurationOfGsSqueakCommon doesn't exist in DataCurator's UserGlobals. Even after the attempted load, it doesn't exist (presumably because the load attempt happens in a transaction).

So… where's it getting this obsolete code from? Is it possible that the fetches mentioned in those transcript lines — the ones that say "cached" at the end — are pulling in old classes?

====

Since I'm not running tODE yet, and since I thought maybe this error was due to something wrong in the official `upgradeSeasideImage` script, I've been writing my own scripts based on `upgradeStone` in GsDevKit_home. I may have missed something, but the steps are as follows:

Yeah, I suggest that you take the topaz scripts from the
upgradeStone script and add them to your own scripts ... there are
also some version dependent steps that is done using tODE scripts
... what versions are involved in your upgrade ... I can decode
the tODE scripts and let you know the additional steps that might
be needed.

Dale

On 02/08/2018 10:30 AM, Ken Treis
wrote:

Thanks, Dale. It turns out I was premature in declaring victory
yesterday, and the "needs recompile" error is still happening even
with my triumphal "fix".

Should I be using the upgradeStone script from
GsDevKit rather than the upgradeSeasideImage script in the
official GS 64-bit distribution? I'm not using GsDevKit / tODE
on this project yet, but migration to tODE is a near-term goal.

I suggest that you take a look at the
GsDevKit_home upgradeStone script[1] for inspiration.
That sounds like a familiar problem and I'm pretty
sure that I've addressed that issue plus others in the
upgradeStone script.

I'm working on upgrading a system from 3.2.15 to 3.3.5
and hit a snag. I'm not sure if this is a bug in
upgradeSeasideImage or just an odd quirk of our setup.

When upgradeSeasideImage tries to
auto-detect ConfigurationOfX classes to remove, it
is apparently deciding to include the base
`ConfigurationOf` class from Metacello in its list
of removals. The script ends up loading it fresh,
removing it, and then bombing when one of the
packages triggers `ConfigurationOf
class>>ensureMetacello`.

I worked around this by building my
#BootstrapExistingConfigurationList that omitted
ConfigurationOf, and then the upgradeSeasideImage
script completed without errors.

Here are some highlights from the
upgradeTo3x.log leading up to the failure:

Automatically generating list of
loaded configuration classes to be removed:

Again, this might be just a quirk of my
setup… but it seems (in my limited understanding)
that the upgrade wouldn't need to remove
ConfigurationOf after having just loaded it.

In case anybody else needs the
workaround, I just copied the auto-generation code
from the log, hacked it as follows, and used it to
set #BootstrapExistingConfigurationList explicitly
before running `upgradeSeasideImage`:

So far we seem to be loading the same packages ... and your error is
presumably popping up while fetching ConfigurationOfGsOB-dkh.93 ....
so it is odd that you've also successfully loaded all of the other
configurations up to this point ...

So I think we need to characterize whats happening with
ConfigurationOf a bit ... the ConfigurationOf cannot be appearing
out of thin air. The 3.3.6 upgradeSeasideImage code removes classes
by unconditionally removing the classes from the symbol dictionary
they were found in:

... and this code is executed right before the
BootstrapApplicationLoadSpecs load ... and I notice that there is
no commit in the script between the removal of the ConfigurationOf
and the BootstrapApplicationLoadSpecs so an ill-timed abort would
restore the old ConfigurationOf... The other possibility is that
there is another ConfigurationOf class floating around in your
symbol list ... so as a first order step I'd like to know the oops
of the ConfigurationOf classes that are involved ...

So record the oop of the class ConfigurationOf from the 3.2.15 db,
then modify the upgradeSeasideImage script to dump the oops of the
ConfigurationOf classes being removed as they are removed ...
Finally we can get the oop of the class that owns the un-recompiled
method #ensureMetacello by inspecting the GsNMethod for
#ensureMetacello and looking at it's inClass instance variable ...
if we are dealing with the same oop in all cases, then we have to
assume that either there is an abort involved or the ConfigurationOf
class is in the symbol list multiple times ...

... and then we'll go from there.

Dale

On 02/13/2018 10:40 AM, Ken Treis
wrote:

Dale,

I'm stumped.

I'm trying to
upgrade from 3.2.15 (non-tODE but with some MetacelloPreview
stuff loaded) to 3.3.6. I get through the regular `upgradeImage`
step without any problems. But I'm still getting the same error
during a GLASS bootstrap when it processes the
BootstrapApplicationLoadSpecs:

In topaz, I can run `ConfigurationOf
ensureMetacello` and it returns immediately. This makes sense,
since it just got loaded fresh from Metacello-Base-dkh.103.mcz
earlier in the bootstrap process.

But it seems that ConfigurationOfGsSqueakCommon —
which it just fetched — somehow ends up referring to an
obsolete version of the ConfigurationOf class. This is weird
on multiple levels:

* ConfigurationOf has only one entry in its
classHistory.

* Before the attempted load,
ConfigurationOfGsSqueakCommon doesn't exist in DataCurator's
UserGlobals. Even after the attempted load, it doesn't exist
(presumably because the load attempt happens in a
transaction).

So… where's it getting this obsolete code from? Is
it possible that the fetches mentioned in those transcript
lines — the ones that say "cached" at the end — are pulling in
old classes?

====

Since I'm not running tODE yet, and since I
thought maybe this error was due to something wrong in the
official `upgradeSeasideImage` script, I've been writing my
own scripts based on `upgradeStone` in GsDevKit_home. I may
have missed something, but the steps are as follows:

Yeah, I suggest that you take the topaz
scripts from the upgradeStone script and add them
to your own scripts ... there are also some
version dependent steps that is done using tODE
scripts ... what versions are involved in your
upgrade ... I can decode the tODE scripts and let
you know the additional steps that might be
needed.

Dale

On 02/08/2018 10:30 AM,
Ken Treis wrote:

Thanks, Dale. It turns out I was premature in
declaring victory yesterday, and the "needs
recompile" error is still happening even with my
triumphal "fix".

Should I be using the upgradeStone
script from GsDevKit rather than the
upgradeSeasideImage script in the official GS
64-bit distribution? I'm not using GsDevKit /
tODE on this project yet, but migration to tODE
is a near-term goal.

I suggest that you take a
look at the GsDevKit_home upgradeStone
script[1] for inspiration. That sounds
like a familiar problem and I'm pretty
sure that I've addressed that issue
plus others in the upgradeStone
script.

I'm working on upgrading a system from
3.2.15 to 3.3.5 and hit a snag. I'm
not sure if this is a bug in
upgradeSeasideImage or just an odd
quirk of our setup.

When upgradeSeasideImage
tries to auto-detect
ConfigurationOfX classes to remove,
it is apparently deciding to include
the base `ConfigurationOf` class
from Metacello in its list of
removals. The script ends up loading
it fresh, removing it, and then
bombing when one of the packages
triggers `ConfigurationOf
class>>ensureMetacello`.

I worked around this by
building my
#BootstrapExistingConfigurationList
that omitted ConfigurationOf, and
then the upgradeSeasideImage script
completed without errors.

Here are some highlights
from the upgradeTo3x.log leading up
to the failure:

Automatically
generating list of loaded
configuration classes to be
removed:

Again, this might be
just a quirk of my setup… but it
seems (in my limited understanding)
that the upgrade wouldn't need to
remove ConfigurationOf after having
just loaded it.

In case anybody else
needs the workaround, I just copied
the auto-generation code from the
log, hacked it as follows, and used
it to set
#BootstrapExistingConfigurationList
explicitly before running
`upgradeSeasideImage`:

Re: ConfigurationOf vs. upgradeSeasideImage

Thanks, Dale. Your tips set me on the right path and I figured it out.

I had two versions of ConfigurationOf in my system; the one in UserGlobals and another that was acting as superclass for BaselineOf and MetacelloBaseConfiguration. Once I recompiled those two subclasses and migrated, I was back to sanity and the upgrades went fine.

I still contend that ConfigurationOf itself shouldn't be included in the auto-generated BootstrapExistingConfigurationList (BECL). We gut all classes from UserGlobals, bootstrap (which loads ConfigurationOf), and then remove everything named in BECL… which (in my case) ends up removing the ConfigurationOf that we just loaded. If somebody didn't have ConfigurationOf pre-upgrade, it wouldn't be an issue.

Also, one thing that threw me a little is that the report of Bootstrap* values in my configuration said:

BootstrapSymbolDictionary: #'BootstrapSymbolDictionary'

BootstrapSymbolDictionaryName: #'BootstrapSymbolDictionary'

I did confirm that BootstrapSymbolDictionary == UserGlobals; it just apparently had some amnesia or a split personality or something.

So far we seem to be loading the same packages ... and your error is
presumably popping up while fetching ConfigurationOfGsOB-dkh.93 ....
so it is odd that you've also successfully loaded all of the other
configurations up to this point ...

So I think we need to characterize whats happening with
ConfigurationOf a bit ... the ConfigurationOf cannot be appearing
out of thin air. The 3.3.6 upgradeSeasideImage code removes classes
by unconditionally removing the classes from the symbol dictionary
they were found in:

... and this code is executed right before the
BootstrapApplicationLoadSpecs load ... and I notice that there is
no commit in the script between the removal of the ConfigurationOf
and the BootstrapApplicationLoadSpecs so an ill-timed abort would
restore the old ConfigurationOf... The other possibility is that
there is another ConfigurationOf class floating around in your
symbol list ... so as a first order step I'd like to know the oops
of the ConfigurationOf classes that are involved ...

So record the oop of the class ConfigurationOf from the 3.2.15 db,
then modify the upgradeSeasideImage script to dump the oops of the
ConfigurationOf classes being removed as they are removed ...
Finally we can get the oop of the class that owns the un-recompiled
method #ensureMetacello by inspecting the GsNMethod for
#ensureMetacello and looking at it's inClass instance variable ...
if we are dealing with the same oop in all cases, then we have to
assume that either there is an abort involved or the ConfigurationOf
class is in the symbol list multiple times ...

... and then we'll go from there.

Dale

On 02/13/2018 10:40 AM, Ken Treis
wrote:

Dale,

I'm stumped.

I'm trying to
upgrade from 3.2.15 (non-tODE but with some MetacelloPreview
stuff loaded) to 3.3.6. I get through the regular `upgradeImage`
step without any problems. But I'm still getting the same error
during a GLASS bootstrap when it processes the
BootstrapApplicationLoadSpecs:

In topaz, I can run `ConfigurationOf
ensureMetacello` and it returns immediately. This makes sense,
since it just got loaded fresh from Metacello-Base-dkh.103.mcz
earlier in the bootstrap process.

But it seems that ConfigurationOfGsSqueakCommon —
which it just fetched — somehow ends up referring to an
obsolete version of the ConfigurationOf class. This is weird
on multiple levels:

* ConfigurationOf has only one entry in its
classHistory.

* Before the attempted load,
ConfigurationOfGsSqueakCommon doesn't exist in DataCurator's
UserGlobals. Even after the attempted load, it doesn't exist
(presumably because the load attempt happens in a
transaction).

So… where's it getting this obsolete code from? Is
it possible that the fetches mentioned in those transcript
lines — the ones that say "cached" at the end — are pulling in
old classes?

====

Since I'm not running tODE yet, and since I
thought maybe this error was due to something wrong in the
official `upgradeSeasideImage` script, I've been writing my
own scripts based on `upgradeStone` in GsDevKit_home. I may
have missed something, but the steps are as follows:

Yeah, I suggest that you take the topaz
scripts from the upgradeStone script and add them
to your own scripts ... there are also some
version dependent steps that is done using tODE
scripts ... what versions are involved in your
upgrade ... I can decode the tODE scripts and let
you know the additional steps that might be
needed.

Dale

On 02/08/2018 10:30 AM,
Ken Treis wrote:

Thanks, Dale. It turns out I was premature in
declaring victory yesterday, and the "needs
recompile" error is still happening even with my
triumphal "fix".

Should I be using the upgradeStone
script from GsDevKit rather than the
upgradeSeasideImage script in the official GS
64-bit distribution? I'm not using GsDevKit /
tODE on this project yet, but migration to tODE
is a near-term goal.

I suggest that you take a
look at the GsDevKit_home upgradeStone
script[1] for inspiration. That sounds
like a familiar problem and I'm pretty
sure that I've addressed that issue
plus others in the upgradeStone
script.

I'm working on upgrading a system from
3.2.15 to 3.3.5 and hit a snag. I'm
not sure if this is a bug in
upgradeSeasideImage or just an odd
quirk of our setup.

When upgradeSeasideImage
tries to auto-detect
ConfigurationOfX classes to remove,
it is apparently deciding to include
the base `ConfigurationOf` class
from Metacello in its list of
removals. The script ends up loading
it fresh, removing it, and then
bombing when one of the packages
triggers `ConfigurationOf
class>>ensureMetacello`.

I worked around this by
building my
#BootstrapExistingConfigurationList
that omitted ConfigurationOf, and
then the upgradeSeasideImage script
completed without errors.

Here are some highlights
from the upgradeTo3x.log leading up
to the failure:

Automatically
generating list of loaded
configuration classes to be
removed:

Again, this might be
just a quirk of my setup… but it
seems (in my limited understanding)
that the upgrade wouldn't need to
remove ConfigurationOf after having
just loaded it.

In case anybody else
needs the workaround, I just copied
the auto-generation code from the
log, hacked it as follows, and used
it to set
#BootstrapExistingConfigurationList
explicitly before running
`upgradeSeasideImage`: