DISPLAY variable is missing... other than that... don't know yet. will try sux as soon as I can figure out why I can't authenticate with su or sux via password..._________________I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.

hi,
i get this now after reemerging libsigc++:
cfg-update --index
Nested quantifiers in regex; marked by <-- HERE in m/dev-libs:libsigc++ <-- HERE -2.0.16:20061002-150435.log/ at /usr/bin/cfg-update line 759.

this is the regex thing that I pointed out earlier. I think it was agreed that using the last log file in /var/log/portage/ was a bad idea and we would be using /var/log/emerge.log to determine last emerged package. When are we going to see the updated version? I think this error is serious enough to warrant a new release, maybe an -r4....

i have also problems with meld and 3way merge. maybe i don't understand the 3way mechanism anyway.
btw. what about setting up a webpage or at least a wiki for cfg-update?

The error will only occur when the last packages that was merged contains ++ in the name.
I have already fixed this in the new version of cfg-update, but it needs some more testing on other points.
Devsk is right, I should release 1.8.0-r4 soon
Will work on it next sunday... don't promise anything though. I hate to rush a release!

A quick fix is to emerge another package (emerge beep) so the last emerged package is something without ++ in the packagename.

About 3-way merging with Meld: I personally use xxdiff and vimdiff, but I think in Meld you have to click on the arrows to move blocks of text from one file to the other. The bad thing about the graphical 3-way diff tools is that most don't support intuitive manual editing. Xxdiff is an exception, I think it opens vim in the background, but I never even noticed that... usually it's enough to simply click on the colored sections in xxdiff.

If you don't want to use xxdiff, I would highly recommend vimdiff. Sure, it can't do 3-way merges but it does work with or without X and it allows you to edit the lines with regular vi-commands. So when you've mastered the vimdiff commands it's easy to update in any situation, even during installation on new systems when X is not available yet, on non-X servers or when the GUI is not available due to problems._________________When all else fails, read the manual...
Registered Linux User #340626

The error will only occur when the last packages that was merged contains ++ in the name.

yes, you are right. the error is gone.

Quote:

but I think in Meld you have to click on the arrows to move blocks of text from one file to the other.

i think i understand the interface of meld. even manual editing seems to work and with 2way-merges everything is fine.
the problem with 3way-merges is, that regardles what i do, for example modifying the middle coloumn, saving it, after quitting meld cfg-update tells me that the update has been cancled. i tried altering all of the three files (and saving it), it makes no difference.
what i still don't understand when does cfg-update decide to do a 3way merge and why is it usefull since it could easier be done with a 2way diff.
i'll try xdiff and the others though to see if they behave different.
thanks for your answer...

The problem with 3way-merges is, that regardles what i do, for example modifying the middle coloumn, saving it, after quitting meld cfg-update tells me that the update has been cancled. i tried altering all of the three files (and saving it), it makes no difference.

I'll do some testing with Meld this weekend...

There are two ways to finish an update:

First method is to save the merged result as foo.merged, cfg-update will look for that file as soon as you exit the mergetool. I believe it even checks if the file is empty or not before deciding to complete or cancel the update. (I have to check the code to be sure)

Second method is to save the results over the original configfile, so if cfg-update doesn't find a foo.merged file it will check if the original foo file has been modified. If the file is not the same as it was before the update, it will conclude that you've saved your changes in the original file and it will complete the update by removing the ._cfg0000_foo file and permanently storing the backups.

These two methods are used by default because not all supported tools can save the results as foo.merged. So don't save the changes in the ._cfg0000_foo file, cfg-update will not detect it and it will cancel the update because it sees no changes.

Quote:

what i still don't understand when does cfg-update decide to do a 3way merge and why is it usefull since it could easier be done with a 2way diff.
i'll try xdiff and the others though to see if they behave different.

The 3-way merge is split into two stages. If cfg-update finds backups of a previous update, it can use those files to automatically determine what changes the devs have made in the new ._cfg0000_foo file and which changes (non-default settings) are made by you in the original file. As long as the devs haven't changed the same lines, cfg-update (by using diff3) can merge the changes so that both the new options/lines and your custom settings end up in the merged result.

If however, you and the devs have changed the same lines differently, a "merge-conflict" occurs. That's when cfg-update decides that you need to manually solve the merge-conflict. If it loads the three files (your_file, the_original_file, the_devs_update) in a GUI mergetool with 3-way merging support, and the non-conflicting lines will already be selected for you. You only have to solve the conflicts manually. xxdiff does this very nicely, don't know if Meld already selects the non-conflicting changes for you.

Ofcourse you can always disable the manual 3-way merge if you rather do this in 2-way merge mode...
Just read the extra info about the 5 stages in /etc/cfg-update.conf and disable the manual 3-way merge stage.
You can leave the automatic 3-way merge stage enabled, because if a merge-conflict occurs, cfg-update will resume the update in the manual 2-way stage.

xentric, I am going to physically find you at the end of the (comcast?) cable and clobber you with jet li kicks....where is my r4 release, man?....

Oh oh, now I'm in trouble...

I'm gonna tell you straight up, my job is really hectic at the moment and my primary focus is on a promotion to a new function before the end of the month. Besides that I just bought a house and I spend a lot of time on painting, and polishing up the place.

You'll have to be a bit patient, but I apologise for not committing to my previous statements that r4 was coming up!
The only thing I can say is, don't count on r4 before the end of january. Only serious bugs will make me shift my priorities._________________When all else fails, read the manual...
Registered Linux User #340626

I was wondering if we could quickly include the changes for /var/log/portage (how you find the last emerge) and the awk thing we discussed before to speed it up a bit. We might forget otherwise.... do you think its a big change?

I was wondering if we could quickly include the changes for /var/log/portage (how you find the last emerge) and the awk thing we discussed before to speed it up a bit. We might forget otherwise.... do you think its a big change?

Actually those fixes are working properly in my test version, it's the other changes that are still not 100%.
I think it's better for me to go back to version r3 and only add those fixes and leave the new stuff for later.
This approach doesn't need a lot of time, because the only thing remaining to be tested is emerging on a clean system.
That's the best I can do for now._________________When all else fails, read the manual...
Registered Linux User #340626

xentric, I am going to physically find you at the end of the (comcast?) cable and clobber you with jet li kicks....where is my r4 release, man?....

I couldn't sleep anymore, got nightmares about being chased by people kicking and screaming that I hadn't updated cfg-update in such a long time... I just had to do something about it

Here you go guys... the ebuild has been submitted on bugzilla and should hit your local mirrors soon!
Edit: version 1.8.0-r4 had some minor flaws, fixing it at the moment, so probably r5 will be released.

When you emerge a new package and portage put's a new config file in one of the
protected directories, it's too late to match the checksum of the current config file with
the checksum from Portage because the CONTENTS file now contains the checksum of
the new file! So we make a backup of the checksums in /usr/lib/cfg-update/checksum.index
We want to keep this backup as current as possible, so before we use emerge we should
check if we can update our checksum-index. We can update it only when there are no
._cfg0000_files in the protected directories otherwise we cannot determine if it has been
modified or not... and we don't want to type "cfg-update --index" before every emerge!

Just some thoughts here, I do not know wheter this would work better or worse than how cfg-config currently works but anyways

Instead of simply backing up the CONTENTS files before every emerge, couldn't we run a "find __DIR__ -type f | xargs md5sum" for each directory found in $CONFIG_PROTECT on initial install of cfg-update. This would build a file with the checksums of all the current config files.

Then when one runs cfg-update, after every config merge, cfg-update would be able to insert the current checksum of the newly merged file into this file thus keeping it updated.

With this scenario there would be no need for the /root/.bashrc hack and no need to run cfg-update --index before and after every emerge operation. A drawback would of course be if users changed the protected config files outside of cfg-update, but then the initial scan could be invoked manually by the user with say cfg-update --index.

Instead of simply backing up the CONTENTS files before every emerge, couldn't we run a "find __DIR__ -type f | xargs md5sum" for each directory found in $CONFIG_PROTECT on initial install of cfg-update. This would build a file with the checksums of all the current config files.

But when you have done this and you have all the checksums, you still don't know if those files have been changed after they were installed. With that index you can only determine which files have been changed after the index creation. So on the very first run it's better to grab all the checksums from the CONTENTS file (as soon as all ._cfg0000_ files have been dealt with), with that index we can determine if a file has been changed after the file was installed. (and I bet that the current indexing method is faster)

Quote:

Then when one runs cfg-update, after every config merge, cfg-update would be able to insert the current checksum of the newly merged file into this file thus keeping it updated.

True, we can update the index with the checksum of the ._cfg0000_file as soon as the updating with cfg-update is completed. I agree that this is more efficient than rebuilding the index from CONTENTS files before every emerge action.

Quote:

A drawback would of course be if users changed the protected config files outside of cfg-update, but then the initial scan could be invoked manually by the user with say cfg-update --index.

Yes, if someone updates a configfile manually and removes the ._cfg0000_file, or also does updates with other tools like dispatch-conf/etc-update, cfg-update will not have the latest checksum in it's index. The result is that it will falsely show the file as being modified on the next update, resulting in a forced manual merge... This can be fixed by periodically grabbing the checksums from the CONTENTS files with "cfg-update --index", maybe with a daily cronjob.

I chose to use an alias to update the index right before the user uses the emerge command to install new packages. Because as soon as there are new ._cfg0000_files on the system, we can't update the index with the checksums from the CONTENTS files anymore. The current alias method catches 100% of all automatic updating opportunities...

Worst case scenario, of the alternative method, is that you'll miss out on a couple of automatic updating opportunities, forcing you to manually merge them with the GUI tool. But actually that's not such a bad trade-off if I can get rid of that ugly alias in /root/.bashrc!

I'll think about it for the next couple of days...

Thanks for bringing this up!

(However, I still think that a pre_emerge_session "hook" would be the best solution. I think that Paludis already has such hooks, but I've read more than once that Portage developers are not willing to implement these hooks into Portage)_________________When all else fails, read the manual...
Registered Linux User #340626

But when you have done this and you have all the checksums, you still don't know if those files have been changed after they were installed. ...

Yes I did realise this after a while, the checksums are supposed to be for the files that portage wants to install and not the checksum of the files currently installed since they can have been modified by the user

To march on to another topic it would be great if one could have the full effect of the automatic updates from the beginning so I am exploring ways to create the backups needed for the 3-way merger. As I see, this could be done in two ways:

Way #1 is the simple one, here we only have to scan the protected directories and compare each files checksum with the content of the CONTENTS file. If they match then we have not modified the file since it was installed and we could copy it to ._new-cfg_foo as backup. This would enable us to get the full benefits of cfg-update if we modify the file later.

The remaining files have been modified by the user so here we have to extract the files from the tarballs (Way #2). But this is extremely more difficult. One can use equery belongs -e foo to find which package owns the file. Then ebuild /usr/portage/xxx-yyy/foo/foo-a.b-r2 unpack to unpack the tarball into /var/lib/portage/xxx-yyy/foo/work/ where we then can scan for the file.

A problem here can be that the file can have a different name in the tarball than in the protected place, it is quite common that it is named example.config or similair in the tarball and gets renamed by the Makefile. So perhaps we have to simply check each file against the CONTENTS checksum for the file we are looking for.

A secondary problem are files that are created at install or which are modified at install. They would however not match the checksum since the checksum is of the file that is installed after creation and modification (at least that is what I think) so the only drawback that we would encounter here is that we would have no way to create the backup file. And this is the main reason for performing way #1 first since there is chance that these files have not been modified by the user and such would be catched by way #1!

One thing with Way #2 is that it would take a very long time due to the amount of work that equery has to do, but then this only have to be done once on the initial install of cfg-update since cfg-update will automatically create the backups as new programs are installed.

Quote:

However, I still think that a pre_emerge_session "hook" would be the best solution. I think that Paludis already has such hooks

I agree completely with you, and I have plans to switch to Paludis anyways My main interests with both Paludis and cfg-update is to enable comfortable maintenance of 100+ Gentoo servers and cfg-update is a real lifesaver in this area due to the many ways it really tries to make the updates automatic. I am also exploring ways to enable automatic updates with the help of cfg-update between machines, that is if I have to manually merge one file on one machine it would be possible for the other machines to perform the same changes on their file thus removing the need for performing manual updater on each machine. But more on that later

Way #1 is the simple one, here we only have to scan the protected directories and compare each files checksum with the content of the CONTENTS file. If they match then we have not modified the file since it was installed and we could copy it to ._new-cfg_foo as backup. This would enable us to get the full benefits of cfg-update if we modify the file later.

I could simply include a new option --create-ancestors that does this. That's a good one!

Way #2 has too many issues as you already pointed out.

Quote:

My main interests with both Paludis and cfg-update is to enable comfortable maintenance of 100+ Gentoo servers and cfg-update is a real lifesaver in this area due to the many ways it really tries to make the updates automatic. I am also exploring ways to enable automatic updates with the help of cfg-update between machines, that is if I have to manually merge one file on one machine it would be possible for the other machines to perform the same changes on their file thus removing the need for performing manual updater on each machine. But more on that later

Actually I already have been thinking about this "taking care of updates on remote hosts"...
I think SSHFS and FUSE would be our best option here. That would give us access to remote filesystems as if they were on the localhost itself.
The only changes to cfg-update would be an index-per-host (all stored on the localhost) and a /etc/cfg-update.hosts file that specifies which mountpoints to use for the remote systems

I get your point, the next step would then be to figure out a way to replicate a manual update on other hosts where no system specific settings are needed... pfff, I need to think a bit longer on that one

I'm going to see if I can get SSHFS and FUSE to work... (will take a while considering my spare time!)

Your last post xentric reminded me of another thing that I think would be an improvement : Right now cfg-update stores all intermediate files in their original folder cluttering e.g. /etc with lots of ._old-cfg / ._new-cfg files - I think they don't belong there. Could those files be stored in their own folder ? The files could be stored in e.g. /var/lib/cfg-update. In that folder the subfolder-structure could be maintained so that cfg-update knows where they were, e.g. /var/lib/cfg-update/etc/init.d/._old-cfg. Furthermore that would make it more maintanable for users / admins as all those files can then be found at one place.

Your last post xentric reminded me of another thing that I think would be an improvement : Right now cfg-update stores all intermediate files in their original folder cluttering e.g. /etc with lots of ._old-cfg / ._new-cfg files - I think they don't belong there. Could those files be stored in their own folder ? The files could be stored in e.g. /var/lib/cfg-update. In that folder the subfolder-structure could be maintained so that cfg-update knows where they were, e.g. /var/lib/cfg-update/etc/init.d/._old-cfg.

Yes the current way of storing the backups in the original directories isn't the most elegant solution. I have always told myself (to avoid having to change this = lame excuse, I know ) that it was just easier for a user if the backupfiles are in the same directory as the current configfile, and they are "hidden" dot-files anyway.

This should be the first thing to change because many more people have requested this...
It's on my TODO-list now