If umount fails, squash_dir-10.6 will now sleep for 1s and retry (and repeat this procedure). Moreover, it attempts now to check whether the directory is already umounted and skip the umount if this is the case. I guess, in most cases this is the desired behavior although perhaps some strange error situations are not cought now.

Are you sure its 0.24 and not 0.25alpha? 0.24 is mostly a bugfix release to 0.23. But 0.25 simplifies path building (unionfs internal) and handles MAX_PATH_LEN better. But this change might have introduced bugs.
However, I'm a Debian user and only found that thread here, as I had been curious what Gentoo is using unionfs-fuse for. So that means I am not familiar with what you are doing.

With both, first I got the empty /usr/portage directory with 0.25-xxx, then I switrched back to 0.24, same effect several times and now I'm using 0.23 again... I think gentoo's /usr/portage-directory is really good for unionfs-fuse stress-testing. I contains >100k small files so a rsync with the original /usr/portage-tree is from the network point of view normally really fast and the unionfs-fuse has a lot to do. And during such rsyncs /usr/portage becomes inaccessible and a "ls /usr/portage" shows an empty directory...

aakef wrote:

There "busy" problem on umount?

The busy umount problem has nothing to do with the 0.24/0.25alpha empty /usr/portage directory problem._________________Train Hard Or Don't Train At All

Are you sure 0.23 vs. 0.24/0.25 are linked against the very same libfuse? I assume the kernel is definitely the same? Are you able to create a test case for me? So I already know the unionfs-fuse command and now I know that ?it is not only the umount issue. But I still do not have the slightest idea, what is actually your problem. Saying that 0.23 works, but that 0.24/0.25 fail is not sufficient, I'm afraid.

1) How does it fail, what are the symptoms?

2) If it fails, can you check using 'ps ax | grep unionfs' if unionfs is still running?

4.1) If it is not easy to reproduce, are you willing to run debug commands yourself? So recompile it with debug symbols (-g) and -DDebug (using cmake that should be easy). And then runing it either in valgrind or gdb and provide me the output.

If umount fails, squash_dir-10.6 will now sleep for 1s and retry (and repeat this procedure). Moreover, it attempts now to check whether the directory is already umounted and skip the umount if this is the case. I guess, in most cases this is the desired behavior although perhaps some strange error situations are not cought now.

Are you sure 0.23 vs. 0.24/0.25 are linked against the very same libfuse?

fuse is not slotted in gentoo, so the answer will almost surely be "yes". Moreover, libfuse has not been upgraded for a while in gentoo: The latest stable (and simultaneously latest unstable) version of fuse in the portage tree is 2.8.1 which is therefore almost surely what js08 is using.

Are you sure 0.23 vs. 0.24/0.25 are linked against the very same libfuse? I assume the kernel is definitely the same? Are you able to create a test case for me? So I already know the unionfs-fuse command and now I know that ?it is not only the umount issue. But I still do not have the slightest idea, what is actually your problem. Saying that 0.23 works, but that 0.24/0.25 fail is not sufficient, I'm afraid.

1) How does it fail, what are the symptoms?

2) If it fails, can you check using 'ps ax | grep unionfs' if unionfs is still running?

4.1) If it is not easy to reproduce, are you willing to run debug commands yourself? So recompile it with debug symbols (-g) and -DDebug (using cmake that should be easy). And then runing it either in valgrind or gdb and provide me the output.

receiving incremental file list
rsync: failed to set times on "/usr/portage/app-accessibility": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/dasher": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/emacspeak": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/espeak": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/festival-freebsoft-utils": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/festival-it": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/gnome-speech": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/gnopernicus": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/mbrola": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/morseall": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/orca": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/perlbox-voice": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/pidgin-festival": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/sound-icons": No such file or directory (2)
rsync: failed to set times on "/usr/portage/app-accessibility/speakup-utils": No such file or directory (2)
...

4) I think you don't need gentoo linux. Get a snapshot of the portage tree from http://gentoo.inode.at/snapshots/, extract it, make a squashfile... and then execute a rsync with one of the gentoo mirrors. I dont know exactly what rsync options emerge --sync, eix-sync use but I think it's more or less this command line...
rsync -av --exclude=/.unionfs rsync://rsync.europe.gentoo.org/gentoo-portage /usr/portage_________________Train Hard Or Don't Train At All

Are you sure 0.23 vs. 0.24/0.25 are linked against the very same libfuse?

fuse is not slotted in gentoo, so the answer will almost surely be "yes". Moreover, libfuse has not been upgraded for a while in gentoo: The latest stable (and simultaneously latest unstable) version of fuse in the portage tree is 2.8.1 which is therefore almost surely what js08 is using.

fuse - 2.8.1,
kernel (at the moment 2.6.34-gentoo, but before a vanilla 2.6.34)_________________Train Hard Or Don't Train At All

I think it is related with the way rsync accesses files - I suppose it keeps some filehandle open which collides with cow or something similiar. Here is a script for which rsync reports a nonexistent dir, although it exists. I tried the same without the squashfs, and the error did no occur. However, with aufs2 there is also no such error, so it is somewhat the interplay between squashfs, unionfs-fuse, and rsync which matters:

strange.
- with unionfs-fuse versions >0.23 it is so reproducable that I'm not able update the portage tree - every rync fails.
- I tried also the next older version in the portage tree libfuse 2.7.4 - same rsync issue

I still think that my environment is not very special. It's a simple 64bit pc linux environment which runs on a pc (amd) and on a laptop (intel)

so my next steps is to compile a debug version and let's see what happens_________________Train Hard Or Don't Train At All

Definitely 'fixed' in 0.25-hg. As it is not a unionfs bug, but a kernel/glibc issue, I simply disabled the error for now. If the issue still exists, I would appreciate further debug logs. However, debugging must be enabled for the build, that way unionfs will provide lots of information what it does internally. Maybe it will be a good idea, if I simply also check for "-d" in unionfs (so far libfuse reads it) and enable debug information without the need to recompile...

Great, thanks for testing it. Btw, the 'official' to hide the .unionfs directory is "-o hide_meta_files". The reason are .fuse... files, which are created, if an open file gets deleted. A ubuntu user who also works with squashfs run into those files and so we need to blacklist those as well. "-o hide_meta_dir" still works, but is deprecated.

I'm currently using these scripts to compress portage. I had no issue with the install, but the majority of the time when I do an emerge --sync or an eix-sync, I get lots of "failed: Too many open files (24)"

"lsof -u root / | wc -l" currently shows >1700 open files. I have not messed with my limits._________________When harmonious relationships dissolve
Then respect and devotion arise;
When a nation falls to chaos
Then loyalty and patriotism are born.
-Lao Tse

Hmm, too many open files by rsync? That's a bit weird. When you get this, could you please check /proc/`pidof unionfs`/fd if there are open filedescriptors left over? If not, or if those reduce within a few seconds, then it is linux cache and the gentoo script will need to specify the option "-o max_files=number" (e.g number=16384").
But if those FDs stay, then it is bug in unionfs-fuse. If that is true, could you please tell me what kind of file those are (the links /proc/fs/fd will tell you).

Yesterday I also pushed an new debug patches into the 0.25alpha branch, if you would enable debugging with that "-o /tmp/debug_file", we could check why those files are not closed.

This is ok: It is first attempted to mount with aufs (which fails, since apparently you do not have the kernel patched), so it falls back to the second choice in the default order which apparently succeeds (so quash_portage has probably succesfully started).

Quote:

after set "ORDER=unionfs-fuse unionfs funionfs aufs" in /etc/conf.d/squash_portage

However, this I cannot reproduce. Are you sure that you did not start with ORDER="funionfs aufs ..."? In my test, after successfully mounting with unionfs-fuse, it exits as it should (and if unionfs-fuse fails,l it prints an error message before trying unionfs, then also failing and printing an error, and only then trying unionfs).