Is it possible to port the init script to systemd ?
I looked over actual openrc script, but it is too far complex to simple migrating the commands _________________Sorry for my English. I'm still learning this language.

I don't know. AFAIK systemd does not have proper dependencies and probably also not a standard route during shutdown which is crucial for the script.
Actually, I consider the whole systemd a very bad idea (running permanently daemons which take resources only to save 1-2 seconds during booting is not sane), so I hope that I will never be forced to use it, and I am not at all interested in porting something to a system with a bad concept.

Quote:

I looked over actual openrc script, but it is too far complex to simple migrating the commands

Essentially, it is the "start" and "stop" which would have to be called at the proper place. The other provided command are only for wrapper scripts to check the current state (e.g. whether it would be saved). Probably one could put a part in a "library" which is sourced.

Thank you for your answer!
Yes, I already tried to simulate the logics behind and, so far, I found no way to start tar and bzip compress of the portage tree BEFORE /usr/portage was umounted.
What was the simulation:
- a separate partition for /usr/portage (without distfiles, of course) mounted automatically at boot via initscript (noauto in fstab)
- during shutdown compress it (tar -cjpf ....) and umount it
As you said, I found no way to prevent systemd to try to umount the partition and kill tar process.
It seems that all thing it's to risky._________________Sorry for my English. I'm still learning this language.

As you said, I found no way to prevent systemd to try to umount the partition and kill tar process.

At least the mount and umount commands from sys-apps/util-linux provide a way to call helper scripts for mount and umount of certain filesystem types. I have never dealt with them and do not know how compatible this setting is. But it might be a possible to invent a "new" filesystem type and to do the actual action in these helper scripts. However, this needs to be well-tested: e.g. aufs also uses such helper scripts, so "recursive" callings of such scripts must be possible.
The advantage of this approach would be that it works with any init-system, the disadvantage is that it probably works only with util-linux.
Currently, I have no time to implement such a thing.

Note: This happens sporadically. Like when I shutdown the computer last night, the portage squashfs file got recompressed and the ones I've got for /var/db and /usr/share/texmf-dist didn't, but at least now (using version 11.6) doing a "/etc/init.d/squash_db restart" made it recompress on the first try. And the need_squash and will_squash variables showed that the tex directory will be recompressed. Guess it's fixed but not sure, given how sporadic it would happen.

edit: PEBKAC. Think it was the threshold setting._________________My political stance/bias
slycordinator != slycoordinator

And it worked before (i.e. with other kernels)? Then it looks to me like a race condition problem in unionfs-fuse or an incompatibility of fuse with the newest kernels.
If the start of the error message is not reproducible, I would guess on the former and suggest that you report the problem upstream at unionfs-fuse.

And it worked before (i.e. with other kernels)? Then it looks to me like a race condition problem in unionfs-fuse or an incompatibility of fuse with the newest kernels.
If the start of the error message is not reproducible, I would guess on the former and suggest that you report the problem upstream at unionfs-fuse.

yes, everything worked fine until 3-21-2012.
and yes the start of the error message is not reproducible. but everything else on two different hosts and 1 virtual machine._________________Train Hard Or Don't Train At All

Hi, since Portage gets too slow on my old ppc Notebook I found this Tip...
Where is the recent version of these init scripts maintained and who does it currently? I just found http://en.gentoo-wiki.com/wiki/Squashed_Portage_Tree, there are already newer versions than the original author wrote in the first post of this thread.

Usually I avoid using non-official approaches which tend to be un-maintained after a while, but since removing the SquashFS approach for the default Portage is just easy and the benefits of this tweak are just too nice.
Wouldn't it be nice to make the Howto a bit more official and centrally maintained by creating an ebuild with the init script in some of the overlays, so I don't need to copy/paste the init-script code from the browser into the editor?_________________ppc:PowerBook5,8 15"(1440)-G4/1.67,2G | amd64:Acer Z5610 (Core2QuadQ8200),i5-3470,VMware VM @ i7-2620M | amd64-prefix:OpenSuse | Lila-Theme

The project (squash_dir, consisting not only of the init-scripts but also of a shell-wrapper with zsh-completion and an extensive README) is currently maintained on github.
The corresponding ebuild is in the mv overlay (available by layman).

Sorry that I overlooked the middle of this thread. I'm also using sys-fs/squash_dir now from your mv overlay. Thank you very much for maintaining these scripts, nice piece of work, especially the very good README.
The only issue for now is that aufs3 sources don't build on PPC. On amd64 its working fine.
Is there any COMPRESSION ("gzip", "xz", "lzo") advice for maximal performance (say slow 1core cpu and slow drive)? This is more important for me than space for some slow old machines and its the main reason that made me come to this thread, after reading Portage_tips._________________ppc:PowerBook5,8 15"(1440)-G4/1.67,2G | amd64:Acer Z5610 (Core2QuadQ8200),i5-3470,VMware VM @ i7-2620M | amd64-prefix:OpenSuse | Lila-Theme

You might try the git sources (aufs-99999999.3, also available from the mv overlay). Let us hope that the kernel developers come to their mind and finally include aufs or at least overlayfs, although the latter has several drawbacks compared to aufs.

Quote:

Is there any COMPRESSION ("gzip", "xz", "lzo") advice for maximal performance

For compression, lzo is undoubtfully the fastest (and least effective) and xz the slowest (and most effective). However, except for the time on shutdown only decompression time plays a role. Concerning decompression, lzo is also the fastest, but surprisingly xz is not much slower. Since xz needs to access less data (because it compresses much better), it can be that on some systems (especially those with not much memory or huge cache) xz is even the winner for decompression, speedwise. Your really have to try.
On the other hand, if you have a really slow system and a huge directory, it can be that compression time of xz is really inacceptable - on some systems, I end up with gtip for this reason...

...which already exceeded my threshold of 40M.
Btw, how do I understand the disk usage? Is it like this, /usr/portage and /usr/portage.readonly are only virtual sizes while the only real consumption on the host fs are these of the *.changes and *.sqfs files only?

I have 3 squash_dir setups now: /usr/portage, /var/db/ and /var/lib/layman.

Because I need to use webrsync on some hosts, I've seen that after a new webrsync of the same snapshot it seems that all has been re-written:

Because I can't afford long startup or stop time, I restart the squashing on daily base:

Are you aware that there is a squash_dir wrapper for this?

Quote:

squash_dir restart

should do the same as your script (only that by default it will stop on errors). Moreover the latter has the advantage that you can easily pass options to e.g. ignore the threshold.

Quote:

Btw, how do I understand the disk usage? Is it like this, /usr/portage and /usr/portage.readonly are only virtual sizes while the only real consumption on the host fs are these of the *.changes and *.sqfs files only?

Exactly.

Quote:

I've seen that after a new webrsync of the same snapshot it seems that all has been re-written.

This is because of the COW mechanism of aufs: If something changes in a file, even if it is only the filestamp or some attributes, the file must be copied into the .changes directory. I guess webrsync changes all filestamps, at least temporarily. With other union-type filesystem the necessary COW might be even more intensive, e.g. (IIRC this was the case in some versions of unionfs-fuse) if only the directory filestamp changes then the whole directory might need to be written to *.changes - it really depends which mechanisms are used to merge the directories.

When a new kernel has no aufs module, then squash_dir scripts fail but has half mounted the read.only parts. This is ok.
After I got the aufs module built and loaded, squash_dir restart or start do not work.

Code:

$ squash_dir restart
/etc/init.d/squash_db is not running
/etc/init.d/squash_layman is not running
/etc/init.d/squash_local_portage is not running
/etc/init.d/squash_portage is not running

When a new kernel has no aufs module, then squash_dir scripts fail but has half mounted the read.only parts.

The problem is that it is not clear what to do:
One possibility is to add an "umount" option which tries to umount even if the start failed. However, this can have horrible undesired effects if the start failed for a different reason or if directories with the temporary name feature are involved.
The other possibility is to start anyway successfully in such a situation, because the directory is mounted but just not writable.
The disadvantage of the latter is that you will not realize that something is broken unless you study the logs (or actually attempt to write).
The real problem is that it is not possible to report to openrc a "partial success".

I would like to improve performance, throughput and disk usage on a WebDAV server, which I mount via net-fs/davfs2. Now I think about if it is a good idea to store the content with squash_dir there.
Disadvantage would be of course that I can't view the files from the web-interface anymore. But I use the WebDAV only as central synchronisation for different sites using net-misc/unison.
Now the problem would be that maybe any change on the re-compressed squashfs would lead to a complete upload of the whole file again instead of sending delta only. Maybe this reason makes squash_dir the wrong solution for that, what do you think?_________________ppc:PowerBook5,8 15"(1440)-G4/1.67,2G | amd64:Acer Z5610 (Core2QuadQ8200),i5-3470,VMware VM @ i7-2620M | amd64-prefix:OpenSuse | Lila-Theme

tried it today but didnt get it to work properly :/
Compression-var seems to be ignored..I got gzip compression all the way
an error message indicated that it tried to umount the readonly before the aufs when shutdown
openrc doesnt seem to launch it altho added to default runlevel..as there was no output in the rc.log and no proper mounts after reboot
I guess the script is a bit outdated
anyway nice idea man!