I have scripts that will install a set of arch linux machines for mewith all the tweaks I want. These scripts run pacman a lot to installbits and pieces. Usually the script will running pacman to install onepackage and then configure that package and then proceed to installthe next (set of) packages. This works great, but a noticeable partof the run time is spent on running pacman hooks (e.g. to update theman-db and similar things).

So I am wondering: Is it possible to stop pacman from running hooksduring package installation? I do know when it is "safe" to not runhooks (when the system is not complete yet) and when I need to run allof them (right after the system has all packages installed that itwill have).

So far I run pacman with a --hookdir that contains symlinks to/dev/null named like some of the more expensive hooks that tend totake long to complete. But a --no-hooks option to pacman would begreat for my use case.

Post by Tobias Hunger via arch-generalHi Arch Community,I have scripts that will install a set of arch linux machines for mewith all the tweaks I want. These scripts run pacman a lot to installbits and pieces. Usually the script will running pacman to install onepackage and then configure that package and then proceed to installthe next (set of) packages. This works great, but a noticeable partof the run time is spent on running pacman hooks (e.g. to update theman-db and similar things).So I am wondering: Is it possible to stop pacman from running hooksduring package installation? I do know when it is "safe" to not runhooks (when the system is not complete yet) and when I need to run allof them (right after the system has all packages installed that itwill have).So far I run pacman with a --hookdir that contains symlinks to/dev/null named like some of the more expensive hooks that tend totake long to complete. But a --no-hooks option to pacman would begreat for my use case.Is it possible to run pacman without it triggering hooks?Best Regards,Tobias

Some hooks take into account the specific files that were installed, so youcannot run them later. Why not just install everything at once?

I am aware of that. This is an optimization that avoids running hooksneedlessly. All the hooks I read so far are safe to run at any time.

Why not just install everything at once?I run a immutable and stateless setup. So I can not actually update systems(they are immutable after all). So I end up having my CI generate imagesfor my systems every couple of hours. Those will then replace the images Irun eventually, getting me to a new updated system state.

The CI builds a very basic system and configured that. It then creates morespecialized systems based on that base image (e.g. one for VMs, one forcontainers and another one for bare metal). It then continues to Branch outfrom those till all the actual systems I want to install are reached.Overall this approach (somewhat I spired by docker files) saves a lot oftime over just creating each system from scratch and it also makes sure allsystems have the same basic features: They all I before them from a commonbase after all.

It would suffice to run all hooks in the leaves of the tree of systems(just before writing the actual HDD image files) and skip them for allothers.

Post by Tobias Hunger via arch-generalI run a immutable and stateless setup. So I can not actually update systems(they are immutable after all). So I end up having my CI generate imagesfor my systems every couple of hours. Those will then replace the images Irun eventually, getting me to a new updated system state.

As a slight side-step, might it be possible to generate the images in aramdisk/tmpfs? That should remove disk I/O as a bottleneck.

Another option, given you know the expected end state, could be tobypass pacman and extract the package file content into place directly(e.g. with tar), then run whatever hooks you want afterwards. Yes, you'dlose package management within the images but that doesn't matter withan immutable image. (Though, there may be side-effects I haven'tconsidered here.)

Post by Tobias Hunger via arch-generalI run a immutable and stateless setup. So I can not actually update systems(they are immutable after all). So I end up having my CI generate imagesfor my systems every couple of hours. Those will then replace the images Irun eventually, getting me to a new updated system state.

As a slight side-step, might it be possible to generate the images in aramdisk/tmpfs? That should remove disk I/O as a bottleneck.

That is possible.

Post by Jonathon FernyhoughAnother option, given you know the expected end state, could be tobypass pacman and extract the package file content into place directly(e.g. with tar), then run whatever hooks you want afterwards. Yes, you'dlose package management within the images but that doesn't matter withan immutable image. (Though, there may be side-effects I haven'tconsidered here.)

The price is loosing package management. I do want to keep that todrag in dependencies.

What makes you say so?I see nothing in the alpm-hooks man page that implies that this would not work.Best Regards,Tobias

Because, as I said earlier, hooks can and do take into account the specificfiles being installed. If you install one package that needs a specific hook,running that hook later will have the correct file list and will not runcorrectly.

What makes you say so?I see nothing in the alpm-hooks man page that implies that this would not work.Best Regards,Tobias

Because, as I said earlier, hooks can and do take into account the specificfiles being installed. If you install one package that needs a specific hook,running that hook later will have the correct file list and will not runcorrectly.

Post by Doug Newgard via arch-generalBecause, as I said earlier, hooks can and do take into account the specificfiles being installed. If you install one package that needs a specific hook,running that hook later will have the correct file list and will not runcorrectly.

Most hooks are just run and do not care for any input. Some of thehook scripts take a list from stdin. By the way: It would be nice ifthat was documented in the alpm-hooks man page.

I see no reason why I can not generate this file list right when Iwant to run the hooks. In my setup I can ignore anything but theInstall hooks. For those I just need to apply the glob patterns in theTarget fields. That should not be too hard.

Post by Tobias Hunger via arch-generalI have scripts that will install a set of arch linux machines for mewith all the tweaks I want. These scripts run pacman a lot to installbits and pieces. Usually the script will running pacman to install onepackage and then configure that package and then proceed to installthe next (set of) packages. This works great, but a noticeable partof the run time is spent on running pacman hooks (e.g. to update theman-db and similar things).

If there are specific hooks you don't want to use, you can use thedefault HookDir (as documented in alpm-hooks(5) *and* in pacman.conf(5),this is "/etc/pacman.d/hooks/") to mask them with symlinks to /dev/null.

You've responded that you need to batch each image installation processusing incremental runs inspired by docker, which I sort of understand,but I don't really see how delaying execution until the end, is a goodidea here.

Consider things like the texinfo package, which installs a hook to runmany install-info processes, once for each file in usr/share/info/ thatthe hook detects. I guess you could write your own custom handling forthis and just try to install the whole directory at the end, but you'dneed to judge based on the hook, and adapt to a unique situation. Andit's irrelevant, since delaying execution will not provide benefits overdoing it every time a file is installed -- on the contrary, delayingexecution means you repeat some of the base image work separately foreach child image.

You should override hooks by hand, when you know what they do, know thatthey don't require targets, and know that you're handling it yourself atthe end.

Post by Tobias Hunger via arch-generalSo I am wondering: Is it possible to stop pacman from running hooksduring package installation? I do know when it is "safe" to not runhooks (when the system is not complete yet) and when I need to run allof them (right after the system has all packages installed that itwill have).So far I run pacman with a --hookdir that contains symlinks to/dev/null named like some of the more expensive hooks that tend totake long to complete. But a --no-hooks option to pacman would begreat for my use case.

Expensive hooks, like the man-db hook? I find that annoying on mystandard system as well, which is why I usehttps://github.com/graysky2/mandb-ondemand

I'm extremely skeptical that we'll add a --no-hooks option.

We have a --noscriptlet option, but that's because there's no other wayto stop a scriptlet from doing specific things you don't want done.

Hooks were designed to be configurable and able to be masked on anindividual level via symlinks. An option to prevent hooks from runningwould therefore serve no purpose except to say "running hooks at all isundesirable to me", which I don't think is the statement we want to say...

You can add a NoExtract directive to pacman.conf, which prevents hooksfrom being installed to the system in the first place. Although I don'tknow how you'd determine what hooks should exist, in order to handletheir actions yourself.