Any collection of installed software on a system should be able to build itself into a working configuration when that system is booted, by the time the packaging operation completes, or at software runtime.

In this post, we’ll cover a more advanced case than last time: true self-assembly, where the configuration can be delivered by multiple add-on packages, if necessary. In particular, we’ll continue to talk about Squid, a package that isn’t normally capable of self-assembly, and will show how we fix that.

How does self-assembly work?

The main premise with self-assembly, is that configuration for an application must be built from a composed view of all fragments of the entire configuration that are present on the system. That can be done either by the application itself, in which case nothing else is required on the part of the application packager, or it can be done with an add-on service to assemble the entire configuration file from the delivered fragments.

When a new package delivers another fragment of the configuration, then the application must have its configuration rebuilt to include that fragment.

Similarly, when a fragment is removed from the system, again, the application must have its configuration rebuilt from the remaining fragments on the system.

A good example of self-assembly is in the Solaris package for pkg:/web/server/apache-22. Solaris ships a default httpd.conf file that has an Include directive that references /etc/apache2/2.2/conf.d.

Packages can deliver a new file to that directory, and use a refresh_fmri actuator causing the system to automatically to refresh the Apache instance
either after a pkg install operation has completed, or after apkg remove operation has completed, causing the webserver to rebuild its configuration.

The reason behind self-assembly, is to replace postinstall, preinstall, preremove, postremove and class action scripts, needed by other packaging systems. Install-time scripting was a common source of errors during application packaging because the scripting had to work in multiple scenarios.

For example, scripts had to correctly run

against alternate image roots, perhaps running on a system that didn’t have
the necessary tools support to correctly run the script

within the confines of a LiveCD environment

when making edits to an offline zone

With IPS, we eliminated those forms of install-time scripting, concentrating on an atomic set of actions (discussed in Chapter 3 of the IPS Developer Guide) that performed common packaging tasks, and allowing for actuators (discussed in Chapter 9 of the IPS Developer Guide) to run during packaging operations.

Actuators enable self-assembly to work on live systems by restarting or refreshing the necessary SMF services. Since the same SMF services they point to run during boot as well, we don’t need to do anything when performing operations on alternate images: the next time the image is booted, our self-assembly is completed.

Making Squid self-assembly aware

As in the previous post, we will start by downloading and modifying our Squid package.

This time, we intend to remove the etc/squid/squid.conf file entirely – our self-assembly service will be constructing this file instead for us. Recall that
Squid delivers some of its configuration files with the following actions:

We’ll use a series of pkgmogrify(1) transforms to edit the package contents, similar to the ones we used in the previous post. We will remove the file action that delivers squid.conf using a drop transform operation, and will also deliver a new directory, etc/squid/conf.d. Here is the transform file that accomplishes that:

The other vital thing needed, is an SMF dependency on the SMF service delivered by the Squid package. We need to add this, so that the Squid application will only be able to start once our self-assembly service has finished producing our configuration file.

First, we’ll create a proto area for the files we’re going to add to our Squid package, and copy the default SMF manifest:

Now that we’ve done this, our next step, is writing the method script for our self-assembly service.

The SMF method script

We need to write a script, such that when it is run, we end up with /etc/squid.conf containing all changes, as defined in all configuration fragments installed on the system.

This step can be as simple or complex as you’d like it to be – essentially we’re performing postinstall scripting here, but on our terms: we know exactly the environment the script is running in – that of a booted OS where our package is installed (defined by the depend actions that accompany the package)

Here is a sample script, written in Python (as short as I could make it, so there’s very little error checking involved here) which takes squid.conf.default copies it to squid.conf, then applies a series of edits to it.

As expected, the configuration file no longer contains the directives configured by connect_ports.conf, since that was removed from the system, but still
contains the changes from change_http_port.conf

Delivering the SMF service

The bulk of the hard work has been done now – to recap:

we have modified the Squid package to drop the shipped squid.conf file

we have an SMF service that can perform self assembly, generatingsquid.conf files from installed fragments on the system

we have added a dependency to the Squid SMF service on our self-assembly SMF service

All that remains, is to ensure that the self-assembly service gets included in
the Squid package.

For that, we’ll add a few more lines to the pkgmogrify(1) transform that we talked about earlier, so that it looks like:

Installing that package, we discover a svc:/config/network/http/squid-assembly service, and verify that when we drop unpackaged files into /etc/squid/conf.d, and restart the self-assembly service, we see what we expect:

We won’t go into details here, but clearly, multiple packages could deliver
configuration fragments at the same time, and they would all contribute to the
configuration of our service.

Conclusion

This has been a pretty fast example of the self-assembly idiom, but we hope this has been useful, and shows complex scripting operations can be performed in IPS.

There may more work to do to make the Squid application fully self-assembly aware – we’ve only covered the main configuration file and have’t looked at whether we also want to allow the other files in /etc/squid to participate in self-assembly. If we did want to do that, it would be a case of ensuring that:

we ship a master template for each configuration file

modify our self-assembly SMF service to copy each template into place

ensure our script can perform edits on that file

Of course, there’s other ways in which a self-assembly service could perform edits – we could use SMF to deliver properties to the service, which are then accessed by a self-assembly script, and placed into a configuration file, but perhaps that’s an example for another day.

Advertisements

Like this:

LikeLoading...

Related

Post navigation

Because user/group actions can’t manipulate existing users, how a package can perform “usermod -G “, and later remove it? postinstall/postremove isn’t support, don’t seem a task for a self-assembly, user/group actions can’t do it, IPS was many flaws at current incarnation.

Perform usermod -G actions in an SMF start method script when the service that package delivers tries to start, then remove the permissions you’ve granted when the service is stopped. Obviously, this would only work for local users on the system.

Thanks for reply, I will do as suggested on “usermod” case, one last case I can’t find a way of doing it IPS way is a package that delivers a service to inetd, because a entry to /etc/services need to be created, and later removed at “pkg uninstall”.

I am concerned that performing any sort of package fixes to resolve conflicts may result in me not being able to receive updates from solaris on those affected packaged that I modified in order to get installed.

For example I have solaris 11 5.11,5.11-0.175.0.0.0.2.1 and wish to update system/kernel to the latest version 5.11-0.175.1.0.0.24.2. However when using the graphical package manager to upgrade it, I got errors saying that some package files had to be modified because multiple packages were installing them.

What in the world am I going to do to get the core to the latest version and get around the following errors?

you helpful sugestions, etc. would be very much appreciated.

Thank you,
Allan

PROBLEMS REPORTED BELOW.
A ‘update’ operation failed for child ‘zone:web1-zone’ with an unexpected
return value of 1 and the following error message:

pkg update: The following packages all deliver file actions to usr/share/man/man3ext/demangle.3ext:

Right there were some changes in the way Solaris was packaged (along with the delivery of a new boot-loader for x86 systems) between early S11 and S11.1, which meant that you have to update to a certain set of packages first, then update to the latest available ones – a “stepping stone” update, if you like.

I believe it should just be a case of doing “pkg update –accept entire@0.5.11,5.11-0.175.0.10; reboot” then do “pkg update” again, but I’ve not tried taking an S11 FCS system up to the latest S11.1 support packages.

I want to create a package that will allow me to install a baseline configuration on multiple systems (had done this with SVR4 packaging on Solaris 8 & 10). Some of these changes will touch files that are part of the core-os pkg. ie: /etc/system, /etc/default/login, sshd config, … Based on this post, it appears I want to use overlays. Any suggestions or links to advice/example? Last thing I want to do is anger IPS to cause a future pkg update to fail.

It really depends on how the files you want to replace or modify were delivered.

If they have the ‘overlay=allow’ attribute, then you can deliver any new package that delivers a replacement file using the ‘overlay=true’ attribute.

If on the other hand, those files only allow user-edits (they have the ‘preserve’ attribute) then you’ll need to use a configuration management solution like Puppet with an SMF service that runs on boot to apply the appropriate edits to those files. You could deliver a package which contains both the SMF service you want to run to applies those edits, as well as the edits to feed to that service.

You’re unlikely to cause the system to fail to update so long as you don’t try to use the same package name as originally delivered those files to deliver your modifications. Choose your own package names, and you should be fine. If you (or your configuration management tool) ends up modifying a file that was not delivered as editable, then you may cause ‘pkg verify’ to emit errors (as it believes the originally delivered file has been corrupted by your edits) which will result your administrators being confused.

As we gain more experience delivering the OS with IPS, we’re gradually delivering more system files as editable, or overlayable, or are using other methods such as the ones I’ve described in this post to use a “foo.d” directory to allow users to deliver fragments of the eventual configuration. In Solaris 11.3, particularly the latest SRUs, many more of the system files are delivered with this in mind.