Make a live-CD with live-sdk

WARNING: This is a work in progress. There should be enough to get you started. There is more to come. (including formatting this page)

INTRODUCTION

Devuan's live-sdk uses shell scripts to create a live-CD/DVD iso. With no configuration, live-sdk will make a small, cli-only live iso. To create a live iso with your choice of packages and configurations, you can make your own blend. A blend consists of a config file and a blend file, and it may also contain customized files for the live system and deb packages or even source code for items that aren't in the devuan repository.

The config file contains lists of packages to install or remove along with any variables you want to set. The blend file contains functions to control the build process. Some of these functions may be copied from the sdk and customized. The customized version will override the stock version during the build. It's also possible to add functions that don't exist in the sdk.

You will also need live-build. (I think this can easily be changed to require either syslinux and isolinux or live-build.)

You probably want these, too:live-config live-config-sysvinit

sudo must be enabled for your user. (If physical access is not a security issue, do yourself a favor and set sudo with no password, or set the timeout longer than the build time, so you only have to enter the password once.)

These are listed as required by libdevuansdk, but I don't have them, and I'm building isos with live-sdk.kpartx cgpt

You can build a minimal live iso without creating a blend. You'll get a very small system with no desktop, the root password will be "toor", there will be no unprivileged user, and most of the standard system utilities will be missing. To do that, use the same commands you would use to build the blend, but leave out the blend_name. You can run this bare build to make sure you have all the requirements; if it doesn't work, you know the problem is not in your blend.

zsh -f
source sdk
load devuan <arch> <blend_name>
build_iso_dist

STEPS TO CREATE A BLENDEach blend gets its own directory in live-sdk/blends. The example we'll use is simple-ice, and we'll make a small live system with icewm and very few apps. We'll be creating two files in live-sdk/blends/simple-ice/. One is config and the other is <blend_name>.blend, or in this example, simple-ice.blend.

Example blends: Download and unpack the tarball in the live-sdk/blends directory. (subject to change and inconsistency with the howto.)simple-ice: (IceWM with a few extra apps) jessie-oblx: (Openbox, custom debs, custom boot menu, uefi bootable)

1. Add a line to the blend map in live-sdk/sdk with the blend name and the location where the blend files can be found. $R is the working directory, live-sdk/. It's also possible to use a URL for the location.

All of these variables except blend_vers are built into the sdk. I added blend_vers for my convenience. It only gets used in the image name here. Whatever you use for an image name will be appended with "-live.iso". You do not need to define the arch here; that will be done when you load the blend at build time.

The package lists exist as arrays in live-sdk. You can add packages to the extra_packages array in the config file as shown in the code box below. Use tabs, not spaces. Note that the syntax here will cause these packages to be added to the extra_packages array if it already exists (i.e. if it's defined elsewhere in the sdk.)

Troubleshooting note: One common reason for a failed build is if one of the listed packages does not exist (or is spelled wrong). It may not be obvious when this happens. Check the logs, but also check your package list.

Keep dbus?EDIT: This probably isn't necessary unless you go deeper into the sdk and change the core_packages and base_packages.

The default purge_packages array contains only dbus. Purging dbus will also remove packages needed for many desktops. Disable or replace that purge with:

purge_packages=()

or

purge_packages=(insert your own purge list here)

or to add to the default purge list

purge_packages+=(insert your own purge list here)

Add custom deb packages:

If you want to install deb packages that aren't in the repository, put them in the blend directory and list them in your config file. In this example, we'll add refractasnapshot-base, refractainstaller-base and palemoon web browser. Palemoon and Yad are listed twice because the packages are arch-specific. Comment out the one you don't want in the build. (NOTE: This is not in the simple-ice example tarball, but you could add your own custom debs list and supply the packages. Uncomment the appropriate lines in blend_preinst() in the blend file. See Step 4. Edit the blend file.)

Package lists will be installed in the order they appear. Packages within a list will be installed alphabetically. In this example, refractainstaller-gui and refractasnapshot-gui depend on yad, so yad must be installed first.

System files (including user files)Custom config files, icons, images or any other files you want to be present in the live system can be added via the rootfs-overlay. Create a directory inside blends/simple-ice called rootfs-overlay. Files that you add should be placed in their relative positions in the directory tree. So, if you wanted to use a custom sources.list, you'd put your custom file in rootfs-overlay/etc/apt/, and it will be added to the system.

The simple-ice example contains an apt config file to install packages without Recommends, modified xdg files to prevent automatic creation of a bunch of directories in your home. (You know the ones I mean - Documents, Music, Video, etc., all of which I consider spam.) There's also a Devuan replacement for the Debian logo in the IceWM applications button and a couple of config files for IceWM.

Add the blend_postinst function. This will copy the rootfs-overlay. If your rootfs-overlay is a git repository, see the heads sdk. This also calls the blend_finalize function, which contains instructions to be run in the chrooted system.

Right now (March 2017) rsyslog is getting removed at the end of the build and alsa-base does not exist. Workaround is to insert the following line into the debootstrap command at lines 42-44 of zlibs/bootstrap.

--include=busybox,busybox-syslogd --exclude=rsyslog,alsa-base \

...and let me know if this changes, so I can keep this doc current. Thanks.

Re: Make a live-CD with live-sdk

See the jessie-oblx example blend. There are some additonal functions in the blend file to customize the boot menu, boot files and other aspects of the iso.

blend_preinst() Same as in previous example, except that the lines for copying the custom deb files are uncommented.

blend_postinst() Added lines to call the iso_make_efi function, which makes the boot files for uefi.

iso_setup_isolinux() Not necessary. This was in the blend before some changes got added to live-sdk. Aside from my comments, this function is identical to the one in lib/libdevuansdk/zlibs/iso, so its presence here is superfluous.

iso_write_isolinux_cfg() Customize your isolinux boot menu here. (If you're an expert in this subject, I would like a consult with you.)

iso_squash_strap() This is identical to the stock iso_squash_strap function with the exception of the additional x86-specific compression option, "-Xbcj x86" which can reduce a CD-size iso file by about 20MB.

iso_xorriso_build() One line was added from the stock function for uefi-compatibility. If you disable uefi in the config file, you also need to remove or comment: -eltorito-alt-boot -e boot/grub/efiboot.img -isohybrid-gpt-basdat -no-emul-boot \

iso_make_efi() Lifted from refractasnapshot and plugged into the sdk. This might get merged into live-sdk. Customize the grub boot menu for uefi boot.

The config file for jessie-oblx is mostly self-explanatory. The base_packages list gets added to the existing base_packages. This makes a bigger bootstrap tarball. The goal of this is to make subsequent builds of the same blend go faster. Sometimes this works, and sometimes it doesn't. Your safest bet for a re-run is to remove everything in live-sdk/tmp/*. You can try deleting only the bootstrap directory for your blend before a re-run. That might work. I'm writing this paragraph off the top of my head, and like this paragraph, some parts of live-sdk are still under construction.

Re: Make a live-CD with live-sdk

APPENDIX

This appendix is not complete. It focuses on the elements used in the example blends.See live-sdk/lib/libdevuansdk/doc/ for more.

VARIABLES

`vars` and `arrs` are global arrays holding other global variables and arrays.this is required for `zuper` and helps a lot with debugging. if you declare newvariables or arrays, add them to `vars` and `arrs`, respectively.

Built-in variables:

Most of the built-in variables are not required to be in your config file. Add them if you want to override the default settings.

blend_name Give your blend a name. You need it for the name of the blend file and for the directory that holds all of the files used in the blend.

os default is "devuan" Changing this will affect the default image name. If you change it to os="$blend_name" you can have separate tarballs and bootstrap directories for each blend.

release default is "jessie" This will affect the build. NOTE: see special instructions for ascii.

version default is currently "1.0.0-RC"

arch whatever arch you use in the command 'load devuan <arch> <blend>' For a live-CD, choices are i386 or amd64.

image_name default is "${os}_${release}_${version}_${arch}"

usernameuserpass These get used with the add-user function.

rootcredentials default is root:toor

i386flavor="586" default i386 flavor is 686-pae. Use this variable if you want a 586 kernel.

section sections of the repo. for adding in /etc/apt/sources.list. separate them with whitespaces. default is "main".

Additional variables used in example blends:

blend_vers - this only exists in the blend config file and is used for the image name.

default_shell this only gets used in the blend file (chsh)

efi_work This is a directory where efi boot files get made. I'm not sure where it will end up, but it will appear in another example.

ARRAYS

core_packagesbase_packages These are part of the sdk that you should not need to change.

extra_packages List of additional packages that you want in the build.

custom_deb_packages Packages you want installed from the .deb packages you put in your blend directory. Only the packages that are listed will be installed. If there are i386 and amd64 versions of a package, both files can be present, and the list needs to be edited to use the right package for the arch you're building.

purge_packages Anything that gets installed that you want removed.

FUNCTIONS

Functions used in the example blendsordered according to build_iso_dist. (in live-sdk/lib/libdevuansdk/zlibs/helpers)

Cool, that's good info, gonna do this then in a clean 64, I have a nice pristine Mate install that I was building from anyway.

So I guess there's no way to do this using "su" instead of sudo? That's one of the things that stopped my first attempt, it's installed on my system but I never use it so my user account isn't in the sudoers file. Heck I can't even remember how to add somebody to sudo, haven't used it in many years.

EDIT: Hey meant to ask, any chance of getting a package list for the "core" and "base" groups of packages? Would be a valuable aid in learning what is and isn't needed to run the basic system.

Re: Make a live-CD with live-sdk

Okay, ran a test making just the cli-iso as detailed in the tutorial, seems all is well as I now have an iso (haven't tested it yet).

But man, iso is 341 mb, does that sound right? CLI-only with just the few packages on there ought to be like 250 or less shouldn't it? At 348 it's 13 mb larger than the fully functional openbox mini I just made.

I looked at some of the scripts and it looks like it's using xz with the extra compression of -xbcj x86, am I wrong about that?

Re: Make a live-CD with live-sdk

Re: Make a live-CD with live-sdk

No, don't mess with them there. You can add to core_packages and base_packages in your config file, and you can remove the stuff you don't want by adding it to the purge_packages list, or replace the purge_packages list as I did below. The following lists will give you approximately what you'd get with a minimal install with standard system checked. The purge list removes things that would not be in the standard install. (Note that grubversion is a variable here.)

- See the comment regarding where to add --no-install-recommends. if that's what you want. (in the jessie-oblx config. Line numbers might be off a bit.) - See the line in blend_finalize that says 'rm -f /etc/fstab'. That should give you some ideas, and around there would be the place to do it.- Look at the lists in the links below. You DO NOT want to list all those packages in your build. Listing some of the ones you want will get you others. Have fun figuring out which ones you really need. (Take careful notes on a VM install in prep for building a blend.)- Run some commands to tell you what the package manager considers Required, Standard or Important.

Re: Make a live-CD with live-sdk

fsmithred wrote:

No, don't mess with them there. You can add to core_packages and base_packages in your config file, and you can remove the stuff you don't want by adding it to the purge_packages list, or replace the purge_packages list as I did below.

Ahh, well there's the rub seemingly for the basic iso i'm trying to make to start with, not using a blend file for it, just running the basic instructions.

So i'll need to make a custom blend for the cli-iso too if I want to adjust packages in/out. And a custom rm list for some things.

For custom configs and removal of extra locales and such, it would almost seem easier if a user could just mod bootstrap-devuan-amd64.tgz prior to feeding it into the build? Or the unpacked bootstrap folder itself.

So the idea is basically, that rootfs-overlay folder is sort of an etc/skel for the entire file system, yes? But only insofar as adding things, not replacing or overwriting things already in the folders/files?

That's a lot of good info, thanks for taking the time to post all that.

Re: Make a live-CD with live-sdk

core_packages and base_packages get installed, and purge_packages get removed, and then the bootstrap tarball is created. So put the additional base_packages in your config, and also put your purge_packages there (either add to it or replace it as described somewhere above.) You should be able to reuse the tarball for both cli and desktop builds, but you should get rid of the bootstrap directory. Unfortunately, that needs to be fixed. Right now, you're better off wiping out tmp/* between runs, but I expect that will get ironed out. If you think this is not a clear answer, you are correct.

Yeah, the overlays are kinda like /etc/skel. They copy files into the bootstrap directory to be packed in the iso. The rsync command to copy the rootfs-overlay is in blend_postinst in the blend file. If you added a delete option to that, then everything that isn't in rootfs-overlay would get deleted. That would be your entire system minus what's in rootfs-overlay. Better to remove stuff in a chroot script in blend_finalize (or earlier if needed.)

Re: Make a live-CD with live-sdk

System files from rootfs-overlay can get copied to the system with user as owner, instead of root.

In the blend file, change the rsync command that's in blend_postinst() to:

sudo rsync -avx --no-o --no-g "$BLENDPATH"/rootfs-overlay/* . || zerr

Or change the rsync to a cp command. The --no-o and --no-g prevent rsync from preserving the file's owner and group. Any files in in the overlay that go into the user's home will need to be changed back to user:user in blend_finalize(). Be careful if some of your system files in the overlay are actually owned by a system account (e.g. mysql, www-data).

Re: Make a live-CD with live-sdk

Ozi,

There is an isolinux-overlay in the jessie-oblx blend and in the devuan-live blend at parazyd's github, although I'm not sure if that one is working right now. He made some changes that we didn't finish testing before the big push on RC2. And for my latest build, I added a hooks directory and a line to copy a hookscript into the iso.

Re: Make a live-CD with live-sdk

Re: Make a live-CD with live-sdk

Hi,

What is the reason for switching from live-build to live-sdk?

Phil

“Property is the fruit of labor; property is desirable; it is a positive good in the world. That some should be rich shows that others may become rich, and hence is just encouragement to industry and enterprise.” — Abraham Lincoln

Re: Make a live-CD with live-sdk

pcalvert wrote:

Hi,

What is the reason for switching from live-build to live-sdk?

Phil

I'm not sure. I know that live-sdk uses some of the same libraries as the devuan sdk that's used for building all the other images, so at least part of the decision was for consistency. I also know that early on, it was impossible to build an iso with live build-that didn't include systemd. I just tried the version of live-build in jessie, and it looks like it was neglected. It still uses debian repos. I just changed that and it's running. I'll know more in a little while.

I'm hoping Ozi will say something here. He uses live-build and knows more about it.

Edit: This is with version 4.0.3-1+devuan2, which looks to be the only one in jessie, ascii and ceres.

The following packages have unmet dependencies:
live-config-systemd : Depends: systemd but it is not installable
Recommends: dbus but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Re: Make a live-CD with live-sdk

I just used live-build ten days ago to build a default Devuan ISO. And according to the package list, there are no systemd files on it, not even libsystemd0. I did have to use a "trick" to get it to build, though. In the past I encountered the same problem that you did, but this time around I was more persistent and found the solution/workaround. More on that shortly...

Phil

“Property is the fruit of labor; property is desirable; it is a positive good in the world. That some should be rich shows that others may become rich, and hence is just encouragement to industry and enterprise.” — Abraham Lincoln

Re: Make a live-CD with live-sdk

Hi Guys

I'm busy with work atm, so haven't noticed your posts.

I don't have a problem with live-build in either Devuan or Debian, still use them both. My Devuan build doesn't use any Debianrepos. and I have pinning to exclude *systemd*. If there are any packages with systemd in the name they would be non functioning in a systemd sense. The last Devuan build I did was 5 hours ago, just worked. I've also build images without dbus as well. I have some well established patterns I use for the build, so the builds are now usually very consistent and stable, unless there are package or dependency changes.

I don't think Frits has done any changes in a while, but I know he was in the process of bringing in live-build 5.x to the repos. Don't know what stage that is in now though.

Also, I'd like to see Daniel Baumann's open-infrastructure-system-build package brought in Devuan.

Re: Make a live-CD with live-sdk

fsmithred wrote:

pcalvert wrote:

Hi,

What is the reason for switching from live-build to live-sdk?

Phil

I'm not sure. I know that live-sdk uses some of the same libraries as the devuan sdk that's used for building all the other images, so at least part of the decision was for consistency. I also know that early on, it was impossible to build an iso with live build-that didn't include systemd. I just tried the version of live-build in jessie, and it looks like it was neglected. It still uses debian repos. I just changed that and it's running. I'll know more in a little while.

I'm hoping Ozi will say something here. He uses live-build and knows more about it.

Edit: This is with version 4.0.3-1+devuan2, which looks to be the only one in jessie, ascii and ceres.

The following packages have unmet dependencies:
live-config-systemd : Depends: systemd but it is not installable
Recommends: dbus but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

I don't actually install live-config-systemd, I only ever install sysvinit-core and use the /etc/apt folder from the latest devuan iso.

I know at one point, I wrote a short script to go through and change the links from debian to devuan. I also tried open-infrastructure. I'll say something about those things in the live-build tips thread.

What do you mean about using the /etc/apt folder from the latest iso. You put that in the chroot includes dir so it gets copied into the iso? Or something else?

Re: Make a live-CD with live-sdk

Very many thanks for this excellent article.

Note on prerequisites:In addition to those mentioned in the article I *think* I found that I had to install the 'isolinux' package to get the build to complete without complaining that isohdpfx.bin was missing from /usr/lib/ISOLINUX/ - although in fact it appeared to be there...

Would it be safe to use our on-site caching proxy server for the archive repository? It would make building from scratch a great deal quicker and I seem to be doing that quite a lot at the moment.