GOTCHA 2: If your VM uses a kernel optimized for virtualization (like the one included in the TurnKey VM builds), you need to replace it with a generic kernel, and also remove vmware-tools if installed.

Generate the ISO

Bonus: By default the ISO will boot automatically. If you want to include the TurnKey bootsplash and bootmenu, extract the cdroot from a TurnKey ISO and tell tklpatch-prepare-cdroot to use it as a template.

Every once in a while it's fun to tackle an issue like this, so thanks for the inspiration. The gotcha's got me, so I assume they got others as well, which makes this post that much more interesting and useful.

I am having a time with trying to get this to work. I have installed the core system, and all the programs, and continue to get the message (qemu-img: Could not open '/vtserver.icanreach.com/vtserver.icanreach.com-flat.vmdk'). I need some help.

I'm in the midst of trying this and I just now slavishly followed the "GOTCHA 2" suggestion of replacing the VM-optimal kernel with a standard kernel. However if the ISO is going to be used exclusively for VMs, would it make sense to skip this step?

The virtual kernel does not include squashfs support, so the resulting ISO won't be able to unpack the root filesystem, and the boot will fail.

Also, an ISO can be installed to bare-metal, so it doesn't make much sense to include the optimized virtual kernel even if it was possible (to be complete, it is possible with a couple of workarounds but not worth the effort, and possible side-effects).

I had partial success with this -- I was able to create an ISO that worked perfectly as a "Live CD". I was able to run my appliance directly off the ISO. The configuration console magically included a new "Install to Disk" option. However when I tried this it got pretty far into the installation -- including seemingly copying the rootfs to the disk -- but then failed while trying to set up grub.

But I'll leave that to another post -- I think even the creation of the "Live ISO" was a good start!

Heres another couple of "gotchas" I experienced:

(1) Alon mentioned that you may find you have multiple partitions on your source VMDK. In fact its probable that you will -- I certainly did on the Turnkey/Postgres-based VM I started with. However I was using another TKL-based VM as the "working machine" to run the commands on and I had trouble installing kpartx here. I switched to a Fedora host and was able to install kpartx ("yum install kpartx" iirc) and complete the rest of the procedure here.

(2) When I chrooted to apply the generic kernel to the rootfs, I found that I had no DNS (so apt-get couldn't find repositories). The same might happen to you depending on how your network is set up. Assuming the machine you're working on has working DNS, doing the following before you chroot should fix this:

cp /etc/resolv.conf turnkey-core.rootfs/var/run

(the desired destination of resolv.conf is likely different on a non-TKL-based rootfs).

Hope that helps! Heres the script I'm running to get as far as the chroot (its more-or-less as per Alon's directions except I've hardcoded the name of the partition in the loop device):

This process is quiet complex and lengthy too. Is there any other easier method? I tried it and halfway through system got rebooted and lost all conversion I did. I had trouble mounting the secondary drive. And is there any preferable location to copy out the root file system. My operating system is windows 7 but the specification of my system is very low, running out of free space. Should I try again, is there any chance of crashing my system and losing all information?

And the TKL .vmdk is a sparse image. In other words the original HDD image contains only data (but can be up to 20GB) but the .raw file includes the empty space as well. Once you mount it and extract the filesystem you should be back to ~1.7GB.

Maybe the .raw is 20GB because the .vmdk was created with 20GB virtual disk?

I created a VM with 20GB virtual disk using VMware fusion with no splitting option and installed linux core and base. I see that .vmdk is ~1.7GB and the following command produced linux.raw file which is 20GB.

qemu-img convert -f vmdk linux.vmdk -O raw linux.raw

What I'm trying to achieve is to install my application and the dependencies in the VM and create an ISO image and distribue it to my customers. It's possible to distribute the virtual appliance in .vmdk itself but if I can distribute an ISO customer can install it on any hardware or any virtualization technologies.

Once you have created the .raw file you then mount it (like you would an .iso) and copy out the contents (which should only be ~1.7GB - ie your original data size). Your resulting .iso should be around that size too. If not then I would assume there is something funny/wrong going on.

It was mentioned that changes were required to do this with version 11 based appliances. What needs to be done to make this process work with those? Now that I have a working iFolder appliance I'd like to make a cd of it so I don't have to compile the packages again.

I posted a comment on the wiki page for the iFolder appliance about the steps i took to compile iFolder on version 11. If the compiled packages could be put in a repo somewhere then making a patch would be very easy. You would not have to wait for it to compile and you would not have to do all the cleanup (removing all the necessary packages, etc.).

That's why it's larger than your original VDI. And actually it's a RAW file (which is a raw harddrive image). If you follow the tutorial it is only halfway through the process. To complete it you need to mount the image you have created and copy out the files, then make an ISO of the files only (that doesn't include the free space). This should bring your ISO back down to a similar (or perhaps smaller) filesize as your original VDI.

This tutorial works with TKL and I would imagine any Ubuntu or Ubuntu based distro. It may even also work on Debian - but perhaps not...

A quick scan of the second link you provided suggests that that particular version is a mishmash of 2 old versions of Debian but I couldn't quite work out what the go was with the new version. The fact that searching/scanning the website, forum and wiki resulted in no idea on what the OS is based on or what sort of licence it's under doesn't bode well (although perhaps I'm wrong). It seems quite closed and as such I'd suspect that the only way you will be able to do what you are hoping to is with the help and/or support of the makers/community. Have you posted on their forums?

But TBH unless there is a compelling reason, personally I'd be looking about for an opensource product that does the same thing thgat does support hacking etc. I haven't tested any but Wikipedia has a list of starters. No doubt there is probably more if you have a good google. My 2c anyway...

Except I've been through ALL of THEM. :) Seriously, I'm running on Zentyal at my office at the moment, but I've been through Smoothwall, pfSense, ClarkConnect, IpCop, IPFire, etc etc etc.

//disclaimer - I am not affiliated with Mikrotik in any way whatsoever. In fact, they would probably look down on what I am trying to do by creating a LiveCD.

Mikrotik is absolutly the best of the best. Its a bit funky to get started with, ( I was actually planning on running pfSense before I read a single post from a random comment on the pfSense forums that said "Try Mikrotik, once you learn it, you will never go back." I spent last weekend learning it, and without a doubt, it frakking rocks.

Just to get started, a web interface and windows interface for the firewall.. Why would you need the windows interface? Well, for starters, it allows you to connect to the firewall to reconfigure it regardless of your machines IP settings. It connects via MAC address, TCP/IP4 or TCP/IP6 as well as autoscans the local network so a freshly installed firewall is accessible without even touching the console.

The 24 hour time limit on the demo is a bummer but you can backup your config, reformat , and reinstall your config to continue using it or learning it. As well as their licensing is confusing and a bit expensive for SOHO usage.. but considering you can buy some of the RouterBoards including the license for less than the licensing alone, I'm not sure where I stand on that.

I highly reccomend it if you do any networking at all, or if you just want something fun to play with.

AFAIK unless the original distro supports running live then this will not work for you to create a live distro.

I'm sure creating a live version of the router OS is possible, but not without some serious hacking. The TKL devs have hacked the default Ubuntu Server OS so it can run live so unlike default Ubuntu Server TKL can run live already. I imagine you would need to do similar hacking first to allow this distro to run live. How you would actually go about that is well beyond me and may also be outside the terms of the Mikrotik licence.

As you've possibly guessed from my previous posts I'm a strong advocate of the open source model. But I also try to be respectful of the relevant licencing. If you like Mikrotik RouterOS that much I suggest you consult with them on what your options are and what it will cost you to do what you want.

I'm guessing that this conversion process could take a really long time. I'm waiting for the command "qemu-img convert -f vmdk Linux\ Mint.vmdk -O raw LinuxMint8Helena.raw" to finish executing. Should it really take so long?

But if it's a big image, yes it will take a while. A good baseline timeframe to consider is how long would it take to copy the file? Then add a bit (depending on hardware). You can see if it's still doing something via a new terminal window. I don't remember the commands off the top of my head but if you google "check progress qemu-img convert commandline linux" or something similar. I haven't actually done it with qemu-img but I am assuming it would be similar to checking progress of dd.

vfat should be a recognised format and depending on what distro you are using, you may need to instal ntfs drivers before it will recognise NTFS FS.

Regardless of that though, because of the nature of how Windows works, I don't think you'll have any joy using this method anyway. Even if you do manage to convert your VM to an ISO I imagine that you'll find that trying to run it on anything that doesn't closely resemble the virtual hardware of your VM will result in a BSOD.

I'm guessing you are trying to create a Win7 ISO with preinstalled apps and/or customised config? I don't know enough about Win7 to give you any really good suggestions, but if you are looking to create a preconfigured image then the MS tool sysprep is probably your best bet and then create an image of your syspreped system. If you hunt about online I'm sure you'll find tons of info detailing the best course of action.

Yeah, I wasn't sure if this would work for Windows, but was having trouble getting anything else to work so thought I'd give it a shot. I'll give sysprep a try! Thanks again for your quick and helpful response!

Another afterthought may be vLite. AFAIK it is the successor to nLite which was a quite cool way of creating a customised XP install CD (vLite is for Vista & 7 IIRC).

I have successfully been using a combination of sysprep and a FOG server to create and deploy Win7 images to multiple machines. It may be overkill (or simply not suitable) for your purposes but it works really well for mine! Possibly worth a look...?

And if not (but still something Linux) it should be possible, although to be honest I have no idea how you would go about that. I would suggest that you post on the forums/mailing list for whatever distro you're using and ask there. That way hopefully you will get some answers relevant to your scenario.

We run a highly virtulized dynamic environment. Even our DNS servers are virtulized. When we have a catastrophic event and have to spin everything up from scratch our VM hosts cant find their back end storage because DNS is down becuase the VM Host cant find its back end storage ..... so I want a bootable ISO of our DNS server, even if I have to tweak some of the entries because they have changed Its still better than what I currently have to do which involves remounting storage devices as an IP bringing up DNS then bringing up a clone on another host and downing the first copy. Its quite time consuming and surprisingly we seem to have several catastrophic events a year.

I use a program called remastersys if your looking at creating bootable OS from working linux system. Allows live OS and install, and options to output config from services/applications so theres no mucking around after install. If was providing read only control on services I couldnt see why it wouldnt run all day live of a cd/dvd. Draw backs is I have only tested this with ubuntu (but most things linux can be tinkered and chaged to suit) and it wont like the iso if it is too big (but again I cant see why the size limitation cant be "forced" to ignore error).

Although I haven't tried it on a server OS. Also I didn't realise that it was back up and running (IIRC it died some time ago, but obviously got revived somewhere along the line) so thanks for that info.

i tried to create the an iso using remastersys, but i was not able to install from it. it was just a live cd. even the install option caused it to boot in live mode. cud u plz help what steps u had followed and which version of remastsys u had used ?

Jeremy you seem to respond to posts so I've got a question/I need help with something.

I have Oracle VM, I have installed ubuntu server 12.10, xfce, other essentials, webmin, samda and basically got it at the stage I'd like to clone it.

The original reason I used VM was because I'm looking to covert and old laptop into a server. I'm waiting on an sdhc card and sd card reader in the post si I can install from usb.

I was originally looking at clonezilla but in oracle vm you can't seem to select a physical disc as a sata controller (only virtual).

If I format the laptop with XP in it, how would i go about cloning my .vdi image onto the usb and then onto the laptop?

I can boot from clonezilla image, use my .vdi as a hard drive. Any suggestions? - Or is the above mentioned way how I should go about it. Does the above way create a live distro?, would I then use bash to unmount, then copy over from the sd? - Kind of confused. I know wmware have a 'physical drive option' in which case the guide @ ;-

Hi,
I was reading through all of this and it all seems very complex to me how you guys are going about this all? Theres a couple of ways we could do this. Like using a FAT32 partition attached to another VM then boot into another live CD in the VM and manipulate it and output to the FAT32 partition. Exit the VM and use Folder2ISO to make your ISO image (obviously removing or adding what ever you want before hand ie. Kernel etc) and/or incorporating anything else that you may want from another ISO distro or what ever.
Remember anything that is generated and output to FAT32 is visible and can be seen in Windows. So you can change things to how you like put it out to FAT32 and when your done use Folder2ISO. This will guarantee to make your ISO bootable.
Second would be to use qemu binary directing in Windows instead of trying to add all this extra stuff and downloading add ons. It accepts the exact same arguments via the command line to make a RAW image from your VMDK. Edit anything you like from the generated img using a cool little GUI app I use called yaffey.
https://code.google.com/p/yaffey
This is perfect cause it maintains all the integrity of the img file and allows you to swap in or take out what ever you like.
Lastly you could use LinuxReader and output your system img to Windows and then rebundle it using Folder2ISO.
As my dad would say 'There's many ways to skin a cat' ;-)
Jarmezrocks

thanks a lot for this solution. Just to add a small bug in the procedure, i was getting error while creating the iso stating that the file size was larger than 2Gb so some flag "-allow-limited-size" had to be added.

finally i create the iso and tried to install with it, but the it didnt work :( from the logs i cud just find that some kernel panic had happened and the whole thing just shut down.

I got the below message after booting the ISO. I was able to follow the instructions and I encountered GOTCHA 1. I was not able to install the linux-image-generic after chroot 'ing so maybe that is my problem but if anyone experience this problem and fixed it then I would greatly appreciate the help!.

Otherwise that's really weird! According to Debian dpkg-divert is provided by the dpkg package. It is a fundamental part of Debian (and it's derivative's) package management system and should certainly be installed!

I've followed this process to the final step and have had success so far, however when I attempt to run tklpatch-geniso foo.cdroot I get: File foo.cdroot//casper/10root.squashfs is larger than 4GiB-1. -allow-limited-size was not specified. There is no way do represent this file size. Aborting. the cdroot directory is 22gig in size. I didn't see anything in this post that excludes large vms. Is there a work around to this?

Did some digging and noticed that the tlkpatch-geniso utilizes the genisoimage command. That led me to this fix mkisofs -allow-limited-size -l -J -r -iso-level 3 -o <output.iso> foo.cdroot which should allow it to generate iso file larger then 4gb, running it now so heres hoping.

In fairness... it would be nice, to leave thouse "once helpful" articles up to date, ore delete 'em ;)
We have too much confusion already, right ?
So how about this one ...https://www.turnkeylinux.org/blog/convert-vm-iso ?
Since Remastersys does no DNS in the backup,
it would be really nice to have a working process
step by step
including /home/USER
Maybe you can help
Regards

The partitions will be as per the output when the -l switch was used. I.e. /dev/mapper/loop0p1 will be the first partition and /dev/mapper/loop0p2 is the second. So you then use mount to mount /dev/mapper/loop0p1 to turnkey-core.mount. When you are done, umount turnkey-core.mount and delete the loopback device with kpartx:

kpartx -d turnkey-core.raw

Actually I just noticed that you are trying to mount /dev/mapper/loop0pl (lower case 'L') whereas it should be /dev/mapper/loop0p1 (number '1'). Perhaps that's your only issue?