Files inside an ISO image are already contiguous (defragmented).
What would be the use of defragging/soritng the content of an ISO image?

There is a -sort parameter for mkisofs:

-sort sort file
Sort file locations on the media. Sorting is controlled by a file that contains pairs of filenames and sorting offset weighting. If the weighting is higher, the file will be located closer to the beginning of the media, if the weighting is lower, the file will be located closer to the end of the media. There must be only one space or tabs character between the filename and the weight and the weight must be the last characters on a line. The filename is taken to include all the characters up to, but not including the last space or tab character on a line. This is to allow for space characters to be in, or at the end of a filename. This option does not sort the order of the file names that appear in the ISO9660 directory. It sorts the order in which the file data is written to the CD image - which may be useful in order to optimize the data layout on a CD. See README.sort for more details.

Icecube wrote:Files inside an ISO image are already contiguous (defragmented).
What would be the use of defragging/soritng the content of an ISO image?

Icecube, I'll try to explain myself better. I'll do it with a comparison between floppy images and iso images.

Suppose you have a clean new floppy, you save some files on it, and you make an image from it. Since you haven't deleted any file from the floppy, the image you have is defragmented. By this, I mean:
a) all the files are saved in the beggining of the data area;
b) the rest of the data area is "clean" (no files)
c) the files are saved in continuous, sorted manner (not fragmented, not unsorted)
d) the FAT area is also "sorted".

If you open this floppy image with Isobuster, and run the "search for missing files and folders" function, you won't find any files/folders "missing". Or alternatively, an undelete program won't find anything.

After some time, you use the floppy again and again, saving, deleting... Now, a new floppy image will show "missing files/folders" (using the term of Isobuster), and the content of this new floppy image can be "resorted" by winimage, using the "defrag" function in it.

The *.ima/*.img file could be defragmented or not in your hard disk. Winimage doesn't care. But until using winimage to "defrag" the contents of the floppy image, those contents are sector-by-sector a copy of the fragmented floppy.

Now, instead of using a floppy, and a floppy image, change the previous paragraphs to a CD-RW and an iso image of it. Winimage can't "defrag" this type of image (the content of it, to be more specific).

If you have Isobuster (or similar alternative software), you can check this "missing" files function on the original UBCD503.iso image. It takes just a few minutes at most, probably less than 2 minutes. And if you try to use the "defrag" function of winimage on this iso image, you will see that it isn't available for it.

Maybe I was more clear now in my explanation. If I wasn't, please ask whatever info you need and I'll do my best to communicate it in a clear manner.

You should NEVER edit an iso with an ISO editor.
If you want to add a file to a bootable ISO, you need to create it from scratch again (with mkisofs).

Floppy images and ISO images aren't comparable. Floppy images use the FAT12 filesystem (at least most of the time) and is meant to be read and written too (and may need defragmenting), ISO images use the iso9660 filesystem (or UDF) which is a read only filesystem and isn't intended to be edited afterwards (and because you can't or shouldn't write to it, you don't need to defragment it)..

Then, you want to change it, and instead of UBCD 5.00, you write UBCD 5.01.

On a normal CD-R, you couldn't do it, but this is a CD-RW. Now, you should erase it first, before burning the new data. This is the recommended method.

But let's suppose you don't erase the CD-RW, but burn on it the new data.

Now you want to test new features, customize it, fix bugs, for the release of the next version. You burn again and again.

At some point, part of the surface of the CD-RW is not going to work correctly, but you keep using the same media.

If you want to distribute this CD-RW, you make an iso image. Everyone who download this image, has exactly the same sector-by-sector information that you have in your master CD-RW.

I want to get rid of those sectors that are unusable, or that have old data from the previous burns. I don't want those sectors to get to a new CD burned from that iso.

So, what winimage was doing by means of its "defrag" function over a floppy image content, was actually "cleaning" the mess. Not only the files got defragmented, but the last sectors were "cleaned". Those sectors, instead of containing deleted data, now contain "F6" (in FAT12 floppies), until you get to the last final sector, where you get "00". (For those interested in looking at this, use an Hex editor like HxD over a "dirty" floppy image, comparing the image before and after winimage "defrag").

So I guess that what I meant was, not so much to align (defrag) the currently existing files/folders inside an iso image, but to get rid of the "old" data.

Maybe, the "missing files and folders" that Isobuster finds, are not actually from the iso image, but from my own hard disk, where those files are saved. I don't know.

By using Isobuster, opening the original UBCD503.iso image, and applying the "search missing files and folders" function, you can see an example of what I am writing about. But, maybe it is just happening in my system, and in yours everything looks fine. I'll appreciate if you can check it.

I don't have the intention of "editing" the iso image. I want to get rid of those old fragments. If there is a method to accomplish that, probably it requires expanding the content of the iso image, applying some function to it, and re-building the iso image once again. It may be possible that the order of this procedure is not as I describe here. I don't know, and that's why I am asking.

I am interested in the method and the tools necessary to accomplish the goal. Probably it requires more than one tool. I only used the winimage+floppy image comparison to explain the goal, not the method.

Now that I explained the goal, maybe someone knows a method to accomplish it.

ady wrote:On a normal CD-R, you couldn't do it, but this is a CD-RW. Now, you should erase it first, before burning the new data. This is the recommended method.

But let's suppose you don't erase the CD-RW, but burn on it the new data.

I have never seen a program that could burn a new ISO image over a CD-RW that already had files on it, without erasing the CD-RW first. I don't know why it isn't supported. If you find a program that is able to do it, tell me. I am interested.

ady wrote:
Now you want to test new features, customize it, fix bugs, for the release of the next version. You burn again and again.

Use virtual machine software (VirtualBox, qemu, kvm, VMware, ...) for testing ISO's. It makes it easy to test most things.

ady wrote:
At some point, part of the surface of the CD-RW is not going to work correctly, but you keep using the same media.

If you want to distribute this CD-RW, you make an iso image. Everyone who download this image, has exactly the same sector-by-sector information that you have in your master CD-RW.

You should keep the original ISO file and distribute that and not an image that you make of the already burned CD-RW. The ISO image is always clean.

ady wrote:
By using Isobuster, opening the original UBCD503.iso image, and applying the "search missing files and folders" function, you can see an example of what I am writing about. But, maybe it is just happening in my system, and in yours everything looks fine. I'll appreciate if you can check it.

Icecube wrote: I have never seen a program that could burn a new ISO image over a CD-RW that already had files on it, without erasing the CD-RW first. I don't know why it isn't supported. If you find a program that is able to do it, tell me. I am interested.

There is such a thing called floppy or hard disk "quick format", as opposed to "full" or "complete" format.

For CD's, there is "quick erase" or "quick blanking". It is basically the same: erase the info that points to the actual blocks of data, instead of deleting the actual data.

Different blanking methods can be used, including "full" blanking in which the entire surface of the disc is cleared, and "fast" blanking in which only meta-data areas are cleared

The "quick blanking" method means there could be "old" data previously burned, that could be found by the "find missing files and folders function" of Isobuster.

Additionally, there is "packet writing" software, which treats a CD-RW as if it were a floppy (with pros and cons). The most known of such software for windows platform is InCD ( http://en.wikipedia.org/wiki/InCD ).

Most good burning software will have an option to "erase" the CD-RW before writing over it. In general, this means "full erase" instead of "quick erase". We are talking about a new complete burning session, not a multisession CD. For a multisession to exist, you would need to preserve the previous session, avoiding the "full erase" of the actual data area.

Use virtual machine software (VirtualBox, qemu, kvm, VMware, ...) for testing ISO's. It makes it easy to test most things.

Yes, I use Mobalivecd (uses kqemu, portable) to test UBCD. But I was trying to explain myself with an example. I don't actually burn again and again the same CD.

You should keep the original ISO file and distribute that and not an image that you make of the already burned CD-RW. The ISO image is always clean.

Again, it was just an example to explain myself. I think my problem to explain the goal is that I used the terms of Isobuster and Winimage.

Please concentrate at the "cleaning the mess" part, instead of thinking about a traditional defrag.

I don't have ISO buster here atm, so I can't check right now.

Well, it seems I will have to wait for someone who tries the "find missing files and folders" of Isobuster over the original UBCD503.iso, since, apparently, experimenting with that function for a few minutes is going to explain my goal better than my words here. (The point being: the original ubcd503.iso in my system does not seem to be "clear/clean")

Icecube, if you can, please just try "find missing files and folders" of Isobuster over the original UBCD503.iso and maybe you will see what I see in my system. Running the function over the iso image should take a couple of minutes at most, assuming you already have Isobuster installed in some system or VM.

I know about the quick erase function and the packet writing software. I only don't know a program that can write to the start of the CD-RW (burn new ISO image) without doing a quick erase first.

The missing files ISObuster found are not left over files inside the ISO image.
Some are real files that are used on UBCD: like the 28.227.584 gzip archive, which is in reality the /ubcd/boot/discwiz/initrd.img file (size: 27.627.228 bytes)

Some files are not even valid files: a png image of 125.646.848 bytes is very unlikely to exist inside a ISO file of 315.994.112 bytes.

In theory it can also list files that are inside a floppy image file which is included in the iso. This will be very unlikely in this case, because all floppy images are compressed with gzip.

Look at the column after the date. It is the LBA address (sector at with the file starts) of the file on the disk and compare it with the output of ISObuster.

The filesize won't always match between ISObuster, photorec and isoinfo, because when searching for missing files, ISObuster and photorec will pad the files so their size is a multiple of 2048 (CD sector size), unles they can reliable determine where a file stops.

The point is that when you use quick erase, you are only erasing the info that points to the actual data, similar to the "delete" or "format /q" commands of FAT. That is how the "undelete" and "data recovery" programs work.

That is why I still think that when using a CD-RW, it is possible to leave info in the data area that is not really part of the last burn session.

When burning ubcd (about 300MB, less than half of the whole capacity), the laser is not going to touch the last half of the surface. If there were previous sessions covering more than 300MB, then the data area is not completely "clean", unless you make a "full erase", instead of a quick erase.

How these "dirty" sectors are getting into the 300MB iso image? I still don't know.

The missing files ISObuster found are not left over files inside the ISO image.

So you are saying there are no "dirty" sectors in the iso image? None at all?

If that's true, then I still don't understand why Isobuster would "mark" part of those "missing files and folders" as "bad sectors".

With "bad sectors", I mean those files that are marked in Isobuster with a specific icon: a big red "X".

Not only those files are "missing" (in terms of Isobuster), but are marked as files (at least partially) burned in physical "bad" sectors.

When I used complete 1440K floppy images, I found "missing files", and using winimage's "defrag" function, the "dirty" sectors of the image were gone.

But the ubcd503.iso image is not a "complete" image (meaning an image of every physical sector of the CD) as the 1440K floppy image.

So I am still confused about how to differentiate actual "dirty" info/sectors from "errors" of the Isobuster algorithm that make it understand that data as "missing files and folders", specially those marked as "bad sectors".

The missing files ISObuster found are not left over files inside the ISO image.

So you are saying there are no "dirty" sectors in the iso image? None at all?.

Yes. The ISO image is created by mkisofs from scratch (takes files from a directory and puts them in an ISO image). It is a process that you could compare with creating a new zip archive. The ISO image isn't a copy of a burned CD-R(W).

ady wrote:
When I used complete 1440K floppy images, I found "missing files", and using winimage's "defrag" function, the "dirty" sectors of the image were gone.

Floppy images are different. The normally use FAT12 as filesystem. Floppy images are readable and writable while ISO images are only readable (once created no extra info is ever written at it). Most of the time, floppy images aren't created from scratch. So leftover files can exists when someone deletes files or when a file is updated but the newer file is smaller than the old one, so you still can retrieve part of the old file.

ady wrote:
So I am still confused about how to differentiate actual "dirty" info/sectors from "errors" of the Isobuster algorithm that make it understand that data as "missing files and folders", specially those marked as "bad sectors".

ISObuster finds files by looking for file signatures (magic bytes) in all sectors:http://www.garykessler.net/library/file_sigs.html
You will see that some files that are in the ISO image aren't detected by ISObuster because it doesn't has a file signature for that file type (e.g.: .c32 modules).
ISObuster tries to find all files only with the file signature without looking at the iso9660 filesystem itself. So in case you have a bad CD-R(W) you can still retrieve some files even if the iso9660 filesystem itself is damaged.

You can confirm that ISObuster finds files that are real files (that are normally visable in the ISO), by comparing the LBA offset column for all found files with the green part of the isoinfo output I posted.

As I said before, I understand the differences between a FAT12 complete floppy image and an iso image.

ISObuster finds files by looking for file signatures (magic bytes) in all sectors

In my system, not all the found files are found by their file signatures. I see an additional list of files as part of a FAT section on the left panel. I wonder if this list is particular to my system, or everyone can see it after running "find missing files and folders" function of Isobuster.

Of course, the particular information of those lists doesn't make any sense.

In this context, the important thing is: I don't see those LBA numbers (or a similar range) as part of the regular files included in the ubcd503.iso in Isobuster.

So:

a ) Are you seeing the same FAT section found by the "find missing files and folders" function of Isobuster? Or only the files found by their signatures?

b ) Even using an algorithm, I can't see why Isobuster marks those blocks as "bad sectors". Moreover, those sectors are far away of the common LBA numbers I find on the "normal" files. Are you also seeing this "bad sectors" in your system? Or is it my particular system?

c ) Does Isobuster and Isoinfo are posting different information about LBA numbers? They should be the same numbers, shouldn't they?

ISObuster just shows bogus information. I don't know why you make such a large problem of it.
You have to ask the ISObuster people, why they show such info. AFAI understand ISObuster can help you so retrieve files from bad CDs. They probably rather want to show some false-positives than don't showing them and missing some files that are really there.

Icecube, thank you very much for your time and answers. I really appreciate it.

Icecube wrote:I see the same strange file in the FAT subsection.

That would suggest that is not a problem of my particular system.

ISObuster just shows bogus information.

It just surprises me that Isobuster is showing "bad sectors" from the original iso image. I understand why the list of files/folders found by their signatures are false positives, but the "bad sectors" surprises me.
I wonder if other recovery software, like CDmage, ICE ECC recovery, DVDisaster or many others also shows this same list of "bad sectors".

I don't know why you make such a large problem of it.

After customizing UBCD, the new iso image also shows exactly the same list of "missing files and folders" in Isobuster.

I was just wondering if those "bad sectors" would get to a burned CD-RW, propagating "missing files/folders", or "bad sectors".

You have to ask the ISObuster people, why they show such info. AFAI understand ISObuster can help you so retrieve files from bad CDs. They probably rather want to show some false-positives than don't showing them and missing some files that are really there.

You are right, but I wanted to be sure that is not something particular of my system, and that is not a problem generated by the ubcd2iso script or something related to UBCD.

I guess I'd have to check it with other tools (like the ones I mentioned a couple of paragraphs above) to be completely sure is a problem with Isobuster algorithm. Or maybe someone else already has those tools installed and want to check it? There must be some similar tool also under Linux, but that is far away from my knowledge.

Something I forgot to mention:
The FAT (floppy, hard disk, ... filesystem) subsection is totally irrelevant for ISO's, or at least for the UBCD ISO.
ISO's use iso9660 or udf as filesystem.

This FAT subsection would only be valid for ISO images which use floppy or hard disk emulation. Then the ISObuster algorithm for the FAT subsection finds the files inside the floppy (e.g.: MemTest Dlx 32 bit iso) or hard disk image (e.g.: DELL diagnostics ISO image) which is embedded in the ISO and is booted when you boot the ISO.

I have never seen a program that could burn a new ISO image over a CD-RW that already had files on it, without erasing the CD-RW first. I don't know why it isn't supported. If you find a program that is able to do it, tell me. I am interested.

Icecube, reading ImgBurn guides:

... ImgBurn will perform a full format. After that the DVD+RW, DVD-RAM or Blu-ray disc is just overwritten without the need for any sort of erase (it's not like DVD-RW in that respect).

This paragraph refers to formatting a DVD+RW the first time, and then you could use it again with no need to erase it every time. I know, it is not a CD-RW. I just thought I could mention it, just in case this info is of any interest.