If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Howto get 44 logical partitions out of a SCSI/SATA/PATA/USB disk

Our Super Mod bwkaz, whom we call the village elder, has alerted me that the Linux kernel 2.6.28 and later have started to support more than 16 partitions and even beyond 64 partitions in a hard disk.

I can confirm that this is indeed the case as I have succeeded in using up to 130th partition when I tried. However not all the current partitioning programs, like cfdisk and fdisk, have been modified to reflect the new change. I only managed it with sfdisk and one or two other desktop-based partitioning tools to reach beyond the traditional 64 device names in a hard disk.

Conclusion : This thread using hidden partition technique to get 44 partitions will be obsolete in future. However the partition changes have not been implemented fully yet in some parts of the kernel, partitioning tools and the installers of many distros so it will be some time before things are sorted. In the mean time enjoy the freedom of the 44 partitions as suggested by this thread.

If you keep an eye here I shall report more findings with the new changes as I discover them. Latest report on 5/3/09 is here.

Howto get 44 logical partitions out of a SCSI/SATA/PATA/USB disk

﻿One line summary – Starting with a raw or empty spare hard disk, create the first extended partition and put 11 logical partitions inside, hide the first extended partition making it to become an unknown primary partition, repeat three times and use Grub to control the access of them.

Primary use – multi booting with Linux distros, using a large hard disk more efficiently, as a security option for saving data in area of the hard disk not available to the operating system or users.

Disadvantage – Only 11 partitions can be accessed at any one time after a system is booted. Grub needs to be hosted elsewhere in another disk, floppy, CD or pen drive.

Tools needed – Terminal partitioning tools like fdisk, cfdisk and sfdisk, Grub and any text editor which can be all inside a Linux Live CD that has Grub support (possibly over than 80&#37; of Live CD would qualify)

Technical details explained

A Linux using 2.6.20 kernel or later now calls every SCSI, Sata, Pata and USB hard disk by the same naming convention, using disk names sda, sdb, sdc, sdd, sde etc. A maximum of 16 device names is now standard for each disk. For the first disk sda the 16 devices names are sda, and sda1 to sda15. The sda1, sda2, sda3 and sda4 are permanently reserved for the 4 primary partitions, even if some of them are not used, and the rest sda5 to sda15 are 11 logical partition names.

Instead of using the entire internal area as one storage unit anyone of the four primaries can be turned into an extended partition to hold the 11 logical partitions inside. Linux, as any other PC system, recognises only one extended partition in the partition table. More than one extended partition is illegal not supported by any of the established PC systems.

However if an extended partition is hidden, by changing its partition ID, then an operating system will regard it as “foreign” and ignore its presence, thereby allowing another extended partition to be created. If all four primaries are used this way a maximum of 44 logical partitions can be created and used in a hard disk. There is an observed standard to hide a partition in a hard disk. Hiding an extended partition is not a common activity but it can be done with the existing software and convention. The 44 logical partitions are achieved simply by using the established partition hiding technique

This is done by the boot loader Grub which has to be hosted outside the 44-partition disk because the partition Grub residing must be available at all time and cannot be hidden, otherwise the control is lost or hidden.

Grub has a “hide” and “unhide” command in its instruction set that can be used on any partition. The resulting hidden non-extended partition ID is compatible with that supported by the Linux’s terminal command “fdisk”, “cfdisk” and “sfdisk”. Therefore there is no hacking and everything described here is the standard facilities in Linux. When an extended partition is formed by cfdiisk the partition ID is given the number "5". When Grub hides this partition its partition ID changes to "15". It will revert to "5" is the instruction is given to Grub to unhide it.

In operation I need 2 hard disks; one for the 44 partitions and another one for hosting the Grub menus. In my case these are sdb and sda. The sdb disk has 44 partitions inside. Both are Sata II disk each has 500Gb.

I need altogether 6 Grub menus.

Menu A for controlling which of the extended partition in sdb to be booted.

Here are the sample of my 6 menus. I managed to find a way to nest them together using the "configfile" statement. I see a future to expand the use of Grub because we can do really complicated arrangements if we can transfer control from menu to menu.

Menu A – master control - the gut that makes the 44 partitions possible

Basically if I boot sdb2 I unhide it and hide the sdb1, sdb3 to sdb4. Thus at any one time Linux sees no more than one extended partition. Once the sdb2 is recognised and so I can then call up Menu D to select which distros to boot.

In the above menu you can see that I could jump from one menu to another to free to select any distro inside the 44 logical partitions of the disk sdb.

The sda disk is intentionally for hosting Grub. I selected sda6 which has a Ubuntu 7.04 inside. The above 6 menus are held in the /boot/grub directory of Ubuntu inside partition sda6.

For application I use the 44 partitions for installing a Linux in each. One common swap for all of then and it is sda5. Again the swap being in sda will be available at all time.

As I can boot up any Linux inside any of the 4 extended partitions, even though not all the logical partitions have been filled yet, the proposed usage of the 44-partition is proved.

My sdb device is a 500Gb Sata II disk. I just partitioned every logical partition to 10Gb using the cfdisk program theoretically amounting to 440Gb. The excess between the 500Gb and 440Gb is dissipated into a few partitions. I wrote down a list of partitions and what systems I had installed.

It go without saying that the partition names of sdb5 and sdb15 have to be repeatedly in each of the four extended partitions. There is no possibility to mount a partition from a hidden extended partition because it is recognised by Linux as an unknown primary partition. When an extended partition is hidden the entire set of logical partitions will disappear.

To host Grub in a partition to carrying the access control of the 44 partitions

Warning : It is a repeat of what I said above but it is vital that Grub and the swap partition are “outside” these 44 logical partitions. The reason is simple. If you hide Grub then you to lose the control too.

The access control can be done on the back of any installed Linux that uses Grub as its boot loader. It is just a matter of amending its menu.lst and adding 5 more optional Grub menus as indicated in the above. In my case I just use the Ubuntu installed in sda6

The menus are always text files inside the /boot/grub directory. For a Linux installed with a separate /boot partition the menus are stored in /grub directory and Red Hat family distros may use grub.conf in lieu of menu.lst although the two are usually symbolically linked (i.e. editing either one will do).

It is noteworthy to point out that I wrote the 6 menu.lst ahead of the Linux installation. The menu.lst is directed to the partitions and marked as “Empty @ sdb7” for example. When a Linux has been installed inside sdb7 it can be instantly bootable by highlight the choice of “Empty @ sdb7”.

I have used chainloading throughtout the menu. The only partition that I did not chainload is sda6 because it would go round in circles if I did. Therefore only Ubuntu in sda6 is booted directly as indicated in Menu A.

To make a Linux chainloadable just tell the installer of every distro that you want its boot loader in the root partition. There is nothing else to do and the Linux will boot immediately after installation regardless of what boot loader it has got.

Lastly I post the partition tables of the sda, sdb1, sdb2, sdb3 and sdb4 here for completeness.

I have been experiemnting with the hidden extended partitions for some time and can confirm that it work satisfactorily. To be frank I haven't run into any difficulty at all.

The principle is sound because the extended partition "is" basically a primary partition with an identical partition table except the Type No is different. Every operating system ignores foreign partitions it doesn't support. This is same for every MS system, Linux and Unix-like system. There is nothing sinister or illegal to the operating systems. We just make use of their known behaviour to do things to suit ourselves.

I had in the past moved the extended partitions successfully by hiding it so that the border can be moved around as one unit.

In the box where I have installed Solaris, BSD and Linux I hide the extended partition (with Linux inside) whenever I run Solaris and BSD. I do the opposite when I run Linux inside the extended partition by hiding Solaris and BSD partitions. So far this arrangement prevents any corruption to the partition table of the hard disk.

No, LVM isn't the same setup as the old DOS partition tables, and you may not be able to interoperate with other OSes. But you can create one small partition for root+boot (or just for /boot, if your distro's initramfs supports LVM), and a large partition for all the Linux systems. Then make that large partition into a physical volume, and add it to a volume group. Then make several logical volumes out of that group, one per Linux system you want to host. Put all their kernels and initramfs images onto the /boot partition (because grub doesn't understand LVM yet, and may never understand it).

AFAIK, LVM isn't limited at all in the number of logical volumes you can create in a group.

I have so far stay away from the LVM but know we can use it to span several disks together to form one volume.

I am not looking for a solution here. It is just something I tried. I possibly still prefer to see each distro installed completely in a self contain partition with its own boot loader. This way I can understand, maintain, resize and migrate the system a lot easier.

Half of the thread is to show how to nest a group of Grub menus.

Hosting Grub is a separte /boot partition is necessary for a LVM as you have indicated. As far as I am aware no Linux boot loader can read a LVM so the kernel must be placed outside the LVM.

I am sure when I come down to a few disros to use them continuously I would need to get into LVM. At the moment I am still installing with every distro I can find. The advance in Linux in the last couple of years has been truly amazing and it is a challenge just to see what is going on.

Right. But I figured if anyone else was looking, LVM might help them if they need tons of partitions on a SCSI device. Hey, whatever.

As far as I am aware no Linux boot loader can read a LVM so the kernel must be placed outside the LVM.

Supposedly LILO can do LVM, but I have no idea how it does it. I have an idea on how to add support to grub for LVM, but I have a feeling it wouldn't be accepted upstream. (Basically, the way grub has a stage1_5 now, I'd add a stage1_1 that loaded LVM. Then it'd look for the stage1_5 inside the logical volume, to load the FS, which would then let it find stage2 and menu.lst.)

I am sure when I come down to a few disros to use them continuously I would need to get into LVM.

Oh, I don't know about that. I've been using just a couple distros (well actually, one LFS, and one other partition that's holding the previous or next LFS version while I build it) for a few years now, and I haven't moved any of my FSes to LVM yet. Mostly that was because I never looked into what all it was able to do, though.

I have an idea on how to add support to grub for LVM, but I have a feeling it wouldn't be accepted upstream. (Basically, the way grub has a stage1_5 now, I'd add a stage1_1 that loaded LVM. Then it'd look for the stage1_5 inside the logical volume, to load the FS, which would then let it find stage2 and menu.lst.)

wets my appetite.

I am not optimistic though. One can host Grub in a floppy or a bootable CD or a pen drive that has stage1.5 and stage2. Thus Grub is equipped with the full intelligence. We have to load a driver into Grub if we want it to read a LVM.

The driver inside a kernel to read a LVM may be small, as there are many modules already available, but for a simple boot loader it may be ill-equipped to do such a job. I am talking above my head here as I have no idea how LVM works except treating it as a management layer.

GRUB is no longer being developed because of messy code and design failures.

Development team is working on GRUB 2 now. Development version of GRUB 2 is usable and works fine on PC. OpenFirmware-based PowerPC machines (PowerMac and Pegasos) and EFI-based PC (IntelMac), and being ported to RiscOS/ARM and UltraSparc. GRUB 2 is working fine in my Test Rig.

I reserve my judgement on Grub 2. The last time I tried I couldn't get anywhere as there was no documentation.

I probably have pushed the Legacy Grub, or the old version developed to 0.97, more than most users have done. In my experience the statement of

GRUB is no longer being developed because of messy code and design failures.

isn't justified in practice.

I can understand the existing Legacy Grub isn't well written and probably very hard to maintain. The truth of the matter is that the maintenance to Legacy Grub was at the point of being stopped when I started learning Linux but through out last few years I have not discovered any major shortcoming to warrant a complaint to Grub.

By any standard I have been able to get better service by Grub than any boot loader from any PC system. The thing that bothers me is I could not single out even one part of Grub that can fail or is known to be unreliable. I am sure there must be instance Grub will show up deficiencies but I just never ran into one in the normal usage, despite having made Grub boot over 150+ systems of any Dos, MS Windows, Linix, BSD and Solaris I could find.

I can only get Grub to hang by repeatedly loading it, i.e. boot to (hd0), then (hd1,2), then back to (hd0).

Chainloading from an older version to a newer version or vice versa does cause erratic Grub behaviour but that is to be expected because features in one version may not be available in another.

I still maintain in a Grub prompt there is no PC system Grub (legacy version) cannot boot. If a PC system is originally healthy and has a boot loader in its root partition it can always be fire it up by a Grub floppy as I have not run into an exception.

Of all the distros I come into contact with, including non-Linux distros, I haven't ran into Grub 2 myself. The Legacy Grub usage is still on the rise as less distros are using Lilo now while more Live CDs switch from isolinux to Grub. In the installation of the above 44-partition hard disk I think I could only find mainly the Slackware/Slax families are hanging onto the Lilo. Legacy Grub has enjoyed an increased popularity since I entered Linux. At the same time I have not seen a distro put the new Grub 2 into practice.

If the GNU/Grub team has done anything to popular Grub 2 then I must have missed it totally because even today I could only find the documentation section marked "GRUB 2 Documentation is under development.". That was the same status possibly 2.5 years ago. I find it strange that users without any instruction could run Grub 2 because it is not possible to correct an error if one crops up.

I like to move with time same as everybody to get my teeth into Grub 2 but from my experience with the versions from 0.91 to 0.97 Grub Legacy I do find the existing Grub "extremely" stable and reliable. If there are messy codes and design failures then in its current "defective" form it still beats all other boot loaders put together.

There is a new challenge facing Grub soon and I am not sure of its outcome. This is due the the use of 2.6.20 and newer kernels forcing Pata disks to use SCSI/SATA/USB disk names. There does not seem to be an agreed standard on the order of how different disk controllers should be scanned by the Bios. Thus one motherboard may assign sda name to a eSata controller while another motherboard favours an internal Sata disk to take the sda name. When mixed with SCSI and Pata disks it can lead to a chaotic situation.

Grub is known to work with the Bios information and may commit mistakes in the recognition the disks in a correct order. This error is not detectable as nobody know or can define a correct sequence of how different disk should be assigned. The whole idea of sda, sdb, sdc, sdd, sde, sdf.... naming convention is to let the motherboard to detect whatever are available. Grub identifies a disk by assigning a number so it is unaffected by the change in the hard disk names but it is hard to tell if Grub has made an error.

I am aware the order detected by Grub is not always in agreement with the bios setting which can also be different from the order determined by Linux once it is booted so there may be a time bomb there when more different type of disks and controllers are introduced into the PC.

Of all the distros I come into contact with, including non-Linux distros, I haven't ran into Grub 2 myself.

Please read my post again. GRUB 2 is being Developed and Stable version is not available yet. You wont find it in any distro and it will released very soon.

Originally Posted by saikee

If the GNU/Grub team has done anything to popular Grub 2 then I must have missed it totally because even today I could only find the documentation section marked "GRUB 2 Documentation is under development.". That was the same status possibly 2.5 years ago. I find it strange that users without any instruction could run Grub 2 because it is not possible to correct an error if one crops up.

Check this Link. its Home Page of GRUB2. I wonder, did you even try to Google for GRUB2?

Originally Posted by saikee

I like to move with time same as everybody to get my teeth into Grub 2 but from my experience with the versions from 0.91 to 0.97 Grub Legacy I do find the existing Grub "extremely" stable and reliable. If there are messy codes and design failures then in its current "defective" form it still beats all other boot loaders put together.

Yes, existing GRUB is extremely stable but it lacks a lot of features. Please check the above link.

There has been progress on GRUB 2, yes. (Of course the most recent "release" (i.e. 1.9x version) is 9 months old...) There has not been any progress on a GRUB 2 manual, or end-user documentation. (Or even a guide for "this is the data to put into your MBR before the partition table; this is how that code finds the next stage, this is how the module loading stuff works, this is how you tell it to read a config file", etc. There's plenty of "just run this command, it'll work, trust us", but not much "here's what that command actually does". It sounds like saikee wants the manual/docs before he'll use it; I want the "this is what this actually does" before I'll trust it.)

Many of us thoroughly understand grub legacy, to the point where we can get it to do almost anything (and even, in some cases, know exactly where to add new features that we'd like to see; not that those new features would ever be accepted upstream, but still). None of us knows grub2 that well -- but that's understandable, as it's new. However, there's no way for us to find out, either, because the documentation doesn't exist yet. (At least, hopefully that's a "yet".)

Now yes, I was able to sorta, kinda figure out how to get it sorta, kinda working, via the TestingOnX86 wiki page. (But I didn't replace my main qemu grub-legacy bootloader with it, I just added a multiboot-type entry to fire up the grub2 core.img that I had it generate.) I'm still nowhere near familiar enough with it to replace grub-legacy, or to recommend it to anyone else (except, perhaps, people that need /boot-on-LVM, and aren't afraid to go read sources).

My problems following these instructions

Hello,

First I want to say thank you for your very useful and informative posts. I've used the ideas presented in some of your other posts to create a multiboot system, with Grub installed on a data partition.

Second this is my first post here although I have been following the forums for some time.

Now I am attempting to follow the idea presented in this post, but can not seem to accomplish my goal, perhaps I am overlooking something and it will be easily resolved.

I have /dev/sdc which is 500 GB and I want to use 4 extended partitions, hiding/unhiding each partition with Grub which is on it's own partition on another disk (sdb2).

With fdisk I've created the 1st extended partition on sdc1. But when I try to change it's type (t) to 15 (hidden extended) I get the message: You cannot change a partition into an extended one or vise versa Delete it first.

There seems to be a number of possible combinations so I unfortunately do not know more than to just ask. Does your method depend on having Grub tied to another Linux? If so I have Ubuntu installed on (sda8) and could try it that way. Although it seems from your instructions that you have previously created and hidden the partitions.

I've tried using parted to set up the partitions first, but it will not allow me to set the flag on the extended partition to hidden. I have also tried cfdisk but it does not directly show the extended partition. I have tried searching on the internet and I find very little, other than your most intriguing post. Will you please assist me, if it is within your grasp?

I think it is timely to talk about the difference between a Grub prompt and a Grub shell.

I believe your problem could be due to the limitation of working in a Grub shell where the integrity of the partitions is protected by Linux. Your Grub is a shell inside the Linux.

At the Grub prompt, which is at the stage before a kernel is booted, you can freely hide and unhide the the extended partition. The change is passed onto the Bios too and so the Linux if booted will conform to the revised changes.

It makes sense if you think about it.

In practical you will have to hide or unhide the extended partition in order to select a system to boot and so you will in a Grub prompt when using the 44 partitions.

I have not had any problem with my 44 partitions in one disk yet and the disk is now fully populated with distros.

The new grub, being graphical should have things like security features added to it like password protection on a OS basis level, and even a password prompt to bring up the menu at all, if set up that way by the user. It should also allow additions to the menu to be made without editing a .lst file. In other words, a person should be able to add an entry through the graphical front end of GRUB.

I have used XOSL for years, and have only recently come across problems with it. The mouse freezes after a few clics once in the loader screen, and it seems to have problems with SATA drives, and will not, of course, see the USB drive(s) I have in the PC at boot time. Anyhow, I just wanted to iterate that this boot loader was great in its day. It allowed me to boot BeOS, Linux, DOS, Windows, etc. without a hitch, and has switches for changing drive ordering, and hiding partitions, etc.

Grub is better, but a marraige between GRUB, and XOSL would be absolute heaven. It should also allow the user to put up his own graphic, and also even individual OS icons.

I noted also that it remarked about PPC versions being worked on. Don't forget to make it work on the Cell based PPC64 system that is a PS3. It has unique requirements for booting to an alternate OS than the Sony XMB front end (which I think is a glorified boot loader as well).

I am aware of XSOL but I was waiting to find out Grub's limit. As it happened Grub has answered all my need. My personal preference is to stick to the terminal mode as it makes the boot loader footprint a lot smaller. I don't think I have used Grub anywhere to its limit yet but probably in a position to make sensible comparison with other boot loaders. In terminal mode a user can communicate with Grub virtually like a mini operating system because a Grub prompt is a command interpretor just a Dos. Not many users know that we could manually boot up any PC system in the terminal mode of Grub. This feature is not available in other boot loaders which tend to be based on an operating system, whereas Grub can be installed unattached to any operating system.

Thus I stay away from any graphic boot loader because I want substance and not the appearance. There are a few graphic interfaces with Grub but they actually inhibit the full power of Grub available to the users.

Grub also has a file called stage2_eltorito enable Grub to be installed in any CD and DVD. Thus one can use one Grub to boot floppies, internal hard disks of any kind, external hard disks like eSata, USB devices, CD and DVD.

I do believe many bootloaders, which XOSL is one of them, are capable too but Grub seems to cover a much broader media yet remaining as the simplest and most flexible, at least to me.

I have not gone into the new Grub2 as there is not enough documentation for us to understand it yet. It is not used as the default boot loader by any distro yet, as far as I am aware.