Friday, April 2, 2010

How to extend LVM on VMWare Guest running Linux

Recently I found my one of virtual Linux machine which running on VMWare ESX server have run out of disk space. And after I googled the Internet, I think it's quite simple as my virtual Linux machine is running on LVM.

And I also found the following solution will also apply to any VMWare virtual machine which running Linux OS with LVM.

Expand the disk

Turn off the virtual machine that you wish to extend disk space.

You can expand the virtual disk size by entering ESX server hidden console mode or using VMware Infrastructure Client to remote manage your ESX Server. I will introduce both way here.

Method One:

Since the GUI mode is easy. Click Your VMWare Guest Machine in your remote management console from VMware Infrastructure Client. Then click "Edit Virtual Machine Settings" from "Getting Started" which is in the Right Panel. Select Hardware -> Hard Disk 1 -> Capacity - New Size (at Right Side), and then expand the Capacity to whatever size that you need. In our case, I expanded from 100GB to 200GB. Then press "OK".

Method Two:

For ESXi Server, using the following command to enter ESXi hidden console mode after press "Alt"+"F1":

unsupported

Then press "Enter", now enter your ESXi server root password. You should be able to successfully log into the hidden ESXi Server console.

For ESX Server, If you press Alt-F1, you can get access to the Linux login prompt of the ESX service console and login to a command prompt.

Now enter the following command to expand the VM Disk:

vmkfstools -X 200G "Your VMDK file here"

To extend an existing Virtual Disk to 200GB.

Note:

If you are running VMware workstation product, then you can expand the virtual disk to expand the virtual disk on your Windows machine by running this command:

vmware-vdiskmanager -x 200G "My harddisk.vmdk"

After above steps, you finished preparing the extra virtual disk space. Now we can start your VMware virtual machine and open a terminal session to continue expanding your LVM.

Issue the df -k command which shows us that the logical volume is at 100% usage. We need to create a partition on /dev/sda. Type ls -al /dev/sda* which lists the existing partitions. Our new partition will be /dev/sda3.

Type fdisk /dev/sda then type n for new partition. Enter p and 3 for the partition number (in this instance, obviously enter the partition number that matches your environment.) Also accept the default First and Last cylinders. Finally type w to write table to disk and exit.

If prompted to reboot then do so to ensure that the new partition table is written. After a restart, type ls -al /dev/sda* to verify whether the new partition was created successfully.

Which shows that there is 99.97GB free that we can add to the volume group. To extend the volume type lvextend -L +99.97G /dev/VolGroup00/LogVol00. If you got an error message that stated that there was "Insufficient free space: 2871 extends needed but only 2870 evailable". After a quick google: Insufficient Free Extents for a Logical Volume. It seems that we needed to select a lightly smaller size. So we changed to 99GB rather than 99.97GB which solved this problem.

29 comments
:

At my job, after doing the same kind of procedure graph, we began to ask ourselves, why are using a LVM on a Linux VM guests?

Since we're no longer living in the physical OS world, we didn't need to use the OS hacks(LVM) to overcome physical disk limitations anymore.We decided to Just let the hypervisor and virtual storage do that work for us.

For example, in our production setup (3 tier commerce with VMs for database , webserver, and appserver), we're see a great improvement in managability and performance (>10%) by just dropping LVM, and most partitions.

Going to a LVM/partition-less setup reduces expansion to 3 steps and we don't need to take the VM OS offline!1. Increase size of VMDK2- Inside the VM, OS, rescan the scsi drive with:'echo 1 >/sys/class/scsi_device//rescan; dmesg' (dmesg will check that you drive isize has grown)3- Resize2fs.

Doing this has simplified resizing so much, I now let the junior admins and my manager expand drive space as needed.

It's also let's us really be spartan on space since expansion is so quick. Instead of increasing systems in 30-50GB chunks, we can do 10-15GB and let our rmonitoring system warn us when space gets tight.

I used the method of Edward ( 7 steps) and I successfully expanded the disk of Linux EL5 guest from 150G to 350G for oracle appsR12.I would like to try out with 3 step but I am truggling with resize2fs.Could you please explain more about the use of resize2fs.I have done:$ echo "1" > /sys/class/scsi_device/0:0:0:0/device/rescan$ dmesgExtract from output of dmesg "SCSI device sda: 734003200 512-byte hdwr sectors (375810 MB)"$ resize2fs /dev/VolGroup00/LogVol00resize2fs 1.39 (29-May-2006)The filesystem is already 38256640 blocks long. Nothing to do!

On the slight chance that anonymous #1 reads this again, I would like to know what Linux distribution you are using? We have been testing the ability to grow a ext3 file system with no partitions and no LVM. We are trying to install RedHat 5 on /dev/sd# (with no partitions). It does not allow you to do this. We receive the error, "Device is a loopback and will need to be formatted before the install". To do some testing, I installed RedHat within partitions on my disks. Once the install is completed, I create a new file system on a standalone disk with no partitions (mkfs.ext3 /dev/sdd to be exact). I am able to then grow this partition exactly as you detailed above. However, I really want to know if there is a way to install RedHat on a partitionless system?

Hello,I think the post from Anonymous #1 mentions LVM/partition-less, which means (once again I think) that they're using the whole disk to run LVM on top of it. So instead of having /dev/sda1 as a Linux LVM partition and running 'pvcreate /dev/sda1', they probably didn't create a partition on sda, and used the whole disk for LVM ('pvcreate /dev/sda'), which then gives you the opportunity to increase the underlying VMDK, rescan the disk ('echo "1" > /sys/class/scsi_device/0:0:0:0/device/rescan' where 0:0:0:0 is the address of /dev/sda, as per 'ls -l /dev/disk/by-path'), and then increase the PV ('pvresize /dev/sda') after which you can increase the size of the LV you want to increase.On the 'lvextend' front, if you want to use all the remaining space, instead of running 'lvextend -L +99GB', you can run 'lvextend -l +100%FREE' (and notice the different in case on the '-L' and '-l' options).I hope this helps.

Although I have not tried this out ( but possibly will do that this week) the Comment #1 meant to create 3 virtual partitions from the vmware:0 - boot1 - / and evrything beneeth 2 - SWAPFrom the guest os perspsective this woul look like either /dev/sda, /dev/sdb, /dev/sdc or maybe /dev/sda1, /dev/sdb1, /dev/sdc1. Depending on that you create FS directly on to these "virtual" partitions without any LVM in-between !!!

What do you mean by 3 partitions from the vmware? If you mean 3 VMDKs, and not using LVM, then I think you'll have to create a partition on each VMDK (which will appear each as a sd disk (/dev/sda, /dev/sdb and /dev/sdc), as I'm not sure you can use the whole disk to install linux directly without partition nor LVM.So, from a VMware point of view, the disks (VMDKs) are "virtual" disk, but as far as the guest OS is concerned, they are physical, SCSI disks. So you could use /dev/sda1 as /boot, /dev/sdb1 as the root filesystem, and /dev/sdc1 as the swap space. But I think you *will* need to create a Linux partition (type 83, from memory), for the /boot and / filesystems and Linux swap partition type (once again from memory, type 82).

Good Post.The 3-step procedure described by anonymous one sounds good, but I think that's (at least with vmware workstation) is a one way road: One can increase the size of drives, but shrinking is no option (afaik). It would be of interest if this is also as easily as the described extension is possible

@ Paul Stone - you should not even need a reboot to update the partition table in the kernel, just re-read the partition table:blockdev --rereadpt /dev/sdxwhere x is the disk you've just changed.

@ Anonymous (September 7), with VMware, you can indeed grow disks, but not reduce them. The way around that, I found, is to create another (smaller) disk, add it to the VG you want to recude, and pvmove all the LVs to the new disk. At least that's how I did it.

Should the "blockdev --rereadpt" command work even on /dev/sda (the boot device containing mounted filesystems)? The 'rescan' option for the '0:0:0:0' device worked so "fdisk -l" sees the larger disk size, but after I deleted and recreated the last partition on the disk (used for LVM PV), the blockdev command returns "BLKRRPART: Device or resource busy"

vmkfstools -X 200G "Your VMDK file here" working fine. I am able to see that space in fdisk -l /dev/sdX

But when creating a partition I am not getting that additional space.

1) -X 110G makes the vmdk size to 110G . am i correct? Say I get 10G additional space by saying -X 110G on a 100G vmdk file. fdisk shows that its 110G , but when creating a partition I am not getting 10G???

Alright, since many are wondering about the first comment (I have, a lot) I found the way of making that setup work, it basicaly involves making a partition less install of your linux. Just tested it, resizing of / on /dev/sdc worked perfectly on production server. Complete series of instructions found here: http://luke.bowbak.org/2013/01/24/partitions-thing-of-the-past-2.html , big fucking thanks to Luke Bakken.