**Contents**
[[!toc levels=3]]
# NetBSD Logical Volume Manager (LVM) configuration
NetBSD LVM allows logical volume management on NetBSD systems, with a well known
user interface, which is the same as the Linux LVM2 tools.
NetBSD LVM is built on Linux lvm2tools and libdevmapper, together with a
BSD-licensed device-mapper kernel driver specially written for NetBSD.
The LVM driver allows the user to manage available disk space effectively and
efficiently. Disk space from several disks, and partitions, known as *Physical
Volumes*, can be added to *Volume Groups*, which is the pool of available disk
space for *Logical Partitions* aka Logical Volumes.
Logical Volumes can be grown and shrunk at will using the LVM utilities.
The basic building block is the Physical Volume. This is a disk, or a part of a
disk, which is used to store data.
Physical Volumes are aggregated together to make Volume Groups, or VGs.
Typically, Volume Groups are used to aggregate storage for the same functional
unit. Typical Volume Groups could thus be named `Audio`, `Multimedia` or
`Documents`. By segregating storage requirements in this functional way, the
same type of resilience and redundancy is applied to the whole of the functional
unit.
The steps required to setup a LVM are as follows:
1. Install physical media
2. Configure kernel support
3. Configure system, install tools
4. *Optional step*
Disklabel each volume member of the LVM
5. Initialize the LVM disk devices
6. Create a volume group from initialized disks
7. Create Logical volume from created Volume group
8. Create a filesystem on the new LV device
9. Mount the LV filesystem
This example features a LVM setup on NetBSD/i386.
## Anatomy of NetBSD Logical Volume Manager
![Anatomy of Logical Volume Management](/guide/images/lvm.png)
1. **Volume Group**
The Volume Group is a disk space pool from which the user creates Logical
Volumes and to which Physical Volumes can be added. It is the basic
administration unit of the NetBSD LVM implementation.
2. **Physical Volume**
A physical volume is the basic unit in a LVM structure. Every PV consists of
small disk space chunks called Physical Extends. Every Volume Group must
have at least one PV. A PV can be created on hard disks or hard disk like
devices such as raid, ccd, or cgd device.
3. **Logical Volume**
The Logical Volume is a logical partition created from disk space assigned
to the Volume Group. LV can be newfsed and mounted as any other pseudo-disk
device. Lvm tools use functionality exported by the device-mapper driver in
the kernel to create the LV.
4. **Physical Extents**
Each physical volume is divided chunks of disk space. The default size is
4MB. Every LV size is rounded by PE size. The LV is created by mapping
Logical Extends in the LV to Physical extends in a Volume group.
5. **Logical Extents**
Each logical volume is split into chunks of disk space, known as logical
extents. The extent size is the same for all logical volumes in the volume
group.
6. **Physical Extents mapping**
Every LV consists of *LEs* mapped to *PEs* mapped by a target mapping.
Currently, the following mappings are defined.
* **Linear Mapping**
will linearly assign range of PEs to LEs.
For example it can map 100 PEs from PV 1 to LV 1 and another 100 PEs from
PV 0.
* **Stripe Mapping**
will interleave the chunks of the logical extents across a number of
physical volumes.
7. **Snapshots**
A facility provided by LVM is 'snapshots'. Whilst in standard NetBSD, the
[fss(3)](http://netbsd.gw.com/cgi-bin/man-cgi?ffs+3+NetBSD-current) driver
can be used to provide file system snapshots at a file system level, the
snapshot facility in the LVM allows the administrator to create a logical
block device which presents an exact copy of a logical volume, frozen at
some point in time. This facility does require that the snapshot be made at
a time when the data on the logical volume is in a consistent state.
*Warning*: Snapshot feature is not fully implemented in LVM in NetBSD and
should not be used in production.
## Install physical media
This step is at your own discretion, depending on your platform and the hardware
at your disposal. LVM can be used with disklabel partitions or even with
standard partitions created with fdisk.
From my `dmesg`:
Disk #1:
probe(esp0:0:0): max sync rate 10.00MB/s
sd0 at scsibus0 target 0 lun 0: SCSI2 0/direct fixed
sd0: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors
Disk #2
probe(esp0:1:0): max sync rate 10.00MB/s
sd1 at scsibus0 target 1 lun 0: SCSI2 0/direct fixed
sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors
Disk #3
probe(esp0:2:0): max sync rate 10.00MB/s
sd2 at scsibus0 target 2 lun 0: SCSI2 0/direct fixed
sd2: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
Disk #4
probe(esp0:3:0): max sync rate 10.00MB/s
sd3 at scsibus0 target 3 lun 0: SCSI2 0
sd3: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
## Configure Kernel Support
The following kernel configuration directive is needed to provide LVM device
support. It is provided as a kernel module, so that no extra modifications need
be made to a standard NetBSD kernel. The dm driver is provided as a kernel
module, it first appeared in the NetBSD 6.0 release.
If your system doesn't use modules you can enable dm driver in NetBSD by adding
this line to kernel configuration file. This will add device-mapper driver to
kernel and link it as statically linked module.
pseudo-device dm
If you do not want to rebuild your kernel only because of LVM support you can
use dm kernel module. The devmapper kernel module can be loaded on your system.
To get the current status of modules in the kernel, the tool
[modstat(8)](http://netbsd.gw.com/cgi-bin/man-cgi?modstat+8+NetBSD-current)
is used:
vm1# modstat
NAME CLASS SOURCE REFS SIZE REQUIRES
cd9660 vfs filesys 0 21442 -
coredump misc filesys 1 2814 -
exec_elf32 misc filesys 0 6713 coredump
exec_script misc filesys 0 1091 -
ffs vfs boot 0 163040 -
kernfs vfs filesys 0 10201 -
ptyfs vfs filesys 0 7852 -
You can use
[modload(8)](http://netbsd.gw.com/cgi-bin/man-cgi?modload+8+NetBSD-current) to
load the dm kernel module by issueing `modload dm`:
vm1# modstat
NAME CLASS SOURCE REFS SIZE REQUIRES
cd9660 vfs filesys 0 21442 -
coredump misc filesys 1 2814 -
dm misc filesys 0 14448 -
exec_elf32 misc filesys 0 6713 coredump
exec_script misc filesys 0 1091 -
ffs vfs boot 0 163040 -
kernfs vfs filesys 0 10201 -
ptyfs vfs filesys 0 7852 -
## Configure LVM on a NetBSD system
For using LVM you have to install lvm2tools and libdevmapper to NetBSD system.
These tools and libraries are not enabled as default.
To enable the build of LVM tools, set `MKLVM=yes` in the `/etc/mk.conf` or
`MAKECONF` file.
## Disklabel each physical volume member of the LVM
Each physical volume disk in LVM will need a special file system established. In
this example, I will need to disklabel:
/dev/rsd0d
/dev/rsd1d
/dev/rsd2d
/dev/rsd3d
It should be borne in mind that it is possible to use the NetBSD vnd driver to
make standard file system space appear in the system as a disk device.
*Note*: Always remember to disklabel the character device, not the block device,
in `/dev/r{s,w}d*`
*Note*: On all platforms except i386 where `d` partition is used for this, the
`c` slice is symbolic of the entire NetBSD partition and is reserved.
You will probably want to remove any pre-existing disklabels on the physical
volume disks in the LVM. This can be accomplished in one of two ways with the
[dd(1)](http://netbsd.gw.com/cgi-bin/man-cgi?dd+1+NetBSD-5.0.1+i386) command:
# dd if=/dev/zero of=/dev/rsd0d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd1d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd2d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd3d bs=8k count=1
If your port uses a MBR (Master Boot Record) to partition the disks so that the
NetBSD partitions are only part of the overall disk, and other OSs like Windows
or Linux use other parts, you can void the MBR and all partitions on disk by
using the command:
# dd if=/dev/zero of=/dev/rsd0d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd1d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd2d bs=8k count=1
# dd if=/dev/zero of=/dev/rsd3d bs=8k count=1
This will make all data on the entire disk inaccessible. Note that the entire
disk is slice `d` on i386 (and some other ports), and `c` elsewhere (e.g. on
sparc). See the `kern.rawpartition` sysctl - `3` means `d`, `2` means `c`.
The default disklabel for the disk will look similar to this:
# disklabel -r sd0
[...snip...]
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 207
total sectors: 208896
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
4 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 208896 0 4.2BSD 0 0 0 # (Cyl. 0 - 207*)
d: 208896 0 unused 0 0 # (Cyl. 0 - 207*)
You will need to create one *slice* on the NetBSD partition of the disk that
consumes the entire partition. The slice must begin at least two sectors after
end of disklabel part of disk. On i386 it is `sector` 63. Therefore, the `size`
value should be `total sectors` minus 2x `sectors`. Edit your disklabel
accordingly:
# disklabel -e sd0
*Note*: The offset of a slice of type `4.2BSD` must be a multiple of the
`sectors` value.
*Note*: Be sure to `export EDITOR=[path to your favorite editor]` before
editing the disklabels.
*Note*: The slice must be fstype `4.2BSD`.
Because there will only be one slice on this partition, you can recycle the `d`
slice (normally reserved for symbolic uses). Change your disklabel to the
following:
3 partitions:
# size offset fstype [fsize bsize cpg]
d: 4197403 65 4.2BSD # (Cyl. 1 - 4020*)
Optionally you can setup a slice other than `d` to use, simply adjust
accordingly below:
3 partitions:
# size offset fstype [fsize bsize cpg]
a: 4197403 65 4.2BSD # (Cyl. 1 - 4020*)
c: 4197405 0 unused 1024 8192 # (Cyl. 0 - 4020*)
Be sure to write the label when you have completed. Disklabel will object to
your disklabel and prompt you to re-edit if it does not pass its sanity checks.
## Create Physical Volumes
Once all disks are properly labeled, you will need to create physical volume on
them. Every partition/disk added to LVM must have physical volume header on
start of it. All informations, like Volume group where Physical volume belongs
are stored in this header.
# lvm pvcreate /dev/rwd1[ad]
Status of physical volume can be viewed with the
[pvdisplay(8)](http://netbsd.gw.com/cgi-bin/man-cgi?pvdisplay+8+NetBSD-current)
command.
# lvm pvdisplay
## Create Volume Group
Once all disks are properly labeled with physical volume header, volume group
must be created from them. Volume Group is pool of PEs from which administrator
can create Logical Volumes *partitions*.
# lvm vgcreate vg0 /dev/rwd1[ad]
* `vg0` is name of Volume Group
* `/dev/rwd1[ad]` is Physical Volume
The volume group can be later extended/reduced with
[vgextend(8)](http://netbsd.gw.com/cgi-bin/man-cgi?vgextend+8+NetBSD-current)
and
[vgreduce(8)](http://netbsd.gw.com/cgi-bin/man-cgi?vgreduce+8+NetBSD-current)
commands. These commands add physical volumes to VG.
# lvm vgextend vg0 /dev/rwd1[ad]
# lvm vgreduce vg0 /dev/rwd1[ad]
The status of Volume group can be viewed with the
[vgdisplay(8)](http://netbsd.gw.com/cgi-bin/man-cgi?vgdisplay+8+NetBSD-current)
command.
# lvm vgdisplay vg0
## Create Logical Volume
Once the volume group was created, the administrator can create `logical
partitions` volumes.
# lvm lvcreate -L 20M -n lv1 vg0
* `vg0` is the name of the volume group
* `-L 20M` is the size of the logical volume
* `-n lv1` is the name of the logical volume
Logical Volume can be later extended/reduced with the
[lvextend(8)](http://netbsd.gw.com/cgi-bin/man-cgi?lvextend+8+NetBSD-current)
and
[lvreduce(8)](http://netbsd.gw.com/cgi-bin/man-cgi?lvreduce+8+NetBSD-current)
commands.
# lvm lvextend -L+20M /dev/vg0/lv1
# lvm lvreduce -L-20M /dev/vg0/lv1
*Note*: To shrink a lv partition you have to shrink filesystem before. See the
manpage of
[resize_ffs(8)](http://netbsd.gw.com/cgi-bin/man-cgi?resize_ffs+8+NetBSD-current)
for how to do this.
The status of Logical Volume can be viewed with the
[lvdisplay(8)](http://netbsd.gw.com/cgi-bin/man-cgi?lvdisplay+8+NetBSD-current)
command
# lvm lvdisplay lv0/lv1
After reboot, all functional LVs in the defined volume group can be activated
with the command:
# lvm vgchange -a y
## Example: LVM with Volume groups located on raid1
The motivation for using raid 1 disk as physical volume disk for Volume Group is
disk reliability. With the PV on raid 1 disk it is possible to use Logical
Volumes even after disk failure.
### Loading Device-Mapper driver
Before we can start work with the LVM tools, we have to be sure that NetBSD dm
driver was properly compiled into the kernel or loaded as a module. The easiest
way to find out if we have dm driver available is to run `modstat`. For more
information, see [[Configure Kernel Support
chapter|guide/lvm#configuring-kernel]].
### Preparing raid1 installation
Following the example raid configuration defined in [[Raid 1
configuration|guide/rf#configuring-raid]], the user will set up a clean raid1
disk device with 2 disks in a mirror mode.
#### Example RAID1 configuration
# vi /var/tmp/raid0.conf
START array
1 2 0
START disks
/dev/wd2a
/dev/wd1a
START layout
128 1 1 1
START queue
fifo 100
# raidctl -v -C /var/tmp/raid0.conf raid0
raid0: Component /dev/wd1a being configured at col: 0
Column: 0 Num Columns: 0
Version: 0 Serial Number: 0 Mod Counter: 0
Clean: No Status: 0
Column out of alignment for: /dev/wd2a
Number of columns do not match for: /dev/wd2a
/dev/wd2a is not clean!
raid0: Component /dev/wd1a being configured at col: 1
Column: 0 Num Columns: 0
Version: 0 Serial Number: 0 Mod Counter: 0
Clean: No Status: 0
Column out of alignment for: /dev/wd1a
Number of columns do not match for: /dev/wd1a
/dev/wd1a is not clean!
raid0: There were fatal errors
raid0: Fatal errors being ignored.
raid0: RAID Level 1
raid0: Components: /dev/wd2a /dev/wd1a
raid0: Total Sectors: 19540864 (9541 MB)
# raidctl -v -I 2004082401 raid0
# raidctl -v -i raid0
Initiating re-write of parity
# tail -1 /var/log/messages
raid0: Error re-writing parity!
# raidctl -v -s raid0
Components:
/dev/wd2a: optimal
/dev/wd1a: optimal
No spares.
Component label for /dev/wd1a:
Row: 0, Column: 1, Num Rows: 1, Num Columns: 2
Version: 2, Serial Number: 2004082401, Mod Counter: 7
Clean: No, Status: 0
sectPerSU: 128, SUsPerPU: 1, SUsPerRU: 1
Queue size: 100, blocksize: 512, numBlocks: 19540864
RAID Level: 1
Autoconfig: No
Root partition: No
Last configured as: raid0
Parity status: DIRTY
Reconstruction is 100% complete.
Parity Re-write is 100% complete.
Copyback is 100% complete.
Component label for /dev/wd2a:
Row: 0, Column: 1, Num Rows: 1, Num Columns: 2
Version: 2, Serial Number: 2004082401, Mod Counter: 7
Clean: No, Status: 0
sectPerSU: 128, SUsPerPU: 1, SUsPerRU: 1
Queue size: 100, blocksize: 512, numBlocks: 19540864
RAID Level: 1
Autoconfig: No
Root partition: No
Last configured as: raid0
Parity status: DIRTY
Reconstruction is 100% complete.
Parity Re-write is 100% complete.
Copyback is 100% complete.
After setting up the raid we need to create a disklabel on the raid disk.
On i386:
# disklabel -r -e -I raid0
type: RAID
disk: raid
label: fictitious
flags:
bytes/sector: 512
sectors/track: 128
tracks/cylinder: 8
sectors/cylinder: 1024
cylinders: 19082
total sectors: 19540864
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
# size offset fstype [fsize bsize cpg/sgs]
a: 19540789 65 4.2BSD 0 0 0 # (Cyl. 0 - 18569)
d: 19540864 0 unused 0 0 # (Cyl. 0 - 19082*)
On sparc64:
# disklabel -r -e -I raid0
[...snip...]
total sectors: 19539968
[...snip...]
2 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 19540793 65 4.2BSD 0 0 0 # (Cyl. 0 - 18799)
c: 19539968 0 unused 0 0 # (Cyl. 0 - 19081)
Partitions should be created with offset 65, because sectors < than 65 sector
are marked as readonly and therefore can't be rewriten.
### Creating PV, VG on raid disk
Physical volumes can be created on any block device (i.e., disklike), and on any
partition on them. Thus, we can use the `a`, `d`, or on sparc64 `c` partitions.
The PV will label the selected partition as LVM-used and add the needed
metainformation for the LVM to it.
The PV is created on the character disk device, as all other disk operations in
NetBSD:
# lvm pvcreate /dev/rraid0a
For our example purpose I will create the `vg00` Volume Group. The first
parameter of `vgcreate` is the name of the volume group, and the second is the
PV created on the raid. If you later found out that the volume group size is not
sufficient, and you need more space, you can extend it with `vgextend`:
# lvm vgcreate vg00 /dev/rraid0a
# lvm vgextend vg00 /dev/rraid1a
**Warning**: If you add a non-raid PV to your Volume Group, your data is not
safe anymore. Therefore you should add a raid based PV to VG if you want to keep
your data safe.
### Creating LVs from VG located on raid disk
For our example purpose we will create Logical Volume named lv0. If you later
found that LV size is not sufficient for you can add it with `lvresize`.
*Note*: You have to resize filesystem, when you have resized LV. Otherwise you
will not see any filesystem change when you mount LV.
**Warning**: Shrinking of ffs file system is not supported in NetBSD be aware
that. If you want to play with file system shrinking you must shrink it before
you shrink LV.
This means that the `-L-*` option is not available in NetBSD.
# lvm lvcreate -n lv0 -L 2G vg00
# lvm lvresize -L+2G vg00/lv0
All lv device nodes are created in the `/dev/vg00/` directory. File system can
be create on LV with this command. After file system creation LV can be mounted
to system.
# newfs -O2 /dev/vg00/rlv0
# mount /dev/vg00/lv0 /mnt/
### Integration of LV's in to the system
For Proper LVM integration you have to enable lvm rc.d script, which detect LVs
during boot and enables them. You have to add entry for Logical Volume to the
`/etc/fstab` file.
# cat /etc/rc.conf
[snip]
lvm=yes
# cat /etc/fstab
/dev/wd0a / ffs rw 1 1
/dev/vg00/lv0 /lv0/ ffs rw 1 1
[snip]