The
vm
utility is used to provide simplified management of
bhyve(8)
virtual machines,
including networking and console access.

Networking is handled by creating one or more virtual switches. Each switch
has a simple name which is referenced in the virtual machine configuration file.
The
vm
utility automatically creates a
bridge(4)
device for each virtual switch and assigns virtual machine
tap(4)
interfaces dynamically.

All configuration for virtual machines is stored in a simple rc style configuration
file. When virtual machines are first created, the configuration file is copied from
a template which can be specified by the user. Multiple templates can be created providing
an easy way to provision guests with specific configurations.

vm
gracefully handles reboot and shutdown commands from inside the guests, whilst providing
full management of the virtual machine from the host system.

Once
vm
is installed, create the directory which will store your virtual machine configuration and data.
This directory will be referred to as
$vm_dir
throughput this man page.

Add the following into
/etc/rc.conf

vm_enable="YES"
vm_dir="/your/vm/path"
vm_list=""
vm_delay="5"

The first and second lines are required to enable the
vm
utiltity. Please see the
startall
command description for more information on the third and fourth settings.

Now run the
vm init
command to finish initialisation. This will create subdirectories inside
$vm_dir
to hold configuration and templates. It will also load any required kernel modules.
This command needs to be run on each boot, which is normally handled by the rc.d script.

Copy the sample templates into the directory
$vm_dir/.templates/.
You can create and edit the templates as required. It is recommended to keep a template called
default.conf,
as this will be used when no template is manually specified.

If you are using a ZFS dataset to store your virtual machines, and want a new child dataset created
for each one, specify the dataset to use in
/etc/rc.conf
as follows:

$vm_dir="zfs:pool/dataset"

In contrast to earlier versions, if
$vm_dir
is a normal path, a standard subdirectory will be created for each virtual machine, regardless
of the file system type. However,
vm
is now able to handle situations where the dataset mountpoint does not match the dataset name.

Create a new guest using the default template and disk size, then start the installation. The
install
subcommand will pause while the guest boot loader completes. Once successfully loaded, use the
console
command to complete the installation.

Windows guests are supported on versions of FreeBSD that have UEFI
support in
bhyve(8).
As of October 2015, this is only FreeBSD-CURRENT, so you will first need
to build and install the head FreeBSD branch manually.

You will also need a copy of the UEFI firmware (see URL below), which needs to be placed in
$vm_dir/.config/BHYVE_UEFI.fd.

As there is no VGA console in
bhyve(8),
an unattended installation ISO is required which allows Windows to install and
boot without any user interaction. Instructions for creating a suitable ISO can
be found at the URL below.

Once the installation ISO is ready, has been placed in the
$vm_dir/.iso
directory, and you have the UEFI firmware, installation can be performed as normal.

Windows installation has been tested with 2012r2 and takes around 20-25 minutes.
During install, the guest will reboot twice (three runs in total). You can see
the guest reboot by watching the log file
$vm_dir/guestname/vm-bhyve.log.
The third run should boot fully into Windows. The
virtio
network adapter will request an IP address using DHCP. Connect to the guest console
and press
i
to see the IP address that has been assigned. The default unattended installation files
should make RDP available, using Administrator and Test123 as the default login details.

The UEFI firmware (BHYVE_UEFI_20151002.fd), as well as instructions for creating an
unattended installation ISO can currently be obtained from
https://people.freebsd.org/~grehan/bhyve_uefi/

This should be run once after each host reboot before running any other
vm
commands. The main function of the
init
command is as follows:

o Load all necessary kernel modules if not already loaded
o Set tap devices to come up automatically when opened
o Create any configured virtual switches

switch list

List virtual switches. This reads all configured virtual switches from the
$vm_dir/.config/switch
file and displays them. If the virtual switches are loaded, it also tries
to display the
bridge(4)
interface that has been assigned to each one.

switch info [name]

This command shows detailed information about the specified virtual switch.
If no switch name is provided, information is output for all configured switches.
Information displayed includes the following:

Create a new virtual switch. The name must be supplied and may only contain
letters, numbers and dashes. However, it may not contain a dash at the beginning
or end.

When a new virtual switch is created, the persistent configuration file is updated
and a new
bridge(4)
interface is provisioned.

switch import name bridge

This command allows you to import an existing bridge interface that has been created
manually and use it for virtual machines. Once a bridge is imported, you can use
the switch
name
in guest configuration. Ideally the manual bridge should be configured in
/etc/rc.conf,
so that it is available on each host boot.

Please note that this creates a manual switch and is designed to allow you to configure your
own bridge. None of the
add,
remove,
vlan
or
nat
commands are supported on manual switches.

If a manual switch is destroyed using the
destroy
command, we remove all vm-bhvye configuration, but leave the
bridge(4)
interface intact.

switch vlan name vlan-id

Assign a VLAN number to a virtual switch. The VLAN number must be between 0-4094.

When adding an interface to a VLAN enabled virtual switch, a new
vlan(4)
interface is created. This interface has the relevent parent interface and VLAN tag
configured. This vlan interface is then added to the virtual switch. As such, all
traffic between guests on the same switch is untagged and travels freely. However,
all traffic exiting via physical interfaces is tagged.

If the virtual switch already has physical interfaces assigned, they are all removed
from the bridge, reconfigured, then re-added.

To remove the VLAN configuration from a virtual switch, specify a
vlan-id
of 0.

switch nat name on|off

Enable or disable NAT functionality on the specified switch. Please note that
pf
is required for this functionality and must be enabled in
/etc/rc.conf.
If DHCP is desired, please install the
dnsmasq
pacakge. vm-bhyve will generate a sample dnsmasq configuration in
/usr/local/etc/dnsmasq.conf.bhyve,
but it is up to the user to either use this configuration directly, or merge with
any existing dnsmasq settings you have configured.

The switch should have no host ports assigned, as these will end up on the private side
of the NAT network.
vm
automatically detects the hosts default gateway, which is used as the forwarding interface
for NAT connections.

Once enabled, a 172.16.X.0/24 network is assigned to the switch (bridge) interface.
X
is chosen based on the ID of the bridge interface. For example, if the switch is using
bridge10, the network will be 172.16.10.0/24.
dnsmasq
can be used to provide DHCP to the guests, and
pf
rules are inserted to provide the NAT translation.

/etc/pf.conf
is created if it doesnt exist, and a single include statement is added. This
include statement can be moved within the file if required.

switch add name interface

Add the specified interface to the named virtual switch.

The interface will immediately be added to the relevant bridge if possible, and
stored in the persistent switch configuration file. If a
vlan-id
is specified on the virtual switch, this will cause a new
vlan(4)
interface to be created.

switch remove name interface

Removes the specified interface from the named virtual switch and updates the
persistent configuration file.

switch destroy name

Completely remove the named virtual switch and all configuration. The associated
bridge(4)
interface will be removed, as well as any
vlan(4)
interfaces if they are not in use by other virtual switches.

create
[-t template]
[-s size]
name

Create a new virtual machine.

Unless specified, the
default.conf
template will be used and a 20GB virtual disk image is created. This command will
created the virtual machine directory
$vm_dir/$name,
and create the configuration file and empty disk image within.

-t template

Specifies the template to use from within the
$vm_dir/.templates
directory. The
.conf
suffix is not required.

-s size

The size of disk image to create in GB. Unless specified, the guest image will
be a sparse file 20GB in size.

List all the virtual machines in the
$vm_dir
directory. This will show the basic configuration for each virtual machine, and whether
they are currently running.

info [name]

Shows detailed information about the specified virtual machine. If no name is given,
information for all virtual machines is displayed.

This output includes detailed information about network and disk devices, including
the space usage for all virtual disks (exluding custom disk devices). If the guest
is running, the output also shows the amount of host memory curently in use,
and additional network details including bytes sent/received for each virtual interface.

install name iso

Start a guest installation for the named virtual machine, using the specified ISO file.
The
iso
argument should be the filename of an ISO file already downloaded into the
$vm_dir/.iso
directory. ISO files in this directory can be managed using the
iso
subcommand described below.

Once started, the guest loader will be booted in the foreground. This allows you to choose
the
Install
boot option for guests that require it. Once the loader has completed, you will be returned
to the shell and bhyve will continue running in the background. Use the
console
subcommand to connect to the guest and complete installation.

After installation, the guest can be rebooted and will restart using its own disk image to boot.
At this point the installation ISO file is still attached, allowing you to use the CD/DVD image
for any post installation tasks. The ISO file will remain attached after each reboot until the
guest is fully stopped.

start name

Start the named virtual machine. The guest will boot and run completely in the background. Use
the
console
subcommand to connect to it if required.

For each network adapter specified in the guest configuration, a
tap(4)
interface will be created. If possible, the tap interface will be attached the relevent
bridge(4)
interface, based on the virtual switch specified in the guest configuration.

stop name

Stop a named virtual machine. All
tap(4)
and
nmdm(4)
devices will be automatically cleaned up once the guest has exited.

console name [com1|com2]

Connect to the console of the named virtual machine. Without network access, this is the primary
way of connecting to the guest once it is running.

By default this will connect to the first com port specified in the client configuration, which
is usually com1. Alternatively you can specify the com port to connect to.

This looks for the
nmdm(4)
device associated with the virtual machine, and connects to it with
cu(1).
Use ~+Ctrl-D to exit the console and return to the host.

rename name new-name

Renames the specified virtual machine. The guest must be stopped to use this function.

add
[-d device]
[-t type]
[-s size|switch]
name

Add a new network or disk device to the named virtual machine. The options depend on
the type of device that is being added:

-d device

The type of device to add. Currently this can either be
disk
or
network

-t type

For disk devices, this specifies the type of disk device to create.
Valid options for this are
zvol,
sparse-zvol
and
file.
If not specified, this defaults to
file.

-s size|switch

For disk devices, this is used to specify the size of the disk image to create. For
network devices, use this option to specify the virtual switch to connect the network interface to.

For both types of device, the emulation type will be chosen automatically based on the
emulation used for the existing guest devices.

reset name

Forcefully reset the named virtual machine. This can cause corruption to the guest file system just
as with real hardware and should only be used if necessary.

poweroff name

Forcefully power off the named virtual machine. As with
reset
above, this does not inform the guest to shutdown gracefully and should only be used if the guest
can not be shut down using normal methods.

startall

Start all virtual machines configured for auto-start. This is the command used by the rc.d scripts
to start all machines on boot.

The list of virtual machines should be specified using the
$vm_list
variable in
/etc/rc.conf.
This allows you to use shared storage for virtual machine data, whilst making sure that the correct
guests are started automatically on each host. (Or to just make sure your required guests start on boot
whilst leaving test/un-needed guests alone)

The delay between starting guests can be set using the
$vm_delay
variable, which defaults to 5 seconds. Too small a delay can cause problems, as each guest doesnt
have enough time to claim a null modem device before the next guest starts. Increasing this value
can be useful if you have disk-intensive guests and want to give each guest a chance to fully
boot before the next starts.

stopall

Stop all running virtual machines. This sends a stop command to all
bhyve(8)
instances, regardless of whether they were starting using
vm
or not.

configure name

The
configure
command simply opens the virtual machine configuration file in your default editor,
allowing you to easily make changes. Please note, changes do not take effect until
the virtual machine is fully shutdown and restarted.

clone name new-name

Create a clone of the virtual machine
name,
as long as it is currently powered off. The new machine will be called
new-name,
and will be ready to boot with a newly assigned UUID and empty log file.

Please note that this function requires ZFS, and a snapshot will be taken of the original
guest, along with any descendant datasets.

snapshot
[-f]
name|name@snapshot

Create a snapshot of the names virtual machine. This command is only supported with ZFS
and will take a snapshot of the guest dataset and any descendent ZVOL devices.

The guest and snapshot name can be specified in the normal
name@snapshot
way familiar to ZFS users. If no snapshot name is given, the snapshot is based on the current timestamp in
Y-m-d-H:M:S
format.

By default the guest must be stopped to use this command, although you can force a snapshot
of a running guest by using the
-f
option.

rollback
[-r]
name@snapshot

Rollback the guest to the specified snapshot. This will roll back the guest dataset and all
descendent ZVOL devices.

Normally, ZFS will only allow you to roll back to the most recent snapshot.
If the snapshot given is not the most recent, ZFS will produce a warning detailing that you
need to use the
-r
option to remove the more recent snapshots. It will also produce a list of the snapshots that
will be destroyed if you use this option. The
-r
option can be passed directly into
vm rollback

The guest must always be stopped to use this command.

iso [url]

List all the ISO files currently stored in the
$vm_dir/.iso
directory. This is often useful during guest installation, allowing you to copy and paste the ISO
filename.

If a
url
is specified, instead of listing ISO files, it attempts to download the given file using
fetch(1).

image list

List available images. Any virtual machine can be packaged into an image, which can then be
used to create additional machines. All images have a globally unique ID (UUID) which is
used to identify them. The list command shows the UUID, the original machine name, the
date it was created and a short description of the image.

Please note that these commands rely on using ZFS featured to package/unpackage the images,
and as such are only available when using a ZFS dataset as the storage location.

image create
[-d description]
name

Create a new image from the named virtual machine. This will create a compressed copy of
the original guest dataset, which is stored in the
$vm_dir/images
directory. It also creates a
UUID.manifest
file which contains details about the image.

Once complete, it will display the UUID which has been assigned to this image.

image provision uuid new-name

Create a new virtual machine, named
new-name,
from the specified image UUID.

Each virtual machine has a configuration file that specifies the hardware configuration. This
uses a similar format to the
rc
files, making them easy to edit by hand. The settings for each guest are stored in
$vm_dir/$vm_name/$vm_name.conf.
An overview of the available configuration options is listed below.

guest

Specify the type of guest to be installed in this virtual machine. Current valid options
for this are generic, freebsd, netbsd, openbsd, windows, centos, ubuntu, debian & alpine linux.

uefi

Set this (any non-empty value) for guests that need UEFI firmware.

cpu

A numeric value specifying the number of virtual CPU cores to assign to the guest.

memory

The amount of memory to assign to the guest. This can be specified in megabytes or
gigabytes using the
M
and
G
suffixes.

hostbridge

This option allows you to specify the type of hostbridge used for the guest hardware.
Normally you can leave this as default, which is to use a standard bhyve hostbridge.

There are two other options.
amd,
which is almost identical to the standard hostbridge, but advertises itself with a
vendor ID of AMD. There are also some special cases where you may require no
hostbridge at all, which can be achieved using the
none
value.

comports

This option allows you to specify which com ports to create for the guest. The default
is to create a single
com1
port. Valid values for this are
com1
and
com2.
You can also connect two com ports by specifying both, separated by a space.

utctime

Set this option to
yes
if the guest RTC should keep UTC time.

debug

If this is set to
yes,
all output from the
bhyve(8)
process will be written to
${vm_dir}/guest/bhyve.log.
This is useful for debugging purposes as it allows you to see any error messages
that are being produced by
bhyve(8)
itself.

network0_type

The emulation to use for the first network adapter. This option can be unspecified
if no guest networking is required. The recommended value for this is
virtio-net.
Additional network interfaces can be configured by adding additional
networkX_type
and
networkX_switch
values, replacing
X
with the next available integer.

network0_switch

The virtual switch to connect interface
0
to. This should correspond to a virtual switch created using the
vm switch create
subcommand. If the virtual switch is not found, an interface will still be assigned,
but not connected to any bridge.

Note that this field is no longer strictly required. If you are using a custom device
for the networking that is already configured, you may not need the interface connected
to a virtual switch. See the
network0_device
configuration option.

network0_device

Normally vm-bhyve will create a
tap(4)
device at run-time for each virtual network interface. This may be an issue in more advanced
configurations where you want to pre-configure the networking manually in a way unsupported by
vm-bhyve. This option allows you to instruct vm-bhyve to use an existing network device for
this virtual interface, rather than creating one dynamically.

network0_mac

This option allows you to specify a mac address to use for this interface. If not
provided,
bhyve(8)
will generate a mac address.

disk0_type

The emulation type for the first virtual disk. At least one virtual disk is required.
Valid options for this are currently
virtio-blk
and
ahci-hd.
Additional disks can be added by adding additional
diskX_type
and
diskX_name
values, replacing
X
with the next available integer.

disk0_name

The filename for the first virtual disk. The first disk is created automatically when
provisioning a new virtual machine. If additional disks are added manually, the image will need
to be created, usually done using the
truncate(1)
or
zfs(8)
commands. Alternatively, you can use the
vm add
command, which will create the disk image for you.

Normally disk images or zvols are stored directly inside the guest. To use a disk
image that is stored anywhere else, you can specify the full path in this option,
and configure the device as
custom

disk0_dev

The type of device to use for the disk. If not specified, this will default to
file,
and a sparse file, located in the guest directory, will be used as the disk image.
Other options include
zvol
&
sparse-zvol,
which will used a ZVOL as the disk image, created directly under the guest dataset.
Alternatively you can specify
custom,
in which case
diskX_name
should be the full path to the image file or device.

disk0_opts

Any additional options to use for this disk device. Multiple options can be specified,
separated by a comma. Please see the
bhyve(8)
man page for more details on supported options.

uuid

This option allows you to specify a fixed UUID for the guests SMBIOS. Normally, the
UUID is generated by
bhyve(8)
based on the hostname and guest name. Because this may change if guests are moved
between systems, the
vm create
command automatically assigns a UUID to all newly created guests.

passthruX

Specify a device to pass through to the guest. You will need to reserve the device first
so that is it claimed by the ppt driver on boot.

Once the device is successfully reserved, you can add it to the guest by adding
passthruX= 1/2/3
to the guest configuration file, where
X
is an integer starting at 0, and
1/2/3
is the Base/Slot/Function of the device. If you are passing through multiple functions on
the same device, make sure they are specified together in the configuration file in the
same sequence as the original device.

Please see https://wiki.freebsd.org/bhyve/pci_passthru for more details on how this works.

virt_random

Set this option to
yes
if you want to create a
virtio-rnd
device for this guest.

grub_commands

Sometimes you may need to provide grub commands that differ from those generated by
vm-bhyve.
In this case, create a text file inside the guests directory containing the commands
you need to run. Usually the last entry in the file should be
boot,
followed by a newline. The newline is important, otherwise the last command may not
actually be executed.

Once you have created the file, set this option to the name of the file. From that point
on, during a normal boot
vm-bhyve
will use the commands from this file rather than attempting to generate them automatically.

zfs_dataset_opts

This allows you to specify one or more ZFS properties to set on the dataset when a
guest is created. Because properties are assigned as the dataset is created, this option
is most useful when specified inside a template. As a guest is created, all properties
listed in this option will be applied to the guest dataset.

Multiple properties can be specified, separated by a space. Please note that spaces
are not currently supported in the property values.

zfs_zvol_opts

Allows you to specify ZFS properties that should be assigned to any ZVOLs that are
created for a guest. As with
zfs_dataset_opts,
this makes most sense when entered into a template, as the properties can be assigned
while a guest is being created. Some ZVOL options, such as
volblocksize
can only be set at creation time.

Multiple properties can be specified, separated by a space. For example, the following
will configure the ZVOL block size to 128k, and turn compression off.

zfs_zvol_opts="volblocksize=128k compress=off"

linux_kernel

CentOS guests require the kernel to be loaded inside the boot loader. This option is
required for these guests and specifies the kernel version number to load. Hopefully
this requirement will be relaxed as
bhyve(8)
development continues.