A cluster-wide overview for Ceph is now displayed in the 'Datacenter View' too.

The activity and state of the placement groups (PGs) is visualized.

The version of all Ceph services is now displayed, making detection of outdated services easier.

Configuration settings from the config file and database are displayed.

You can now select the public and cluster networks in the GUI with a new network selector.

Easy encryption for OSDs with a checkbox.

ZFS 0.8.1

Native encryption for datasets with comfortable key-handling by integrating the encryption directly into the `zfs` utilities. Encryption is as flexible as volume creation and adding redundancy - the gained comfort w.r.t. dm-crypt is comparable to the difference between mdadm+lvm to zfs.

Allocation-classes for vdevs: you can add a dedicated fast device to a pool which is used for storing often accessed data (metadata, small files).

You can now set certain performance and security related mount options for each container mountpoint.

Linux Kernel

Updated 5.0 Kernel based off the Ubuntu 19.04 "Disco" kernel with ZFS.

Intel in-tree NIC drivers are used:

Many recent improvements to the kernel networking subsystem introduced incompatibilities with the out of tree drivers provided by Intel, which sometimes lag behind on support for new kernel versions. This can lead to a change of the predictable network interface names for Intel NICs.

Automatic cleanup of old kernel images

Old kernel images are not longer marked as NeverAutoRemove - preventing problems when /boot is mounted on a small partition.

By default the following images are kept installed (all others can be automatically removed with `apt autoremove`):

the currently running kernel

the version being newly installed on package updates

the two latest kernels

the latest version of each kernel series (e.g. 4.15, 5.0)

Guest status display in the tree view: Additional states for guests (migration, backup, snapshot, locked) are shown directly in the tree overview.

Improved ISO detection in the installer: The way how the installer detects the ISO was reworked to include more devices, alleviating problems of detection on certain hardware.

Pool level backup: It is now possible to create a backup task for backing up a whole pool. By selecting a pool as backup target instead of an explicit list of guests, new members of the pool are automatically included, and removed guests are automatically excluded from the backup task.

New User Settings and Logout menu.

Automatic rotation of authentication key every 24h: by limiting the key lifetime to 24h the impact of key leakage or a malicious administrator are reduced.

The nodes Syslog view in the GUI was overhauled and is now faster.

Sheepdog is no longer maintained, and thus not supported anymore as Storage plugin.

`ceph-disk` has been removed in Ceph Nautilus - use `ceph-volume` instead.

Proxmox VE 5.4

Ceph can now be installed via the GUI, thus closing the gap of needing the CLI for this initial step, before configuring a complete (hyperconverged) Ceph setup via PVE's GUI.

QDevice support via `pvecm`

primarily for small 2-node clusters adding a qdevice can help mitigate the downside of not being able to reboot one node without losing quorum (and thus the ability to make any changes in the cluster) ** Can also help in clusters with a larger even number of nodes by providing a tie-break vote.

Integration into pvecm and PVE stack vastly simplifies adding a qdevice (it was possible manually before as well)

Wake On Lan support via `pvenode`

It is now possible to associate a MAC-Address with a PVE-node.

This can be used to send a WOL (wake on lan) packet to that node from any other node in the PVE-Cluster, and thus power it on on-demand

Provides a comfortable way of saving power by keeping only part of the nodes permanently online, and starting others on demand (e.g. increased load, or for running guests during upgrades to the PVE-nodes.)

GUI Container wizard creates unprivileged containers by default

HA improvements and added flexibility

It is now possible to set a datacenter wide HA policy which can change the way guests are treated upon a Node shutdown or reboot. The choices are:

failover: never freeze services, this means that a service will get recovered to another node if possible and if the current node does not comes back up in the grace period of 1 minute.

default: this is the current behavior, freeze on reboot but do not freeze on poweroff

Suspend to disk/hibernate support for Qemu/KVM guests

qemu guests can be 'hibernated' (have their RAM contents and internal state saved to permanent storage) and resumed on the next start.

This enables users to preserve the running state of the qemu-guests across most upgrades to and reboots of the PVE-node.

Additionally it can speed up the startup of guests running complex workloads/ workloads which take lots of resources to setup initially, but which need not run permanently.

Support for guest (both Qemu and LXC) hookscripts:

Hook-scripts are small executables which can be configured for each guest, and are called at certain steps of the guest's lifetime ('pre-start', 'post-start', 'pre-stop', 'post-stop').

This gives Administrators great flexibility in the way they can prepare the environment for the guest (e.g. adding necessary network resources (routes, vlans), firewall-rules, unlocking encrypted files/devices,...) and cleaning them up when the guest is shutdown or stopped.

Improved Qemu Guest creation wizard:

Certain often requested options (e.g. Machine-type (q35, pc-i440fx), Firmware (Seabios, UEFI), SCSI controller) can now be selected directly in the VM Creation Wizard, and dependent options get set to sensible values directly.

Clusterwide Subscription Check and Display

The Datacenter Summary page now gives Administrators a global overview of the subscription status of all nodes in a PVE cluster, thus helping to verify the support-status of the complete cluster on one quick glance

Improved UI in the installer

It is now possible to go back to a previous screen in the pve-installer, and adapt choices made there, without the need to restart the complete installation process

Before the actual installation is performed an information-page is displayed containing all relevant information

U2F Authentication for the WebUI

PVE's WebUI now can optionally use two-factor authentication for users.

This is generally considered good security practice, and in certain domains and environments it is required.

The new U2F authentication and the TOTP second factor authentication can be configured by each user for themselves (before all 2FA solutions needed to be configured by a user with User.Modify permission).

Improved reference documentation

Countless bug fixes and package updates (for all details see bugtracker and GIT)

Proxmox VE 5.3

Released 4.12.2018: See Downloads, updated ISO image with installer and apt bug fixes and updated packages on 23.1.2019

Based on Debian Stretch 9.6

Kernel 4.15.18

QEMU 2.12.1

LXC 3.0.2

ZFS 0.7.12

Ceph 12.2.8 (Luminous LTS, stable), packaged by Proxmox

Installer with ZFS: no swap space is created by default, instead an optional limit of the used space in the advanced options can be defined, thus leaving unpartitioned space at the end for a swap partition.

Disk Management on GUI (ZFS, LVM, LVMthin, xfs, ext4)

Create CephFS via GUI (MDS)

CephFS Storage plugin (supports backups, templates and ISO images)

LIO support for ZFS over iSCSI storage plugin

ifupdown2 package and partial GUI support

Delete unused disk images on the storage view

Enable/disable the local cursor in noVNC

Enable/disable autoresize in noVNC

Edit /etc/hosts/ via GUI, which is necessary whenever the IP or hostname is changed

VM Generation ID: Users can set a 'vmgenid' device, which enables safe snapshots, backup and clone for various software inside the VM, like Microsoft Active Directory. This vmgenid will be added to all newly created VMs automatically. Users have to manually add it to older VMs.

qemu-server: add ability to fstrim a guest filesystem after moving a disk (during move_disk or migrate with local disks)

vGPU/MDev and PCI passthrough. GUI for configuring PCI passthrough and also enables the use of vGPUs (aka mediated devices) like Intel KVMGT (aka GVT-g) or Nvidias vGPUS.

pvesh rewrite to use the new CLI Formatter framework - providing a configurable output format for all API calls (formats include: json, yaml and a pleasant table output with borders). Provides unified support for json output for the PVE CLI Tools.

Note: This includes updates for fence_ipmilan (fence_ilo3, fence_ilo4, fence_imm, and fence_idrac), and some parameter names changed (see 'man fence_ipmilan'). Please verify that your fence device still works if you use HA.

based on Debian Wheezy 7.6

countless bug fixes and package updates, for all details see bugtracker and GIT