Currently Ganeti supports two configuration modes for instance NICs:
routed and bridged mode. The ip NIC parameter, which is mandatory
for routed NICs and optional for bridged ones, holds the given NIC’s IP
address and may be filled either manually, or via a DNS lookup for the
instance’s hostname.

This approach presents some shortcomings:

It relies on external systems to perform network resource
management. Although large organizations may already have IP pool
management software in place, this is not usually the case with
stand-alone deployments. For smaller installations it makes sense to
allocate a pool of IP addresses to Ganeti and let it transparently
assign these IPs to instances as appropriate.

The NIC network information is incomplete, lacking netmask and
gateway. Operating system providers could for example use the
complete network information to fully configure an instance’s
network parameters upon its creation.

Furthermore, having full network configuration information would
enable Ganeti nodes to become more self-contained and be able to
infer system configuration (e.g. /etc/network/interfaces content)
from Ganeti configuration. This should make configuration of
newly-added nodes a lot easier and less dependant on external
tools/procedures.

Instance placement must explicitly take network availability in
different node groups into account; the same link is implicitly
expected to connect to the same network across the whole cluster,
which may not always be the case with large clusters with multiple
node groups.

In order to deal with the above shortcomings, we propose to extend
Ganeti with high-level network management logic, which consists of a new
NIC mode called managed, a new “Network” configuration object and
logic to perform IP address pool management, i.e. maintain a set of
available and occupied IP addresses.

We propose the introduction of a new high-level Network object,
containing (at least) the following data:

Symbolic name

UUID

Network in CIDR notation (IPv4 + IPv6)

Default gateway, if one exists (IPv4 + IPv6)

IP pool management data (reservations)

Default NIC connectivity mode (bridged, routed). This is the
functional equivalent of the current NIC mode.

Default host interface (e.g. br0). This is the functional equivalent
of the current NIC link.

Tags

Each network will be connected to any number of node groups, possibly
overriding connectivity mode and host interface for each node group.
This is achieved by adding a networks slot to the NodeGroup object
and using the networks’ UUIDs as keys.

A new helper library is introduced, wrapping around Network objects to
give IP pool management capabilities. A network’s pool is defined by two
bitfields, the length of the network size each:

reservations

This field holds all IP addresses reserved by Ganeti instances, as
well as cluster IP addresses (node addresses + cluster master)

externalreservations

This field holds all IP addresses that are manually reserved by the
administrator, because some other equipment is using them outside the
scope of Ganeti.

The bitfields are implemented using the python-bitarray package for
space efficiency and their binary value stored base64-encoded for JSON
compatibility. This approach gives relatively compact representations
even for large IPv4 networks (e.g. /20).

It should be noted that IP pool management is performed only for IPv4
networks, as they are expected to be densely populated. IPv6 networks
can use different approaches, e.g. sequential address asignment or
EUI-64 addresses.

In order to be able to use the new network facility while maintaining
compatibility with the current networking model, a new network mode is
introduced, called managed to reflect the fact that the given NICs
network configuration is managed by Ganeti itself. A managed mode NIC
accepts the network it is connected to in its link argument.
Userspace tools can refer to networks using their symbolic names,
however internally, the link argument stores the network’s UUID.

We also introduce a new ip address value, constants.NIC_IP_POOL,
that specifies that a given NIC’s IP address should be obtained using
the IP address pool of the specified network. This value is only valid
for managed-mode NICs, where it is also used as a default instead of
constants.VALUE_AUTO. A managed-mode NIC’s IP address can also be
specified manually, as long as it is compatible with the network the NIC
is connected to.

In order to keep the hypervisor-visible changes to a minimum, and
maintain compatibility with the existing network configuration scripts,
the instance’s hypervisor configuration will have host-level link and
mode replaced by the connectivity mode and host interface of the
given network on the current node group.

The managed mode can be detected by the presence of new environment
variables in network configuration scripts:

gnt-network modify --cidr=192.0.2.0/25 public (only allowed if all current reservations fit in the new network)
gnt-network modify --gateway=192.0.2.126 public
gnt-network modify --host_interface=test --nic_connectivity=routed public (issues warning about instances that need to be rebooted)
gnt-network rename public public2

gnt-network connect public nodegroup1
gnt-network connect --host_interface=br1 public nodegroup2
gnt-network disconnect public nodegroup1 (only permitted if no instances are currently using this network in the group)

The IAllocator protocol can be made network-aware, i.e. also consider
network availability for node group selection. Networks, as well as
future shared storage pools, can be seen as constraints used to rule out
the placement on certain node groups.