Communication Service Providers (CSP) are rapidly adopting NFV and one of the key requirements of NFV is to achieve the same performance and scalability that is currently being offered by hardware appliances. In order to address this, several carrier grade features were added to OpenStack and various hypervisors. Among them an important feature in context of I/O virtualization is SR-IOV (Single-Root I/O Virtualization) which provides the ability to attach a virtual machine`s vNIC directly to a physical interface on the host compute node. This is required to achieve the near line rate performance from the VNFs (Virtual Network Function), otherwise the latency would increase due to the additional path (vSwitch, Host OS Kernel etc.) the VNF packets have to traverse in order to do network communication with their peer components.

To use SR-IOV on Intel platforms, the BIOS option VT-d must be enabled. SR-IOV based physical NIC can expose multiple virtual functions and each of these functions (VFs) can be attached to a VM`s virtual NIC by using the PCI device assignment concept supported by the hypervisors. The Physical NIC internally contains a switch (in technical terms – Hardware Virtual Ethernet Bridge) to manage the traffic for each of these Virtual Functions.

While SR-IOV has several advantages, it also comes with few issues in the context of security and reliability which can be solved with proper VIM orchestration and Switch configuration. Following are the important vulnerabilities, a SR-IOV based NFV deployment should consider, in order to cater to carrier grade security and reliability requirements, especially when untrusted VNFs need to be deployed.

Following logical diagram captures the various actors involved and rules to be enforced.

MAC / VLAN Spoofing

When VFs are enabled on a Physical NIC, there is no MAC or VLAN assigned to the VFs. If this VF is assigned to a VM, any VNF app on the VM can change the Source MAC or VLAN tag in outgoing packets. This can be avoided if the VIM / Orchestrator sets the MAC and the VLAN for a VF as part of the VM Port Provisioning. For example Intel 82599 based platforms, provides APIs to set these on a per VF basis. Along with this, the spoof check flags should also be enabled so that the internal switch on the physical NIC can check the outgoing packets for invalid MAC and VLANs. ip command can also be used to set the above parameters. Following is an example on how to set the same for a PF eth0 / VF 3.

ip link set eth0 vf 3 mac xx:yy:xx:yy:xx:yy vlan 100 spoofchk on

To prevent MAC spoofing on the switch side, port security feature can be used to add list of MAC addresses allowed on the port. This approach requires the switch to provide API/CLI to dynamically modify this list so that whenever a VF is assigned to a VM and a MAC is allocated, the corresponding MAC can be added to this list. This approach doesn’t provide finer security compared to the NIC based approach since a VNF can still spoof any of the MAC part of the port-security list assigned on the switch side (provided the VNF knows about the list).

To prevent VLAN spoofing on the switch side, MAC based VLAN feature can be used to assign specific VLAN for a MAC. When the switch receives any untagged packets with this MAC, it’s automatically assigned a VLAN. If a VNF app sends tagged packets it’s automatically dropped since there is no VLAN assigned at port level.

<switch-prompt> mac-vlan mac-address <VF MAC 1> vlan <vlan1>

<switch-prompt> mac-vlan mac-address <VF MAC 2> vlan <vlan2>

Ethernet Flow Control

Flow Control at Ethernet level exists to provide lossless layer 2 network communications. This feature enables a receiver to send a signal to the transmitter to temporarily pause the traffic. A special frame called PAUSE frame is used for this signal whenever the receiver side runs out of buffers. Since SR-IOV enables a VM to share the physical link on the host, a malicious VM can halt the traffic of all VMs for some time by manipulating this flow control feature. Please refer to the Intel Advisory for more details.

At NIC side, the Ethernet flow control can be turned off as below. But this may have impact on the CPU usage of the VM since TCP layer has to do the flow control. It may also impact any VNF workloads using Ethernet level protocols like FCoE, RoCE etc.

ethtool -A <PF ethX> autoneg off rx off tx off

At switch side, the Ethernet flow control is turned off by default on HP switches like 6125 and 5930. Otherwise it can be turned off by going to the interface prompt and using command like below

[switch-prompt] flow-control disable

Rate Limiting

Another issue with the physical link being shared with the VFs is that a single VNF can use the entire bandwidth available, thus impacting the SLAs of other VNFs sharing the same NIC. To avoid this, bandwidth limit can be set at individual VF level as shown below

ip link set eth2 vf 0 rate 100 (where rate is mentioned in Mbps)

At switch side, if the switch where the SR-IOV physical NIC is connected supports MAC based rate limiting, then the same restriction can be applied on the switch.

In summary, SR-IOV base NFV deployments need additional security and reliability settings in order to cater to the carrier grade SLAs and requirements. The NFV Platform comprising of the Orchestrator, VIM, hardware switches and physical NIC interfaces should work in unison to achieve this so that dynamic nature of VNF deployment can still be maintained without any manual intervention. For more information on how HP is providing an integrated NFV platform, please refer to HP`s NFV offerings.