SR-IOV is required for QAT to function in TNSR. SR-IOV enables Virtual Functions
which are required for binding by crypto devices.

The procedure to enable SR-IOV varies by platform. Generally this involves
rebooting the hardware and entering the BIOS setup, making the change, and then
saving and rebooting. The exact location of the SR-IOV option also varies in
different BIOS implementations.

Note

Netgate devices which ship with a CPIC card preinstalled will have
this step completed at the factory, but double check the BIOS to ensure it
is set as expected.

IOMMU (Input–Output Memory Management Unit), which in this context is also known
as Intel VT-d, must be enabled in grub for QAT to function. It functions
similar to PCI passthrough, allowing the dataplane to access the QAT device.

To enable IOMMU in grub:

Open /etc/default/grub in a text editor (as root or with sudo)

Locate the line starting with GRUB_CMDLINE_LINUX

Check if that line includes intel_iommu=oniommu=pt

If those parameters are not included on the line, append them to the end,
before the end quote.

When viewing the XML configuration with showconfigurationrunning, it will
contain settings similar to the following example. Note that if other dataplane
options are present in the configuration, those will also be visible. Here is
how it looks once configured:

If the QAT device does not appear in the showdpdkcryptodevices output, or
it only shows an AES-NI device, then VPP can not see the crypto device. To
correct this, first verify the QAT drivers are loaded, VFs exist for the QAT
device, and grub BOOT_IMAGE is passing the necessary iommu parameters.

Verify IOMMU parameters:

$ dmesg | grep iommu

The following parameters should appear somewhere on the BOOT_IMAGE line in the
dmesg output:

The number of listings are dependent on how many threads VPP uses to process
packets. At minimum there will be at least three entries, but there may be many
more. The lines will look similar to this example:

TNSR stores the device Physical Function (PF), 04:00.0 for example, in its
configuration because the VFs do not yet exist at boot time. They are created
by clixon-backend when it processes the crypto device. Then, the
allocated VFs on the PF have their addresses written to startup.conf.

The VFs are bound to igb_uio because igb_uio is a driver which allows a
userspace process to do RDMA on buffers that are used by a PCI device.

If the drivers are loaded and the VFs show under lspci, then verify
/etc/vpp/startup.conf has the appropriate dpdk settings. The igb_uio
driver must be present and the PCI IDs of TNSR interfaces along with one of the
VFs for the QAT device:

Physical TNSR interfaces will display there in addition to the QAT VF ID, which
matches the QAT VF ID configured for dpdk in /etc/vpp/startup.conf.

If any of those tests do not provide the expected output, then reboot the system
and check again. Ensure the TNSR services and VPP are running, and then check
the VPP QAT status again.

$ sudo vppctl show dpdk crypto devices

If there is still no output, verify the PCI ID for the crypto device specified
in TNSR is accurate. It must be the first PCI ID displayed by
lspci|grepqat. Then verify the PCI ID of the next listing in that output
(first VF device) is specified in /etc/vpp/startup.conf properly and also
the same PCI ID seen by VPP when running: