*[PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters@ 2018-08-08 14:44 Tony Krowiak
2018-08-08 14:44 ` [PATCH v8 01/22] s390/zcrypt: Add ZAPQ inline function Tony Krowiak
` (23 more replies)0 siblings, 24 replies; 60+ messages in thread
From: Tony Krowiak @ 2018-08-08 14:44 UTC (permalink / raw)
To: linux-s390, linux-kernel, kvm
Cc: freude, schwidefsky, heiko.carstens, borntraeger, cohuck,
kwankhede, bjsdjshi, pbonzini, alex.williamson, pmorel, alifm,
mjrosato, jjherne, thuth, pasic, berrange, fiuczy, buendgen,
akrowiak, frankja, Tony Krowiak
From: Tony Krowiak <akrowiak@linux.ibm.com>
Several major objections were raised to design changes introduced in the v7
patch series, so in order to avoid an extended discussion around these
objections and to expedite acceptance of the series, the following changes
have been made for v8:
1. Removed the AP bus's ability to designate queues as 'used by host' or as
'used by alternate driver(s)'. The bind/unbind sysfs interfaces will be
used for managing the connection of AP queue devices to a zcrypt driver
or the VFIO AP driver.
2. Removed the 'activate' sysfs interfaces which allowed for over
provisioning of the mediated device as well as creation of mdevs with
overlapping matrixes. It was pointed out that both of these enhancements
break the mdev model. Consistency checking of the mdev matrix has
therefore been returned to the mediated matrix device's sysfs interfaces
for assigning adapters and domains:
* Verify that APQNs assigned to the mediated device are bound to the
VFIO AP device driver
* Verify that no APQN assigned to the mediated matrix device is assigned
to any other mediated matrix device.
4. Reworked the handling of the CRYCB in vSIE based upon patches introduced
by David in the mainline.
Notes:
=====
Patches 1-4 (by Harald) posted with this series are forthcoming via
Martins tree and are based on changes in the ap driver/bus that we use as a
foundation. They have been included here because some of the functions
in this patch series are dependent upon them.
Patches 5-6 (by David) are posted with this series because they are not
currently in our master branch. Patches 19 and 20 of this series are
dependent upon them. I believe David's patches are available in the
mainline now.
This patch series works with the v6 QEMU patches. There is no new QEMU
patchset version yet because there have been no review comments worthy of
creating a new series; only a couple of extremely minor nits.
Abstract:
========
On s390, we have cryptographic coprocessor cards, which are modeled on
Linux as devices on the AP bus. Each card can be partitioned into domains
which can be thought of as a set of hardware registers for processing
crypto commands. Crypto commands are sent to a specific domain within a
card is via a queue which is identified as a (card,domain) tuple. We model
this something like the following (assuming we have access to cards 3 and
4 and domains 1 and 2):
AP -> card3 -> queue (3,1)
-> queue (3,2)
-> card4 -> queue (4,1)
-> queue (4,2)
If we want to virtualize this, we can use a feature provided by the
hardware. We basically attach a satellite control block to our main
hardware virtualization control block and the hardware takes care of
most of the rest.
For this control block, we don't specify explicit tuples, but a list of
cards and a list of domains. The guest will get access to the cross
product.
Because of this, we need to take care that the lists provided to
different guests don't overlap; i.e., we need to enforce sane
configurations. Otherwise, one guest may get access to things like
secret keys for another guest.
The idea of this patch set is to introduce a new device, the matrix
device. This matrix device hangs off a different root and acts as the
parent node for mdev devices.
If you now want to give the tuples (4,1) and (4,2), you need to do the
following:
- Make sure the queues (4,1) and (4,2) belong to vfio_ap (see patches
#5 and #6)
- Create the mediated device.
- Assign card 4 and domains 1 and 2 to the mediated device
- Optionally activate the mediated device.
QEMU will now simply consume the mediated device and things should work.
For a complete description of the architecture and concepts underlying
the design, see the Documentation/s390/vfio-ap.txt file included with this
patch set.
v7 => v8 Change log:
===================
* Removed the AP bus's ability to designate queues as 'used by host' or as
'used by alternate driver(s)' and relying on bind/unbind for connecting
AP queue devices to a zcrypt driver or the VFIO AP driver.
* Removed 'activate' attribute from mediated device.
* Do consistency checking during device assignment:
1. Verify that APQNs assigned to the mediated device are bound to the
VFIO AP device driver
2. Verify that no APQN assigned to the mediated matrix device is assigned
to any other mediated matrix device.
* The attributes of a mediated matrix device that is in use by a guest can
not be changed - i.e., no device assignment/unassignment allowed
* A mediated matrix device that is in use by a guest can not be removed.
* Removed all printk logging from VFIO AP driver; allowing return codes
from interfaces to describe the error.
* Reworked the handling of the CRYCB in vSIE based upon patches introduced
by David in the mainline.
v6 => v7 Change log:
===================
* The AP bus gained the ability to designate queues as 'used by host'
or as 'used by alternate driver(s)'. This allows us to authorise access
(via the CRYCB) to queues that are not currently bound to the vfio_ap
driver. If a vfio_ap owned queue diss- and reapears it's guaranteed
to get bound back to the vfio_ap driver.
* The mediated device gained an 'activate' attribute. Sharing conflicts are
checked on activation now. If the device was not activated, the mdev
open still implies activation. An active ap_matrix_mdev device claims
it's resources -- an inactive does not.
* An active ap_matrix_mdev device can not be removed. An ap_matrix_mdev
that is hooked up with a guest can not be deactivated.
* An active ap_matrix_mdev device rejects assign_* and deassign_*
operations. Thus changing the CRYCB masks of a guest in order to
accomplys certain hotplug scenarios is planned, but not supported yet. In
previous versions it was possible to do those operations on a ap_matrix_mdev
that is hooked up to a guest, but the changes would take effect on the next
mdev_open.
* Synchronisation was reworked.
* The sysfs path of the parent device changed from /sys/devices/vfio_ap/matrix/
to /sys/devices/virtual/misc/vfio_ap/. The parent device is a misc
device now.
* The severity for most of the messages were reduced form error to
warning.
* We are not as thick headed about the zapq as we used to be in v6.
v5 => v6 Change log:
===================
* Added VSIE support - thanks to Pierre Morel
* Added VFIO_DEVICE_RESET ioctl
* Zeroizing AP queues when mediated device released and when
VFIO_DEVICE_RESET ioctl is invoked
* Removed /arch/s390/kvm/kvm-ap.c and arch/s390/include/asm/kvm-ap.h and
moved guest matrix configuration into vfio driver
* Removed temporary interfaces to be supplied by AP bus
* Made the variable that keeps track of mdev instance count an atomic_t
type
* Removed code iterating through vm_list to determine if another guest has
a queue .... not keep a list of matrix_mdev devices and verify against
that list. Removes the need for the kvm_lock.
* Added a sysfs attribute for the mediated matrix device to display the
matrix contained in the guest's CRYCB if a guest is using the mdev.
v4 => v5 Change log:
===================
* Verify AP queues bound to driver in mediated device open callback, prior
to configuring the matrix in the CRYCB
* Implement VFIO_DEVICE_RESET ioctl
* Zeroize queues on guest entry and exit
* Removed vnet from all email IBM email addresses referenced
* Add synchronization in mdev create/remove and open/release.
v4 => v5 Change log:
===================
* Added code to mdev open callback to ensure not more than one vfio-ap
device can be opened by a guest.
* Interpret AP instructions by default
* Removed patch implementing interface to enable/disable AP interpretation,
since that will now be done by default
* Removed patch to reset crypto attributes for ALL vcpus. That will be
submitted as a single patch since it will not be needed in this series -
i.e., it was called from the interface to enable/disable AP instructions
* All code for initializing crypto for a guest has been moved back to
kvm-s390.c, kvm_s390_crypto_init(kvm) function
* Maintaining a module reference count for the vfio_ap module so it is not
removed while a guest with AP devices is running.
v3 => v4 Change log:
===================
* Resolved issue with enabling ZCRYPT when KVM is enabled by using
#ifdef ZCRYPT in relevant functions
* Added patch with a new function for resetting the crypto attributes
for all vcpus to resolve the issue raised with running vcpus getting out
of sync.
* Removed KVM_S390_VM_CRYPTO_INTERPRET_AP: Setting interpretive exec mode
from vfio_ap driver when mdev device is opened.
v2 => v3 Change log:
===================
* Set APIE in VCPU setup function
* Renamed patch 13/15:
KVM: s390: Configure the guest's CRYCB
KVM: s390: Configure the guest's AP devices
* Fixed problem with building arch/s390/kvm/kvm-ap.c when CONFIG_ZCRYPT
not selected
* Removed patch introducing VSIE support for AP pending further
investigation
* Initialized AP maximum mask sizes - i.e., APM, AQM and ADM - from info
returned from PQAP(QCI) function
* Introduced a new device attribute to the KVM_S390_VM_CRYPTO attribute
group for setting a flag via the KVM_SET_DEVICE_ATTR ioctl to indicate
whether ECA_APIE should be set or not. The flag is used in the
kvm_s390_vcpu_crypto_setup() function to set ECA_APIE in the SIE block.
v1 => v2 Change log:
===================
* Added documentation vfio-ap.txt
* Renamed vfio_ap_matrix module and device driver to vfio_ap
* Use device core device list instead of maintaining list of matrix
devices in driver
* Added VSIE support for AP
* Create matrix device before registering VFIO AP device driver with the
AP bus
* Renamed the following files in drivers/s390/crypto:
* vfio_ap_matrix.drv -> vfio_ap_drv
* vfio_ap_matrix_private.h -> vfio_ap_private.h
* vfio_ap_matrix_ops.c -> vfio_ap_ops.c
* arch/s390/include/asm/kvm/ap-matrix-config.h
* Renamed to kvm-ap.h
* Changed the data type of the bit mask fields for the matrix structure
to unsigned long and create them with DECLARE_BITMAP
* Changed #define prefixes from AP_MATRIX to KVM_AP
* Changed function and structure prefixes from ap_matrix to kvm_ap
* Added function interface to check if AP Extended Addressing (APXA)
facility is installedCRYCB_FORMAT_MASK
* Added function interface to get the maximum ID for AP mask type
* Added function interface to set the AP execution mode
* arch/s390/kvm/ap-matrix-config.c
* Renamed to kvm-ap.c
* Changed function prefixes from ap_matrix to kvm_ap
* Added function to check if AP Extended Addressing (APXA) facility is
installed
* Added function to get the maximum ID for AP mask type
* Added function to set the AP execution mode
* Added a boolean parameter to the functions that retrieve the APM, AQM
and ADM bit mask fields from the CRYCB. If true, then the function
will clear the bits in the mask before returning a reference to it
* Added validation to verify that APM, AQM and ADM bits that are set do
not exceed the maximum ID value allowed
*
* arch/s390/include/asm/kvm_host.h
* Changed defined for ECA_AP to ECA_APIE - interpretive execution mode
* Added a flag to struct kvm_s390_crypto to indicate whether the
KVM_S390_VM_CPU_FEAT_AP CPU model feature for AP facilities is set
* Added two CPU facilities features to set STFLE.12 and STFLE.15
* arch/s390/kvm/kvm-s390.c
* Added initialization for new KVM_S390_VM_CPU_FEAT_AP CPU model feature
* Removed kvm_s390_apxa_installed() function
* Changed call to kvm_s390_apxa_installed() which has been removed to a
call to new kvm_ap_apxa_installed() function.
* Added code to kvm_s390_vcpu_crypto_setup() to set the new CPU model
feature flag in the kvm_s390_crypto structure
* Added CRYCB_FORMAT_MASK to mask CRYCBD
* arch/s390/tools/gen_facilities.c
* Added STFLE.12 and STFLE.15 to struct facility _def
* drivers/s390/crypto/vfio_ap_matrix_private.h
* Changed name of file to vfio_ap.private.h
* Changed #define prefixes from VFIO_AP_MATRIX to VFIO_AP
* struct ap_matrix: removed list fields and locks
* struct vfio_ap_queue: removed list field
* Renamed functions ap_matrix_mdev_register and ap_matrix_mdev_unregister
to vfio_ap_mdev_register and vfio_ap_mdev_unregister respectively
* drivers/s390/crypto/vfio_ap_matrix_drv.c
* Renamed file to drivers/s390/crypto/vfio_ap_drv.c
* Changed all #define, structure and function prefixes to vfio_ap
* probe function
* Changed root device name for the matrix device to vfio_ap:
i.e., /sys/devices/vfio_ap/matrix
* No longer storing the AP queue device in a list, it is retrievable via
the device core
* Removed unnecessary check whether matrix device exists
* Store the vfio_ap_queue structure in the private field of the ap_queue
structure rather than using list interface
* remove function
* Retrieve vfio_ap_queue structure from the struct ap_queue private
data rather than from a list
* Removed unnecesary check
* drivers/s390/crypto/vfio_ap_matrix_ops.c
* Renamed file to vfio_ap_ops.c
* Changed #define prefixes from AP_MATRIX to VFIO_AP
* Changed function name prefixes from ap_matrix to vfio_ap
* Removed ioctl to configure the CRYCB
* create function
* Removed ap_matrix_mdev_find_by_uuid() function - function is provided
by mdev core
* Removed available_instances verification, provided by mdev core
* Removed check to see if mediated device exists, handled by mdev core
* notifier function
* Configuring matrix here instead of via ioctl
* Set interpretive execution mode for all VCPUs
* Removed R/O attributes to display adapters and domains
* Added an R/O attribute to display the matrix
* assign_control_domain mdev attribute:
* Removed check to see if the domain is installed on the linux host
* Added check to verify the control domain ID does not exceed the max
value
* assign_adapter mdev attribute:
* Added check to verify the adapter ID does not exceed the max
value
* If any APQNs configured for the mediated matrix device that
have an APID matching the adapter ID being assigned are not
bound to the vfio_ap device driver then it is assumed that the APQN is
bound to another driver and assignment will fail
* assign_domain mdev attribute:
* Added check to verify the domain ID does not exceed the max
value
* If any APQNs configured for the mediated matrix device that
have an APQI matching the domain ID being assigned are not
bound to the vfio_ap device driver then it is assumed that the APQN is
bound to another driver and assignment will fail
* tools/arch/s390/include/uapi/asm/kvm.h
* removed KVM_S390_VM_CPU_FEAT_AP feature definition
David Hildenbrand (2):
KVM: s390: vsie: simulate VCPU SIE entry/exit
KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
Harald Freudenberger (4):
s390/zcrypt: Add ZAPQ inline function.
s390/zcrypt: Review inline assembler constraints.
s390/zcrypt: Show load of cards and queues in sysfs
s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.
Pierre Morel (2):
KVM: s390: Clear Crypto Control Block when using vSIE
KVM: s390: Handling of Cypto control block in VSIE
Tony Krowiak (14):
KVM: s390: refactor crypto initialization
s390: vfio-ap: base implementation of VFIO AP device driver
s390: vfio-ap: register matrix device with VFIO mdev framework
s390: vfio-ap: sysfs interfaces to configure adapters
s390: vfio-ap: sysfs interfaces to configure domains
s390: vfio-ap: sysfs interfaces to configure control domains
s390: vfio-ap: sysfs interface to view matrix mdev matrix
KVM: s390: interfaces to clear CRYCB masks
s390: vfio-ap: implement mediated device open callback
s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
s390: vfio-ap: zeroize the AP queues.
s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
KVM: s390: CPU model support for AP virtualization
s390: doc: detailed specifications for AP virtualization
Documentation/s390/vfio-ap.txt | 616 +++++++++++++++++++++++
MAINTAINERS | 12 +
arch/s390/Kconfig | 11 +
arch/s390/include/asm/ap.h | 284 ++++++++++-
arch/s390/include/asm/kvm_host.h | 7 +
arch/s390/include/uapi/asm/kvm.h | 1 +
arch/s390/kvm/kvm-s390.c | 131 ++++--
arch/s390/kvm/kvm-s390.h | 1 +
arch/s390/kvm/vsie.c | 244 ++++++++-
arch/s390/tools/gen_facilities.c | 2 +
drivers/s390/crypto/Makefile | 4 +
drivers/s390/crypto/ap_asm.h | 236 ---------
drivers/s390/crypto/ap_bus.c | 21 +-
drivers/s390/crypto/ap_bus.h | 1 +
drivers/s390/crypto/ap_card.c | 1 -
drivers/s390/crypto/ap_queue.c | 1 -
drivers/s390/crypto/vfio_ap_drv.c | 137 +++++
drivers/s390/crypto/vfio_ap_ops.c | 885 +++++++++++++++++++++++++++++++++
drivers/s390/crypto/vfio_ap_private.h | 106 ++++
drivers/s390/crypto/zcrypt_card.c | 12 +
drivers/s390/crypto/zcrypt_queue.c | 12 +
include/uapi/linux/vfio.h | 2 +
22 files changed, 2373 insertions(+), 354 deletions(-)
create mode 100644 Documentation/s390/vfio-ap.txt
delete mode 100644 drivers/s390/crypto/ap_asm.h
create mode 100644 drivers/s390/crypto/vfio_ap_drv.c
create mode 100644 drivers/s390/crypto/vfio_ap_ops.c
create mode 100644 drivers/s390/crypto/vfio_ap_private.h
^permalinkrawreply [flat|nested] 60+ messages in thread

*Re: [PATCH v8 21/22] KVM: s390: CPU model support for AP virtualization
2018-08-09 8:17 ` David Hildenbrand
2018-08-09 8:34 ` Harald Freudenberger@ 2018-08-09 20:27 ` Tony Krowiak1 sibling, 0 replies; 60+ messages in thread
From: Tony Krowiak @ 2018-08-09 20:27 UTC (permalink / raw)
To: David Hildenbrand, Tony Krowiak, linux-s390, linux-kernel, kvm
Cc: freude, schwidefsky, heiko.carstens, borntraeger, cohuck,
kwankhede, bjsdjshi, pbonzini, alex.williamson, pmorel, alifm,
mjrosato, jjherne, thuth, pasic, berrange, fiuczy, buendgen,
frankja
On 08/09/2018 04:17 AM, David Hildenbrand wrote:
> On 08.08.2018 16:44, Tony Krowiak wrote:
>> From: Tony Krowiak <akrowiak@linux.ibm.com>
>>
>> Introduces a new CPU model feature and two CPU model
>> facilities to support AP virtualization for KVM guests.
>>
>> CPU model feature:
>>
>> The KVM_S390_VM_CPU_FEAT_AP feature indicates that
>> AP instructions are available on the guest. This
>> feature will be enabled by the kernel only if the AP
>> instructions are installed on the linux host. This feature
>> must be specifically turned on for the KVM guest from
>> userspace to use the VFIO AP device driver for guest
>> access to AP devices.
>>
>> CPU model facilities:
>>
>> 1. AP Query Configuration Information (QCI) facility is installed.
>>
>> This is indicated by setting facilities bit 12 for
>> the guest. The kernel will not enable this facility
>> for the guest if it is not set on the host. This facility
>> must not be set by userspace if the KVM_S390_VM_CPU_FEAT_AP
>> feature is not installed.
> Could it happen on real HW (not frankenstein CPU models), that
>
> !AP, but STFL 12 or 15 are indicated?
I suspect that if AP instructions are not installed, then neither
STFLE.12 nor STFLE.15 would be set in the real hardware. This
comment, however, is talking about an administrator setting the
facility for a guest, not real hardware. Having said that and after
thinking about it a little more, I am going to remove that last sentence
because the CPU model configuration (e.g., as specified on the qemu
command line) will fail if the QCI or APFT facility is enabled when the
CPU model feature KVM_S390_VM_CPU_FEAT_AP is not.
>
>> If this facility is not set for the KVM guest, then only
>> APQNs with an APQI less than 16 will be used by a Linux
>> guest regardless of the matrix configuration for the virtual
>> machine. This is a limitation of the Linux AP bus.
>>
>> 2. AP Facilities Test facility (APFT) is installed.
>>
>> This is indicated by setting facilities bit 15 for
>> the guest. The kernel will not enable this facility for
>> the guest if it is not set on the host. This facility
>> must not be set by userspace if the KVM_S390_VM_CPU_FEAT_AP
>> feature is not installed.
>>
>> If this facility is not set for the KVM guest, then no
>> AP devices will be available to the guest regardless of
>> the guest's matrix configuration for the virtual
>> machine. This is a limitation of the Linux AP bus.
>>
>> Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
>> Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
>> Tested-by: Michael Mueller <mimu@linux.ibm.com>
>> Tested-by: Farhan Ali <alifm@linux.ibm.com>
>> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> ---
>> arch/s390/kvm/kvm-s390.c | 7 +++++++
>> arch/s390/tools/gen_facilities.c | 2 ++
>> 2 files changed, 9 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 9203f0b..7d4fe9b 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -367,6 +367,13 @@ static void kvm_s390_cpu_feat_init(void)
>>
>> if (MACHINE_HAS_ESOP)
>> allow_cpu_feat(KVM_S390_VM_CPU_FEAT_ESOP);
>> +
>> + /*
>> + * Check if AP instructions installed on host
>> + */
> Make this a one-line comment, please.
Will do
>
>> + if (ap_instructions_available() == 0)
>> + allow_cpu_feat(KVM_S390_VM_CPU_FEAT_AP);
>> +
>> /*
>> * We need SIE support, ESOP (PROT_READ protection for gmap_shadow),
>> * 64bit SCAO (SCA passthrough) and IDTE (for gmap_shadow unshadowing).
>> diff --git a/arch/s390/tools/gen_facilities.c b/arch/s390/tools/gen_facilities.c
>> index 90a8c9e..a52290b 100644
>> --- a/arch/s390/tools/gen_facilities.c
>> +++ b/arch/s390/tools/gen_facilities.c
>> @@ -106,6 +106,8 @@ struct facility_def {
>>
>> .name = "FACILITIES_KVM_CPUMODEL",
>> .bits = (int[]){
>> + 12, /* AP Query Configuration Information */
>> + 15, /* AP Facilities Test */
>> -1 /* END */
>> }
>> },
>>
>
^permalinkrawreply [flat|nested] 60+ messages in thread

*[PATCH v8 22/22] s390: doc: detailed specifications for AP virtualization
2018-08-08 14:44 [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters Tony Krowiak
` (20 preceding siblings ...)
2018-08-08 14:44 ` [PATCH v8 21/22] KVM: s390: CPU model support for AP virtualization Tony Krowiak
@ 2018-08-08 14:44 ` " Tony Krowiak
2018-08-08 15:06 ` [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters Janosch Frank
2018-08-08 16:25 ` Cornelia Huck23 siblings, 0 replies; 60+ messages in thread
From: Tony Krowiak @ 2018-08-08 14:44 UTC (permalink / raw)
To: linux-s390, linux-kernel, kvm
Cc: freude, schwidefsky, heiko.carstens, borntraeger, cohuck,
kwankhede, bjsdjshi, pbonzini, alex.williamson, pmorel, alifm,
mjrosato, jjherne, thuth, pasic, berrange, fiuczy, buendgen,
akrowiak, frankja, Tony Krowiak
From: Tony Krowiak <akrowiak@linux.ibm.com>
This patch provides documentation describing the AP architecture and
design concepts behind the virtualization of AP devices. It also
includes an example of how to configure AP devices for exclusive
use of KVM guests.
Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
Documentation/s390/vfio-ap.txt | 616 ++++++++++++++++++++++++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 617 insertions(+), 0 deletions(-)
create mode 100644 Documentation/s390/vfio-ap.txt
diff --git a/Documentation/s390/vfio-ap.txt b/Documentation/s390/vfio-ap.txt
new file mode 100644
index 0000000..c08c1c9
--- /dev/null
+++ b/Documentation/s390/vfio-ap.txt
@@ -0,0 +1,616 @@
+Introduction:
+============
+The Adjunct Processor (AP) facility is an IBM Z cryptographic facility comprised
+of three AP instructions and from 1 up to 256 PCIe cryptographic adapter cards.
+The AP devices provide cryptographic functions to all CPUs assigned to a
+linux system running in an IBM Z system LPAR.
+
+The AP adapter cards are exposed via the AP bus. The motivation for vfio-ap
+is to make AP cards available to KVM guests using the VFIO mediated device
+framework. This implementation relies considerably on the s390 virtualization
+facilities which do most of the hard work of providing direct access to AP
+devices.
+
+AP Architectural Overview:
+=========================
+To facilitate the comprehension of the design, let's start with some
+definitions:
+
+* AP adapter
+
+ An AP adapter is an IBM Z adapter card that can perform cryptographic
+ functions. There can be from 0 to 256 adapters assigned to an LPAR. Adapters
+ assigned to the LPAR in which a linux host is running will be available to
+ the linux host. Each adapter is identified by a number from 0 to 255. When
+ installed, an AP adapter is accessed by AP instructions executed by any CPU.
+
+ The AP adapter cards are assigned to a given LPAR via the system's Activation
+ Profile which can be edited via the HMC. When the system is IPL'd, the AP bus
+ module is loaded and detects the AP adapter cards assigned to the LPAR. The AP
+ bus creates a sysfs device for each adapter as they are detected. For example,
+ if AP adapters 4 and 10 (0x0a) are assigned to the LPAR, the AP bus will
+ create the following sysfs entries:
+
+ /sys/devices/ap/card04
+ /sys/devices/ap/card0a
+
+ Symbolic links to these devices will also be created in the AP bus devices
+ sub-directory:
+
+ /sys/bus/ap/devices/[card04]
+ /sys/bus/ap/devices/[card04]
+
+* AP domain
+
+ An adapter is partitioned into domains. Each domain can be thought of as
+ a set of hardware registers for processing AP instructions. An adapter can
+ hold up to 256 domains. Each domain is identified by a number from 0 to 255.
+ Domains can be further classified into two types:
+
+ * Usage domains are domains that can be accessed directly to process AP
+ commands.
+
+ * Control domains are domains that are accessed indirectly by AP
+ commands sent to a usage domain to control or change the domain; for
+ example, to set a secure private key for the domain.
+
+ The AP usage and control domains are assigned to a given LPAR via the system's
+ Activation Profile which can be edited via the HMC. When the system is IPL'd,
+ the AP bus module is loaded and detects the AP usage and control domains
+ assigned to the LPAR. The domain number of each usage domain will be coupled
+ with the adapter number of each AP adapter assigned to the LPAR to identify
+ the AP queues (see AP Queue section below). The domain number of each control
+ domain will be represented in a bitmask and stored in a sysfs file
+ /sys/bus/ap/ap_control_domain_mask created by the bus. The bits in the mask,
+ from most to least significant bit, correspond to domains 0-255.
+
+ A domain may be assigned to a system as both a usage and control domain, or
+ as a control domain only. Consequently, all domains assigned as both a usage
+ and control domain can both process AP commands as well as be changed by an AP
+ command sent to any usage domain assigned to the same system. Domains assigned
+ only as control domains can not process AP commands but can be changed by AP
+ commands sent to any usage domain assigned to the system.
+
+* AP Queue
+
+ An AP queue is the means by which an AP command-request message is sent to a
+ usage domain inside a specific adapter. An AP queue is identified by a tuple
+ comprised of an AP adapter ID (APID) and an AP queue index (APQI). The
+ APQI corresponds to a given usage domain number within the adapter. This tuple
+ forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP
+ instructions include a field containing the APQN to identify the AP queue to
+ which the AP command-request message is to be sent for processing.
+
+ The AP bus will create a sysfs device for each APQN that can be derived from
+ the cross product of the AP adapter and usage domain numbers detected when the
+ AP bus module is loaded. For example, if adapters 4 and 10 (0x0a) and usage
+ domains 6 and 71 (0x47) are assigned to the LPAR, the AP bus will create the
+ following sysfs entries:
+
+ /sys/devices/ap/card04/04.0006
+ /sys/devices/ap/card04/04.0047
+ /sys/devices/ap/card0a/0a.0006
+ /sys/devices/ap/card0a/0a.0047
+
+ The following symbolic links to these devices will be created in the AP bus
+ devices subdirectory:
+
+ /sys/bus/ap/devices/[04.0006]
+ /sys/bus/ap/devices/[04.0047]
+ /sys/bus/ap/devices/[0a.0006]
+ /sys/bus/ap/devices/[0a.0047]
+
+* AP Instructions:
+
+ There are three AP instructions:
+
+ * NQAP: to enqueue an AP command-request message to a queue
+ * DQAP: to dequeue an AP command-reply message from a queue
+ * PQAP: to administer the queues
+
+AP and SIE:
+==========
+Let's now take a look at how AP instructions executed on a guest are interpreted
+by the hardware.
+
+A satellite control block called the Crypto Control Block (CRYCB) is attached to
+our main hardware virtualization control block. The CRYCB contains three fields
+to identify the adapters, usage domains and control domains assigned to the KVM
+guest:
+
+* The AP Mask (APM) field is a bit mask that identifies the AP adapters assigned
+ to the KVM guest. Each bit in the mask, from most significant to least
+ significant bit, corresponds to an APID from 0-255. If a bit is set, the
+ corresponding adapter is valid for use by the KVM guest.
+
+* The AP Queue Mask (AQM) field is a bit mask identifying the AP usage domains
+ assigned to the KVM guest. Each bit in the mask, from most significant to
+ least significant bit, corresponds to an AP queue index (APQI) from 0-255. If
+ a bit is set, the corresponding queue is valid for use by the KVM guest.
+
+* The AP Domain Mask field is a bit mask that identifies the AP control domains
+ assigned to the KVM guest. The ADM bit mask controls which domains can be
+ changed by an AP command-request message sent to a usage domain from the
+ guest. Each bit in the mask, from least significant to most significant bit,
+ corresponds to a domain from 0-255. If a bit is set, the corresponding domain
+ can be modified by an AP command-request message sent to a usage domain
+ configured for the KVM guest.
+
+If you recall from the description of an AP Queue, AP instructions include
+an APQN to identify the AP adapter and AP queue to which an AP command-request
+message is to be sent (NQAP and PQAP instructions), or from which a
+command-reply message is to be received (DQAP instruction). The validity of an
+APQN is defined by the matrix calculated from the APM and AQM; it is the
+cross product of all assigned adapter numbers (APM) with all assigned queue
+indexes (AQM). For example, if adapters 1 and 2 and usage domains 5 and 6 are
+assigned to a guest, the APQNs (1,5), (1,6), (2,5) and (2,6) will be valid for
+the guest.
+
+The APQNs can provide secure key functionality - i.e., a private key is stored
+on the adapter card for each of its domains - so each APQN must be assigned to
+at most one guest or to the linux host.
+
+ Example 1: Valid configuration:
+ ------------------------------
+ Guest1: adapters 1,2 domains 5,6
+ Guest2: adapter 1,2 domain 7
+
+ This is valid because both guests have a unique set of APQNs: Guest1 has
+ APQNs (1,5), (1,6), (2,5) and (2,6); Guest2 has APQNs (1,7) and (2,7).
+
+ Example 2: Invalid configuration:
+ Guest1: adapters 1,2 domains 5,6
+ Guest2: adapter 1 domains 6,7
+
+ This is an invalid configuration because both guests have access to
+ APQN (1,6).
+
+The Design:
+===========
+The design introduces three new objects:
+
+1. AP matrix device
+2. VFIO AP device driver (vfio_ap.ko)
+3. AP mediated matrix passthrough device
+
+The VFIO AP device driver
+-------------------------
+The VFIO AP (vfio_ap) device driver serves the following purposes:
+
+1. Provides the interfaces to bind APQNs for exclusive use of KVM guests.
+
+2. Sets up the VFIO mediated device interfaces to manage a mediated matrix
+ device and creates the sysfs interfaces for assigning adapters, usage
+ domains, and control domains comprising the matrix for a KVM guest.
+
+3. Configures the APM, AQM and ADM in the CRYCB referenced by a KVM guest's
+ SIE state description to grant the guest access to a matrix of AP devices
+
+Reserve APQNs for exclusive use of KVM guests
+---------------------------------------------
+The following block diagram illustrates the mechanism by which APQNs are
+reserved:
+
+ +------------------+
+ remove | |
+ +------------------->+ cex4queue driver +
+ | | |
+ | +------------------+
+ |
+ |
+ | remove +------------------+
+ | +-----------------+ |<---------------+
+ | | probe | Device core | |
+ | | +--------------+ +<-----------+ |
+ | | | +--------+---------+ | |
+ | | | ^ | |
+ | | | register | | |
+ | | | vfio_ap device | bind | | unbind
+ | v v | vfio_ap | | cex4queue
++--------+-----+---+ +--------+---------+ +-+---+---+--+
+| | register | | | |
+| ap_bus +<---------+ vfio_ap driver + + admin |
+| +--------->+ | | |
++------------------+ probe +---+--------+-----+ +------------+
+ | |
+ create | | assign
+ | | adapters/domains/control domains
+ v v
+ +---+--------+-----+
+ | |
+ | mediated device |
+ | |
+ +------------------+
+
+The process for reserving an AP queue for use by a KVM guest is:
+
+* The vfio-ap driver during its initialization will perform the following:
+ * Create a single 'vfio_ap' device, /sys/devices/virtual/misc/vfio_ap,
+ otherwise known as the matrix device.
+ * Register the matrix device with the device core
+* Register with the ap_bus for AP queue devices of type 10 (CEX4 and
+ newer) and to provide the vfio_ap driver's probe and remove callback
+ interfaces. Devices older than CEX4 queues are not supported to simplify the
+ implementation and because older devices will be going out of service in the
+ relatively near future.
+* The admin needs to unbind AP Queues to be reserved for use by guests from
+ the cex4queue device driver and bind them to the vfio_ap device driver.
+
+
+Set up the VFIO mediated device interfaces
+------------------------------------------
+The VFIO AP device driver utilizes the common interface of the VFIO mediated
+device core driver to:
+* Register an AP mediated bus driver to add a mediated matrix device to and
+ remove it from a VFIO group.
+* Create and destroy a mediated matrix device
+* Add a mediated matrix device to and remove it from the AP mediated bus driver
+* Add a mediated matrix device to and remove it from an IOMMU group
+
+The following high-level block diagram shows the main components and interfaces
+of the VFIO AP mediated matrix device driver:
+
+ +-------------+
+ | |
+ | +---------+ | mdev_register_driver() +--------------+
+ | | Mdev | +<-----------------------+ |
+ | | bus | | | vfio_mdev.ko |
+ | | driver | +----------------------->+ |<-> VFIO user
+ | +---------+ | probe()/remove() +--------------+ APIs
+ | |
+ | MDEV CORE |
+ | MODULE |
+ | mdev.ko |
+ | +---------+ | mdev_register_device() +--------------+
+ | |Physical | +<-----------------------+ |
+ | | device | | | vfio_ap.ko |<-> matrix
+ | |interface| +----------------------->+ | device
+ | +---------+ | callback +--------------+
+ +-------------+
+
+During initialization of the vfio_ap module, the matrix device is registered
+with an 'mdev_parent_ops' structure that provides the sysfs attribute
+structures, mdev functions and callback interfaces for managing the mediated
+matrix device.
+
+* sysfs attribute structures:
+ * supported_type_groups
+ The VFIO mediated device framework supports creation of user-defined
+ mediated device types. These mediated device types are specified
+ via the 'supported_type_groups' structure when a device is registered
+ with the mediated device framework. The registration process creates the
+ sysfs structures for each mediated device type specified in the
+ 'mdev_supported_types' sub-directory of the device being registered. Along
+ with the device type, the sysfs attributes of the mediated device type are
+ provided.
+
+ The VFIO AP device driver will register one mediated device type for
+ passthrough devices:
+ /sys/devices/virtual/misc/vfio_ap/mdev_supported_types/vfio_ap-passthrough
+ Only the read-only attributes required by the VFIO mdev framework will
+ be provided:
+ ... name
+ ... device_api
+ ... available_instances
+ ... device_api
+ Where:
+ * name: specifies the name of the mediated device type
+ * device_api: the mediated device type's API
+ * available_instances: the number of mediated matrix passthrough devices
+ that can be created
+ * device_api: specifies the VFIO API
+ * mdev_attr_groups
+ This attribute group identifies the user-defined sysfs attributes of the
+ mediated device. When a device is registered with the VFIO mediated device
+ framework, the sysfs attributes files identified in the 'mdev_attr_groups'
+ structure will be created in the mediated matrix device's directory. The
+ sysfs attributes for a mediated matrix device are:
+ * assign_adapter:
+ * unassign_adapter:
+ Write-only attributes for assigning/unassigning an AP adapter to/from the
+ mediated matrix device. To assign/unassign an adapter, the APID of the
+ adapter is written to the respective attribute file.
+ * assign_domain:
+ * unassign_domain:
+ Write-only attributes for assigning/unassigning an AP usage domain to/from
+ the mediated matrix device. To assign/unassign a domain, the APQI of the
+ AP queue corresponding to a usage domain is written to the respective
+ attribute file.
+ * matrix:
+ A read-only file for displaying the APQNs derived from the cross product
+ of the adapters and domains assigned to the mediated matrix device.
+ * assign_control_domain:
+ * unassign_control_domain:
+ Write-only attributes for assigning/unassigning an AP control domain
+ to/from the mediated matrix device. To assign/unassign a control domain,
+ the ID of a domain to be assigned/unassigned is written to the respective
+ attribute file.
+ * control_domains:
+ A read-only file for displaying the control domain numbers assigned to the
+ mediated matrix device.
+
+* functions:
+ * create:
+ allocates the ap_matrix_mdev structure used by the vfio_ap driver to:
+ * Store the reference to the KVM structure for the guest using the mdev
+ * Store the AP matrix configuration for the adapters, domains, and control
+ domains assigned via the corresponding sysfs attributes files
+ * remove:
+ deallocates the mediated matrix device's ap_matrix_mdev structure. This will
+ be allowed only if a running guest is not using the mdev.
+
+* callback interfaces
+ * open:
+ The vfio_ap driver uses this callback to register a
+ VFIO_GROUP_NOTIFY_SET_KVM notifier callback function for the mdev matrix
+ device. The open is invoked when QEMU connects the VFIO iommu group
+ for the mdev matrix device to the MDEV bus. Access to the KVM structure used
+ to configure the KVM guest is provided via this callback. The KVM structure,
+ is used to configure the guest's access to the AP matrix defined via the
+ mediated matrix device's sysfs attribute files.
+ * release:
+ unregisters the VFIO_GROUP_NOTIFY_SET_KVM notifier callback function for the
+ mdev matrix device and deconfigures the guest's AP matrix.
+
+Configure the APM, AQM and ADM in the CRYCB:
+-------------------------------------------
+Configuring the AP matrix for a KVM guest will be performed when the
+VFIO_GROUP_NOTIFY_SET_KVM notifier callback is invoked. The notifier
+function is called when QEMU connects to KVM. The CRYCB is configured by:
+* Setting the bits in the APM corresponding to the APIDs assigned to the
+ mediated matrix device via its 'assign_adapter' interface.
+* Setting the bits in the AQM corresponding to the APQIs assigned to the
+ mediated matrix device via its 'assign_domain' interface.
+* Setting the bits in the ADM corresponding to the domain dIDs assigned to the
+ mediated matrix device via its 'assign_control_domains' interface.
+
+The CPU model features for AP
+-----------------------------
+The AP stack relies on the presence of the AP instructions as well as two
+facilities: The AP Facilities Test (APFT) facility; and the AP Query
+Configuration Information (QCI) facility. These features/facilities are made
+available to a KVM guest via the following CPU model features:
+
+1. ap: Indicates whether the AP instructions are installed on the guest. This
+ feature will be enabled by KVM only if the AP instructions are installed
+ on the host.
+
+2. apft: Indicates the APFT facility is available on the guest. This facility
+ can be made available to the guest only if it is available on the host.
+
+3. apft: Indicates the AP QCI facility is available on the guest. This facility
+ can be made available to the guest only if it is available on the host.
+
+Note that if the user chooses to specify a CPU model different than the 'host'
+model to QEMU, the CPU model features and facilities need to be turned on
+explicitly; for example:
+
+ /usr/bin/qemu-system-s390x ... -cpu z13,ap=on,apqci=on,apft=on
+
+A guest can be precluded from using AP features/facilities by turning them off
+explicitly; for example:
+
+ /usr/bin/qemu-system-s390x ... -cpu host,ap=off,apqci=off,apft=off
+
+Example:
+=======
+Let's now provide an example to illustrate how KVM guests may be given
+access to AP facilities. For this example, we will show how to configure
+two guests such that executing the lszcrypt command on the guests would
+look like this:
+
+Guest1
+------
+CARD.DOMAIN TYPE MODE
+------------------------------
+05 CEX5C CCA-Coproc
+05.0004 CEX5C CCA-Coproc
+05.00ab CEX5C CCA-Coproc
+06 CEX5A Accelerator
+06.0004 CEX5A Accelerator
+06.00ab CEX5C CCA-Coproc
+
+Guest2
+------
+CARD.DOMAIN TYPE MODE
+------------------------------
+05 CEX5A Accelerator
+05.0047 CEX5A Accelerator
+05.00ff CEX5A Accelerator
+
+These are the steps:
+
+1. Install the vfio_ap module on the linux host. The dependency chain for the
+ vfio_ap module is:
+ * vfio
+ * mdev
+ * vfio_mdev
+ * KVM
+ * vfio_ap
+
+2. Secure the AP queues to be used by the two guests so that the host can not
+ access them. Only type 10 adapters (i.e., CEX4 and later) are supported
+ for the following reasons: To simplify the implementation; a lack of older
+ systems on which to test; and because the older hardware will go out of
+ service in a relatively short time.
+
+ To secure the AP queues each, each AP Queue device must first be unbound from
+ the cex4queue device driver. The sysfs location of the driver is:
+
+ /sys/bus/ap
+ --- [drivers]
+ ------ [cex4queue]
+ --------- [05.0004]
+ --------- [05.0047]
+ --------- [05.00ab]
+ --------- [05.00ff]
+ --------- [06.0004]
+ --------- [06.00ab]
+ --------- unbind
+
+ To unbind AP queue 05.0004 for example;
+
+ echo 05.0004 > unbind
+
+ The AP queue devices must then be bound to the vfio_ap driver. The sysfs
+ location of the driver is:
+
+ /sys/bus/ap
+ --- [drivers]
+ ------ [cex4queue]
+ ---------- bind
+
+ To bind AP queue 05.0004 to the vfio_ap driver:
+
+ echo 05.0004 > bind
+
+ Take note that the AP queues bound to the vfio_ap driver will be available
+ for guest usage until the vfio_ap module is unloaded, or the host system is
+ shut down.
+
+3. Create the mediated devices needed to configure the AP matrixes for the
+ two guests and to provide an interface to the vfio_ap driver for
+ use by the guests:
+
+ /sys/devices/virtual/misc
+ --- [vfio_ap]
+ --------- [mdev_supported_types]
+ ------------ [vfio_ap-passthrough] (passthrough mediated matrix device type)
+ --------------- create
+ --------------- [devices]
+
+ To create the mediated devices for the two guests:
+
+ uuidgen > create
+ uuidgen > create
+
+ This will create two mediated devices in the [devices] subdirectory named
+ with the UUID written to the create attribute file. We call them $uuid1
+ and $uuid2:
+
+ /sys/devices/virtual/misc/
+ --- [vfio_ap]
+ --------- [mdev_supported_types]
+ ------------ [vfio_ap-passthrough]
+ --------------- [devices]
+ ------------------ [$uuid1]
+ --------------------- assign_adapter
+ --------------------- assign_control_domain
+ --------------------- assign_domain
+ --------------------- matrix
+ --------------------- unassign_adapter
+ --------------------- unassign_control_domain
+ --------------------- unassign_domain
+
+ ------------------ [$uuid2]
+ --------------------- assign_adapter
+ --------------------- assign_control_domain
+ --------------------- assign_domain
+ --------------------- matrix
+ --------------------- unassign_adapter
+ --------------------- unassign_control_domain
+ --------------------- unassign_domain
+
+4. The administrator now needs to configure the matrixes for mediated
+ devices $uuid1 (for Guest1) and $uuid2 (for Guest2).
+
+ This is how the matrix is configured for Guest1:
+
+ echo 5 > assign_adapter
+ echo 6 > assign_adapter
+ echo 4 > assign_domain
+ echo 0xab > assign_domain
+
+ For this implementation, all usage domains - i.e., domains assigned
+ via the assign_domain attribute file - will also be configured in the ADM
+ field of the KVM guest's CRYCB, so there is no need to assign control
+ domains here unless you want to assign control domains that are not
+ assigned as usage domains.
+
+ If a mistake is made configuring an adapter, domain or control domain,
+ you can use the unassign_xxx files to unassign the adapter, domain or
+ control domain.
+
+ To display the matrix configuration for Guest1:
+
+ cat matrix
+
+ This is how the matrix is configured for Guest2:
+
+ echo 5 > assign_adapter
+ echo 0x47 > assign_domain
+ echo 0xff > assign_domain
+
+ In order to successfully assign an adapter:
+
+ * All APQNs that can be derived from the adapter ID and the IDs of
+ the previously assigned domains must be bound to the vfio_ap device
+ driver. If no domains have yet been assigned, then there must be at least
+ one APQN with the specified APID bound to the vfio_ap driver.
+
+ No APQN that can be derived from the adapter ID and the IDs of the
+ previously assigned domains can be assigned to another mediated matrix
+ device.
+
+ In order to successfully assign a domain:
+
+ * All APQNs that can be derived from the domain ID and the IDs of
+ the previously assigned adapters must be bound to the vfio_ap device
+ driver. If no domains have yet been assigned, then there must be at least
+ one APQN with the specified APQI bound to the vfio_ap driver.
+
+ No APQN that can be derived from the domain ID and the IDs of the
+ previously assigned adapters can be assigned to another mediated matrix
+ device.
+
+5. Start Guest1:
+
+ /usr/bin/qemu-system-s390x ... -cpu xxx,ap=on,apqci=on,apft=on \
+ -device vfio-ap,sysfsdev=/sys/devices/virtual/misc/vfio_ap/$uuid1 ...
+
+7. Start Guest2:
+
+ /usr/bin/qemu-system-s390x ... -cpu xxx,ap=on,apqci=on,apft=on \
+ -device vfio-ap,sysfsdev=/sys/devices/virtual/misc/vfio_ap/$uuid2 ...
+
+When the guest is shut down, the mediated matrix device may be removed.
+
+Using our example again, to remove the mediated matrix device $uuid1:
+
+ /sys/devices/virtual/misc
+ --- [vfio_ap]
+ --------- [mdev_supported_types]
+ ------------ [vfio_ap-passthrough]
+ --------------- [devices]
+ ------------------ [$uuid1]
+ --------------------- remove
+
+
+ echo 1 > remove
+
+ This will release all the AP queues configured for the mediated device and
+ remove all of the mdev matrix device's sysfs structures including the mdev
+ device itself. To recreate and reconfigure the mdev matrix device, all of the
+ steps starting with step 3 will have to be performed again. Note that the
+ remove will fail if a guest using the mdev is still running.
+
+ It is not necessary to remove an mdev matrix device, but one may want to
+ remove it if no guest will use it during the lifetime of the linux host. If
+ the mdev matrix device is removed, one may want to unbind the AP queues the
+ guest was using from the vfio_ap device driver and bind them back to the
+ default driver. Alternatively, the AP queues can be configured for another
+ mdev matrix (i.e., guest).
+
+
+Limitations
+===========
+* The KVM/kernel interfaces do not provide a way to prevent unbinding an AP
+ queue that is still assigned to a mediated device. Even if the device
+ 'remove' callback returns an error, the device core detaches the AP
+ queue from the VFIO AP driver. It is therefore incumbent upon the
+ administrator to make sure there is no mediated device to which the
+ APQN - for the AP queue being unbound - is assigned.
+
+* Hot plug/unplug of AP devices is not supported for guests.
+
+* Live guest migration is not supported for guests using AP devices.
\ No newline at end of file
diff --git a/MAINTAINERS b/MAINTAINERS
index 3597910..dc25477 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12428,6 +12428,7 @@ S: Supported
F: drivers/s390/crypto/vfio_ap_drv.c
F: drivers/s390/crypto/vfio_ap_private.h
F: drivers/s390/crypto/vfio_ap_ops.c
+F: Documentation/s390/vfio-ap.txt
S390 ZFCP DRIVER
M: Steffen Maier <maier@linux.ibm.com>
--
1.7.1
^permalinkrawreply [flat|nested] 60+ messages in thread

*Re: [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters
2018-08-08 14:44 [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters Tony Krowiak
` (22 preceding siblings ...)
2018-08-08 15:06 ` [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters Janosch Frank
@ 2018-08-08 16:25 ` Cornelia Huck
2018-08-08 22:52 ` Tony Krowiak23 siblings, 1 reply; 60+ messages in thread
From: Cornelia Huck @ 2018-08-08 16:25 UTC (permalink / raw)
To: Tony Krowiak
Cc: linux-s390, linux-kernel, kvm, freude, schwidefsky,
heiko.carstens, borntraeger, kwankhede, bjsdjshi, pbonzini,
alex.williamson, pmorel, alifm, mjrosato, jjherne, thuth, pasic,
berrange, fiuczy, buendgen, frankja, Tony Krowiak
On Wed, 8 Aug 2018 10:44:10 -0400
Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote:
> From: Tony Krowiak <akrowiak@linux.ibm.com>
>
> Several major objections were raised to design changes introduced in the v7
> patch series, so in order to avoid an extended discussion around these
> objections and to expedite acceptance of the series, the following changes
> have been made for v8:
>
> 1. Removed the AP bus's ability to designate queues as 'used by host' or as
> 'used by alternate driver(s)'. The bind/unbind sysfs interfaces will be
> used for managing the connection of AP queue devices to a zcrypt driver
> or the VFIO AP driver.
I don't think the idea of pools is bad per se; I mainly did not like
the sysfs interface and the dynamic interactions.
We can probably reintroduce something like that later on, if it is
still useful.
>
> 2. Removed the 'activate' sysfs interfaces which allowed for over
> provisioning of the mediated device as well as creation of mdevs with
> overlapping matrixes. It was pointed out that both of these enhancements
> break the mdev model. Consistency checking of the mdev matrix has
> therefore been returned to the mediated matrix device's sysfs interfaces
> for assigning adapters and domains:
>
> * Verify that APQNs assigned to the mediated device are bound to the
> VFIO AP device driver
>
> * Verify that no APQN assigned to the mediated matrix device is assigned
> to any other mediated matrix device.
Ok, that makes sense.
Where's point 3? :)
>
> 4. Reworked the handling of the CRYCB in vSIE based upon patches introduced
> by David in the mainline.
I had reviewed David's patches and they looked good to me.
>
> Notes:
> =====
>
> Patches 1-4 (by Harald) posted with this series are forthcoming via
> Martins tree and are based on changes in the ap driver/bus that we use as a
> foundation. They have been included here because some of the functions
> in this patch series are dependent upon them.
I don't remember anything contentious in these.
>
> Patches 5-6 (by David) are posted with this series because they are not
> currently in our master branch. Patches 19 and 20 of this series are
> dependent upon them. I believe David's patches are available in the
> mainline now.
I don't see them queued yet, but as said, they looked fine to me.
>
> This patch series works with the v6 QEMU patches. There is no new QEMU
> patchset version yet because there have been no review comments worthy of
> creating a new series; only a couple of extremely minor nits.
Once the kernel part is merged, I'd need a respin anyway due to the
kernel headers updates.
^permalinkrawreply [flat|nested] 60+ messages in thread

*Re: [PATCH v8 00/22] vfio-ap: guest dedicated crypto adapters
2018-08-08 16:25 ` Cornelia Huck@ 2018-08-08 22:52 ` Tony Krowiak0 siblings, 0 replies; 60+ messages in thread
From: Tony Krowiak @ 2018-08-08 22:52 UTC (permalink / raw)
To: Cornelia Huck, Tony Krowiak
Cc: linux-s390, linux-kernel, kvm, freude, schwidefsky,
heiko.carstens, borntraeger, kwankhede, bjsdjshi, pbonzini,
alex.williamson, pmorel, alifm, mjrosato, jjherne, thuth, pasic,
berrange, fiuczy, buendgen, frankja
On 08/08/2018 12:25 PM, Cornelia Huck wrote:
> On Wed, 8 Aug 2018 10:44:10 -0400
> Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote:
>
>> From: Tony Krowiak <akrowiak@linux.ibm.com>
>>
>> Several major objections were raised to design changes introduced in the v7
>> patch series, so in order to avoid an extended discussion around these
>> objections and to expedite acceptance of the series, the following changes
>> have been made for v8:
>>
>> 1. Removed the AP bus's ability to designate queues as 'used by host' or as
>> 'used by alternate driver(s)'. The bind/unbind sysfs interfaces will be
>> used for managing the connection of AP queue devices to a zcrypt driver
>> or the VFIO AP driver.
> I don't think the idea of pools is bad per se; I mainly did not like
> the sysfs interface and the dynamic interactions.
>
> We can probably reintroduce something like that later on, if it is
> still useful.
That may very well be the case, but we decided on bare bones expedite
acceptance.
>
>> 2. Removed the 'activate' sysfs interfaces which allowed for over
>> provisioning of the mediated device as well as creation of mdevs with
>> overlapping matrixes. It was pointed out that both of these enhancements
>> break the mdev model. Consistency checking of the mdev matrix has
>> therefore been returned to the mediated matrix device's sysfs interfaces
>> for assigning adapters and domains:
>>
>> * Verify that APQNs assigned to the mediated device are bound to the
>> VFIO AP device driver
>>
>> * Verify that no APQN assigned to the mediated matrix device is assigned
>> to any other mediated matrix device.
> Ok, that makes sense.
>
> Where's point 3? :)
That is the invisible point. Only the all-knowing, all-seeing can
discern its
presence ;)
>
>> 4. Reworked the handling of the CRYCB in vSIE based upon patches introduced
>> by David in the mainline.
> I had reviewed David's patches and they looked good to me.
Excellent!
>
>> Notes:
>> =====
>>
>> Patches 1-4 (by Harald) posted with this series are forthcoming via
>> Martins tree and are based on changes in the ap driver/bus that we use as a
>> foundation. They have been included here because some of the functions
>> in this patch series are dependent upon them.
> I don't remember anything contentious in these.
There weren't any issues. They are included here solely because they are
needed to
build the kernel and are not yet available in our master branch.
>
>> Patches 5-6 (by David) are posted with this series because they are not
>> currently in our master branch. Patches 19 and 20 of this series are
>> dependent upon them. I believe David's patches are available in the
>> mainline now.
> I don't see them queued yet, but as said, they looked fine to me.
I was told they are available in the mainline, but I'm not entirely sure
what that means
and I didn't verify it. They are included here precisely
because they are not yet available in our master branch and our code is
dependent upon
them.
>
>> This patch series works with the v6 QEMU patches. There is no new QEMU
>> patchset version yet because there have been no review comments worthy of
>> creating a new series; only a couple of extremely minor nits.
> Once the kernel part is merged, I'd need a respin anyway due to the
> kernel headers updates.
Got it.
>
^permalinkrawreply [flat|nested] 60+ messages in thread