On Thu, Oct 3, 2013 at 11:19 PM, Bhushan Bharat-R65777
<R65777@freescale.com> wrote:
>> I don't know enough about VFIO to understand why these new interfaces are>> needed. Is this the first VFIO IOMMU driver? I see vfio_iommu_spapr_tce.c and>> vfio_iommu_type1.c but I don't know if they're comparable to the Freescale PAMU.>> Do other VFIO IOMMU implementations support MSI? If so, do they handle the>> problem of mapping the MSI regions in a different way?>> PAMU is an aperture type of IOMMU while other are paging type, So they are completely different from what PAMU is and handle that differently.
This is not an explanation or a justification for adding new
interfaces. I still have no idea what an "aperture type IOMMU" is,
other than that it is "different." But I see that Alex is working on
this issue with you in a different thread, so I'm sure you guys will
sort it out.
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Tue, Oct 08, 2013 at 10:47:49AM -0600, Bjorn Helgaas wrote:
> I still have no idea what an "aperture type IOMMU" is,> other than that it is "different."
An aperture based IOMMU is basically any GART-like IOMMU which can only
remap a small window (the aperture) of the DMA address space. DMA
outside of that window is either blocked completly or passed through
untranslated.
Joerg
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Tue, 2013-10-08 at 10:47 -0600, Bjorn Helgaas wrote:
> On Thu, Oct 3, 2013 at 11:19 PM, Bhushan Bharat-R65777> <R65777@freescale.com> wrote:> > >> I don't know enough about VFIO to understand why these new interfaces are> >> needed. Is this the first VFIO IOMMU driver? I see vfio_iommu_spapr_tce.c and> >> vfio_iommu_type1.c but I don't know if they're comparable to the Freescale PAMU.> >> Do other VFIO IOMMU implementations support MSI? If so, do they handle the> >> problem of mapping the MSI regions in a different way?> >> > PAMU is an aperture type of IOMMU while other are paging type, So they are completely different from what PAMU is and handle that differently.> > This is not an explanation or a justification for adding new> interfaces. I still have no idea what an "aperture type IOMMU" is,> other than that it is "different." But I see that Alex is working on> this issue with you in a different thread, so I'm sure you guys will> sort it out.
PAMU is a very constrained IOMMU that cannot do arbitrary page mappings.
Due to these constraints, we cannot map the MSI I/O page at its normal
address while also mapping RAM at the address we want. The address we
can map it at depends on the addresses of other mappings, so it can't be
hidden in the IOMMU driver -- the user needs to be in control.
Another difference is that (if I understand correctly) PCs handle MSIs
specially, via interrupt remapping, rather than being translated as a
normal memory access through the IOMMU.
-Scott
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

> -----Original Message-----> From: joro@8bytes.org [mailto:joro@8bytes.org]> Sent: Tuesday, October 08, 2013 10:32 PM> To: Bjorn Helgaas> Cc: Bhushan Bharat-R65777; alex.williamson@redhat.com; benh@kernel.crashing.org;> galak@kernel.crashing.org; linux-kernel@vger.kernel.org; linuxppc-> dev@lists.ozlabs.org; linux-pci@vger.kernel.org; agraf@suse.de; Wood Scott-> B07421; iommu@lists.linux-foundation.org> Subject: Re: [PATCH 1/7] powerpc: Add interface to get msi region information> > On Tue, Oct 08, 2013 at 10:47:49AM -0600, Bjorn Helgaas wrote:> > I still have no idea what an "aperture type IOMMU" is, other than that> > it is "different."> > An aperture based IOMMU is basically any GART-like IOMMU which can only remap a> small window (the aperture) of the DMA address space. DMA outside of that window> is either blocked completly or passed through untranslated.
It is completely blocked for Freescale PAMU.
So for this type of iommu what we have to do is to create a MSI mapping just after guest physical address, Example: guest have a 512M of memory then we create window of 1G (because of power of 2 requirement), then we have to FIT MSI just after 512M of guest.
And for that we need
1) to know the physical address of MSI's in interrupt controller (for that this patch was all about of).
2) When guest enable MSI interrupt then we write MSI-address and MSI-DATA in device. The discussion with Alex Williamson is about that interface.
Thanks
-Bharat
> > > Joerg> >
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

>> - u32 msiir_offset; /* Offset of MSIIR, relative to start of CCSR */>> + dma_addr_t msiir; /* MSIIR Address in CCSR */>> Are you sure dma_addr_t is right here, versus phys_addr_t? It implies> that it's the output of the DMA API, but I don't think the DMA API is> used in the MSI driver. Perhaps it should be, but we still want the raw> physical address to pass on to VFIO.
I don't know what "msiir" is used for, but if it's an address you
program into a PCI device, then it's a dma_addr_t even if you didn't
get it from the DMA API. Maybe "bus_addr_t" would have been a more
suggestive name than "dma_addr_t". That said, I have no idea how this
relates to VFIO.
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

On Tue, 2013-10-08 at 17:25 -0600, Bjorn Helgaas wrote:
> >> - u32 msiir_offset; /* Offset of MSIIR, relative to start of CCSR */> >> + dma_addr_t msiir; /* MSIIR Address in CCSR */> >> > Are you sure dma_addr_t is right here, versus phys_addr_t? It implies> > that it's the output of the DMA API, but I don't think the DMA API is> > used in the MSI driver. Perhaps it should be, but we still want the raw> > physical address to pass on to VFIO.> > I don't know what "msiir" is used for, but if it's an address you> program into a PCI device, then it's a dma_addr_t even if you didn't> get it from the DMA API. Maybe "bus_addr_t" would have been a more> suggestive name than "dma_addr_t". That said, I have no idea how this> relates to VFIO.
It's a bit awkward because it gets used both as something to program
into a PCI device (and it's probably a bug that the DMA API doesn't get
used), and also (if I understand the current plans correctly) as a
physical address to give to VFIO to be a destination address in an IOMMU
mapping. So I think the value we keep here should be a phys_addr_t (it
comes straight from the MMIO address in the device tree), which gets
trivially turned into a dma_addr_t by the non-VFIO code path because
there's currently no translation there.
-Scott
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

> -----Original Message-----> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-> owner@vger.kernel.org] On Behalf Of Bhushan Bharat-R65777> Sent: Tuesday, October 08, 2013 10:40 PM> To: joro@8bytes.org; Bjorn Helgaas> Cc: alex.williamson@redhat.com; benh@kernel.crashing.org;> galak@kernel.crashing.org; linux-kernel@vger.kernel.org; linuxppc-> dev@lists.ozlabs.org; linux-pci@vger.kernel.org; agraf@suse.de; Wood> Scott-B07421; iommu@lists.linux-foundation.org> Subject: RE: [PATCH 1/7] powerpc: Add interface to get msi region> information> > > > > -----Original Message-----> > From: joro@8bytes.org [mailto:joro@8bytes.org]> > Sent: Tuesday, October 08, 2013 10:32 PM> > To: Bjorn Helgaas> > Cc: Bhushan Bharat-R65777; alex.williamson@redhat.com;> > benh@kernel.crashing.org; galak@kernel.crashing.org;> > linux-kernel@vger.kernel.org; linuxppc- dev@lists.ozlabs.org;> > linux-pci@vger.kernel.org; agraf@suse.de; Wood Scott- B07421;> > iommu@lists.linux-foundation.org> > Subject: Re: [PATCH 1/7] powerpc: Add interface to get msi region> > information> >> > On Tue, Oct 08, 2013 at 10:47:49AM -0600, Bjorn Helgaas wrote:> > > I still have no idea what an "aperture type IOMMU" is, other than> > > that it is "different."> >> > An aperture based IOMMU is basically any GART-like IOMMU which can> > only remap a small window (the aperture) of the DMA address space. DMA> > outside of that window is either blocked completly or passed through> untranslated.> > It is completely blocked for Freescale PAMU.> So for this type of iommu what we have to do is to create a MSI mapping> just after guest physical address, Example: guest have a 512M of memory> then we create window of 1G (because of power of 2 requirement), then we> have to FIT MSI just after 512M of guest.
[Sethi Varun-B16395] PAMU (FSL IOMMU) has a concept of primary window and subwindows. Primary window corresponds to the complete guest iova address space (including MSI space), with respect to IOMMU_API this is termed as geometry . IOVA Base of subwindow is determined from the number of subwindows (configurable using iommu API). Subwindows allow for handling physically discontiguous memory. PAMU translates device iova accesses to actual physical address. MSI mapping would be addressed by a subwindow, with iova base starting at the end of the guest iova space.
VFIO code creates a PAMU window (also defines number of subwindow) to map the guest iova space + msi space. The interface defined by this patch queries the PAMU driver to get the iova mapping for the msi region assigned to the PCIe device (assigned to the guest).
-Varun
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html