On 08/18/2011 10:12 PM, Wen Congyang wrote:
> >>> >> The following patch can fix this problem, but I'm not sure whether it> >> is right.> >> > It's correct but insufficient, the filtering code (pci_bridge_filter)> > needs to be updated to use the memory API.>> I read the function pci_bridge_filter(), and the function only read> PCI bridge's config space(command, base and limit). If base> limit,> it will set addr to PCI_BAR_UNMAPPED.>> I do not find anything that needs to updated to use the memory API.
Currently it doesn't do any filtering at all. Bridges need to create a
new address space, then attach aliases of this region (corresponding to
the filtered area and to the legacy vga space) to the parent bus'
address space.
> I add a scsi controller on pci bus1, and a scsi disk on this controller.> I can read and write this disk, and I do not meet any problem.>
However, filtering doesn't work. You could put a BAR outside the
filtered area and it would be visible to the guest.

At 08/19/2011 11:26 PM, Avi Kivity Write:
> On 08/18/2011 10:12 PM, Wen Congyang wrote:>> >>>> >> The following patch can fix this problem, but I'm not sure whether it>> >> is right.>> >>> > It's correct but insufficient, the filtering code (pci_bridge_filter)>> > needs to be updated to use the memory API.>>>> I read the function pci_bridge_filter(), and the function only read>> PCI bridge's config space(command, base and limit). If base> limit,>> it will set addr to PCI_BAR_UNMAPPED.>>>> I do not find anything that needs to updated to use the memory API.> > Currently it doesn't do any filtering at all. Bridges need to create a> new address space, then attach aliases of this region (corresponding to> the filtered area and to the legacy vga space) to the parent bus'> address space.
Hmm, does this problem exist before memory API is introduced?
> >> I add a scsi controller on pci bus1, and a scsi disk on this controller.>> I can read and write this disk, and I do not meet any problem.>>> > However, filtering doesn't work. You could put a BAR outside the> filtered area and it would be visible to the guest.
How to put a BAR outside the filtered area and confirm whether it would be
virible to the guest?

On 08/22/2011 06:13 AM, Wen Congyang wrote:
> At 08/19/2011 11:26 PM, Avi Kivity Write:> > On 08/18/2011 10:12 PM, Wen Congyang wrote:> >> >>> >> >> The following patch can fix this problem, but I'm not sure whether it> >> >> is right.> >> >> >> > It's correct but insufficient, the filtering code (pci_bridge_filter)> >> > needs to be updated to use the memory API.> >>> >> I read the function pci_bridge_filter(), and the function only read> >> PCI bridge's config space(command, base and limit). If base> limit,> >> it will set addr to PCI_BAR_UNMAPPED.> >>> >> I do not find anything that needs to updated to use the memory API.> >> > Currently it doesn't do any filtering at all. Bridges need to create a> > new address space, then attach aliases of this region (corresponding to> > the filtered area and to the legacy vga space) to the parent bus'> > address space.>> Hmm, does this problem exist before memory API is introduced?
Yes. There was code to handle it, but I don't think it was correct.
>> >> >> I add a scsi controller on pci bus1, and a scsi disk on this controller.> >> I can read and write this disk, and I do not meet any problem.> >>> >> > However, filtering doesn't work. You could put a BAR outside the> > filtered area and it would be visible to the guest.>> How to put a BAR outside the filtered area and confirm whether it would be> virible to the guest?>>
You could use something like kvm-unit-tests.git to write a simple test
that sets up a BAR (say from hw/ivshmem.c), writes and reads to see that
it is visible, programs the bridge to filter part of the BAR out, then
writes and reads again to verify that the correct part is filtered out.

On Thu, Aug 18, 2011 at 08:15:43AM -0700, Avi Kivity wrote:
> It's correct but insufficient, the filtering code> (pci_bridge_filter) needs to be updated to use the memory API.> > Basically it gets simpler and correcter.
I've been struggling with the following problem: bridges have two memory
ranges: prefetcheable and non-prefetcheable.
Memory in the device can be behind the prefetcheable and
non-prefetcheable memory range, but things only work correctly if
non-prefetcheable memory on the device is put behind a non-prefetcheable
range. Prefetcheable memory can go anywhere I think.
This didn't work correctly before the memory API change,
but it was easy to fix ... Now I'm not sure how.

On 08/26/2011 12:43 PM, Michael S. Tsirkin wrote:
> On Thu, Aug 18, 2011 at 08:15:43AM -0700, Avi Kivity wrote:> > It's correct but insufficient, the filtering code> > (pci_bridge_filter) needs to be updated to use the memory API.> >> > Basically it gets simpler and correcter.>> I've been struggling with the following problem: bridges have two memory> ranges: prefetcheable and non-prefetcheable.>> Memory in the device can be behind the prefetcheable and> non-prefetcheable memory range, but things only work correctly if> non-prefetcheable memory on the device is put behind a non-prefetcheable> range. Prefetcheable memory can go anywhere I think.>> This didn't work correctly before the memory API change,> but it was easy to fix ... Now I'm not sure how.
If it really matters, you can add a prefetchability attribute to
MemoryRegions. Does it though?

On Sun, Aug 28, 2011 at 10:50:49AM +0300, Avi Kivity wrote:
> On 08/26/2011 12:43 PM, Michael S. Tsirkin wrote:> >On Thu, Aug 18, 2011 at 08:15:43AM -0700, Avi Kivity wrote:> >> It's correct but insufficient, the filtering code> >> (pci_bridge_filter) needs to be updated to use the memory API.> >>> >> Basically it gets simpler and correcter.> >> >I've been struggling with the following problem: bridges have two memory> >ranges: prefetcheable and non-prefetcheable.> >> >Memory in the device can be behind the prefetcheable and> >non-prefetcheable memory range, but things only work correctly if> >non-prefetcheable memory on the device is put behind a non-prefetcheable> >range. Prefetcheable memory can go anywhere I think.> >> >This didn't work correctly before the memory API change,> >but it was easy to fix ... Now I'm not sure how.> > If it really matters, you can add a prefetchability attribute to> MemoryRegions. Does it though?
Well, its another one of these things that
guests *probably* won't in practice use.
But I don't see a way to be sure.
If the guest puts a prefetcheable memory BAR behind
a non-prefetcheable range in the bridge, it won't
be able to access that BAR, and it should.
Prefetcheable BARs on devices are less common than
non-prefetcheable, but they do exist:
I have a system with 2 devices: a VGA controller from Matrox
and an ethernet card from Mellanox have
prefetcheable BARs.
I'm not sure how prefetcheability attribute will help.
Could you explain pls?
> -- > I have a truly marvellous patch that fixes the bug which this> signature is too narrow to contain.

On 08/28/2011 02:41 PM, Michael S. Tsirkin wrote:
> >> > If it really matters, you can add a prefetchability attribute to> > MemoryRegions. Does it though?>> Well, its another one of these things that> guests *probably* won't in practice use.> But I don't see a way to be sure.>> If the guest puts a prefetcheable memory BAR behind> a non-prefetcheable range in the bridge, it won't> be able to access that BAR, and it should.
Not sure I understand - on real hardware, does it see the BAR or not?
>> Prefetcheable BARs on devices are less common than> non-prefetcheable, but they do exist:> I have a system with 2 devices: a VGA controller from Matrox> and an ethernet card from Mellanox have> prefetcheable BARs.>> I'm not sure how prefetcheability attribute will help.> Could you explain pls?>
Have an attribute for a region "does not allow prefetchable memory" in
subregions
Have an attribute for a region "prefetchable memory"
When rendering the memory map, ignore any "prefetchable memory" regions
that are subregions of a "does not allow prefetchable memory" region.
(if I understood correctly - not sure)

On Sun, Aug 28, 2011 at 04:10:14PM +0300, Avi Kivity wrote:
> On 08/28/2011 02:41 PM, Michael S. Tsirkin wrote:> >>> >> If it really matters, you can add a prefetchability attribute to> >> MemoryRegions. Does it though?> >> >Well, its another one of these things that> >guests *probably* won't in practice use.> >But I don't see a way to be sure.> >> >If the guest puts a prefetcheable memory BAR behind> >a non-prefetcheable range in the bridge, it won't> >be able to access that BAR, and it should.> > Not sure I understand - on real hardware, does it see the BAR or not?
It does.
> >> >Prefetcheable BARs on devices are less common than> >non-prefetcheable, but they do exist:> >I have a system with 2 devices: a VGA controller from Matrox> >and an ethernet card from Mellanox have> >prefetcheable BARs.> >> >I'm not sure how prefetcheability attribute will help.> >Could you explain pls?> >> > Have an attribute for a region "does not allow prefetchable memory"> in subregions> Have an attribute for a region "prefetchable memory"> > When rendering the memory map, ignore any "prefetchable memory"> regions that are subregions of a "does not allow prefetchable> memory" region.> > (if I understood correctly - not sure)
Sorry, I've been unclear. Or maybe I misunderstood?
ATM we have each BAR as a memory region, which is
in turn within io or memory address space region.
With bridges, each bridge has a single range
covering legal io addresses below it, and two ranges for memory.
Example from a real system:
Memory behind bridge: 98200000-982fffff
Prefetchable memory behind bridge: 0000000097000000-00000000977fffff
And a device can have:
Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]
Region 2: Memory at 97000000 (64-bit, prefetchable) [size=8M]
guest can in theory reprogram this:
Memory behind bridge: 98200000-98afffff
Prefetchable memory behind bridge: 0000000097000000-00000000977fffff
and
Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]
Region 2: Memory at 98300000 (64-bit, prefetchable) [size=8M]
and the device will work (presumably, guests will try to avoid this
as they assume prefetchable ranges are faster).
Thus, which range the device BAR is behind depends on the
programmed values. How do we model that?
A side note on bus filtering:
In cases of bus range partially hinding the BAR from the guest, one can
even have multiple non-contigious bits of the BAR visible and the rest
hidden.
I'm not saying it's very important to model this,
I'm guessing the only important cases are all of the
BAR visible and all of the BAR hidden.
> -- > error compiling committee.c: too many arguments to function

On 08/28/2011 04:42 PM, Michael S. Tsirkin wrote:
> On Sun, Aug 28, 2011 at 04:10:14PM +0300, Avi Kivity wrote:> > On 08/28/2011 02:41 PM, Michael S. Tsirkin wrote:> > >>> > >> If it really matters, you can add a prefetchability attribute to> > >> MemoryRegions. Does it though?> > >> > >Well, its another one of these things that> > >guests *probably* won't in practice use.> > >But I don't see a way to be sure.> > >> > >If the guest puts a prefetcheable memory BAR behind> > >a non-prefetcheable range in the bridge, it won't> > >be able to access that BAR, and it should.> >> > Not sure I understand - on real hardware, does it see the BAR or not?>> It does.
Ok, was different from what I thought. So anything that matches the two
windows is exposed (after clipping). If the guest enables the legacy
vga range, then there are three regions, with equal treatment, yes?
> ATM we have each BAR as a memory region, which is> in turn within io or memory address space region.> With bridges, each bridge has a single range> covering legal io addresses below it, and two ranges for memory.>> Example from a real system:> Memory behind bridge: 98200000-982fffff> Prefetchable memory behind bridge: 0000000097000000-00000000977fffff>> And a device can have:>> Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]> Region 2: Memory at 97000000 (64-bit, prefetchable) [size=8M]>>> guest can in theory reprogram this:>> Memory behind bridge: 98200000-98afffff> Prefetchable memory behind bridge: 0000000097000000-00000000977fffff>> and> Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]> Region 2: Memory at 98300000 (64-bit, prefetchable) [size=8M]>> and the device will work (presumably, guests will try to avoid this> as they assume prefetchable ranges are faster).
> Thus, which range the device BAR is behind depends on the> programmed values. How do we model that?
Create a memory region for the bridge's address space. This region is
not directly added to system_memory or its descendants. Devices under
the bridge see this region as its pci_address_space(). The region is as
large as the entire address space - it does not take into account any
windows.
For each of the three windows (pref, non-pref, vga), create an alias
with the appropriate start and size. Map the alias into the bridge's
parent's pci_address_space(), as subregions.
fx440 does exactly this, with the following cosmetic changes:
- the windows are different (vga, pci hole, 64-bit pci area, PAMx, SMRAM)
- instead of mapping them to the parent bridge's pci_address_space(), we
map them to get_system_memory()
> A side note on bus filtering:> In cases of bus range partially hinding the BAR from the guest, one can> even have multiple non-contigious bits of the BAR visible and the rest> hidden.
The memory API will deal with this perfectly.
> I'm not saying it's very important to model this,> I'm guessing the only important cases are all of the> BAR visible and all of the BAR hidden.
It should all work. Including a sub-bridge's windows being fragmented
by the parent bridge. Too bad it doesn't matter in practice, because
it's a really neat solution to this non-problem.

At 08/22/2011 02:23 PM, Avi Kivity Write:
> On 08/22/2011 06:13 AM, Wen Congyang wrote:>> At 08/19/2011 11:26 PM, Avi Kivity Write:>> > On 08/18/2011 10:12 PM, Wen Congyang wrote:>> >> >>>> >> >> The following patch can fix this problem, but I'm not sure>> whether it>> >> >> is right.>> >> >>> >> > It's correct but insufficient, the filtering code>> (pci_bridge_filter)>> >> > needs to be updated to use the memory API.>> >>>> >> I read the function pci_bridge_filter(), and the function only read>> >> PCI bridge's config space(command, base and limit). If base> limit,>> >> it will set addr to PCI_BAR_UNMAPPED.>> >>>> >> I do not find anything that needs to updated to use the memory API.>> >>> > Currently it doesn't do any filtering at all. Bridges need to>> create a>> > new address space, then attach aliases of this region>> (corresponding to>> > the filtered area and to the legacy vga space) to the parent bus'>> > address space.>>>> Hmm, does this problem exist before memory API is introduced?> > Yes. There was code to handle it, but I don't think it was correct.> >>>> >>> >> I add a scsi controller on pci bus1, and a scsi disk on this>> controller.>> >> I can read and write this disk, and I do not meet any problem.>> >>>> >>> > However, filtering doesn't work. You could put a BAR outside the>> > filtered area and it would be visible to the guest.>>>> How to put a BAR outside the filtered area and confirm whether it>> would be>> virible to the guest?>>>>> > > You could use something like kvm-unit-tests.git to write a simple test> that sets up a BAR (say from hw/ivshmem.c), writes and reads to see that> it is visible, programs the bridge to filter part of the BAR out, then> writes and reads again to verify that the correct part is filtered out.>
I test it with rtl8139(Because I have a such real hardware).
I add some fprintf in the callback function: rtl8139_mmio_writeb(),
rtl8139_mmio_writel(), rtl8139_mmio_writew(), rtl8139_mmio_readb(),
rtl8139_mmio_readw(), rtl8139_mmio_readl().
My test way:
1. unbind the device from rtl8139 driver
2. modify the BAR
3. reprobe the device
If the BAR is visible, the OS will access it when we reprobe the device.
If the BAR is not visible, the OS will access it.
I can not reprobe the real device if the BAR is not visible. But I can
reprobe the virtual device if the BAR is not visible.
Thanks
Wen Congyang

At 08/22/2011 02:23 PM, Avi Kivity Write:
> On 08/22/2011 06:13 AM, Wen Congyang wrote:>> At 08/19/2011 11:26 PM, Avi Kivity Write:>> > On 08/18/2011 10:12 PM, Wen Congyang wrote:>> >> >>>> >> >> The following patch can fix this problem, but I'm not sure>> whether it>> >> >> is right.>> >> >>> >> > It's correct but insufficient, the filtering code>> (pci_bridge_filter)>> >> > needs to be updated to use the memory API.>> >>>> >> I read the function pci_bridge_filter(), and the function only read>> >> PCI bridge's config space(command, base and limit). If base> limit,>> >> it will set addr to PCI_BAR_UNMAPPED.>> >>>> >> I do not find anything that needs to updated to use the memory API.>> >>> > Currently it doesn't do any filtering at all. Bridges need to>> create a>> > new address space, then attach aliases of this region>> (corresponding to>> > the filtered area and to the legacy vga space) to the parent bus'>> > address space.>>>> Hmm, does this problem exist before memory API is introduced?> > Yes. There was code to handle it, but I don't think it was correct.> >>>> >>> >> I add a scsi controller on pci bus1, and a scsi disk on this>> controller.>> >> I can read and write this disk, and I do not meet any problem.>> >>>> >>> > However, filtering doesn't work. You could put a BAR outside the>> > filtered area and it would be visible to the guest.>>>> How to put a BAR outside the filtered area and confirm whether it>> would be>> virible to the guest?>>>>> > > You could use something like kvm-unit-tests.git to write a simple test> that sets up a BAR (say from hw/ivshmem.c), writes and reads to see that> it is visible, programs the bridge to filter part of the BAR out, then> writes and reads again to verify that the correct part is filtered out.
I am testing ivshmem now. But I do not know how to access the memory
specified in the BAR.
Thanks
Wen Congyang
>

On 09/02/2011 05:56 AM, Wen Congyang wrote:
> >> > You could use something like kvm-unit-tests.git to write a simple test> > that sets up a BAR (say from hw/ivshmem.c), writes and reads to see that> > it is visible, programs the bridge to filter part of the BAR out, then> > writes and reads again to verify that the correct part is filtered out.>> I am testing ivshmem now. But I do not know how to access the memory> specified in the BAR.>>
Use the uio driver -
http://docs.blackfin.uclinux.org/kernel/generated/uio-howto/. You just
mmap() the BAR from userspace and play with it.

On Sun, Aug 28, 2011 at 04:53:33PM +0300, Avi Kivity wrote:
> On 08/28/2011 04:42 PM, Michael S. Tsirkin wrote:> >On Sun, Aug 28, 2011 at 04:10:14PM +0300, Avi Kivity wrote:> >> On 08/28/2011 02:41 PM, Michael S. Tsirkin wrote:> >> >>> >> >> If it really matters, you can add a prefetchability attribute to> >> >> MemoryRegions. Does it though?> >> >> >> >Well, its another one of these things that> >> >guests *probably* won't in practice use.> >> >But I don't see a way to be sure.> >> >> >> >If the guest puts a prefetcheable memory BAR behind> >> >a non-prefetcheable range in the bridge, it won't> >> >be able to access that BAR, and it should.> >>> >> Not sure I understand - on real hardware, does it see the BAR or not?> >> >It does.> > Ok, was different from what I thought. So anything that matches the> two windows is exposed (after clipping). If the guest enables the> legacy vga range, then there are three regions, with equal> treatment, yes?> > >ATM we have each BAR as a memory region, which is> >in turn within io or memory address space region.> >With bridges, each bridge has a single range> >covering legal io addresses below it, and two ranges for memory.> >> >Example from a real system:> > Memory behind bridge: 98200000-982fffff> > Prefetchable memory behind bridge: 0000000097000000-00000000977fffff> >> >And a device can have:> >> > Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]> > Region 2: Memory at 97000000 (64-bit, prefetchable) [size=8M]> >> >> >guest can in theory reprogram this:> >> > Memory behind bridge: 98200000-98afffff> > Prefetchable memory behind bridge: 0000000097000000-00000000977fffff> >> >and> > Region 0: Memory at 98200000 (64-bit, non-prefetchable) [size=1M]> > Region 2: Memory at 98300000 (64-bit, prefetchable) [size=8M]> >> >and the device will work (presumably, guests will try to avoid this> >as they assume prefetchable ranges are faster).> > >Thus, which range the device BAR is behind depends on the> >programmed values. How do we model that?> > Create a memory region for the bridge's address space. This region> is not directly added to system_memory or its descendants.
I do this for each bridge in the hierarchy, right?
> Devices> under the bridge see this region as its pci_address_space(). The> region is as large as the entire address space - it does not take> into account any windows.> > For each of the three windows (pref, non-pref, vga), create an alias> with the appropriate start and size. Map the alias into the> bridge's parent's pci_address_space(), as subregions.> > fx440 does exactly this, with the following cosmetic changes:> > - the windows are different (vga, pci hole, 64-bit pci area, PAMx, SMRAM)> - instead of mapping them to the parent bridge's> pci_address_space(), we map them to get_system_memory()> > >A side note on bus filtering:> >In cases of bus range partially hinding the BAR from the guest, one can> >even have multiple non-contigious bits of the BAR visible and the rest> >hidden.> > The memory API will deal with this perfectly.> > >I'm not saying it's very important to model this,> >I'm guessing the only important cases are all of the> >BAR visible and all of the BAR hidden.> > It should all work. Including a sub-bridge's windows being> fragmented by the parent bridge. Too bad it doesn't matter in> practice, because it's a really neat solution to this non-problem.
Hmm, what ties the windows of a child bridge
to be within the windows of a parent?
> -- > error compiling committee.c: too many arguments to function

On 09/04/2011 03:30 PM, Michael S. Tsirkin wrote:
> >> > Create a memory region for the bridge's address space. This region> > is not directly added to system_memory or its descendants.>> I do this for each bridge in the hierarchy, right?
Each bridge does this independently (so yes).
> > fx440 does exactly this, with the following cosmetic changes:> >> > - the windows are different (vga, pci hole, 64-bit pci area, PAMx, SMRAM)> > - instead of mapping them to the parent bridge's> > pci_address_space(), we map them to get_system_memory()>> Hmm, what ties the windows of a child bridge> to be within the windows of a parent?>
system_memory
|
+--- pci0_alias0 (aliases part of pci0)
pci0
|
+--- pci1_alias0 (a bridge)
pci1
|
+--- pci2_alias0 (another bridge)
pci2
|
+--- BAR
When rendering the memory hierarchy, the only parts of BAR which are
visible are those that fit the clipping regions pci0_alias0 ^
pci1_alias0 ^ pci2_alias0. If there are multiple aliases (like the low
and high PCI holes, and PAM, it becomes (pci0_alias0 v pci0_alias1) ^
(pci1_alias0v pci1_alias1) ^ (pci2_alias0 v pci2_alias1). ( "^" ==
intersection, "v" == union )

At 09/04/2011 04:25 PM, Avi Kivity Write:
> On 09/02/2011 05:56 AM, Wen Congyang wrote:>> >>> > You could use something like kvm-unit-tests.git to write a simple test>> > that sets up a BAR (say from hw/ivshmem.c), writes and reads to see>> that>> > it is visible, programs the bridge to filter part of the BAR out, then>> > writes and reads again to verify that the correct part is filtered>> out.>>>> I am testing ivshmem now. But I do not know how to access the memory>> specified in the BAR.>>>>> > Use the uio driver -> http://docs.blackfin.uclinux.org/kernel/generated/uio-howto/. You just> mmap() the BAR from userspace and play with it.
When I try to bind ivshmem to uio_pci_generic, I get the following messages:
uio_pci_generic 0000:01:01.0: No IRQ assigned to device: no support for interrupts?
Thanks
Wen Congyang

On 09/06/2011 06:06 AM, Wen Congyang wrote:
> > Use the uio driver -> > http://docs.blackfin.uclinux.org/kernel/generated/uio-howto/. You just> > mmap() the BAR from userspace and play with it.>> When I try to bind ivshmem to uio_pci_generic, I get the following messages:> uio_pci_generic 0000:01:01.0: No IRQ assigned to device: no support for interrupts?>
No idea what this means.

At 09/06/2011 03:45 PM, Avi Kivity Write:
> On 09/06/2011 06:06 AM, Wen Congyang wrote:>> > Use the uio driver ->> > http://docs.blackfin.uclinux.org/kernel/generated/uio-howto/. You>> just>> > mmap() the BAR from userspace and play with it.>>>> When I try to bind ivshmem to uio_pci_generic, I get the following>> messages:>> uio_pci_generic 0000:01:01.0: No IRQ assigned to device: no support>> for interrupts?>>> > No idea what this means.
PCI 3.0 6.2.4
For x86 based PCs, the values in this register correspond to IRQ numbers (0-15) of the standard dual
8259 configuration. The value 255 is defined as meaning "unknown" or "no connection" to the interrupt
controller. Values between 15 and 254 are reserved.
The register is interrupt line.
I read the config of this device, the interrupt line is 0. It means that it uses the IRQ0.
The following is the uio_pci_generic's code:
static int __devinit probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
struct uio_pci_generic_dev *gdev;
int err;
err = pci_enable_device(pdev);
if (err) {
dev_err(&pdev->dev, "%s: pci_enable_device failed: %d\n",
__func__, err);
return err;
}
if (!pdev->irq) {
dev_warn(&pdev->dev, "No IRQ assigned to device: "
"no support for interrupts?\n");
pci_disable_device(pdev);
return -ENODEV;
}
...
}
This function will be called when we write 'domain:bus:slot.function' to /sys/bus/pci/drivers/uio_pci_generic/bind.
pdev->irq is 0, it means the device uses IRQ0. But we refuse it. I do not why.
To Michael S. Tsirkin
This code is writen by you. Do you know why you check whether pdev->irq is 0?
Thanks
Wen Congyang
>

On Wed, Sep 07, 2011 at 12:39:09PM +0800, Wen Congyang wrote:
> At 09/06/2011 03:45 PM, Avi Kivity Write:> > On 09/06/2011 06:06 AM, Wen Congyang wrote:> >> > Use the uio driver -> >> > http://docs.blackfin.uclinux.org/kernel/generated/uio-howto/. You> >> just> >> > mmap() the BAR from userspace and play with it.> >>> >> When I try to bind ivshmem to uio_pci_generic, I get the following> >> messages:> >> uio_pci_generic 0000:01:01.0: No IRQ assigned to device: no support> >> for interrupts?> >>> > > > No idea what this means.> > PCI 3.0 6.2.4> For x86 based PCs, the values in this register correspond to IRQ numbers (0-15) of the standard dual> 8259 configuration. The value 255 is defined as meaning "unknown" or "no connection" to the interrupt> controller. Values between 15 and 254 are reserved.> > The register is interrupt line.> > I read the config of this device, the interrupt line is 0. It means that it uses the IRQ0.> > The following is the uio_pci_generic's code:> static int __devinit probe(struct pci_dev *pdev,> const struct pci_device_id *id)> {> struct uio_pci_generic_dev *gdev;> int err;> > err = pci_enable_device(pdev);> if (err) {> dev_err(&pdev->dev, "%s: pci_enable_device failed: %d\n",> __func__, err);> return err;> }> > if (!pdev->irq) {> dev_warn(&pdev->dev, "No IRQ assigned to device: "> "no support for interrupts?\n");> pci_disable_device(pdev);> return -ENODEV;> }> ...> }> > This function will be called when we write 'domain:bus:slot.function' to /sys/bus/pci/drivers/uio_pci_generic/bind.> pdev->irq is 0, it means the device uses IRQ0. But we refuse it. I do not why.> > To Michael S. Tsirkin> This code is writen by you. Do you know why you check whether pdev->irq is 0?> > Thanks> Wen Congyang> > >
Well I see this in linux:
/*
* Read interrupt line and base address registers.
* The architecture-dependent code can tweak these, of course.
*/
static void pci_read_irq(struct pci_dev *dev)
{
unsigned char irq;
pci_read_config_byte(dev, PCI_INTERRUPT_PIN, &irq);
dev->pin = irq;
if (irq)
pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &irq);
dev->irq = irq;
}
Thus a device without an interrupt pin will get irq set to 0,
and this seems the right way to detect such devices.
I don't think PCI devices really use IRQ0 in practice,
its probably used for PC things. More likely the system is
misconfigured. Try lspci -vv to see what went wrong.

On Fri, Sep 09, 2011 at 02:43:24PM +0800, Wen Congyang wrote:
> > However, filtering doesn't work. You could put a BAR outside the> > filtered area and it would be visible to the guest.> > > > I test it on real hardware. If I put a BAR outside the filterer area, and> then run 'lspci -vv', the BAR does not change:
...
> The BAR1 is feafbc00, and it is in the bus2's range.> I map the BAR(mmap /sys/bus/pci/devices/0000\:03\:01.0/resource1), and find> I can read and write the memory.> > Thanks> Wen Congyang
So, it's as expected. Nothing seems wrong with this picture. But
this is not the test that Avi suggested.

At 09/09/2011 03:12 PM, Michael S. Tsirkin Write:
> On Fri, Sep 09, 2011 at 02:43:24PM +0800, Wen Congyang wrote:>>> However, filtering doesn't work. You could put a BAR outside the>>> filtered area and it would be visible to the guest.>>>>>>> I test it on real hardware. If I put a BAR outside the filterer area, and>> then run 'lspci -vv', the BAR does not change:> > ...> > >> The BAR1 is feafbc00, and it is in the bus2's range.>> I map the BAR(mmap /sys/bus/pci/devices/0000\:03\:01.0/resource1), and find>> I can read and write the memory.>>>> Thanks>> Wen Congyang> > So, it's as expected. Nothing seems wrong with this picture. But> this is not the test that Avi suggested.
Sorry for my misunderstand.
My question is: How to put a BAR outside the filterer area, and how to know
whether it is visible?
Thanks
Wen Congyang

On Fri, Sep 09, 2011 at 03:24:54PM +0800, Wen Congyang wrote:
> At 09/09/2011 03:12 PM, Michael S. Tsirkin Write:> > On Fri, Sep 09, 2011 at 02:43:24PM +0800, Wen Congyang wrote:> >>> However, filtering doesn't work. You could put a BAR outside the> >>> filtered area and it would be visible to the guest.> >>>> >>> >> I test it on real hardware. If I put a BAR outside the filterer area, and> >> then run 'lspci -vv', the BAR does not change:> > > > ...> > > > > >> The BAR1 is feafbc00, and it is in the bus2's range.> >> I map the BAR(mmap /sys/bus/pci/devices/0000\:03\:01.0/resource1), and find> >> I can read and write the memory.> >>> >> Thanks> >> Wen Congyang> > > > So, it's as expected. Nothing seems wrong with this picture. But> > this is not the test that Avi suggested.> > Sorry for my misunderstand.> My question is: How to put a BAR outside the filterer area,
Write into address/limit registers on the bridge to make them
not cover the BAR behind.
> and how to know> whether it is visible?> > Thanks> Wen Congyang
Read the BAR memory as you did.

At 09/09/2011 03:34 PM, Michael S. Tsirkin Write:
> On Fri, Sep 09, 2011 at 03:24:54PM +0800, Wen Congyang wrote:>> At 09/09/2011 03:12 PM, Michael S. Tsirkin Write:>>> On Fri, Sep 09, 2011 at 02:43:24PM +0800, Wen Congyang wrote:>>>>> However, filtering doesn't work. You could put a BAR outside the>>>>> filtered area and it would be visible to the guest.>>>>>>>>>>>>> I test it on real hardware. If I put a BAR outside the filterer area, and>>>> then run 'lspci -vv', the BAR does not change:>>>>>> ...>>>>>>>>>> The BAR1 is feafbc00, and it is in the bus2's range.>>>> I map the BAR(mmap /sys/bus/pci/devices/0000\:03\:01.0/resource1), and find>>>> I can read and write the memory.>>>>>>>> Thanks>>>> Wen Congyang>>>>>> So, it's as expected. Nothing seems wrong with this picture. But>>> this is not the test that Avi suggested.>>>> Sorry for my misunderstand.>> My question is: How to put a BAR outside the filterer area,> > Write into address/limit registers on the bridge to make them> not cover the BAR behind.> >> and how to know>> whether it is visible?>>>> Thanks>> Wen Congyang> > Read the BAR memory as you did.
Thanks for your explain.
Wen Congyang