> On Sat, Nov 21, 2009 at 10:49:50AM -0800, Eric W. Biederman wrote:> > Dimitri Sivanich <sivanich@sgi.com> writes:> > > > > This patch allows for hard numa restrictions to irq affinity on x86 systems.> > >> > > Affinity is masked to allow only those cpus which the subarchitecture> > > deems accessible by the given irq.> > >> > > On some UV systems, this domain will be limited to the nodes accessible> > > to the irq's node. Initially other X86 systems will not mask off any cpus> > > so non-UV systems will remain unaffected.> > > > Is this a hardware restriction you are trying to model?> > If not this seems wrong.> > Yes, it's a hardware restriction.

Nevertheless I think that this is the wrong approach.

What we really want is a notion in the irq descriptor which tells us:this interrupt is restricted to numa node N.

The solution in this patch is just restricted to x86 and hides thatinformation deep in the arch code.

Further the patch adds code which should be in the generic interruptmanagement code as it is useful for other purposes as well:

Driver folks are looking for a way to restrict irq balancing to agiven numa node when they have all the driver data allocated on thatnode. That's not a hardware restriction as in the UV case but requiresa similar infrastructure.

One possible solution would be to have a new flag: IRQF_NODE_BOUND - irq is bound to desc->node

When an interrupt is set up we would query with a new irq_chipfunction chip->get_node_affinity(irq) which would default to an emptyimplementation returning -1. The arch code can provide its ownfunction to return the numa affinity which would express the hardwarerestriction.

The core code would restrict affinity settings to the cpumask of thatnode without any need for the arch code to check it further.

That same infrastructure could be used for the software restriction ofinterrupts to a node on which the device is bound.

Having it in the core code also allows us to expose this informationto user space so that the irq balancer knows about it and does not tryto randomly move the affinity to cpus which are not in the allowed setof the node.