Comments

The current implementation of lazy interrupts handling has some
issues that this tries to address.
Except on iSeries, we don't do the various workarounds we need to
do on re-enable when returning from an interrupt, which can do an
implicit re-enable, and thus we may still lose or get delayed
decrementer or doorbell interrupts.
The current scheme also makes it much harder to handle the external
"edge" interrupts provided by some BookE processors when using the
EPR facility (External Proxy) and the Freescale Hypervisor.
We also hard mask on decrementer interrupts which is sub-optimal.
This is an attempt at fixing it all in one go by reworking the way
we do the lazy interrupt disabling.
The base idea is to replace the "hard_enabled" field with a
"irq_happened" field in which we store a bit mask of what interrupt
occurred while soft-disabled.
When re-enabling, either via arch_local_irq_restore() or when returning
from an interrupt, we can now decide what to do by testing bits in that
field. We then implement re-emitting of the lost interrupts via either
a re-use of the existing exception frame (exception exit case) or via
the creation of a new one from assembly code (arch_local_irq_enable),
without the need to trigger a fake one using set_dec() or similar.
In addition, this adds a few refinements:
- We no longer hard disable decrementer interrupts that occur
while soft-disabled. We now simply bump the decrementer back to max
(on BookS) or leave it stopped (on BookE) and continue with hard interrupts
enabled, which means that we'll potentially get better sample quality from
performance monitor interrupts.
- Timer, decrementer and doorbell interrupts now hard-enable
shortly after removing the source of the interrupt, which means
they no longer run entirely hard disabled. Again, this will improve
perf sample quality.
- On Book3E 64-bit, we now make the performance monitor interrupt
act as an NMI like Book3S (the necessary C code for that to work
appear to already be present in the FSL perf code, notably calling
nmi_enter instead of irq_enter).
There are additional refinements that we can do on top of this patch:
- We could remove the ps3 workaround from arch_local_irq_enable(),
I believe that it should no longer be necessary
- We could make "masked" decrementer interrupts act as NMIs when doing
timer-based perf sampling to improve the sample quality.
- There are additional simplifications of the exception entry/exit path
that I've spotted along the way, such as merging fast_exception_return
with the normal code path.
This patch needs a LOT more testing & review than it had so far !!!
Not-signed-off-by-yet: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2:
- Add hard-enable to decrementer, timer and doorbells
- Fix CR clobber in masked irq handling on BookE
- Make embedded perf interrupt act as an NMI
- Add a PACA_HAPPENED_EE_EDGE for use by FSL if they want
to retrigger an interrupt without preventing hard-enable
v3:
- Fix or vs. ori bug on Book3E
- Fix enabling of interrupts for some exceptions on Book3E
v4:
- Fix resend of doorbells on return from interrupt on Book3E
---
arch/powerpc/include/asm/exception-64s.h | 21 ++-
arch/powerpc/include/asm/hw_irq.h | 51 +++++-
arch/powerpc/include/asm/irqflags.h | 13 +-
arch/powerpc/include/asm/paca.h | 2 +-
arch/powerpc/kernel/asm-offsets.c | 2 +-
arch/powerpc/kernel/dbell.c | 12 ++
arch/powerpc/kernel/entry_64.S | 106 +++++++-----
arch/powerpc/kernel/exceptions-64e.S | 209 ++++++++++++++++-------
arch/powerpc/kernel/exceptions-64s.S | 90 ++++++----
arch/powerpc/kernel/head_64.S | 9 -
arch/powerpc/kernel/idle_book3e.S | 8 +-
arch/powerpc/kernel/idle_power4.S | 17 ++-
arch/powerpc/kernel/idle_power7.S | 20 ++-
arch/powerpc/kernel/irq.c | 187 ++++++++++++++------
arch/powerpc/kernel/time.c | 15 ++-
arch/powerpc/platforms/iseries/Makefile | 2 +-
arch/powerpc/platforms/iseries/exception.S | 11 +-
arch/powerpc/platforms/iseries/misc.S | 26 ---
arch/powerpc/platforms/pseries/processor_idle.c | 24 +++-
19 files changed, 549 insertions(+), 276 deletions(-)
delete mode 100644 arch/powerpc/platforms/iseries/misc.S