i.MX6Q/UL, broken-cd causes many interrupts from uSDHC

I am working on a platform based on the i.MX6Q which does not have the card detect line connected to the uSD socket. For that reason, it uses the 'broken-cd' standard kernel funcitonality to periodically poll the status of the interface and be able to support hotplug card detection.

The problem is that, when a card is removed, I constantly get interrupts from the uSDHC controller used for the uSD at a high rate (about 50 interrupts per second). After adding some debugging, the interrupt is:

This has a performance hit on the processsor, especially on i.MX6UL, where the behavior is also reproducible.

When the card is inserted back, the interrupts rate goes back to 1 per second (as expected, one per each CMD the MMC core framework is polling). I would expect the same rate when there the uSD card is not removed.

This is reproducible in the i.MX6Q EVK with the following device tree: