I tried to extract your vmlinuz to a vmlinux to disassemble and look at stack
usage of functions, but that was painful and did not seem to work out in the
end. :)
Can you do:
objdump -d vmlinux | scripts/checkstack.pl
and similar for xfs if it's a module:
objdump -d xfs.ko | scripts/checkstack.pl
and see how big each of the functions on the above backtrace is, in your kernel?

Sadly, I didn't have the vmlinux file around anymore. I'll be glad to
recreate it when I get in tomorrow. However, I have revered commit
3a3675b7f23f83ca8c67c9c2b6edf707fd28d1ba and the problem seems to have
vanished. I'm guessing the stack at this point is a little to fragile
for a memset. The patch is: