Friday, 23 October 2015

finding UI crashes by fuzzing input events with american fuzzy lop

As mentioned previously I've been experimenting using afl as a fuzzing engine to fuzz a stream of serialized keyboard events which LibreOffice reads and dispatches.

Performance is still pretty poor, but by tweaking our headless mode to allow dialogs to be created, then using that headless mode for eventtesting and then hacking out the actual rendering of the UI to the headless backend I've got something that performs reasonably well enough to enable me to set a far higher limit of 50 input events per cycle and start to discover real bugs in impress.

because I've seen that happen plenty in the real world, and is the problem I was hoping to find.

Turns out its been so difficult to track down because there's a timer involved which is triggered by earlier modifications to the document. To get it to crash by undoing insert slide you have to modify an object in the document, which triggers an object-modified timer, and then very quickly, before the modified-timer fires, undo insert slide. Which has the effect of impress not registering that the slide has been deleted. Some time later, in far away code, impress will crash on use of the deleted slide.

afl-eventtesting was able to find a sequence of keystrokes (which isn't a huge surprise seeing as I primed it with some insert and undo slide sequences so it didn't have to mutate things too far before it modified a document object after an insert and before the undo) to trigger the crash and the eventtesting + headless modes of LibreOffice gave a reproducible platform where the same events happen in the same sequence without any intermediate system-ui events to throw off the delicate timing. Once the thing is reliably reproducible then its just a matter of grinding through the debugging.