Re: [gsoc] syscall/libc fuzzer proposal

On Mar 20, 2010, at 3:35 PM, David Holland wrote:
> On Sat, Mar 20, 2010 at 12:40:12PM -0400, Thor Lancelot Simon wrote:
>>> As a part of my work I would like to write a translator for C
>>> language and a small library. Their goal would be to detect
>>> integer overflows, stack overflows, problems with static array
>>> indexing, etc (when such occur during the program execution). It
>>> will enable me to uncover more bugs in the software.
>>
>> What is the benefit of this when compared to existing static-analysis
>> tools such as Coverity Scan, splint, or the Clang static analyzer? Will
>> this cover any cases they don't? If so, which ones?
>
> AIUI from chat, the idea is to increase the probability that if the
> testing causes something bogus to happen, the bogus behavior will
> result in an easily identifiable abort.
>
> This seems like a valid line of reasoning; all the same, implementing
> such a tool is a fairly big (and annoying) pile of grunt work. Plus
> various variations on it have been done before. (Some of which might
> be worth looking into, actually.)
Yes. Beyond that, though, I think that fuzzing is a very useful, valid
security *testing* tool.
Thor asked how this compared to Coverity and the like. Unless I badly
misunderstand what Coverity is about, it's a static analysis tool. Static
analysis is great -- when it does the job. It can find bugs that would require
really unlikely test cases and timing. But there are things that static
analysis can't do, even in principle. Fuzzing is one tool, among many, for
dynamic testing.
Now -- as has been noted, just saying "fuzzing" isn't enough. There are tools
around that do it; there's also the NetBSD testing framework. Integrating all
that, plus new code and perhaps new analysis, would probably be a better way to
proceed. Note carefully that I said "probably" -- I haven't fleshed out my
ideas any more than the original poster. But I do encourage the original
poster to proceed and develop the proposal some more.
--Steve Bellovin, http://www.cs.columbia.edu/~smb