News Analysis: Why Vista Was Vulnerable

Microsoft Corp. likes to promote Windows Vista as its most secure operating
system release ever. And with good reason. After all, the company put
a lot of software-development elbow grease into its top-to-bottom redesign
of Vista's default security experience.

Microsoft officials were surprised, too. As it turned out, the errant code
happened to be very, very old errant code.

"First of all, this code is pretty old; [it's] in Windows 2000 and predates
the SDL," wrote Michael Howard on Microsoft's new Security Development
Lifecycle blog.

Microsoft's SDL practice is part of its much-ballyhooed Trustworthy Computing
push. As Howard pointed out, no SDL is perfect, and -- in the case of last month's
Windows Animated Cursor exploit -- some vulnerabilities will still slip through.

"In the Windows Vista process, we banned certain APIs, like strcpy and
strncpy, and changed well over 140,000 calls to use safer calls. Memcpy [the
affected call] wasn't on that list," Howard said.

So how did Microsoft miss the Windows Animated Cursor Handling flaw during
its Vista DWM coding (and legacy GDI recoding) processes? For one thing, Howard
said, the errant call in this place didn't trigger a -GS cookie, which makes
it difficult to identify in the first place.

"Because there are no candidate buffers on the function's stack, there
is no -GS cookie added to the stack, even though the code is compiled with -GS,"
Howard wrote. "This is not the first time we've seen code with no cookie,
and this has made us rethink the heuristics used by the compiler when it determines
whether to place a cookie on the stack or not. But changing the compiler is
a long-term task. In the short-term, we have a new compiler pragma that forces
the compiler to be much more aggressive, and we will start using this pragma
on new code."

"The next issue is that the code is wrapped in an exception handler that
catches code failures," he said.

Enter ASLR: "[T]he goal of ASLR is to reduce the likelihood that an attacker
can determine the address of critical functions," Howard said. "This
makes it harder to make exploits run correctly. But if the vulnerable code is
wrapped in an exception handler that catches many errors, a failed attempt will
not crash the component and the attacker can try again with a different set
of addresses."

Elsewhere, Howard explained, Microsoft missed the vulnerability because of
an utterly pragmatic reason.

"Code that uses calls such as memcpy is hard to flag as vulnerable without
generating a great many false positives. This is a research problem that no
one has solved, here or elsewhere," he wrote. "Another angle is to
replace calls to memcpy with memcpy_s, which forces the developer to think about
the destination buffer size. We may ban memcpy for new code, but we still need
to analyze this further. Stay tuned."

About the Author

Stephen Swoyer is a Nashville, TN-based freelance journalist who writes about technology.