The security of non-exec files

This article is based on a speech I gave couple of months ago at DeepSec. I wrote it during the summer, which means I would now expand on some of the paragraphs. Nonetheless, I hope you’ll enjoy the read.

Introduction

As we know there’s has been a huge increase of malware attacks carried out with files other than executable ones. I’m aware that this is a very generic definition. If we consider a PDF with JavaScript stored inside, would you call it an executable? Probably you wouldn’t, although the script might be executed. Even saying that an executable can only be a file which contains native machine code isn’t accurate. A .NET assembly which contains only managed code would still be considered an executable. But a Shockwave Flash file (with its SWF extension) may not be regarded as standing in the same category. Of course, a Shockwave Flash file is not the same thing as a .NET assembly, but they both contain byte code which at some point is converted into machine code and is executed.

This means that the barriers between executable and non-executable files are thin and in many cases there’s a problem of perception, hence the difficulty of giving this article a completely accurate title. A more appropriate one would have been: the security of all those files generally perceived as harmless or, at least, less dangerous than applications. You may guess why I opted for the other title.

Does this look infected? (no, I’m talking about the file)

This is the most feared issue. How can a non-exec file infect a system? Basically through:

Scripting or byte code

Shellcode (buffer overflows)

Dangerous format features

These vectors are the most common for infection.

Scripting and byte code (security α 1/functionality)

Many file types offer the capability to execute code. However, a distinction has to be drawn between those file formats which offer it just as an additional feature and those formats which completely rely on it.

Shockwave Flash has been a very popular infection vector thanks to its powerful byte code. While it may be apparent even to an unskilled user that a Flash game on the internet is a sort of application, it’s not as apparent under other circumstances.

Very often playing a video in a web browser involves Flash. And I’ve heard many users referring to this as “Flash videos”. They don’t know that what actually happens is that a Flash file is downloaded and its ActionScript code executed.