Yeah the "smell test" is the only real heuristics that an AV can use, and pieces of an OS packed like such obviously smells fishy. There's little reason not to detect such shenanigans given that there isn't really a legit use case either ...

But first, the AVs don't seem to detect packed UPX system files or some of packed BEP system files but only those of Upack and NsPack. So there is some bias in what they flag as malware. So if taskmgr.exe is packed using UPX or BEP, some AVs may not flag it as suspicious.

And third, this experiment hardly took me any time. The AV software vendors supposedly have several employees and all they need to do is collect all unpacked benign executables (which would be available in their huge corpus), pack them with known packers, obtain their signatures (or even just hashes) and flag them as benign (or at least not as malware!).

Yeah the "smell test" is the only real heuristics that an AV can use, and pieces of an OS packed like such obviously smells fishy. There's little reason not to detect such shenanigans given that there isn't really a legit use case either ...

I very much addressed this in the blog:
"While VT clearly mentions that using their service for AV comparison is a bad idea, it is well known that VT is popular both in the security industry as well as academia. One of the reasons that VT states in the above link is that they use command line versions of AV software. But from my experience, even latest versions of some AV software installed on my Windows still label the packed system files as malicious. Plus, in this experiment I'm not really comparing AV labels but only seeing how many AVs give labels to packed system files."

I did scan the packed system files with a few AV software installed on my computer (the latest version of course) and the trend was similar. The packed files were flagged as malware.

Although, I'm not sure why it would be difficult for AVs to pack system files (or all benign files in their corpus) using known packers, fingerprint (or hash) them and whenever they find those fingerprints (or similar ones), they can just flag them as benign (or at least not as malware).

Because they can't. Most of those packers (except UPX who repeatedly stated that they won't add encryption) implement some kind of encryption, meaning the resulting file can not be fingerprinted.

That's why for certain files they have to switch to the heuristic approach and that is why certain packers give a higher score then others. The other reason is that there is no good reason for someone to repack a system executable.

"Because they can't. Most of those packers (except UPX who repeatedly stated that they won't add encryption) implement some kind of encryption, meaning the resulting file can not be fingerprinted."- That's a very good point. But wouldn't that depend on what type of encryption the other packers use? If they use simple encryption (which most packers do), can they not be fingerprinted?

You'd have to know the secret that was used for the encryption, so no.

I'm getting a little out of my comfort zone but AFAIK most of them use at least XOR with some secret as a key or something transpositional. They, of course, can be decrypted but not on the fly with little effort like it would be necessary for the amount of data VT has to process.

I think you're right. After packing, they usually have packed data (usually constant) and a packer stub which varies. This way, they create millions of packed variants and hashing all of them would be a waste. However, we could go fuzzy hash fingerprints or image processing based fingerprints. These fingerprints more or less remain the same for all the variants.

I have seen numerous packed variants of the same family which only differ in a few bytes (packer stub). Here are two such families with many packed variants:
Instantaccess
,Obfuscator.AD

yeah, I know. Usually I will temp enable javascript on the root domain if the content seems worthwhile enough, but it's such a PITA to do it for blogspot and the experience is so crap I usually don't bother now.

This makes plenty sense. Most AVs are shipped with a UPX unpacker, but not likely any other unpacker. If the AV can't unpack the code it cannot scan it until it is executed and unpacked into memory. If you can't scan a file, why not mark it as suspicious?

Unpacking at execution time is a common technique for hiding malicious code and obfuscating its activities. Some malware makers will purchase commercial packers like Armadillo (probably old school by now). These commercial packers are frequently used to protect shareware and software with built in registration processes, but malware creators have found them invaluable.

UPX is great though, it really cuts down the size of most executables and libraries by quite a bit. The last version I've installed still credits John Reiser, as in ReiserFS, who is currently in prison for murdering his wife, which gives me a creepy chuckle.

Also, I don't why it would be difficult for AVs to take all unpacked benign exes from their database, pack them with known packers, create an MD5 or SHA hash and flag them as benign. Some packers create new executables every time, but even in that case other algorithms can be used to see how similar they are to the previously packed files.