Kaspersky Lab has defended its handling of a controversial experiment criticised by some as a marketing exercise of questionable technical value.
The Russian anti-virus firm created 20 innocent executable files, adding fake malware detections for ten of the sample, before uploading the files to online online malware scanning …

Dodgy

I'd like to know who was the 14 antivirus publishers that updated their signatures with the fake detection. Once I know, I can completely avoid them ones, because they clearly are not checking the files themselves.

Re: Dodgy

Given the large number of variants that all these products claim to detect, and the small number of people with the experience to determine what they do, I'd say it was odds on that none of the publishers fully check all the files that they block.

Excellent advice, though. Since they *aren't* checking, and since new variants are arriving faster than the update cycle, all AV products are broken by design. To secure your system, you have to stop executing untrusted code. That means switching off the "user-friendly" features in your browser and email client, and forcing less-expert users to use less trusted accounts. AV software's "post-infection detection" simply doesn't have a role to play here, so if you think you need AV then you've completely mis-understood the problem and probably already lost the battle.

Regardless of how you look at it ...

For example

I "reported" a file KVM.EXE (I think) on my PC to F-Secure, which claimed (DeepGuard) to have detected the program fiddling in the Registry and blocked that. But I think the file actually may be an on-screen caps lock indicator and/or function key display for my netbook. That doesn't explain the Registry thing but if I've accidentally made it recognised as a virus worldwide, well, whoops.

F-Secure also objected to the VLC media player's Registry activity, once. Apparently VLC is respectable freeware, although it seems to have attached itself without asking(?) in more places in Windows than I like, i.e. Windows Explorer context menus.

It's a no-win situation

If we implement detection of some sample based solely on the fact that XYZ's scanner detects it, we're being accused of not doing proper analysis and copying other company's detection. If we don't detect the sample because our analysis has shown it is obviously not malicious, it gets into the testes' test sets and our detection rate in the tests is lowered. When we protest, we're being told that "but half a gazillion other products already detect it".

Welcome to the world of anti-virus research, where your only choices are bad ones and worse ones.

@DrBontchev

"If we don't detect the sample because our analysis has shown it is obviously not malicious"

If it's obviously not malicious, it's not malicious, QED. Flag it as "other vendors report this as malicious, but we believe it safe". There is precedent ... see the EICAR Standard Anti-Virus Test File.

"it gets into the testes' test sets and our detection rate in the tests is lowered"

So the fuck what? The tests are obviously flawed if they are showing false positives.

"When we protest, we're being told that "but half a gazillion other products already detect it"."

Again, so the fuck what? Are you good at AV (malware detection, whatever), or are you good at spinning how you want the media to see you? Pick one, ignore the other, the rest will take care of itself.

False positives...

"The goal was not to show any problems with VirusTotal or AV [anti-virus] vendors, but to show that AV vendors detecting a sample does not automatically guarantee that it's really malware - simply because false positives can happen, and they duplicate quickly," Kalkuhl told El Reg.

Well, Kaspersky should know all about false positives sib ce their product has produced some of the most damaging false positives in recent years, deleting system files and generally screwing up systems.

Do what's right

Dr Bontchev: surely any ethical third party who tests and comments on your threat-scanner will only use genuine threat software as test samples... or perhaps not? Does a magazine reviewer just take some suspect files from the same repository that was used in this exercise? For that matter, what about cookies? Some scanners do say, "Ugh! You have a cookie on your computer! It can be used to track your use of web sites!" And I genuinely don't understand why that's a problem. So perhaps it isn't.

Doing the right thing.

Actually I reckon that Kaspersky have done us all a favour here. Imagine what fun we would have had if someone with a sense of humour and a malicious streak had figured out this wonderful "how to game the world's AV engines into producing false positives" loophole instead of them?

[Random musings] Would it help if...

...the likes of VirusTotal had a mandatory option you had to select when uploading a file:

* I believe this is a suspicious file

* I believe this is a false positive

And if others had submitted the same file (by checksum) you could view the proportion who chose each button? And the AV firms could also see that of course.

Also, it would be nice if Virustotal maintained a distinct list of the filenames submitted for each identical file (by checksum). If they appear random then it's more likely a real infection but if they are all named the same...

Oh, perhaps the volume of hapless users submitting files versus the small number of geeks doing the same would make the above ideas useless?!

A Tit is a tit for awe that!

False positives render a beyond the safety requirement status, whereas, false negatives will really screw up the system. So, better that it thinks a few things are viruses which aren't, than it thinks a few viruses aren't !!

Seems so clear to me that I had to put on dark glasses just to look at it !