We are introducing Proactive Security Challenge Awards. Every product tested against the full set of 148 tests that reaches at least Very good protection level will receive a Proactive Security Challenge Award. The vendor of such a product may request for the PSC Award logo that can be placed on the product's web page or printed on its box.

We have also retested two products in Proactive Security Challenge, namely Malware Defender and McAfee Internet Security. Malware Defender has become the first product that received Proactive Security Challenge Award. Check out the latest results!

Who is "we"? anyway is flawed only if you dont know how to interpret the results. So please dont start again another "war", if you dont like it say it but we dont need to discuss again the same.

Click to expand...

thats exactly why i dont like matousec, first of all, it is kinda just a POC test, but nothing wrong with it out of curiosity, its just most people, even on wilders dont understand what the test does, they assume this test is showing a good firewall, which its not even testing that area.

thats really the only reason i dont like matousec tests, gives people the wrong idea. now whether thats matousecs fault or people's fault is up for argument.

Matousec should explicitly state that this is a HIPS test and only test HIPS products, then everything will be OK.

This is the problem:

Emsi Software GmbH – the vendor of Mamutu

2008-11-29 (Mamutu 1.7.0.23 scored 2%): In our opinion Mamutu is completely misplaced in this test. Mamutu is not a firewall, but a behavior blocker, designed to detect and block real malware samples only, not to pass a firewall leaktest. For us, the test results are useless because the product was tested for features that Behavior Blockers are not intended to provide in general. In oposite to Matousec, we think that Firewalls, Behavior Blockers and HIPS are not the same type of software and therefore they can not be tested and compared as they were the same.

Christian Mairoll
Managing Director
Emsi Software GmbH

Our response: We are testing a specific kind of security software which must meet some fixed criteria in order to be included to our project. The main criterion is to implement a process-based security. Proactive Security Challenge is designed to test Internet security suites, personal firewalls, HIPS products, behavior blockers and other behavior based systems. Mamutu met all the required criteria and hence there was no why not to include Mamutu to our project after we received several requests from our visitors. All the products included to our project implement similar features. These security features are tested in our project. We believe that using a set of open tests is the only objective way to compare all the products that implement the very same features. There are various tests used in Proactive Security Challenge, only a part of the used testing suite is based on leak-tests.

Click to expand...

Product requirements

Question: What kind of products are suitable for Proactive Security Challenge testing and which are not?

Answer: We often receive requests to test security products that are not suitable for Proactive Security Challenge. It is important to understand what kind of products do we test. The primary requirements are that the product implements application-based security model and behavior blocking. This means that it allows its users to control selected actions of applications. Among behavior blocking capabilities, the product must be able to control applications' network access. Then we require the product's project to be alive. We are not interested in already dead projects without a future although exceptions may appear. Finally, we require the tested version of the product to be stable, publicly available in English and run on Windows OS that is currently supported by the challenge. Most of the products called an Internet security suite, a personal firewall, a HIPS, a behavior blocker do meet all these criteria and hence they are suitable for Proactive Security Challenge testing.

thats exactly why i dont like matousec, first of all, it is kinda just a POC test, but nothing wrong with it out of curiosity, its just most people, even on wilders dont understand what the test does, they assume this test is showing a good firewall, which its not even testing that area.

thats really the only reason i dont like matousec tests, gives people the wrong idea. now whether thats matousecs fault or people's fault is up for argument.

Click to expand...

I know all this but we can still take profit from this results and if somebody want to know how to differenciate the garbage from the useful data from the matousec website only have to ask here.
I just want to see how to Comodo and OA and maybe any other hips will do it in the 148 test all the other tests are quite useless (AV and not HIPS soft) but at least you can see which AV's care about the HIPS like kaspersky and which not.

Do you understand what is HIPS?
That's funny you could show to wilders where is your 100% effective AV

Click to expand...

I do understand what HIPS means and what it brings to the security aspect. But the whole point with a security product is to prevent infection in the first place. That's why Matousecs tests aren't really important - it just shows how good a certrain application is when the computer is infected. And in most cases when you're infected, you won't be able to get rid of the virus/adware/spyware anyway. See my point? Security Suites are so much more important than some tests checking the self-protection of a firewall.

I do understand what HIPS means and what it brings to the security aspect. But the whole point with a security product is to prevent infection in the first place. That's why Matousecs tests aren't really important - it just shows how good a certrain application is when the computer is infected. And in most cases when you're infected, you won't be able to get rid of the virus/adware/spyware anyway. See my point? Security Suites are so much more important than some tests checking the self-protection of a firewall.

Click to expand...

Ok, reading your answer is evident that you dont know how an HIPS works and what is being tested in Matousec tests
You can easily prevent an infection when the malware bypass the AV.

I know all this but we can still take profit from this results and if somebody want to know how to differenciate the garbage from the useful data from the matousec website only have to ask here.
I just want to see how to Comodo and OA and maybe any other hips will do it in the 148 test all the other tests are quite useless (AV and not HIPS soft) but at least you can see which AV's care about the HIPS like kaspersky and which not.

Click to expand...

ye the test is fine to just see, but id have to disagree with the part that they just need to ask here, cuz ive seen countless discussions here wer people still dont realize wat theyre looking at. also if u check quite a few other forums, namely Remove-Malware forums, they use Matousec as a firewall bible without even understanding its results. Matousec shuld have a big bold disclaimer saying "THIS IS NOT A TEST OF FIREWALLS, BUT A TEST OF HIPS"

Just curious why people post this in the firewall section.. Even Mat has admitted they do not test "Firewall Portion" of the firewall. Only hip's this test is completely pointless when applied to a firewall. This test is nothing but a lame excuse for testing hips.

Just curious why people post this in the firewall section.. Even Mat has admitted they do not test "Firewall Portion" of the firewall. Only hip's this test is completely pointless when applied to a firewall. This test is nothing but a lame excuse for testing hips.

Click to expand...

because 90% of people still think its a firewall test and if Matt thinks matousec in that way, why does his entire forum think otherwise? lol

Just curious why people post this in the firewall section.. Even Mat has admitted they do not test "Firewall Portion" of the firewall. Only hip's this test is completely pointless when applied to a firewall. This test is nothing but a lame excuse for testing hips.

Click to expand...

Probably because the top HIPS are all of them integrated in firewalls, the next time I will do it in "other anti-malware software"

Probably because the top HIPS are all of them integrated in firewalls, the next time I will do it in "other anti-malware software"

Click to expand...

Next time you may want to keep posting it in the same thread. Much better for someone that wants to follow the matousec saga or someone else that does not want to see the spreading of different posts everytime a new software tool is matousec tested.

Just curious why people post this in the firewall section.. Even Mat has admitted they do not test "Firewall Portion" of the firewall. Only hip's this test is completely pointless when applied to a firewall. This test is nothing but a lame excuse for testing hips.

Click to expand...

It is not pointless if one value his/her private data (passes, bank accounts etc. etc.), that is why personal FW should have good outbound filtering per app. and thus must have good anti-leak protection which can be done only by behavior blocker component or by complete behavioral HIPS...

It is not pointless if one value his/her private data (passes, bank accounts etc. etc.), that is why personal FW should have good outbound filtering per app. and thus must have good anti-leak protection which can be done only by behavior blocker component or by complete behavioral HIPS...

Click to expand...

I'm sorry that is not the JOB of a firewall. A firewall is there to protect you from Network based attacks. Not watch your system for every little change. I think they have a thing called a Anti-Virus for those, But then again I could be wrong. Hip's is ONLY as strong as the person clicking Yes or No. Hell even I could write a program that pops up everytime I move the darn mouse and ask if its ok.

I'm sorry that is not the JOB of a firewall. A firewall is there to protect you from Network based attacks. Not watch your system for every little change. I think they have a thing called a Anti-Virus for those, But then again I could be wrong. Hip's is ONLY as strong as the person clicking Yes or No. Hell even I could write a program that pops up everytime I move the darn mouse and ask if its ok.

Click to expand...

Yes, you should write this program, but the secret is only ask when is needed not always
This is simple, if you dont like the HIPS dont use them.

"A firewall is there to protect you from Network based attacks", yes
A Firewall + HIPS is a Firewall + HIPS
And an AV + HIPS is an AV + HIPS
And an AV + Firewall Is an av or a firewall? xD
And AV is an AV...
And AV + Firewall + Parental Control + System Backup + registry scanner + privacy protector + ..... what the hell is this?

I'm sorry that is not the JOB of a firewall. A firewall is there to protect you from Network based attacks. Not watch your system for every little change. I think they have a thing called a Anti-Virus for those, But then again I could be wrong. Hip's is ONLY as strong as the person clicking Yes or No. Hell even I could write a program that pops up everytime I move the darn mouse and ask if its ok.

Click to expand...

To be honest, I don't care, one thing is certain, good personal FW must distinguish which application goes to net, how will do that? it is not my problem ...

Correct, they are highly ineffective for the majority of the users and this is why there are thousands of users still getting infected everyday whatever HIPS they run (A nice empirical evidence was the results of initial Prevx releases where 80% users were allowing malware to install regardless of prevx pop-ups).

In a nutshell, pure HIPS approaches (i.e. leave decision to a third party) are for experience users while sandbox and virtualization (i.e. isolation and damage control) are more adaptable to novice operators.