Microsoft Security Essentials Fails Second AV-Test, Microsoft Responds

For the second time in a row Microsoft's Security Essentials failed to win certification from AV-Test, an independent testing lab that evaluates the efficacy of antivirus products. In total twenty-three products received AV-TEST certification when used with Windows 7, but Microsoft's own Security Essentials suite failed back to back certification testing.

AV-Test runs several independent tests on several platforms, publishing the labs tests results every two months, and for this latest installment for November and December, the firm evaluated 25 consumer antivirus security programs. In the previous testing, Microsoft Security Essentials 4.1 was the only one to fail certification, falling well below the testing baseline set for a recommended product.

According to the results, Microsoft Security Essentials 4.1 scored a 1.5 out of 6.0 in the Protection category, caused by its lower-than-average protection against 0-day malware attacks, inclusive of web and e-mail threats (Real-World Testing). Its detection of a "representative set of malware discovered in the last 2-3 months" was also lower than the industry average. In the Repair department, which evaluates the suite's ability to remove all active components of widespread malware (including Rootkits and stealth malware), Microsoft Security Essentials 4.1 again scored well below the standard. Scoring a 3.0 out of 6.0. The s fell under the industry average, and its ability to detect actively running widespread malware (including Rootkits and stealth malware) also fell under the industry standard.

Microsoft responded to the test via a blog posted yesterday, challenging its findings.

"Our review showed that 0.0033 percent of our Microsoft Security Essentials and Microsoft Forefront Endpoint Protection customers were impacted by malware samples not detected during the test," Joe Blackbird, program manager for Microsoft's Malware Protection Center, said. "In addition, 94 percent of the malware samples not detected during the test didn't impact our customers."

In an e-mail to CNET, AV-Test CEO Andreas Marx said that Microsoft brought up some good points in its blog and that the group has been discussing these items with the company and other antivirus vendors. According to Marx, the issues cited by Microsoft aren't specific to testing but rather to the prevalence of certain viruses and their impact on actual users. "Depending on what you count, you will get different results," he said.

The testing is also dependent on the amount of data AV-Test receives, both from antivirus vendors and users.

"To create meaningful data, we are already using telemetry data from a wide range of antivirus companies and users of these products," Marx said. "Our tests can only be as good as the prevalence data we're getting in a timely manner."

Either way you look at it the results aren't good news for Microsoft which has taken more than it's share of its in the security vulnerability department. The initial version of Security Essentials scored well in the group's testing. But has quickly fallen from those high marks and in more recent tests conducted last year found the product much less effective.