Security/Features/Application Reputation/Preliminary Results: Difference between revisions
< Security | Features | Application Reputation
Jump to navigation
Jump to search
(Created page with "== Goal == Find out if future versions of Firefox with application reputation (a new-to-Firefox safebrowsing feature using Google's safebrowsing API) improves malware detectio...") |
|||
| Line 4: | Line 4: | ||
== Testing steps == | == Testing steps == | ||
* Enable all reputation application checks in FF 31 | * Enable all reputation application checks in FF 31 | ||
* Test this special build with the 657 false negative URLs | * Test this special build with the 657 false negative URLs from NSS labs. | ||
* Test Chrome with the same URLs as a sanity check. | * Test Chrome with the same URLs as a sanity check. | ||
Latest revision as of 17:10, 23 July 2014
Goal
Find out if future versions of Firefox with application reputation (a new-to-Firefox safebrowsing feature using Google's safebrowsing API) improves malware detection. Using Google's API, we can never do better than Chrome. We refer to a malware download not caught by the malware check as a false negative.
Testing steps
- Enable all reputation application checks in FF 31
- Test this special build with the 657 false negative URLs from NSS labs.
- Test Chrome with the same URLs as a sanity check.
Results
- 657 false negative URLs reported by NSS Labs
- Of these, 330 URLs were not found (404, timeout, or host not found)
- Of these, 327 legitimate false negative URLs not caught by the version of FF that NSS Labs tested
- Of these, the special FF 31 build caught 155 of them, resulting in 172 false negatives.
- Of the 327 URLs that the release version of FF did not catch, Chrome caught 286 of them. The difference in coverage is due to certain metadata in the downloads (such as the redirect chain, and whether or not the user initiated the download) that we don't currently emulate.
We expect application reputation to reduce false negatives by approximately half.