Confirmed users
908
edits
Security/Features/Application Reputation Design Doc: Difference between revisions
Security/Features/Application Reputation Design Doc (view source)
Revision as of 21:07, 31 March 2016
, 31 March 2016→How to turn off this feature: update the pref names
(→How to turn off this feature: update the pref names) |
|||
| (18 intermediate revisions by one other user not shown) | |||
| Line 3: | Line 3: | ||
== Background == | == Background == | ||
Google has offered an application reputation feature to detect malicious downloads as part of Google Safe Browsing since 2012 [http://blog.chromium.org/2012/01/all-about-safe-browsing.html]. Although this part of the Safe Browsing API is not documented, they have offered it to us for use in Firefox. Malicious download detection is separate from detection of phishing and malware pages, though both features use some of the same mechanisms. | Google has offered an application reputation feature to detect malicious downloads as part of Google Safe Browsing since 2012 [http://blog.chromium.org/2012/01/all-about-safe-browsing.html]. Although this part of the Safe Browsing API is not documented, they have offered it to us for use in Firefox. Malicious download detection is separate from detection of phishing and malware pages (present in Firefox since 2.0), though both features use some of the same mechanisms. | ||
This document attempts to document all of the things that Google Chrome does, so that even in the absence of official API documentation from Google, we collectively have a better chance of implementing this feature correctly. | This document attempts to document all of the things that Google Chrome does, so that even in the absence of official API documentation from Google, we collectively have a better chance of implementing this feature correctly. | ||
== How to turn off this feature == | |||
Do any one of the following: | |||
* Turn off malware detection in Preferences > Security > "Block reported attack sites." This disables all Safebrowsing malware protection, including the warning interstitial that appears when the user navigates to a malware site. | |||
* Set <tt>browser.safebrowsing.downloads.remote.enabled</tt> to false in <tt>about:config</tt>. This disables application reputation remote lookupsbut leaves other Safebrowsing malware protection intact. To turn off all download protection checks, use <tt>browser.safebrowsing.downloads.enabled</tt> instead. | |||
== High-level overview == | == High-level overview == | ||
| Line 12: | Line 19: | ||
When a binary file is downloaded, the user-agent extracts several pieces of metadata about the file, including: | When a binary file is downloaded, the user-agent extracts several pieces of metadata about the file, including: | ||
# The target URL from which the file was downloaded, its referrer URL and any URLs in the redirect chain. | # The target URL from which the file was downloaded, its referrer URL and any URLs in the redirect chain. These URLs are checked against goog-badbinurl-shavar. | ||
# The SHA-256 hash of the contents of the file. | # The SHA-256 hash of the contents of the file. | ||
# Any certificate verification information obtained through the Windows Authenticode APIs. | # Any certificate verification information obtained through the Windows Authenticode APIs. This certificate information is used to construct synthetic URLs with which to check goog-downloadwhite-digest256. | ||
# The length of the file in bytes. | # The length of the file in bytes. | ||
# The suggested filename for the download. | # The suggested filename for the download. | ||
| Line 20: | Line 27: | ||
Using this metadata, the user-agent performs a local lookups against a blocklist. In Chrome, this happens in DownloadProtectionService::CheckDownloadUrl [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=813]. If no match is found, the user-agent checks whether the binary is supported. In Chrome, the entry point is in DownloadProtectionService::CheckClientDownloadRequest [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&l=271]. If the download is supported, it checks an allowlist. In Chrome, these checks happen in CheckClientDownloadRequest::CheckWhitelists [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&l=488]. If the binary is signed, then the code signing information is checked against the allowlist. In Chrome, these checks begin inside CheckWhitelists [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=584]. | Using this metadata, the user-agent performs a local lookups against a blocklist. In Chrome, this happens in DownloadProtectionService::CheckDownloadUrl [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=813]. If no match is found, the user-agent checks whether the binary is supported. In Chrome, the entry point is in DownloadProtectionService::CheckClientDownloadRequest [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&l=271]. If the download is supported, it checks an allowlist. In Chrome, these checks happen in CheckClientDownloadRequest::CheckWhitelists [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&l=488]. If the binary is signed, then the code signing information is checked against the allowlist. In Chrome, these checks begin inside CheckWhitelists [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=584]. | ||
If any blocklist matches are found, the user-agent must not save the file to disk. If any allowlist matches are found, the user-agent may treat the binary as trusted and skip the remote lookup. If no matches are found and the binary was unsigned or signature extraction completed successfully (on Windows only), the user-agent may send a remote lookup to the application reputation service. | If any blocklist matches are found, the user-agent must not save the file to disk. If any allowlist matches are found, the user-agent may treat the binary as trusted and skip the remote lookup. If no matches are found and the binary was unsigned or signature extraction completed successfully (on Windows only) and the filename ends with an executable extension, the user-agent may send a remote lookup to the application reputation service. | ||
== Metadata extraction == | == Metadata extraction == | ||
| Line 34: | Line 41: | ||
The Chrome implementation for the digest256 list can be found in SafeBrowsingProtocolParser [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/protocol_parser.cc]. The Firefox implementation for the digest256 list may be found in ProtocolParser [http://mxr.mozilla.org/mozilla-central/source/toolkit/components/url-classifier/ProtocolParser.cc]. | The Chrome implementation for the digest256 list can be found in SafeBrowsingProtocolParser [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/protocol_parser.cc]. The Firefox implementation for the digest256 list may be found in ProtocolParser [http://mxr.mozilla.org/mozilla-central/source/toolkit/components/url-classifier/ProtocolParser.cc]. | ||
== Local lookups == | == Local lookups (present in FF 31) == | ||
URL-based lookups for the target URL and its redirect chain happen in the same way for published Safe Browsing API. For certificate information lookups, the user-agent generates URLs based on signing information and looks up the resulting URL and its fragments in the same way. | URL-based lookups for the target URL and its redirect chain happen in the same way for published Safe Browsing API. For certificate information lookups, the user-agent generates URLs based on signing information and looks up the resulting URL and its fragments in the same way. | ||
| Line 45: | Line 52: | ||
Thus every for every certificate chain of length n, n-1 whitelist strings are constructed. In Chrome, certificate whitelist strings are constructed in CertificateChainIsWhitelisted [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=703]. | Thus every for every certificate chain of length n, n-1 whitelist strings are constructed. In Chrome, certificate whitelist strings are constructed in CertificateChainIsWhitelisted [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=703]. | ||
== Remote lookup == | == Remote lookup (present in FF 32, Windows-only) == | ||
The user-agent | These lookups are Windows-only, because we rely on signature information in order to suppress remote lookups and signature APIs are only available on Windows. If the binary is unsigned or its signature does not match a known good publisher and the filename ends in a known executable extension, Firefox sends a remote lookup to the application reputation service. | ||
The user-agent encodes file metadata into a ClientDownloadRequest protocol buffer and sends it to the remote service. If a MALWARE verdict is received, the downloaded file should not be saved and the UI should report that the download may be malware. In Chrome, this occurs in CheckClientDownloadRequest::SendRequest [https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/safe_browsing/download_protection_service.cc&rcl=1392695364&l=612]. In Firefox, this happens in ApplicationReputation::PendingLookup::SendRemoteQuery [http://mxr.mozilla.org/mozilla-central/source/toolkit/components/downloads/ApplicationReputation.cpp] and in the JS DownloadIntegration.shouldBlockForApplicationReputation [http://mxr.mozilla.org/mozilla-central/source/toolkit/components/jsdownloads/src/DownloadIntegration.jsm#491]. | |||
== Preliminary results == | == Preliminary results == | ||
| Line 53: | Line 62: | ||
== Divergence from Chrome implementation == | == Divergence from Chrome implementation == | ||
* We currently have no way to determine if a user initiated the download. | |||
* We strip url params. | |||
== Implementation status == | |||
The tracking bug is {{bug|662819}}. This feature is enabled on Firefox Desktop for all OSes, except for remote lookups which are Windows only. Local lookups are turned on in FF 31, and remote lookups are scheduled to be turned on in FF 32. For more information, see | |||
* http://monica-at-mozilla.blogspot.com/2014/07/download-files-more-safely-with-firefox.html | |||
* https://blog.mozilla.org/security/2014/07/23/improving-malware-detection-in-firefox/ | |||