Security/Safe Browsing/V4 Implementation: Difference between revisions

← Older edit

Security/Safe Browsing/V4 Implementation (view source)

Revision as of 19:48, 17 August 2017

1,277 bytes added , 17 August 2017

Split Milestone 9 into two (i.e. move the Fennec work to 58)

Fmarier

Confirmed users

908

edits

@@ Line 1: / Line 1: @@
-= Quick Links =
 * [[Security/Safe_Browsing/V4_Implementation/Dashboard|Dashboard]]
 * [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1167038&hide_resolved=1 Bug Dependency Tree]
 = Introduction =
 Safe Browsing v4 is designed to reduce network bandwidth and disk storage (mainly for mobile devices). The usage of partial URL hashes (aka prefixes) and complete hashes remains the same but the way we get those prefixes and complete hashes is changed. The update and hash completion API will be based on protobuf and content compression (in additional to HTTP compression) is introduced. For further information, see https://developers.google.com/safe-browsing/v4 (The public specification is not complete. For example, the protobuf is not mentioned.)
+= Timeline =
+Given that ESR 52.8 will be the last supported version of Firefox using the V2 servers, we have the following '''deadlines''':
+* Firefox 59 (Desktop): merge on 2017-11-13, release on 2018-03-06
+* Firefox 61 (Android): merge on 2018-03-05, release on 2018-06-26
+We will therefore '''target''' the following releases:
+* Firefox 58 (Desktop): merge on 2017-09-20, release on 2018-01-16
+* Firefox 60 (Android): merge on 2018-01-15, release on 2018-05-01
 = Design and Implementation =
@@ Line 11: / Line 23: @@
 When '''ListManager''' and '''HashCompleter''' (and any other related components like ProtocolParser) sees table names suffixed by "-proto", they would behave differently. For example, in '''listmanager.js''', while making update request for table '''goog-phish-proto''', ''nsIUrlClassifier.makeUpdateRequestV4'' will be called to build a v4 specific request. (See {{Bug|1264885}} and {{Bug|1275507}} for more information.)
-= Using custom API key =
+== Using custom API key ==
-We are required to have a API key to access google services like geolocation and safe browsing. The nightly/beta/aurora/release Firefox builds will have a proper API key associated with mozilla. However, if you are using the Firefox built on your own, the API key will be substituted with "no-google-api-key", which is apparently not available. There are a couple of ways to use a working API key for Safe Browsing. The most recommended way is to prepare a key file and add the path to mozconfig:
+We are required to have a API key to access Google services like geolocation and Safe Browsing. The nightly/beta/aurora/release Firefox builds will have a proper API key associated with Mozilla. However, if you build your own Firefox, the API key will be substituted with the string <tt>"no-google-api-key"</tt>, which is not a valid key.
+Here's how to get your own key:
+# Obtain a Safe Browsing Google API key from https://console.developers.google.com and put it in <tt>~/.gapi.data</tt>
+# Add <tt>ac_add_options --with-google-api-keyfile=/home/foobar/.gapi.data</tt> to your <tt>.mozconfig</tt>
+# Rebuild and check if <tt>config.status</tt> has something like <tt>'MOZ_GOOGLE_API_KEY': b'AIzaSooxxxxoxoxoxoxooxx'</tt>
+== Try Google Safe Browsing V4! ==
+To use Google Safe Browsing V4, set the following preferences:
-# Obtain a Safe Browsing google API key from https://console.developers.google.com
+# <tt>urlclassifier.downloadAllowTable</tt> to <tt>goog-downloadwhite-proto</tt>
-#* If you don't know how to get a API key, buy hchang@mozilla.com a beer and ask him :p
+# <tt>urlclassifier.downloadBlockTable</tt> to <tt>goog-badbinurl-proto</tt>
-# Create a key file which only contains the API key.
+# <tt>urlclassifier.phishTable</tt> to <tt>goog-phish-proto,test-phish-simple</tt>
-# Add "ac_add_options --with-google-api-keyfile=/path/to/your/keyfile" to mozconfig
+# <tt>urlclassifier.malwareTable</tt> to <tt>goog-malware-proto,goog-unwanted-proto,test-malware-simple,test-unwanted-simple</tt>
-# Rebuild and check if config.status has something like 'MOZ_GOOGLE_API_KEY': b'AIzaSooxxxxoxoxoxoxooxx'
-= Try it on! =
+To use both V4 and V2 at the same time:
-V4 has been function ready (but pref'ed off) since Bug 1312339 was resolved. By changing preferences
-# '''urlclassifier.malwareTable''' to '''goog-phish-shavar,goog-phish-proto,test-phish-simple'''
+# <tt>urlclassifier.downloadAllowTable</tt> to <tt>goog-downloadwhite-proto,goog-downloadwhite-digest256</tt>
-# '''urlclassifier.phishTable''' to '''goog-malware-shavar,goog-malware-proto,goog-unwanted-shavar,goog-unwanted-proto,test-malware-simple,test-unwanted-simple'''
+# <tt>urlclassifier.downloadBlockTable</tt> to <tt>goog-badbinurl-proto,goog-badbinurl-shavar</tt>
-# '''browser.safebrowsing.provider.google4.gethashURL''' to '''https://safebrowsing.googleapis.com/v4/fullHashes:find?$ct=application/x-protobuf&key=%GOOGLE_API_KEY%'''
+# <tt>urlclassifier.phishTable</tt> to <tt>goog-phish-proto,goog-phish-shavar,test-phish-simple</tt>
-you can turn on v4 feature (while v2 is still up and running).
+# <tt>urlclassifier.malwareTable</tt> to <tt>goog-malware-proto,goog-unwanted-proto,goog-malware-shavar,goog-unwanted-shavar,test-malware-simple,test-unwanted-simple</tt>
 = Milestones =
@@ Line 89: / Line 110: @@
 [https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M4 Dashboard for M4 bugs]
-== M55 (2017/3/6) ==
+== M5 (2017/3/6) ==
 === Deliverables ===
 # Enable v4 completion on nightly but ignore the result
@@ Line 96: / Line 117: @@
 === Bugs ===
-[https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M55 Dashboard for M55 bugs]
+[https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M5 Dashboard for M5 bugs]
-== M6 (Deadline TBD) ==
+== M6 (2017/4/18) ==
 === Deliverables ===
-# Cache
+# Caching
-# Fennec
+# Enable both V4 and V2 on Nightly 55
-# Download protection
-# Make v2 testcases can also test v4
 === Bugs ===
 [https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M6 Dashboard for M6 bugs]
-== M7 (Deadline TBD) ==
+== M7 (2017/6/12) ==
 === Deliverables ===
-# Enable v4 by default!
+# V4 is feature complete
 === Bugs ===
 [https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M7 Dashboard for M7 bugs]
+== M8 (2017/8/7) ==
+=== Deliverables ===
+# V4 only in Nightly 56
+# Initial Soft Vision tests
+=== Bugs ===
+[https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M8 Dashboard for M8 bugs]
+== M9 (2017/09/20) ==
+=== Deliverables ===
+# Shipping to release
+# Crash comparison on beta
+# Gradual roll-out on release
+=== Bugs ===
+[https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M9 Dashboard for M9 bugs]
+== M10 (2017/11/13) ==
+=== Deliverables ===
+# V4 working on Fennec
+# Follow-ups completed
+=== Bugs ===
+[https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation/Dashboard#M10 Dashboard for M10 bugs]