Firefox/Accounts/Progress Report/2017Q3: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Updated comments for objectives 1 and 2)
(completed updates for objective 3)
 
Line 50: Line 50:
|
|
*Done!
*Done!
*Completed 13 out of 20.
*Completed 13 out of 22, or 59%.
*[https://waffle.io/mozilla/fxa/metrics/burndown?milestone=FxA-131:%20Q3%20Quality Quality P1s Burndown]
*[https://waffle.io/mozilla/fxa/metrics/burndown?milestone=FxA-131:%20Q3%20Quality Quality P1s Burndown]
|-
|-
Line 66: Line 66:
*[https://github.com/mozilla/fxa-content-server/pull/5332 Issue 5332: ABC test for CAD phase 3 (deep link)]  
*[https://github.com/mozilla/fxa-content-server/pull/5332 Issue 5332: ABC test for CAD phase 3 (deep link)]  
|
|
*Still not seeing any difference in outcomes between the different phases in our ABC test. Landed our iOS deeplink patch on Sept 7th so signinCodes consumption should have increased since that date. Form views on iOS should also have increased since that date thanks to deeplinks. Leif will compare form views per phase.
*Didn't achieve this. Had a lot of questions about the metrics.
*The bottom charts of our dashboard are not measuring phase 2 and 3 the same way. They show very different results because phase 3 uses the flow_id. There may be a bug with the flow_id so Leif will copy the phase 2 chart for phase 3. Will see what difference that makes.
|-
|-
|align="left"| 1.4 Fix newsletter subscription when email is confirmed on a 2nd device.
|align="left"| 1.4 Fix newsletter subscription when email is confirmed on a 2nd device.
Line 84: Line 83:
|rowspan=2| 2. To have improved the quality of the Firefox Account experience on mobile.
|rowspan=2| 2. To have improved the quality of the Firefox Account experience on mobile.
   
   
| align="left"|2.1 Measure and A/B test best push opt-in screen on iOS.  
| align="left"|2.1 Measure and A/B test best push opt-in screen on iOS.  
|align="center"|50%
|align="center"|50%
Line 109: Line 107:
|align="center"|0.8
|align="center"|0.8
|align="left"|  
|align="left"|  
Identification and fixing of the most impactful bugs on iOS
Identification and fixing of the most impactful bugs on iOS  
*[https://waffle.io/mozilla/fxa?milestone=FxA-132:%20Q3%20Mobile Mobile Bugs for Q3]
Currently working on:
*[https://github.com/mozilla/fxa-content-server/issues/5279  Issue 5279: Update Confirm view to handle when verification is using Push]
*[https://github.com/mozilla/fxa-bugzilla-mirror/issues/335 Issue 335: Display FxA avatar in settings [bz1382756]]
*[https://github.com/mozilla/fxa-bugzilla-mirror/issues/332 Issue 332: Use Firefox Account server's /.well-known/fxa-client-configuration to configure servers in Firefox for iOS [bz1237407]]
|
|
*8 out of 13 are done - 80% of 80% done as of 9/25
*8 out of 13 are done - 80% of 80% done as of 9/25
Line 130: Line 123:
|align="left"|
|align="left"|
*Allow users to test out device pairing. This does not necessarily mean that QR codes are used.
*Allow users to test out device pairing. This does not necessarily mean that QR codes are used.
*Feature doc and issues coming soon.
|  
|  
*Pushed forward the auto-focus
*Pushed forward the challenge with auto-focus.
*Things we got done this quarter that we can roll forward.
*There are things we got done this quarter that we can roll forward.
*Great analysis
*Great analysis.
|-
|-
| align="left"|3.2 Ship secondary email.
| align="left"|3.2 Ship secondary email.
Line 146: Line 138:
|
|
*We are allowing users to add a secondary email. Allows users to change primary email.
*We are allowing users to add a secondary email. Allows users to change primary email.
*Issues for changing primary email:
**[https://github.com/mozilla/fxa-bugzilla-mirror/issues/278 Issue 278: Add ability for a FxA user to change their primary email]
**[https://github.com/mozilla/fxa-bugzilla-mirror/issues/341 Issue 341: Detect when account email has changed, and update local state to match]
**[https://github.com/mozilla/fxa-auth-server/issues/1999 Issue 1999: (node:48) DeprecationWarning: DeprecationWarning for mem.emailRecord: Use mem.accountRecord]
**[https://github.com/mozilla/fxa-bugzilla-mirror/issues/341 Issues 341: Detect when account email has changed, and update local state to match]
|
|
*Caching bug fixed
*Caching bug fixed
Client code is in
*Client code is in
Works for AMO,and they have an issue to fix one little thing
*Works for AMO, and they have an issue to fix one little thing
Pocket was the only remaining thing.
*Pocket was the only remaining thing.
Confident we’ll be able to turn it on next quarter.
Removed feature flag for add secondary email last week.
We’ll pref it on sometime in November after 57 has shipped
|-
|-
| align="left"|3.3 Land support for relier encryption keys.  
| align="left"|3.3 Land support for relier encryption keys.  
Line 171: Line 155:
Relier encryption keys  
Relier encryption keys  
|
|
*Crypto consultant started today.
*Have done a lot of good work on it but it’s not totally closed out.
*Still working on the reviews.
*Currently under scrutiny by two crypto specialists.
*Can’t ship until we clear the security reviews.
*[https://github.com/mozilla/fxa/issues/245 Issue 245: Prototype key support api for Firefox Notes]
*[https://github.com/mozilla/fxa/issues/245 Issue 245: Prototype key support api for Firefox Notes]
|-
|-
Line 185: Line 172:
Implementation of Amplitude
Implementation of Amplitude
|
|
*Feel good about having data in place in time for the training.
*Should be collecting real-time data this week.
*Logging 2.0 is sending the data over.
*We’ve haven’t completely tied everything off, but it’s basically done.
*Should be able to start doing the backfill using the same process.
*Working in stage and currently fine tuning.
|-
|-
|}
|}

Latest revision as of 01:53, 30 September 2017

September 25, 2017

Q3 OKR Progress

Objective Key Result Confidence Features Notes
Jul 10 Jul 24 Aug 8 Aug 21 Sep 5 Sep 18 Score
1. To have improved the quality for the Firefox Account experience on desktop. 1.1 Close ALL (100%) issues in the Quantum/Photon milestone. (Crouton) 60% 65% 70% 75% 90% 100% 1.0

Priorities for support of Quantum/Photon in 57:

1.2 Close 50% of our Q3 quality backlog. 50% 75% 75% 90% 100% 1.0

Quality Priorities for Q3:

1.3 Increase by 15% the rate of users that become multi-device as a result of going through our first run flow. 50% 50% 50% 40% 0% 0% 0.0
  • Didn't achieve this. Had a lot of questions about the metrics.
1.4 Fix newsletter subscription when email is confirmed on a 2nd device. 95% 95% 95% 95% 90% 90% 1.0

Fixes a problem with getting marketing newsletter opt-in data over to Salesforce.

  • Done!
2. To have improved the quality of the Firefox Account experience on mobile. 2.1 Measure and A/B test best push opt-in screen on iOS. 50% 40% 55% 55% 50% 50% 0.7
  • PR reviewed but not merged.
  • Review brought up some interesting points to address.
2.2 Complete 80% of Q3 mobile bugs. N/A 60% 60% 60% 65% 90% 0.8

Identification and fixing of the most impactful bugs on iOS

3. To have completed all features that are still in flight from Q1 and Q2. 3.1 Have a device pairing flow ready to A/B test. 75% 75% 50% 25% 0% 0% 0.3
  • Allow users to test out device pairing. This does not necessarily mean that QR codes are used.
  • Pushed forward the challenge with auto-focus.
  • There are things we got done this quarter that we can roll forward.
  • Great analysis.
3.2 Ship secondary email. 80% 80% 90% 90% 90% 90% 0.9
  • We are allowing users to add a secondary email. Allows users to change primary email.
  • Caching bug fixed
  • Client code is in
  • Works for AMO, and they have an issue to fix one little thing
  • Pocket was the only remaining thing.
3.3 Land support for relier encryption keys. 70% 70% 75% 75% 75% 80% 0.9

Relier encryption keys

3.4 Integrate 100% of high priority Amplitude events. 75% 75% 75% 75% 75% 90% 0.9

Implementation of Amplitude

  • Should be collecting real-time data this week.
  • We’ve haven’t completely tied everything off, but it’s basically done.
  • Working in stage and currently fine tuning.