Firefox/Accounts/Progress Report: Difference between revisions

Jump to navigation Jump to search

Firefox/Accounts/Progress Report (view source)

Revision as of 01:26, 17 June 2017

153 bytes added ,  17 June 2017
Updated with June 12th assessments
(Updated confidence levels as of May 30th)
(Updated with June 12th assessments)
Line 1: Line 1:
__NOTOC__
__NOTOC__
<big><big>'''May 30, 2017'''</big></big>
<big><big>'''June 12, 2017'''</big></big>


== OKR Progress ==
== OKR Progress ==
Line 18: Line 18:
|style="text-align: center; background-color: #f9d9a8"|May 15
|style="text-align: center; background-color: #f9d9a8"|May 15
|style="text-align: center; background-color: #f9d9a8"|May 30
|style="text-align: center; background-color: #f9d9a8"|May 30
|style="text-align: center; background-color: #f9d9a8"|June 5
|style="text-align: center; background-color: #f9d9a8"|June 12
|style="text-align: center; background-color: #f9d9a8"|June 19
|style="text-align: center; background-color: #f9d9a8"|June 26
|-
|-
|rowspan=2| 1. To enable webextension developers to provide value with Firefox Accounts on desktop.
|rowspan=2| 1. To enable webextension developers to provide value with Firefox Accounts on desktop.
Line 28: Line 28:
|align="center"|70%
|align="center"|70%
|align="center"|75%
|align="center"|75%
|align="center"|
|align="center"|80%
|align="center"|
|align="center"|
|align="left"|
|align="left"|
Line 34: Line 34:
*[https://github.com/mozilla/fxa-features/issues/121 Issue 121: OAuth Support for Web Extensions]  
*[https://github.com/mozilla/fxa-features/issues/121 Issue 121: OAuth Support for Web Extensions]  
|
|
*New design for hoverpad will be integrated next week.
*Will ship what we can for Hoverpad. Implementing Auth Server PKCE, the "Proof Key for Code Exchange" spec **[https://openid.net/2015/05/26/enhancing-oauth-security-for-mobile-applications-with-pkse/ Enhancing OAuth Security for Mobile Applications with PKCE]
**[https://www.authlete.com/documents/article/pkce/index Proof Key for Code Exchange (RFC 7636)]
**[https://github.com/mozilla/fxa-oauth-server/pull/466/files#r121266395 Issue 466 - feat(pkce): add PKCE support to the oauth server]
**[https://github.com/mozilla/fxa-oauth-server/issues/465 Issue 465 - Implement PKCE to support public clients using the code grant flow]
**[https://github.com/mozilla/fxa-content-server/pull/5126 Issue 5126 - feat(oauth): support PKCE oauth parameters]
|-
|-
|align="left"| 1.2 To have at least 1 internal project consuming our API on desktop.
|align="left"| 1.2 To have at least 1 internal project consuming our API on desktop.
Line 42: Line 46:
|align="center"|65%
|align="center"|65%
|align="center"|70%
|align="center"|70%
|align="center"|
|align="center"|80%
|align="center"|
|align="center"|
|
|
Hoverpad or ScreenShot integration
Hoverpad or ScreenShot integration
|
|
*Deployed box for hoverpad devs to test login and sync.
*Actively collaborating.
*Plan to meet with Screenshots team at SF Work Week. Will make sure they have everything they need to start working before hand. Want to make them autonomous.
*Plan to meet with Screenshots team at SF Work Week.
|-
|-
|rowspan=3| 2. To increase the adoption of mobile as a second device.
|rowspan=3| 2. To increase the adoption of mobile as a second device.
Line 57: Line 61:
|align="center"|25%
|align="center"|25%
|align="center"|0%
|align="center"|0%
|align="center"|
|align="center"|0%
|align="center"|
|align="center"|
|rowspan=3; align="left"|
|rowspan=3; align="left"|
Line 69: Line 73:
*[https://github.com/mozilla/fxa-features/issues/52 Issue 52]
*[https://github.com/mozilla/fxa-features/issues/52 Issue 52]
|
|
*At the time confidence level was measured, we didn't understand what the metrics were telling us. In the days following, we identified a couple fixes (adding "Maybe later? link back in and removing rate limit for Nexmo) that we think will improve the metrics and give us confidence in them.  
*Still need to figure out what caused the numbers to go down on May 24th.
*We'll re-assess confidence after the fixes have been in place long enough to give the metrics a chance to move.
*Good metrics were the result of users being incorrectly identified as being in the experiment.
*[https://groups.google.com/a/mozilla.com/forum/?hl=en#!topic/fxa-core/Dg3HPIDItrY Current working theory documented in an email thread after confidence level was estimated]:
|-
|-
| align="left"|2.2 Increase the volume of new mobile devices being setup as a second device within 48h of FxA sign-up by 20% (from X to Y).
| align="left"|2.2 Increase the volume of new mobile devices being setup as a second device within 48h of FxA sign-up by 20% (from X to Y).
Line 78: Line 83:
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|
|align="center"|0%
|align="center"|
|align="center"|
|rowspan=2; align="left"|  
|rowspan=2; align="left"|  
*Some numbers say we are and others say we’re not. Focus on getting deep link in place. See 2.1.
*Graphs are looking bad - went backwards in our numbers
*As for the rate of increase for new multi-device users within 48 hours of sign-up, it depends on what graph you look at.  
*Still have phase 3 to go.
|-
|-
| align="left"|2.3 Increase the rate of new multi-device users from 10.9% (March 27th) to 16.35% within 48h of FxA sign-up. (50% improvement)  
| align="left"|2.3 Increase the rate of new multi-device users from 10.9% (March 27th) to 16.35% within 48h of FxA sign-up. (50% improvement)  
Line 89: Line 96:
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|
|align="center"|50%
|align="center"|
|align="center"|
|-
|-
Line 99: Line 106:
|align="center"|60%
|align="center"|60%
|align="center"|60%
|align="center"|60%
|align="center"|
|align="center"|60%
|align="center"|
|align="center"|
|Email on Acid
|Email on Acid
|
|
*Current spam test run shows we’re holding steady at the improved value of 93%.
*Works well for Outlook and Hotmail.
*2 failures still reported are Outlook 2007 and 2013
*"Broken OKR."
 
|-
|-
| align="left"|3.2 To have improved login success rate of Outlook to match that of Gmail users
| align="left"|3.2 To have improved login success rate of Outlook to match that of Gmail users
Line 113: Line 119:
|align="center"|80%
|align="center"|80%
|align="center"|95%
|align="center"|95%
|align="center"|
|align="center"|100%
|align="center"|
|align="center"|
|
|
|
|
*Graphs show improved results. Seems we've hit the goal already
|-
|-
| align="left"|3.3 To have improved sign up success rate of Outlook to match that of Gmail users  
| align="left"|3.3 To have improved sign up success rate of Outlook to match that of Gmail users  
Line 125: Line 130:
|align="center"|80%
|align="center"|80%
|align="center"|95%
|align="center"|95%
|align="center"|
|align="center"|100%
|align="center"|
|align="center"|
|  
|  
|
|
*Same explanation as for 3.2
|-
|-
| align="left"|3.4 Allow to change primary email of account  
| align="left"|3.4 Allow to change primary email of account  
Line 137: Line 141:
|align="center"|55%
|align="center"|55%
|align="center"|25%
|align="center"|25%
|align="center"|
|align="center"|0%
|align="center"|
|align="center"|
|align="left"|
|align="left"|
Line 143: Line 147:
*[https://github.com/mozilla/fxa-features/issues/27 Issue 27]
*[https://github.com/mozilla/fxa-features/issues/27 Issue 27]
|
|
* Didn’t think about how it would affect android and ios. We’ll be able to ship some code but don’t have confidence we’ll be able to change email on all platforms.
*Still needs validation of android and ios client. Needs more testing.
*Need to sort out email migration by end of quarter.
|-
|-
|rowspan=4| 4. To have improved our confidence in the quality of our deploys.
|rowspan=4| 4. To have improved our confidence in the quality of our deploys.
Line 152: Line 157:
|align="center"|75%
|align="center"|75%
|align="center"|100%
|align="center"|100%
|align="center"|
|align="center"|100%
|align="center"|
|align="center"|
|New metrics tool selection
|New metrics tool selection
|
|
*Working on integration questions now.
|-
|-
| align="left"|4.2 Average 7 days or less between cutting a train and having it live in production (decrease from 12 days in Q1)
| align="left"|4.2 Average 7 days or less between cutting a train and having it live in production (decrease from 12 days in Q1)
Line 164: Line 168:
|align="center"|65%
|align="center"|65%
|align="center"|85%
|align="center"|85%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|
|
|
|rowspan=2; align="left"|
|rowspan=2; align="left"|
*Had issues in train-87 but were able to resolve them quickly. Complexity in train-87, but it was well managed and didn’t slow us down.  
*Expect to hit it by the end of the quarter.  
|-
|-
| align="left"|4.3 Average less than 2 bustage-fixing point release per train. (decrease from 6 in Q1)  
| align="left"|4.3 Average less than 2 bustage-fixing point release per train. (decrease from 6 in Q1)  
Line 176: Line 180:
|align="center"|65%
|align="center"|65%
|align="center"|85%
|align="center"|85%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|
|
|
*Not a lot of bustage, but only half-way through the quarter.
|-
|-
| align="left"|4.4 Close 30% of bugs in devops bug backlog   
| align="left"|4.4 Close 30% of bugs in devops bug backlog   
Line 187: Line 190:
|align="center"|100%
|align="center"|100%
|align="center"|100%
|align="center"|100%
|align="center"|
|align="center"|100%
|align="center"|
|align="center"|
|
|
|
|
*Have closed 33%. Great variability in the amount of work for each bug.
|-
|-
|}
|}
Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1173717"

Navigation menu