Fennec/NativeUI/GAReadinessReview: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(62 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== [Fennec Native] GA Readiness Review ==
== [Fennec Native] GA Readiness Review, June 20, 2012 ==


== June 18, 2012 ==
The goal of this meeting is very simple: to be sure all x-functional teams are aligned on required activities in preparation for and on the day of GA and provide a forum for any questions and concerns.


 
==GA Release Schedule ==  
=== GA + Release Schedule ===  
Moco confidential: pls. refer to weekly status email or feel free to contact elancaster directly.
Moco confidential: pls. refer to weekly status email or feel free to contact elancaster directly.


=== GA Release Criteria ===
*https://wiki.mozilla.org/Mobile/NativeFennecEndGame


=== GA Go-To Market Plan ===
=== GA Go-To Market Plan ===
Promotion timeline: https://www.smartsheet.com/b/publish?EQBCT=1b9a19bd30804b03ae1bcb3da43f7b01
Promotion timeline: ping jfu


==== Web pages being updated/localized/launched on June 26th [Jaclyn to Update for GA]====
==== Web pages being updated/localized/launched on GA day ====
NOTE: If there are existing pages that need updating but are not the responsibility of PMM; please state below :)
NOTE: If there are existing pages that need updating but are not the responsibility of PMM; please state below :)


Line 20: Line 21:
*mozilla.org/mobile/platforms
*mozilla.org/mobile/platforms
*mozilla.org/mobile
*mozilla.org/mobile
* Bug 755411 - "About Firefox Beta" Release Notes link is 404<br>
* {{bug|755411}} - "About Firefox Beta" Release Notes link is 404<br>
* Bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page<br>
* {{bug|756908}} - Link to FAQ on About Firefox Beta page points to a missing page<br>
**[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes/ Release notes]<br>
**[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes/ Release notes]<br>


Line 29: Line 30:


==== Other Promos ====
==== Other Promos ====
Reddit IamA campaign (June 25 as a primer)  
Reddit IamA campaign (Day prior to GA as a primer)


==== Other recommended Blog Posts & Activities ====
==== Other recommended Blog Posts & Activities ====
Line 35: Line 36:
*Eideticker blog post (William already has this drafted)
*Eideticker blog post (William already has this drafted)


=== Summary of Beta Releases ===
[5/25] Beta 1
[6/15] Beta 7 [https://wiki.mozilla.org/QA/Fennec/14/Beta/7#Native_Fennec_14_Beta_7_Test_Plan_.28build_3.29 QA sign off wiki]
=== Marketing Distribution Plan ===
===List of Benchmarks===
===List of Benchmarks===
https://intranet.mozilla.org/FN14GAbenchmarks
https://intranet.mozilla.org/FN14GAbenchmarks


Since performance is a large part of our story, we need to have all benchmark numbers ready to go (JS, Eideticker, Ringmark, HTML5)
*N.B.: Some of our scores are already better post-FN14. As a general rule, we promote the product we're shipping, not the future. Still, when we compile the details for PR we should include the fact that, for instance, we fail ring mark 0 in FN14, but already pass it on Aurora and Nightly. <== Johnath Speaks
 
* Since performance is a large part of our story, we need to have all benchmark numbers ready to go (JS, Eideticker, Ringmark, HTML5)


=== PR Intro Plan ===  
===Eideticker===
[[image:clock.png]]
[[image:cnn1.png]]
[[image:cnn2.png]]


*"Future of Firefox" blog post <br>
=== PR Intro Plan  ===
*Friendly reporters have been given a head's up <br>
 
*FAQ for inbound press questions <br>
*Mozilla Blog post <br>
*Reviewer's Program in progress<br>  
*Friendly reporters will get a head's up <br>  
*FAQ for inbound press questions <br>  
*Press coverage report <br>
*Press coverage report <br>


=== Support Plan ===  
=== Support Plan ===  
*[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes/ Release notes]<br>
*[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes/ Release notes] <== akeybl handles this webpage<br>
*[https://wiki.mozilla.org/Mobile/Platforms/Android#System_Requirements Minimum System Requirements & Device Compatibility]
*[https://wiki.mozilla.org/Mobile/Platforms/Android#System_Requirements Minimum System Requirements & Device Compatibility]
*https://www.mozilla.org/en-US/mobile/platforms/ Updated Platforms page targeted for a beta very soon]
*https://www.mozilla.org/en-US/mobile/platforms/ Updated Platforms page targeted for a beta very soon]
Line 69: Line 73:


=== Issues and Risks ===  
=== Issues and Risks ===  
*en-us only build for beta 1<br>
THESE HAVE BEEN RESOLVED
*{{Bug|760098}} - Default (minTwips; 120) text-inflation on YouTube makes us look bad
*{{Bug|766397}}
*[https://bugzilla.mozilla.org/show_bug.cgi?id=765254 Sync on Android is not translated at all in Suomi (fi)]<br>
*[https://bugzilla.mozilla.org/show_bug.cgi?id=765232 Sync on Android is not translated completely in Português (Brasil)(pt-BR)]
 
Cross-Functional<br> <== This are in progress
*Web site compatibility effort is ongoing<br>
*Web site compatibility effort is ongoing<br>
*Crash rate (although trending in the direction)<br>
*Need to grow add-on developer adoption for compatibility with new architecture
*GFX work is ongoing
*Telemetry is disabled for GA due to {{Bug|762620}} - Evidence that idle-daily can start misfiring
*Add-On support for beta: {{Bug|744555}}, need add-on developer adoption for compatibility with new architecture
 
*Telemetry being disabled
=== Success Factors ===
*[[image:ratingstrends.png]]
 
==Bug Metrics and Testing==
[https://bugzilla.mozilla.org/buglist.cgi?list_id=3469578;field0-0-0=cf_blocking_fennec10;type0-0-0=equals;value0-0-0=%2B;resolution=---;resolution=FIXED;resolution=INVALID;resolution=WONTFIX;resolution=DUPLICATE;resolution=WORKSFORME;resolution=INCOMPLETE;resolution=SUPPORT;resolution=EXPIRED;resolution=MOVED;query_format=advanced;bug_status=RESOLVED;bug_status=VERIFIED;bug_status=CLOSED 396 Release Blockers Resolved Since Beta (all inclusive)]
 
=== Minimum System Requirements and Device Filtering ===
*{{Bug|764518}} - Fennec Native Go Live: Be sure blocked devices are properly filtered (minimum system requirements, 7" tablets) (edit)
**Blocking 7" tablets with any OS other than ICS due to Flash support
 
=== GA Test Coverage ===
*{{Bug|765871}} Please test 10.0.5 -> 14.0 (beta 7? RC?) migration one last time Final Migration Verification in progress
*Test cases based on each functional area will be 100% complete; this includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
*Installation, update, and migration testing will be 100% complete
*Google Play pre-testing will be 100% complete
 
Beta Build Verification Testing/Sign Offs<br>
[5/15] https://wiki.mozilla.org/QA/Fennec/14/Beta/1<br>
[5/18] https://wiki.mozilla.org/QA/Fennec/14/Beta/2<br>
[5/25] https://wiki.mozilla.org/QA/Fennec/14/Beta/3<br>
[6/01] https://wiki.mozilla.org/QA/Fennec/14/Beta/4 & https://wiki.mozilla.org/QA/Fennec/14/Beta/5<br>
[6/8]  https://wiki.mozilla.org/QA/Fennec/14/Beta/6 <br>
[6/14] https://wiki.mozilla.org/QA/Fennec/14/Beta/7#Native_Fennec_14_Beta_7_Test_Plan_.28build_3.29 <br>


=== Deferred Bugs Scheduled for GA ===  
=== Deferred Bugs Scheduled for 14.0.1 ===  
*[https://bugzilla.mozilla.org/buglist.cgi?quicksearch=717096%2C725389%2C736123%2C748531%2C752426%2C752444%2C752688%2C753534%2C753845%2C754276%2C725389%2C736123%2C739648%2C748531%2C750846%2C751479%2C752052%2C752444%2C752688%2C753534%2C753845%2C754276%2C738676%2C746860%2C747231%2C747857%2C751262%2C752492%2C752514;list_id=3083671 We will continue to approve until the cut-off for go-to build]
*Strawperson PRD: https://etherpad.mozilla.org/d7OACrquVo <br>
*[https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking-fennec1.0%3AbetaN%2B;list_id=3083812 tracking issues we want fixed early in the beta cycle with a specific flag
*[https://bugzilla.mozilla.org/buglist.cgi?field0-0-0=cf_blocking_fennec10;resolution=---;query_format=advanced;type0-0-0=equals;value0-0-0=.N%2B Intended fixes for 14.0.1]
*[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes Other notable bugs in Known Issues]
 
=== Web Compatibility ===
[https://docs.google.com/document/d/1ruX5DBS7-4GQpshtPhderVsC_LZGMb0kMFWuSw2atNY/edit DOs/DONTs when talking about website compatibility in public]<br>
*[http://lmandel.github.com/mobilewebcompat/ Are We Compatible Yet?]
*[https://bugzilla.mozilla.org/buglist.cgi?list_id=3468301;resolution=---;query_format=advanced;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;component=Evangelism;product=Fennec%20Native Query for all open Web Compat Bugs reported against Fennec Native]
*[http://people.mozilla.org/~atrain/mobile/Evangelism/chrome-compare/chrome-compare.html Product Sign Off for Beta]
*Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues
*High level goals for k9o are captured in {{bug|747123}}. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
*We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues:
{{Bug|668288}} - [meta] Google evangelism issues in Firefox for Android<br>
{{Bug|759986}} - [meta] Issues with facebook.com<br>
{{Bug|763510}} - [meta] Issues with twitter.com<br>
{{Bug|763512}} - [meta] Issues with microsoft.com<br>


=== Features Summary ===  
=== Features Summary ===  
Line 86: Line 130:
*Readability  
*Readability  
*Refined UI, better tab management,smoother in-content UI
*Refined UI, better tab management,smoother in-content UI
 
*Sync
=== Test Coverage ===
*Add-On Support
#Test cases based on each functional area 100% complete
#This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
#Installation, update, and migration testing 100% complete
#Google Play pre-testing 100% complete


=== Reliability ===  
=== Reliability ===  
#[https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0b7?duration=3 As of June 19th, we are at 4 crashes per 100 ADU]
#[https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0b7?duration=3 As of June 19th, we are at 4 crashes per 100 ADU]
#[https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0a2?duration=3 This is down from 22 crashes per 100 adu on May 14th]
#[https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0a2?duration=3 This is down from 22 crashes per 100 adu on May 14th]
#Criteria for GA is no more than 5 crashes per 100 ADU
#[https://wiki.mozilla.org/Mobile/NativeFennecEndGame Criteria for GA is no more than 5 crashes per 100 ADU]
 
*[[image:crash-stats620.png]]
=== Performance ===  
=== Performance ===  
====Goals====
====Goals====
Line 104: Line 144:
*Responsive browsing: page load, panning & zooming performance
*Responsive browsing: page load, panning & zooming performance
*Better Memory Usage
*Better Memory Usage
====Performance State====
 
*[http://people.mozilla.org/~johnath/fnWe think we are within acceptable beta quality see state of Fennec Perf Section for more details]
====Status====
*Latest numbers highlighted in weekly status
*Perception of start-up time is within an acceptable range: continued optimization is planned<br>
“This native version feels ten times faster than the XUL-based version. Starts up very quickly on my old Nexus S”<br>
“Better than Chrome Beta”<br>
“Makes my work much easier”<br>
“Downloading is very stable”<br>
“Love the way it syncs tabs and bookmarks with my laptop”<br>
 
*Canvas and Content Rendering
Eideticker Measures What the Users See<br>
http://wrla.ch/blog/2011/11/measuring-what-the-user-sees/<br>
 
CNN.com performance (lower checkerboarding than Chrome, smoother<br>
panning/zooming than any other browser):<br>
 
http://wrla.ch/eideticker/benchmarks/metric.html?data=cnn-jun12.json#/checkerboard<br>
http://wrla.ch/eideticker/benchmarks/metric.html?data=cnn-jun12.json#/fps<br>
 
Clock benchmark (better canvas framerate than any other browser):<br>
 
http://wrla.ch/eideticker/benchmarks/metric.html?data=clock-jun13.json#/fps<br>
 
*[http://people.mozilla.org/~johnath/fn Beta Sign Off]<br>
{{Bug|723361}} - Benchmarks for native UI launch <br>


=== l10n ===
=== l10n ===
[https://bugzilla.mozilla.org/show_bug.cgi?id=760169 Shipping the Following Locales:]
[https://bugzilla.mozilla.org/show_bug.cgi?id=760169 Shipping the Following Locales:]
#English (en-US)
#Czech (cs)
#Czech (cs)
#Danish (da)
#Danish (da)
Line 126: Line 189:
#Russian(ru)
#Russian(ru)


All available locales, those not included above to be hosted:
=== Sync Server High Availability ===
https://l10n.mozilla.org/teams/
* Signed off by Server Ops
 
QA sign off:
 
Two bugs being tracked for possible ride-alongs before we GA. If not, they will go into 14.0.1.
#[https://bugzilla.mozilla.org/show_bug.cgi?id=765254 Sync on Android is not translated at all in Suomi (fi)]<br>
#[https://bugzilla.mozilla.org/show_bug.cgi?id=765232 Sync on Android is not translated completely in Português (Brasil)(pt-BR)]


=== Mobile Test Drivers ===  
=== Mobile Test Drivers ===  
Line 141: Line 198:
*Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
*Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
*We are *keeping* MTD testers on Aurora
*We are *keeping* MTD testers on Aurora
=== Web Compatibility ===
*Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
*We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
*High level goals for k9o are captured in {{bug|747123}}. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
*Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.

Latest revision as of 03:48, 23 June 2012

[Fennec Native] GA Readiness Review, June 20, 2012

The goal of this meeting is very simple: to be sure all x-functional teams are aligned on required activities in preparation for and on the day of GA and provide a forum for any questions and concerns.

GA Release Schedule

Moco confidential: pls. refer to weekly status email or feel free to contact elancaster directly.

GA Release Criteria

GA Go-To Market Plan

Promotion timeline: ping jfu

Web pages being updated/localized/launched on GA day

NOTE: If there are existing pages that need updating but are not the responsibility of PMM; please state below :)

Product pages:

  • mozilla.org/firefox/fx#mobile
  • mozilla.org/mobile/features
  • mozilla.org/mobile/faq
  • mozilla.org/mobile/platforms
  • mozilla.org/mobile
  • bug 755411 - "About Firefox Beta" Release Notes link is 404
  • bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page

Website promos:

  • mozilla.org
  • Tabzilla

Other Promos

Reddit IamA campaign (Day prior to GA as a primer)

Other recommended Blog Posts & Activities

  • Performance blog post (similar to something like this)
  • Eideticker blog post (William already has this drafted)

List of Benchmarks

https://intranet.mozilla.org/FN14GAbenchmarks

  • N.B.: Some of our scores are already better post-FN14. As a general rule, we promote the product we're shipping, not the future. Still, when we compile the details for PR we should include the fact that, for instance, we fail ring mark 0 in FN14, but already pass it on Aurora and Nightly. <== Johnath Speaks
  • Since performance is a large part of our story, we need to have all benchmark numbers ready to go (JS, Eideticker, Ringmark, HTML5)

Eideticker

Clock.png Cnn1.png Cnn2.png

PR Intro Plan

  • Mozilla Blog post
  • Reviewer's Program in progress
  • Friendly reporters will get a head's up
  • FAQ for inbound press questions
  • Press coverage report

Support Plan

-Positives
-Negatives
-Rating change
-Interesting comments/trending

Issues and Risks

THESE HAVE BEEN RESOLVED

Cross-Functional
<== This are in progress

  • Web site compatibility effort is ongoing
  • Need to grow add-on developer adoption for compatibility with new architecture
  • Telemetry is disabled for GA due to bug 762620 - Evidence that idle-daily can start misfiring

Success Factors

  • Ratingstrends.png

Bug Metrics and Testing

396 Release Blockers Resolved Since Beta (all inclusive)

Minimum System Requirements and Device Filtering

  • bug 764518 - Fennec Native Go Live: Be sure blocked devices are properly filtered (minimum system requirements, 7" tablets) (edit)
    • Blocking 7" tablets with any OS other than ICS due to Flash support

GA Test Coverage

  • bug 765871 Please test 10.0.5 -> 14.0 (beta 7? RC?) migration one last time Final Migration Verification in progress
  • Test cases based on each functional area will be 100% complete; this includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
  • Installation, update, and migration testing will be 100% complete
  • Google Play pre-testing will be 100% complete

Beta Build Verification Testing/Sign Offs
[5/15] https://wiki.mozilla.org/QA/Fennec/14/Beta/1
[5/18] https://wiki.mozilla.org/QA/Fennec/14/Beta/2
[5/25] https://wiki.mozilla.org/QA/Fennec/14/Beta/3
[6/01] https://wiki.mozilla.org/QA/Fennec/14/Beta/4 & https://wiki.mozilla.org/QA/Fennec/14/Beta/5
[6/8] https://wiki.mozilla.org/QA/Fennec/14/Beta/6
[6/14] https://wiki.mozilla.org/QA/Fennec/14/Beta/7#Native_Fennec_14_Beta_7_Test_Plan_.28build_3.29

Deferred Bugs Scheduled for 14.0.1

Web Compatibility

DOs/DONTs when talking about website compatibility in public

bug 668288 - [meta] Google evangelism issues in Firefox for Android
bug 759986 - [meta] Issues with facebook.com
bug 763510 - [meta] Issues with twitter.com
bug 763512 - [meta] Issues with microsoft.com

Features Summary

  • Performance
  • click-2-play Flash support
  • Readability
  • Refined UI, better tab management,smoother in-content UI
  • Sync
  • Add-On Support

Reliability

  1. As of June 19th, we are at 4 crashes per 100 ADU
  2. This is down from 22 crashes per 100 adu on May 14th
  3. Criteria for GA is no more than 5 crashes per 100 ADU
  • Crash-stats620.png

Performance

Goals

  • Fast start up time
  • UI is up and running and ready for user interaction quickly
  • Responsive browsing: page load, panning & zooming performance
  • Better Memory Usage

Status

  • Perception of start-up time is within an acceptable range: continued optimization is planned

“This native version feels ten times faster than the XUL-based version. Starts up very quickly on my old Nexus S”
“Better than Chrome Beta”
“Makes my work much easier”
“Downloading is very stable”
“Love the way it syncs tabs and bookmarks with my laptop”

  • Canvas and Content Rendering

Eideticker Measures What the Users See
http://wrla.ch/blog/2011/11/measuring-what-the-user-sees/

CNN.com performance (lower checkerboarding than Chrome, smoother
panning/zooming than any other browser):

http://wrla.ch/eideticker/benchmarks/metric.html?data=cnn-jun12.json#/checkerboard
http://wrla.ch/eideticker/benchmarks/metric.html?data=cnn-jun12.json#/fps

Clock benchmark (better canvas framerate than any other browser):

http://wrla.ch/eideticker/benchmarks/metric.html?data=clock-jun13.json#/fps

bug 723361 - Benchmarks for native UI launch

l10n

Shipping the Following Locales:

  1. English (en-US)
  2. Czech (cs)
  3. Danish (da)
  4. German (de)
  5. Spanish (es-ES)
  6. Finnish (fi)
  7. French (fr)
  8. Japanese (ja)
  9. Korean (ko)
  10. Italian (it)
  11. Norwegian bokmal (nb-NO)
  12. Dutch (nl)
  13. Polish (pl)
  14. Brazilian Portuguese (pt-BR)
  15. Portuguese (pt-PT)
  16. Russian(ru)

Sync Server High Availability

  • Signed off by Server Ops

Mobile Test Drivers

  • Number of testers: 453 (down from 500 around May 14th)
  • Number of bugs filed to date 158 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
  • Areas of focus: Web compatibility, panning and zooming, flash,
  • Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
  • We are *keeping* MTD testers on Aurora
Retrieved from "https://wiki.allizom.org/index.php?title=Fennec/NativeUI/GAReadinessReview&oldid=444730"