Fennec/NativeUI/BetaReadinessReview

May 14, 2012

Deferred Bug Metrics

1. Bugs scheduled for Beta2
2. Notable bugs in known issues

Features Summary

1. click-2-play Flash support

Test Coverage

1. Test cases based on each functional area 100% complete
2. This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
3. Installation, update, and migration testing 100% complete
4. Google Play pre-testing 100% complete

Reliability

1. As of May 14th, we are at 28 crashes per 100 ADU
2. Criteria initially defined was no more than 10 crashes per 100 ADU
3. We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes

[Clint, Wlach, Geoff, Erin] Performance

1. List the performance criteria defined for the project
2. State results of performance testing (measured against the criteria)

Mobile Test Drivers

1. Number of testers: 500
2. Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
3. Areas of focus: Web compatibility, panning and zooming, flash,
4. Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
5. We are *keeping* MTD testers on Aurora

[AaronMT, Lawrence] Web Compatibility

1. Where we are today
   Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
2. Where we want to be when we ship
   We are working on top sites. Goals are aligned around k9o. Haven't received specific guidance from product about Fennec 1.0 launch.
3. If goals can't be obtained when we ship in June, provide high level for k9o
   High level goals for k9o are captured in bug 747123. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.

Release Plan

Friday- May 11

[Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure
[Aki] 13 beta 2 XUL build is kicked off for tablets
[QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions
[Erin, Alex] Draft of release notes available for review
[Erin] Status and timeline released

Monday - May 14

AM
[QA] Complete 13 beta 2 XUL for tablet verification and sign-off: https://wiki.mozilla.org/QA/Fennec/Milestones/13.0/Beta/Beta_2/Build_2
[Alex] Formal in-person beta release sign-off confirming all x-functional teams have tasks complete (mobile dev, sync, QA, releng, marketing, michelle luna)
[All] Submit blog posts for review to communications@mozilla.com

PM
[Alex] Final release notes on staging
[Jaclyn] All marketing web site changes on staging
[Michelle] All support materials complete

Tuesday- May 15

AM
[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)

Weekly Beta Release Schedule

•Go-to build every Tuesday, at the latest on Wed afternoons
•Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree

Marketing Distribution Plan

Websites being updated/localized/launched on May 15th

• hero image update
• [1] Google Play Store content update
• FAQ for versions and l10n plan

List text (general location is fine) that is being updated and localized)

Other Promos

• Facebook ads for messaging testing
• Beta newsletter
• Beta channel Snippets
• Core Mozillians communication
• Reddit IamA campaign

PR Intro Plan

• 5/15 Introduction date
  
• "Future of Firefox" blog post
  
• Friendly reporters have been given a head's up
  
• FAQ for inbound press questions
  

Support Plan

• Release notes
  
• en-us support docs complete for Google Play launch; l10n ready
• New Firefox 14 for Mobile Support Site; 60 new articles and new tutorials
• firefoxforandroid@mozilla.com: one-time push notification about Flash
• army of awesome will monitor social sites and direct problem solving on sumo;
• Keep feedback flowing into Google Play for best support; monitoring throughout the day
• Daily reports for the first week or so, depending upon traffic:

-Positives
-Negatives
-Rating change
-Interesting comments/trending

Issues and Risks

• en-us only build for beta 1
• Web site compatibility effort is ongoing
• Crash rate (although trending in the direction)