[Fennec Native] Beta Readiness Review
May 14, 2012
Beta Release Plan
Friday- May 11
[Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure
[Aki] 13 beta 2 XUL build is kicked off for tablets
[QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions
[Erin, Alex] Draft of release notes available for review
[Erin] Status and timeline released
Monday - May 14
AM
√ [QA] Complete 13 beta 2 XUL for tablet verification and sign-off
√ [Alex] Formal in-person beta release sign-off confirming all x-functional teams have tasks complete
[All] Submit (optional) blog posts for review to communications@mozilla.com
PM
√ [Alex] Final release notes on staging
√ [Jaclyn] All marketing web site changes on staging
√ [Michelle] All support materials complete
Tuesday- May 15
AM
[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)
Weekly Beta Release Schedule
[05/18] Beta 2
[05/25] Beta 3
[06/01] Beta 4
[06/08] Beta 5
[06/15] Beta 6
[06/22] Beta 7
[06/29] GA, LAST DAY OF Q2
- Go-to build every Tuesday, at the latest on Wed afternoons
- Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree
Marketing Distribution Plan
Websites being updated/localized/launched on May 15th
- hero image update
- [1] Google Play Store content update
- FAQ for versions and l10n plan
List text (general location is fine) that is being updated and localized)
Other Promos
- Facebook ads for messaging testing
- Beta newsletter
- Beta channel Snippets
- Core Mozillians communication
- Reddit IamA campaign
PR Intro Plan
- 5/15 Introduction date
- "Future of Firefox" blog post
- Friendly reporters have been given a head's up
- FAQ for inbound press questions
Support Plan
- Release notes
- Minimum System Requirements
- en-us support docs complete for Google Play launch; l10n ready
- New Firefox 14 for Mobile Support Site; 60 new articles and new tutorials
- firefoxforandroid@mozilla.com: one-time push notification about Flash
- army of awesome will monitor social sites and direct problem solving on sumo;
- Keep feedback flowing into Google Play for best support; monitoring throughout the day
- Daily reports for the first week or so, depending upon traffic:
-Positives
-Negatives
-Rating change
-Interesting comments/trending
Issues and Risks
• en-us only build for beta 1
• Web site compatibility effort is ongoing
• Crash rate (although trending in the direction)
Deferred Bug Metrics
- Bugs scheduled for Beta2
- Notable bugs in known issues
Features Summary
- click-2-play Flash support
Test Coverage
- Test cases based on each functional area 100% complete
- This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
- Installation, update, and migration testing 100% complete
- Google Play pre-testing 100% complete
Reliability
- As of May 14th, we are at 22 crashes per 100 ADU
- Criteria initially defined was no more than 10 crashes per 100 ADU
- We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes
Performance
- Fast start up time
- Responsive browsing: page load, panning & zooming performance
Mobile Test Drivers
- Number of testers: 500
- Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
- Areas of focus: Web compatibility, panning and zooming, flash,
- Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
- We are *keeping* MTD testers on Aurora
Web Compatibility
- Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
- We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
- High level goals for k9o are captured in bug 747123. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
- Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.