B2G/QA/Smoketest

< B2G‎ | QA
Revision as of 23:37, 4 June 2014 by Jsmith (talk | contribs)

For each branch that is 2.0 or later:

  1. Run the smoketests that are not covered by automation manually (http://mzl.la/T4x9vi)
  2. Spend one hour exploratory testing areas called out by the day of the week
  • Monday - Comms (e.g. Messages, Usage, Dialer, Contacts, WAP Push)
  • Tuesday - Media (e.g. Camera, Gallery, YouTube, Music, FM Radio, Ringtones, Video, Wallpaper)
  • Wednesday - System (e.g. Homescreen, FTE, Cards View, Power Menu, System Update, Browser, Notifications)
  • Thursday - Productivity (e.g. Email, Calendar, Clock)
  • Friday - Misc (e.g. Bluetooth, Browse Cell Data/Wifi, FxAccounts, USB Storage, FindMyDevice, Keyboard, Settings)
  1. Formulate a smoketest report that calls out:
  • The automated smoketest results from today's run
  • The manual smoketest results from today's run

For [3], an example report would be something like the following:

 Title: 05/30/2014 Flame 2.0 Smoke Test Results - 62/64 smoketests, 2 new blockers
 Test Results
 * 40/41 automated smoketests passing
 * 22/23 manual smoketests passing
 * 0 reboots
 * 0 crashes
 New Bugs Breaking Manual Smoketests:
 - [B2G][Dialer] Inbound call icon missing in call log 
 https://bugzilla.mozilla.org/show_bug.cgi?id=1018474
 - Settings has a broken icon on the homescreen
 https://bugzilla.mozilla.org/show_bug.cgi?id=1018183
 **Note: Resolved Fixed, should be fixed in tomorrow's build
 Existing Bugs Breaking Manual Smoketests:
 - None reported
 New Bugs Breaking Automated Smoketests:
 - None reported
 Existing Bugs Breaking Automated Smoketests:
 - [v2.0] Geolocation enabled by default in FTU app
 https://bugzilla.mozilla.org/show_bug.cgi?id=1013859
 Build Information
 * Build ID: 20140530040207
 * Version: 32.0a1
 * Gecko: e6f113c83095
 * Gaia: 26d8fcab9b61f46451600f39c51e0387ef3c4f88
 * Base Image: v10G-2 
 Links
 * Daily Smoketest Logs: https://docs.google.com/a/qanalydocs.com/spreadsheet/ccc?key=0AjRc6aVFoOW9dGZLdVdiWGZLbVdORlFpUGt5XzZ3Znc&usp=drive_web#gid=0
 * MozTrap Results: https://moztrap.mozilla.org/results/cases/?filter-run=4214