QA/Loop

From MozillaWiki
< QA
Revision as of 21:17, 7 October 2014 by Ashughes (talk | contribs) (→‎Testing)
Jump to navigation Jump to search

Summary

Loop is a project to deliver rich real-time communication to the web. This page details the strategy to ensure this project delivers on time and to a high-level of quality.

People Involved

QA Lead Anthony Hughes
Product Anthony Hughes
Tony Chung 		QA contact for the Desktop client
QA contact for the Mobile client
Platform Geo Mealer
Nils Ohlmeier
Syd Polk
Jason Smith
Martijn Wargers
Paul Yang 		Automation support for mobile performance
QA contact for WebRTC, Screensharing, and Tokbox SDK support
Automation support for WebRTC
QA contact for Privileged API support
QA contact for MSISDN support
QA contact for H264 decoding support
Services James Bonacci
Peter DeHaan
Richard Pappalardo
Edwin Wong 		QA contact for server scalability
QA contact for server APIs
QA contact for Firefox Accounts integration
QA Lead for Services support

Meetings

Meeting When Where Notes Last Meeting
Project Status 11:30h Pacific on Tuesday Shell Escalante's Vidyo room Agenda
Partner Testing Coordination 09:30h Pacific on Wednesday Shell Escalante's Vidyo Room
Tokbox Testing Coordination 14:00h Pacific on Wednesday Opentok Agenda
QA Status 13:00h Pacific on Thursday Anthony Hughes' Vidyo room Agenda 2014-08-21

Top-level Status

Milestone: Firefox 34 Beta Readiness

Component Tracking Bug QA Contact Health Details
Client
Desktop bug 971986 Anthony Hughes [ON TRACK] Tracking uplifts to Aurora for pre-Beta sign-off
Mobile bug 988276 Jason Smith Tony Chung [ON TRACK] detailed status/blockers
Platform
MSISDN Support Martijn Wargers [ON TRACK] detailed status/blockers
Performance (B2G) Geo Mealer [ON TRACK] detailed status/blockers
Privileged API Support bug 988285 Jason Smith [DONE]
Tokbox Support Nils Ohlmeier [ON TRACK] detailed status/blockers
WebRTC/H264 Support bug 1004761 Nils Ohlmeier [ON TRACK] detailed status/blockers
WebRTC Automation Syd Polk [ON TRACK] detailed status/blockers
Services
Firefox Accounts Richard Pappalardo detailed status/blockers
Server APIs Peter DeHaan detailed status/blockers
Server Scalability James Bonacci detailed status/blockers

Details

How to Get Involved

Feature Tracking Bugs

Full Query
ID Summary Priority Status
938467 gUM permission grant scenario - remember my choice support P1 RESOLVED
945111 [B2G getUserMedia] Show audio-capture and video-capture in the settings app -- RESOLVED
978660 Setting "GRANT" for audio-capture or video-capture permissions is equivalent to DENY -- RESOLVED
979716 webrtc audio connection uses high cpu -- RESOLVED
988212 Change the attention screen permission from |certified| to |privileged| -- RESOLVED
988392 Allow Loop to be added to the contact details -- RESOLVED
988469 MSISDN verification API for privileged apps -- RESOLVED
989945 [B2G][WebRTC] long video lag when using H.264 codec. P2 RESOLVED
991011 Perform security testing on Gaia attention screen and audio changes -- RESOLVED
1000782 Loop Server — Implements HAWK for anonymous user session -- VERIFIED
1000789 Implement a PoC of an MSISDN verification service -- RESOLVED
1003330 Mobile identity UI -- RESOLVED
1003712 [B2G][WebRTC] Fall back to VP8 when H.264 HW codec is in use. -- RESOLVED
1006380 Set phone in PHONE_STATE_IN_COMMUNICATION audio state when telephony audio channel is in use -- RESOLVED
1007066 The permission prompt appears below the attention screen -- RESOLVED
1015085 Loop server - Implement "User Calls Other User by ID" flow P1 VERIFIED
1016277 If two different apps try to use the 'telephony' channel at the same time both apps can play audio. -- RESOLVED
1017031 navigator.mozApps.checkInstalled is not working for checking if another app is installed even if I have 'webapps-manage' permission -- RESOLVED
1017420 Add support for certified apps to get the list of apps matching an activity request -- RESOLVED
1017756 Update Loop buttons & state (color, name, visibility...) in contact details when a branding/UX decision is taken -- RESOLVED
1021550 [Phone][Ring Tone] The ring tone is muted after first call used speaker. -- RESOLVED
1021595 Set MobileID service https dev URL and force https -- RESOLVED
1021643 Make gUM permissions (audio-capture, video-capture) allow by default for certified apps P1 RESOLVED
1022008 Connect SDP negotiation to CodecSpecific structures for GMP codecs -- RESOLVED
1022181 Mobile ID Tests -- RESOLVED
1022193 Add a "forceIdSelection" to navigator.getMobileIdAssertion -- RESOLVED
1022480 permission-request screen has a bonus message about "Select a camera" (but with no ability to do so, or reason for doing so) -- VERIFIED
1022908 [B2G][Flame][Browser] Camera and microphone permissions are truncated -- VERIFIED
1022981 [B2G][Flame][Browser][Microphone] User will still hear sound from microphone stream if it is paused -- RESOLVED
1023175 AudioContext should have attribute EventHandler onmozinterruptend/begin in the webIDL interface -- RESOLVED
1023238 "Remember my choice" is not the default option pre-selected in GUM permission -- VERIFIED
1023266 Make the Mobile ID API privileged -- RESOLVED
1023780 Ensure that Mobile ID client support the 401/110 error (invalid token) properly -- RESOLVED
1023794 Dialer][Call Screen] UI moved up when entering another apps and back to call screen -- VERIFIED
1023853 Canceling the MobileID UI flow should result in a rejection of the getMobileIdAssertion() promise -- RESOLVED
1023860 Canceling the UI flow should clean up pending content requests -- VERIFIED
1024396 Allow blocking certain permissions for privileged apps P1 RESOLVED
1025309 FxA does not fire onlogout for FMD P2 RESOLVED
1025872 WebSockets based protocol for call progress P1 VERIFIED
1025876 Loop server - Timer supervision P1 VERIFIED
1025881 Loop server - calls need to store the name of the calling user P2 VERIFIED
1025883 Loop server - calls need to store whether the calling user is going to send audio and video, or audio only P3 VERIFIED
1025895 Loop server - Allow URL associated data to be modified via PUT /call-url/{token} P3 VERIFIED
1026072 xpconnect wrapped JSObject: DOMIdentity.jsm:354 - TypeError: this.getContextForMM(...).RP is undefined -- RESOLVED
1026494 Loop server - Add URL info to call data P1 VERIFIED
1026549 Require implicit "mobileid" permission in the manifest and allow users to revoke the permission -- RESOLVED
1026999 Mobile ID - SIM change scenario is not working -- RESOLVED
1027172 AudioContext should dispatch call AudioChannelAgent::StopPlaying() when the destination node doesn't have any input or it's muted. -- RESOLVED
1027595 TypeError: deriveHawkCredentials is not a function after bug 1020859 -- VERIFIED
1028218 Allow apps to be offered only to certain FFOS versions -- RESOLVED
1028398 FxA will silently provide user's email to privileged apps in 2.0 -- RESOLVED
1032266 ALSA crash in AudioStream::Shutdown when seeking during playback -- RESOLVED
1032871 [Contacts] There is no way to pick a contact with filter 'webcontacts/contact' -- RESOLVED
1033238 Cannot revoke Mobile ID permission -- RESOLVED
1033318 [MSISDN Authentication][System] International prefix duplicated -- VERIFIED
1033407 Set MobileID service production URL -- RESOLVED
1036490 FxOS Loop Mobile testing bugs -- RESOLVED
1040029 Expose a "marketplace-app" like activity that returns the control to the caller P1 RESOLVED
1040278 Expose a postMessage based API to allow app management to trusted 3rd party apps. P1 RESOLVED
1042345 Problems with DNS resolution for ice configuration on B2G -- RESOLVED
1042791 H264 SDP in WebRTC has non-standard fmtp line -- RESOLVED
1045581 [MobileID] The Mobile ID flow can't be completed with a manually inserted phone number P1 VERIFIED
1046578 [Loop] In a call users cannot change volume -- VERIFIED
1046736 [MobileID] Set default country code in phone number manual selection. -- RESOLVED
1049302 B2G audio+video captures cause major audio delay (regression) -- RESOLVED
1059488 [MobileID] Country code manual selection dialog seems broken P1 VERIFIED
1064249 [MobileID][2.0] The Mobile ID flow can't be completed with a manually inserted phone number P1 VERIFIED
1086880 Mobile ID is broken with v188 RIL -- RESOLVED
1091775 network-activity.png framerate is too high (30), should be lower (~4) -- RESOLVED

69 Total; 0 Open (0%); 50 Resolved (72.46%); 19 Verified (27.54%);


Firefox 34

Health

Project QA Lead - Anthony Hughes

Client

Lead - Anthony Hughes

  • [ON TRACK] Desktop client and standalone interface (Anthony Hughes)
  • [ON TRACK] Firefox OS client (Jason Smith)

Platform

Lead - Syd Polk

  • [ON TRACK] Tokbox Integration (Nils Ohlmeier)
  • [ON TRACK] Privileged API (Jason Smith)
  • [ON TRACK] Performance Automation (Geo Mealer)
  • [ON TRACK] MSISDN Support (Martijn Wargers)
  • [ON TRACK] H.264 Support (Paul Yang)

Services

Lead - Edwin Wong (wiki)

Tracked Bugs

For each milestone:

  • review that no unresolved bugs block that milestone
  • review that all fixed bugs have been triaged for qe-verify+/-
  • review that all qe-verify+ bugs have been verified
  • review that all qe-verify+ bugs have appropriate test coverage (automated & manual)

Mobile Client (Tony Chung)

Full Query
ID Summary Status Resolution Target milestone
984239 [meta][user story] Adding H264 hardware support to WebRTC in Firefox OS RESOLVED FIXED mozilla32
988279 [Meta] Loop Client integration in Gaia functionality (Contacts) for Loop Mobile Client RESOLVED FIXED ---
988285 [meta] Changes in FirefoxOS to allow Loop Client to be developed as a privileged application RESOLVED FIXED ---
988389 Loop REST client helper RESOLVED FIXED ---
988402 Make sure that Tokbox JS library works well in Firefox OS RESOLVED FIXED ---
990546 [Loop] [App] Create basic skeleton RESOLVED FIXED ---
990579 [Loop] [App] Create push helper RESOLVED FIXED ---
1002728 [Meta] Sign-Up, Log-in, Log-out and ID Management for Loop Mobile Client RESOLVED FIXED ---
1002742 [Meta] for Communication history in Loop Mobile Client (Call Log and Shared URLs) RESOLVED FIXED ---
1002756 [Meta] for Starting Loop Calls from Loop Mobile Client RESOLVED FIXED ---
1002763 [Meta] for Opening URL Callbacks from Loop Mobile Client RESOLVED FIXED ---
1002768 [Meta] for Receiving Calls in Loop in Loop Mobile Client RESOLVED FIXED ---
1002774 [Meta] for In-Call Features in Loop Mobile Client RESOLVED FIXED ---
1002782 [Meta] for Post-Call Features in Loop Mobile Client RESOLVED FIXED ---
1010185 [Meta] Settings for Loop Mobile Client RESOLVED FIXED ---
1016423 Use Hawk for making authenticated HTTP requests and support FxA assertions RESOLVED FIXED ---
1019698 Add TB v2.2.5 lib RESOLVED FIXED 2.0 S4 (20june)
1021594 [meta] Mobile ID WebAPI RESOLVED INCOMPLETE ---
1023772 [Loop] Enable test & app shortcuts RESOLVED FIXED 2.0 S4 (20june)
1023773 [Loop] Remove 'Test' entry point RESOLVED FIXED ---
1027027 [Loop] Share by URL implemented in the final App. RESOLVED FIXED ---
1027032 [Loop] Enable MobileID authentication RESOLVED FIXED ---
1028218 Allow apps to be offered only to certain FFOS versions RESOLVED DUPLICATE ---
1028398 FxA will silently provide user's email to privileged apps in 2.0 RESOLVED FIXED 2.0 S6 (18july)
1028848 Add the attention screen to the test app RESOLVED FIXED ---
1030782 [Loop] Enable activities in Loop App. RESOLVED FIXED ---
1032380 FxOS Loop client side of bug 1025872 — Setup websockets for call setup information RESOLVED FIXED ---
1032779 Add 'callType' to POST /calls request and s/call_url/callUrl RESOLVED FIXED ---
1033418 Include new 2.2.6 OpenTok library version in Loop Mobile Client RESOLVED FIXED ---
1034544 [Loop][CallLog] ActionsLogDB Helper RESOLVED FIXED ---
1034550 [Loop][CallLog] Implement UI based on the data retrieved from the ActionsLogDB Helper RESOLVED FIXED ---
1034982 Add call progress protocol bits to the FxOS Loop client app (follow-up 1032380) RESOLVED FIXED ---
1036490 FxOS Loop Mobile testing bugs RESOLVED WONTFIX ---
1036860 FxOS Loop Client needs to send channel in call setup messages RESOLVED WONTFIX ---
1037022 [Loop] Add 'webview' screen to load external links RESOLVED FIXED ---
1037368 [Loop] Logger RESOLVED WONTFIX ---
1037391 Loop cannot be installed via make reset-gaia/install-gaia RESOLVED FIXED 2.0 S6 (18july)
1038224 [Loop][Settings] Enable video on/off by default in outgoing calls RESOLVED FIXED ---
1038668 [Loop] Add settings in the call screen RESOLVED FIXED ---
1038827 Contacts helper returns a wrong contact RESOLVED FIXED ---
1039168 FxOS Loop client app needs to reject incoming calls with a reason of "busy" if already on a call RESOLVED FIXED ---
1039971 Add commit hash information to the setting panel RESOLVED FIXED ---
1040204 Loop - implement a "loop-call" activity that accepts URL tokens. RESOLVED FIXED ---
1041965 [Loop] We should be able to recover from an invalid token situation RESOLVED FIXED ---
1042568 Loop - Update BB to latest release RESOLVED FIXED ---
1042574 Loop - Add components using Bower RESOLVED FIXED ---
1043861 Follow-up to bug 1038749: Release AudioContext/audio resources RESOLVED FIXED ---
1046170 Eliminate the permissions settings in the Wizard and the example of a Loop call RESOLVED FIXED ---
1046736 [MobileID] Set default country code in phone number manual selection. RESOLVED FIXED 2.1 S1 (1aug)
1046796 Add more information about the call progress in the call screen RESOLVED FIXED ---
1047245 Loop - Apply visuals to Wizard screen RESOLVED FIXED ---
1048928 [Loop] Implement visual design in Share screen RESOLVED FIXED ---
1050336 [Loop][Optimization] Implement infinite scrolling in the call log RESOLVED FIXED ---
1053699 Call screen. Polish markup RESOLVED FIXED ---
1054309 [Loop][Regression] Can't share URL anymore VERIFIED FIXED ---
1054962 Need to add a tone player helper RESOLVED FIXED ---
1055003 [Loop][l10n] Apply l10n strings to Loop. RESOLVED FIXED ---
1056910 [Loop] Add icons to the Manifest & Splash screen RESOLVED FIXED ---
1057263 [Loop] Force library to use the 100% of the volume available RESOLVED FIXED ---
1060389 [Meta] Mobile Loop Client reporting RESOLVED FIXED ---
1060751 Play both dialing and ringback tone (follow-up bug 1054962) RESOLVED FIXED ---
1060757 Notify user when remote party not reachable VERIFIED FIXED ---
1062883 [meta]Partner cherry pick for 2.0 Loop bugs RESOLVED FIXED ---
1064429 [Loop] Performance logger RESOLVED FIXED ---
1064741 [Loop] replace web components with BB RESOLVED FIXED ---
1066532 Update OpenTok library to 2.2.9 in Loop Mobile client RESOLVED FIXED ---
1067614 [User Story] Ensure users are always on the latest loop mobile version RESOLVED FIXED ---
1068661 [Loop] Entering an invalid FxA password and then re-entering the app makes it impossible to log in via FxA RESOLVED DUPLICATE ---
1069227 [Loop] Make sure that we recover from an expired token situation RESOLVED FIXED ---
1069404 Mobile Loop application new sounds RESOLVED FIXED ---
1069500 [Loop][Regression] Can't receive calls with 2.2 VERIFIED FIXED 2.1 S7 (24Oct)
1080889 [meta] ensure LOOP displays in Marketplace only to FxOS devices with matching device capabilities RESOLVED FIXED ---
1081060 [Loop] change icon to answer a call in loop RESOLVED FIXED ---
1082517 [Loop] Can't make outgoing calls on 2.1 or later VERIFIED FIXED 2.1 S9 (21Nov)
1083096 [Loop][UI] Change local video asset of 'shadow' VERIFIED FIXED ---
1083138 [Meta] Localization work for Loop app version 1.1 RESOLVED FIXED ---
1083184 [Loop] Global debug flag RESOLVED FIXED ---
1085179 [Meta] Market Place work for Loop app version 1.1 RESOLVED FIXED ---
1085300 [Loop][Testing] Basic test infrastructure with grunt RESOLVED FIXED ---
1085485 [Loop][Build] Add grunt tasks to generate release and push to the device RESOLVED FIXED ---
1087797 [User Story] Ensure that 1.1 Loop Mobile application version can only be installed in Fire E devices with (at least) 2.0 FxOS version RESOLVED FIXED ---
1090103 [Loop] FxOS Loop client needs to send channel in call setup messages RESOLVED FIXED ---
1093497 Fix Version Attributes in Loop Mobile Client VERIFIED FIXED ---
1093929 Improve grunt tasks and add additional options RESOLVED FIXED ---

84 Total; 0 Open (0%); 78 Resolved (92.86%); 6 Verified (7.14%);


Loop Server (James Bonacci)

Full Query
ID Summary Status Resolution Target milestone
972028 [meta] System needs automated integration testing RESOLVED DUPLICATE ---
984949 Loop server — Reactivate FxA assertions checking and leave hawk sessions. VERIFIED FIXED ---
1000782 Loop Server — Implements HAWK for anonymous user session VERIFIED FIXED ---
1015085 Loop server - Implement "User Calls Other User by ID" flow VERIFIED FIXED mozilla33
1024920 [meta] Loop server needs reporting RESOLVED FIXED mozilla33
1025779 Generation (at POST /call-url) and retrieval (GET /calls/{token}) of the call-url should require/provide an optional "issuer" parameter VERIFIED FIXED ---
1025869 Loop server is storing some DB keys with a dup user HMAC VERIFIED FIXED ---
1025872 WebSockets based protocol for call progress VERIFIED FIXED ---
1025881 Loop server - calls need to store the name of the calling user VERIFIED FIXED ---
1025883 Loop server - calls need to store whether the calling user is going to send audio and video, or audio only VERIFIED DUPLICATE ---
1025895 Loop server - Allow URL associated data to be modified via PUT /call-url/{token} VERIFIED FIXED ---
1026426 Handle short URLS VERIFIED FIXED mozilla33
1026494 Loop server - Add URL info to call data VERIFIED FIXED ---
1029580 Add 'callType' to POST /calls VERIFIED FIXED mozilla33
1034068 [Loop][Server] Trying to call a non-registered identity is not working as expected VERIFIED FIXED ---
1034537 [Loop][Server] Trying to call an array of identities is not working as expected VERIFIED FIXED ---
1034590 MobileID-based registration requests are not working correctly VERIFIED FIXED ---
1035980 [Server] POST /call-url should return the token inside a "callToken" property. VERIFIED FIXED ---
1035983 [Server][DEV] Dev server is returning wrong 'expireAt' date when requesting a call url VERIFIED WORKSFORME ---
1036893 Loop server needs to disable an account which matches a set of rules VERIFIED WONTFIX mozilla35
1037979 Connection Timer should be more that five seconds VERIFIED FIXED ---
1038676 Do not select SMS MO+MT flow unless there is an available charge free number RESOLVED FIXED ---
1040702 Add a fallback to websockets for the call progress protocol VERIFIED INVALID ---
1046114 Change the URL format for the webapp for calls / rooms RESOLVED FIXED ---
1068742 Change shared URL format to https://webrtc.firefox.com/<token> VERIFIED WONTFIX ---

25 Total; 0 Open (0%); 4 Resolved (16%); 21 Verified (84%);


Privileged API (Jason Smith)

Full Query
ID Summary Status Resolution Target milestone
938467 gUM permission grant scenario - remember my choice support RESOLVED FIXED 2.0 S3 (6june)
988212 Change the attention screen permission from |certified| to |privileged| RESOLVED FIXED mozilla32
988427 Allow audio-channel-ringer to be used by |privileged| apps RESOLVED FIXED 2.0 S2 (23may)
988469 MSISDN verification API for privileged apps RESOLVED FIXED mozilla32
990552 Grant |privileged| apps use the Telephony channel by requesting the audio-channel-telephony permission RESOLVED FIXED 2.0 S2 (23may)
1006380 Set phone in PHONE_STATE_IN_COMMUNICATION audio state when telephony audio channel is in use RESOLVED FIXED mozilla32
1016277 If two different apps try to use the 'telephony' channel at the same time both apps can play audio. RESOLVED FIXED mozilla33
1017031 navigator.mozApps.checkInstalled is not working for checking if another app is installed even if I have 'webapps-manage' permission RESOLVED WONTFIX ---
1021643 Make gUM permissions (audio-capture, video-capture) allow by default for certified apps RESOLVED FIXED mozilla32
1024396 Allow blocking certain permissions for privileged apps RESOLVED FIXED mozilla33

10 Total; 0 Open (0%); 10 Resolved (100%); 0 Verified (0%);

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Loop&oldid=1023214"