Releases/Firefox 3.6.4/Test Plan:OOPP Testing

OOPP Focused Testing for 3.6.4

We are going to ship OOPP to a wider audience in this branch. By default it will have three specific plugins whitelisted: Flash, Silverlight and Quicktime. These will run out of process in Windows and Linux. Out of process will not be suppported on Mac.

Plugin versions to focus on:

• Flash 10.0 r45 (current) and 10.1 (rc).
• Silverlight 3 / 4
• Quicktime 7.6.3

Run the plugins tests in the BFT:

• Plugins BFT

Focus on:

• Exploratory Testing:
  • End to end crashed-plugin UI
    • Flash 10.x and 10.1rc, Silverlight 3/4, Quicktime 7.6
  • Crashed-plugin UI elements and their properties
  • Quicktime 7.6 testing on Windows (any other version)
  • Bing toolbar (uses Silverlight)
  • Any other popular toolbars (Google, Yahoo)
  • Flashblock or a couple of popular extensions
  • One P1 locale (de, fr, pt-br, it... for example)
  • Popular sites compatibility
  • Top sites automated testing (Tomcat)

• Performance Testing:
  • Plugin Testing (Quicktime, Flash and Silerlight) in full screen mode
  • Plugin Testing - HD where appropriate or varying the resolution
  • Visual comparison with other browsers in cases where a performance lag is seen

Results

Windows 7 (juanb)

• Status: Completed (no major issues)
• Build ID:
• Plugins Tested
  • Flash 10.0 r45, 10.1 rc
  • Silverlight 3 and 4
  • Quicktime 7.6.6
• Ran the plugins section of the BFT.
• New bugs filed during testing:
  • bug 559484 - crash in [@ mozilla::ipc::RPCChannel::..]
• Issues Noted:
  • Unable to reproduce: crash on http://weblogs.mozillazine.org/marcia/archives/plugin-test.html (with Flash, Quicktime, and Java installed).
  • Unable to reproduce: Under certain conditions, with several pages with different plugins open, killing the plugin-container for Flash, and then reloading the page will kill the whole browser.
  • Investigating: Auto resume while watching a Flash video in full screen doesn't seem to refresh the video, but it will keep going. If you re-size to the original it will pick up where it left as far as video rendering. We noticed this with the network latencies we're experiencing yesterday.

Linux (juanb)

• Status: Completed (no major issues)
• Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugins tested:
  • Flash 10.0 r45, 10.1 rc
  • Novel Moonlight 2.2 (in process), Silverlight 3.0x
  • QuakeLive Game Launcher 1.0.277
• Add-ons used:
  • Yahoo Toolbar 2.1.3.20100310105313
• New bugs filed during testing:
  • bug 559703 - non reproducible crash while login in to quakelive.com
    • crash report
  • bug 559704 - reproducible crash within quakelive.com (happens in 3.6.3 too)
    • crash report

Windows XP (marcia) spot check

• Status: result
• Build ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugins
  • Shockwave Flash 10.0 r45
  • Shockwave Flash 10.1 r53
  • Silverlight Version: 3.0.50106.0
  • Quicktime Version: 7.6.6.0
• New Bugs Filed during testing
  • Bug 559375 OOPP: Difficulty reloading page when Quicktime Plugin crashes
  • [Bug 559412 Crash in @ nsPluginNativeWindow::CallSetWindow(nsCOMPtr<nsIPluginInstance>&) ]
  • Bug 559451 Crash plugin UI notification bar is not showing every time I crash
  • Bug 559456 Crash in [@ fts3ColumnMethod ]Crash in [@ fts3ColumnMethod ]
  • Bug 559690 OOPP: Crashed plugin UI occasionally shows in the wrong tab
• Existing Bugs Seen during testing
  • Bug 532565
  • Bug 532700nsNPAPIPluginInstance can outlive its nsNPAPIPlugin [@DoStopPlugin]
    • https://crash-stats.mozilla.com/report/index/bp-370d75cb-1443-4bbb-a1df-99fcd2100414 for that crash
• Issues Noted
  • I don't consistently get the yellow plugin notification bar when I crash. For example, I get it using http://www.newgrounds.com/portal/view/533001 but not using http://www.miniclip.com/games/rubble-trouble/en/. Shouldn't it be consistent? Will file a bug.
  • http://www.miniclip.com/games/forest-temple/en/ freezes the browser every time.
• Litmus Test Cases Run:
  • I ran the Plugins section of the BFT
• Video Sites Explored
  • http://www.hulu.com/
  • http://dailymotion.com
  • http://www.youtube.com/
  • http://vimeo.com/

Windows Vista (marcia) spot check

• Status: result
• Build ID: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugin Versions:
  • Shockwave Flash 10.1 r53
  • Shockwave Flash 10.0 r45
  • Silverlight Version: 3.0.50106.0
  • Quicktime Version: 7.6.6.0
• New Bugs Filed
  • Bug 559433 OOOP: Runtime error when killing quicktime process
  • Bug 559680 Crash in [@ NPSWF32.dll@0x1619ae ] while flash video is reloading
• Issues noted:
  • I get freezes and hangs when browsing sites Using Adobe Shockwave Version: 11.5.6.606
    • Example http://www.shockwave.com/
  • Crashes using r45
    • Firefox 3.6.4 Crash Report [@ @0x733d763f ] ->http://crash-stats.mozilla.com/report/index/e703e5e7-2a47-4494-aee7-993d62100415
  • Error: uncaught exception: [Exception... "Node was not found" code: "8" nsresult: "0x80530008 (NS_ERROR_DOM_NOT_FOUND_ERR)" location: "http://trailers.apple.com/trailers/global/scripts/lib/overlay_panel.js Line: 427"]

Mac OS X (marcia)

• Status: result
• BuildID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugin Versions tested
  • Shockwave Flash 10.1 r53
• Bugs Seen
  • Bug 510846 Flash Player settings/preferences don't receive clicks/mouse events on certain sites
    • Highly annoying since sites that require webcam access such as tokbox require accessing settings.

• Status: result
• BuildID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugin Versions tested
  • Shockwave Flash 10.0 r45
  • Silverlight 4.0.50401.0
• Issues noted:
  • http://crash-stats.mozilla.com/report/index/734b1340-7c8b-46e6-9f76-860bb2100416 after installing updated version of Silverlight
• Bugs Seen