Releases/Firefox 3.6.4/Test Plan:OOPP Testing

OOPP Focused Testing for 3.6.4

We are going to ship OOPP to a wider audience in this branch. By default it will have three specific plugins whitelisted: Flash, Silverlight and Quicktime. These will run out of process in Windows and Linux. Out of process will not be suppported on Mac.

Plugin versions to focus on:

• Flash 10.0 r45 (current) and 10.1 (rc).
• Silverlight 3 / 4
• Quicktime 7.6.3

Run the plugins tests in the BFT:

• Plugins BFT

Focus on:

• Exploratory Testing:
  • End to end crashed-plugin UI
    • Flash 10.x and 10.1rc, Silverlight 3/4, Quicktime 7.6
  • Crashed-plugin UI elements and their properties
  • Quicktime 7.6 testing on Windows (any other version)
  • Bing toolbar (uses Silverlight)
  • Any other popular toolbars (Google, Yahoo)
  • Flashblock or a couple of popular extensions
  • One P1 locale (de, fr, pt-br, it... for example)
  • Popular sites compatibility
  • Top sites automated testing (Tomcat)

• Performance Testing:
  • Plugin Testing (Quicktime, Flash and Silerlight) in full screen mode
  • Plugin Testing - HD where appropriate or varying the resolution
  • Visual comparison with other browsers in cases where a performance lag is seen

Results

Windows 7 (juanb)

• Try OOPP: Flash 10.0.(latest), Flash 10.1rc, Silverlight 3 and 4, Quicktime
• Try non-OOPP: Quakelive, Java
• Status: RESULT
• Build ID:
• Plugins Tested
  • Flash
  • Silverlight
  • Quicktime
• Ran the plugins section of the BFT.
• New bugs filed during testing:
• Issues Noted:
  • Investigating: crash on http://weblogs.mozillazine.org/marcia/archives/plugin-test.html (with Flash, Quicktime, and Java installed).
  • Investigating: Under certain conditions, with several pages with different plugins open, killing the plugin-container for Flash, and then reloading the page will kill the whole browser.

Linux (juanb)

• Try OOPP: Flash 10.0.(latest), Flash 10.1rc
• Try non-OOPP: Quakelive, Java, Silverlight 3 (Moonlight)

Windows XP (marcia) spot check

• Status: result
• Build ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugins
  • Shockwave Flash 10.0 r45
  • Shockwave Flash 10.1 r53
  • Silverlight Version: 3.0.50106.0
  • Quicktime Version: 7.6.6.0
• New Bugs Filed during testing
  • Bug 559375 OOPP: Difficulty reloading page when Quicktime Plugin crashes
  • [Bug 559412 Crash in @ nsPluginNativeWindow::CallSetWindow(nsCOMPtr<nsIPluginInstance>&) ]
  • Bug 559451 Crash plugin UI notification bar is not showing every time I crash
  • Bug 559456 Crash in [@ fts3ColumnMethod ]Crash in [@ fts3ColumnMethod ]
• Existing Bugs Seen during testing
  • Bug 532565
  • Bug 532700nsNPAPIPluginInstance can outlive its nsNPAPIPlugin [@DoStopPlugin]
    • https://crash-stats.mozilla.com/report/index/bp-370d75cb-1443-4bbb-a1df-99fcd2100414 for that crash
• Issues Noted
  • I don't consistently get the yellow plugin notification bar when I crash. For example, I get it using http://www.newgrounds.com/portal/view/533001 but not using http://www.miniclip.com/games/rubble-trouble/en/. Shouldn't it be consistent? Will file a bug.
• Litmus Test Cases Run:
  • I ran the Plugins section of the BFT

Windows Vista (marcia) spot check

• Build ID: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
• Plugin Versions:
  • Shockwave Flash 10.1 r53
  • Silverlight Version: 3.0.50106.0
  • Quicktime Version: 7.6.6.0
• New Bugs Filed
  • Bug 559433 OOOP: Runtime error when killing quicktime process
• Issues noted:
  • I get freezes and hangs when browsing sites Using Adobe Shockwave Version: 11.5.6.606
    • Example http://www.shockwave.com/

Mac OS X (marcia)

• Try a number of plugins. These will run in-process.