Auto-tools/Projects/Platform Quality/External Media Tests: Difference between revisions
< Auto-tools | Projects | Platform Quality
Jump to navigation
Jump to search
m (Sydpolk moved page Auto-tools/Projects/Platform Quality/Firefox Media Tests to Auto-tools/Projects/Platform Quality/External Media Tests: External Media Tests is a better name given the source is in mozilla-central now.) |
(Updated for current usage.) |
||
| Line 1: | Line 1: | ||
[https:// | [https://dxr.mozilla.org/mozilla-central/source/dom/media/test/external external-media-tests] is a suite of Marionette Python tests for media playback in Mozilla Firefox. Its aim is to test MSE (Media Source Extensions) and EME (Encrypted Media Extensions) on specific websites such as YouTube. | ||
== Results Reported to Treeherder == | == Results Reported to Treeherder == | ||
| Line 10: | Line 10: | ||
* Group Symbol: VP | * Group Symbol: VP | ||
* Jobs: | * Jobs: | ||
** b-m - "media-tests" short tests using standard video API on YouTube; equivalent to 'm' below | ** b-m - "media-tests" | ||
** b-y - "media-youtube-tests" short tests using YouTube JS API on YouTube; equivalent to 'y' below | *** short tests using standard video API on YouTube; equivalent to 'm' below | ||
** b-y - "media-youtube-tests" | |||
*** short tests using YouTube JS API on YouTube; equivalent to 'y' below | |||
All available test jobs also run in our Jenkins instance, [http://pf-jenkins.qa.mtv2.mozilla.com:8080/ pf-jenkins]. (Soon-to-move to mozmill-ci) | All available test jobs also run in our Jenkins instance, [http://pf-jenkins.qa.mtv2.mozilla.com:8080/ pf-jenkins]. (Soon-to-move to mozmill-ci) | ||
* Repos | * Repos | ||
* Platforms: OS X 10.10 opt | ** mozilla-central | ||
** mozilla-aurora | |||
* Platforms: | |||
** Mac OS X 10.10 opt | |||
** Windows 7 opt | |||
** Windows 8 opt | |||
* Group Name: VideoPuppeteer | * Group Name: VideoPuppeteer | ||
* Group Symbol: VP | * Group Symbol: VP | ||
| Line 31: | Line 38: | ||
Also take a look at the [http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/scripts/firefox_media_tests_buildbot.py mozharness script] and [http://hg.mozilla.org/mozilla-central/file/default/testing/config package requirements] | Also take a look at the [http://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/scripts/firefox_media_tests_buildbot.py mozharness script] and [http://hg.mozilla.org/mozilla-central/file/default/testing/config package requirements] | ||
== How to disable an individual failing test on pf-jenkins. == | == How to disable an individual failing test on pf-jenkins. == | ||
| Line 43: | Line 48: | ||
== Where to file bugs == | == Where to file bugs == | ||
Test failures seen on Treeherder should be filed under [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_file_loc=http%3A%2F%2F&bug_ignored=0&bug_severity=normal&bug_status=NEW&cc=mjzffr%40gmail.com&cc=spolk%40mozilla.com&cf_blocking_b2g=---&cf_blocking_fennec=---&cf_feature_b2g=---&cf_fx_iteration=---&cf_fx_points=---&cf_status_b2g_2_0=---&cf_status_b2g_2_0m=---&cf_status_b2g_2_1=---&cf_status_b2g_2_1_s=---&cf_status_b2g_2_2=---&cf_status_b2g_2_2r=---&cf_status_b2g_master=---&cf_status_firefox39=---&cf_status_firefox40=---&cf_status_firefox41=---&cf_status_firefox42=---&cf_status_firefox_esr31=---&cf_status_firefox_esr38=---&cf_tracking_b2g=---&cf_tracking_e10s=---&cf_tracking_firefox39=---&cf_tracking_firefox40=---&cf_tracking_firefox41=---&cf_tracking_firefox42=---&cf_tracking_firefox_esr31=---&cf_tracking_firefox_esr38=---&cf_tracking_firefox_relnote=---&cf_tracking_p11=---&cf_tracking_relnote_b2g=---&component=Audio%2FVideo%3A%20Playback&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-203=X&flag_type-37=X&flag_type-4=X&flag_type-41=X&flag_type-5=X&flag_type-607=X&flag_type-720=X&flag_type-721=X&flag_type-737=X&flag_type-781=X&flag_type-787=X&flag_type-791=X&flag_type-799=X&flag_type-800=X&flag_type-835=X&flag_type-846=X&flag_type-855=X&flag_type-856=X&flag_type-857=X&flag_type-863=X&flag_type-864=X&flag_type-870=X&flag_type-875=X&flag_type-889=X&flag_type-892=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Core&rep_platform=Unspecified&target_milestone=---&version=unspecified Core::Audio/Video: Playback] | Test failures seen on Treeherder should be filed under [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_file_loc=http%3A%2F%2F&bug_ignored=0&bug_severity=normal&bug_status=NEW&cc=mjzffr%40gmail.com&cc=spolk%40mozilla.com&cf_blocking_b2g=---&cf_blocking_fennec=---&cf_feature_b2g=---&cf_fx_iteration=---&cf_fx_points=---&cf_status_b2g_2_0=---&cf_status_b2g_2_0m=---&cf_status_b2g_2_1=---&cf_status_b2g_2_1_s=---&cf_status_b2g_2_2=---&cf_status_b2g_2_2r=---&cf_status_b2g_master=---&cf_status_firefox39=---&cf_status_firefox40=---&cf_status_firefox41=---&cf_status_firefox42=---&cf_status_firefox_esr31=---&cf_status_firefox_esr38=---&cf_tracking_b2g=---&cf_tracking_e10s=---&cf_tracking_firefox39=---&cf_tracking_firefox40=---&cf_tracking_firefox41=---&cf_tracking_firefox42=---&cf_tracking_firefox_esr31=---&cf_tracking_firefox_esr38=---&cf_tracking_firefox_relnote=---&cf_tracking_p11=---&cf_tracking_relnote_b2g=---&component=Audio%2FVideo%3A%20Playback&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-203=X&flag_type-37=X&flag_type-4=X&flag_type-41=X&flag_type-5=X&flag_type-607=X&flag_type-720=X&flag_type-721=X&flag_type-737=X&flag_type-781=X&flag_type-787=X&flag_type-791=X&flag_type-799=X&flag_type-800=X&flag_type-835=X&flag_type-846=X&flag_type-855=X&flag_type-856=X&flag_type-857=X&flag_type-863=X&flag_type-864=X&flag_type-870=X&flag_type-875=X&flag_type-889=X&flag_type-892=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Core&rep_platform=Unspecified&target_milestone=---&version=unspecified Core::Audio/Video: Playback] | ||
== Older versions == | |||
The original repo for these tests is at [https://github.com/mjzffr/firefox-media-tests.git firefox-media-tests]. This repo is no longer supported officially, although most jobs in [https://pf-jenkins.qa.mtv2.mozilla.com:8080 pf-jenkins] still use it. That will gradually, and nothing in mozill-ci will use the github repo. | |||
Revision as of 19:41, 25 February 2016
external-media-tests is a suite of Marionette Python tests for media playback in Mozilla Firefox. Its aim is to test MSE (Media Source Extensions) and EME (Encrypted Media Extensions) on specific websites such as YouTube.
Results Reported to Treeherder
These are Tier-2 jobs, meaning they are visible by default but not sheriff-managed.
There are two short test jobs that run in buildbot
- Repos: trunk
- Platforms: Linux x64 (opt only), Windows 7, Windows 8 x64, OS X 10.10
- Group Name: VideoPuppeteer
- Group Symbol: VP
- Jobs:
- b-m - "media-tests"
- short tests using standard video API on YouTube; equivalent to 'm' below
- b-y - "media-youtube-tests"
- short tests using YouTube JS API on YouTube; equivalent to 'y' below
- b-m - "media-tests"
All available test jobs also run in our Jenkins instance, pf-jenkins. (Soon-to-move to mozmill-ci)
- Repos
- mozilla-central
- mozilla-aurora
- Platforms:
- Mac OS X 10.10 opt
- Windows 7 opt
- Windows 8 opt
- Group Name: VideoPuppeteer
- Group Symbol: VP
- Jobs:
- m - short tests using standard video API on YouTube
- y - short tests using YouTube JS API on YouTube
- ym1-60 - hour-long tests using YouTube JS API on YouTube
- e - short tests using standard video API to test EME
- e-f - full tests using standard video API to test EME
- *-e10s - job that is run with e10s enabled e.g. m-e10s, y-e10s
- *-bw - tests that runs for various bandwidths
Instructions for running locally.
Please see the project's readme
Also take a look at the mozharness script and package requirements
How to disable an individual failing test on pf-jenkins.
Visit pf-jenkins (VPN required), open the summary page of the job you want to disable and click 'Disable Project'. When a job is reported to Treeherder, its Jenkins build url and build id are listed in the Job Details pane.
Contacts
- :sydpolk (primary contact)
- :maja_zf
Where to file bugs
Test failures seen on Treeherder should be filed under Core::Audio/Video: Playback
Older versions
The original repo for these tests is at firefox-media-tests. This repo is no longer supported officially, although most jobs in pf-jenkins still use it. That will gradually, and nothing in mozill-ci will use the github repo.