|
|
| (22 intermediate revisions by the same user not shown) |
| Line 6: |
Line 6: |
| I was introduced to open source (and Mozilla) in 2006 by David Humphrey, a professor at Seneca College in Toronto, Ontario, Canada. After volunteering for a few years, I transitioned to being a paid contributor in 2010. Starting out as an add-on reviewer I became involved with QA (quality assurance), testing Firefox features, driving Firefox releases, developing strategies, and running test events for the community. I lead a QA strategy to ship the first version of Firefox Hello (WebRTC based calling in the browser) and developed experiments to test GPU Process (a foundational piece of the Quantum architecture). | | I was introduced to open source (and Mozilla) in 2006 by David Humphrey, a professor at Seneca College in Toronto, Ontario, Canada. After volunteering for a few years, I transitioned to being a paid contributor in 2010. Starting out as an add-on reviewer I became involved with QA (quality assurance), testing Firefox features, driving Firefox releases, developing strategies, and running test events for the community. I lead a QA strategy to ship the first version of Firefox Hello (WebRTC based calling in the browser) and developed experiments to test GPU Process (a foundational piece of the Quantum architecture). |
|
| |
|
| I am currently a member of the Product Integrity group working on improvements in automation coverage and transitioning to a more robust tool for managing incoming requests. | | I am currently a member of the Product Integrity group working on improvements in automation coverage. |
|
| |
|
| When I'm not trying to make Firefox better, I'm usually hiking up some mountain or just traveling somewhere new, always with my camera in tow. | | When I'm not working on Firefox I volunteer with the Green Party of Canada, Green Party of British Columbia, and Green Party of Vancouver. |
| | |
| | In my spare time I'm usually hiking up some mountain or just traveling somewhere new with my partner. |
|
| |
|
| = Contact Info = | | = Contact Info = |
| Line 17: |
Line 19: |
| * Availability: I'm typically online between 09:00 and 17:00 PDT/PST (UTC-8/UTC-9) | | * Availability: I'm typically online between 09:00 and 17:00 PDT/PST (UTC-8/UTC-9) |
|
| |
|
| = Contributions = | | = Work Plan = |
| == 2018 ==
| | '''Legend''': ☑ Completed task | ☐ Upcoming task |
| === PI Requests v2 ===
| | {| class="wikitable mw-collapsible" style="overflow:auto; width:800px" |
| ; OKR: Drive creation of a proof-of-concept tool to replace the mailing list / spreadsheet ([https://wiki.mozilla.org/User:Ashughes/PI_Requests_v2 wiki])
| | ! style="text-align:left" | <big>KEY DELIVERABLES</big> |
| | | |- |
| ; Tasks
| | | |
| : ☑ Conduct user interviews
| | ; Professional Development |
| : ☑ Document requirements of the ideal system based on user feedback
| | : ☐ Read ''Peopleware'' |
| : ☑ Fix low-hanging fruit within the current system
| | : ☐ Read ''Managing Humans'' |
| : ☑ Draft a workflow design encompassing the minimum viable end-to-end workflow
| | ; Mozilla IRIS |
| : ☑ Get feedback on workflow design from key stakeholders (Softvision, PI Management, Service Now)
| | : ☑ Vet the installation process for Mozilla IRIS |
| : ☑ Revise MVP workflow design
| | : ☑ Assist in recruiting an Outreachy intern to improve the control panel UI |
| : ⇒ Discuss implementation of MVP workflow design in Service Now ''-- Pending return of Sean Rich from PTO (2018-04-03)''
| | : ☐ Develop and execute a deployment strategy |
| : ☐ Review implementation of MVP workflow design in Service Now
| | : ☐ Develop and execute a communications strategy |
| : ☐ Test implementation of MVP workflow design in Service Now with real users
| | : ☐ Develop and execute a community participation strategy |
| | | : ☐ Develop a plan to deploy Mozilla IRIS to at least one team outside the Product Integrity group |
| === PI Automation ===
| | : ☐ Set up a mini deployment at the YVR office |
| ; <big>''OKR: Lead project teams in developing and maintaining automated tests''</big>
| | ; Carry Over |
| | | : ☑ Transition Alin Rus from WebRender benchmarking to IRIS test development |
| What are the metrics for this project?
| | : ☑ Transition Roxana Robotin from WebRTC test maintenance to IRIS test development |
| * number of defects found by automation?
| | |} |
| * time spent on implementation of a specific test?
| |
| * time spent on maintenance of a specific test?
| |
| * time saved vs manual testing?
| |
| * key deliverable such as a dashboard or audit?
| |
| | |
| ===== WebRTC =====
| |
| * '''Owners''': Roxana Robotin (reporting to Jan-Yvar)
| |
| * '''Code''': [https://hg.mozilla.org/mozilla-central mozilla-central]
| |
| * '''Tracking''': [https://docs.google.com/spreadsheets/d/1bi89fKZ8rGkHxi_4aZwz1ZnxfUYQa7oODsTR_WmC61s/edit#gid=1282602213 spreadsheet]
| |
| * '''Dashboard''': ?
| |
| * '''Status''':
| |
| ** '''2018-04-03''': 6 Open, 0 Resolved, 0% Complete
| |
| | |
| <bugzilla>{"blocks":"1405876"}</bugzilla>
| |
| | |
| ===== Firefox UI Tests =====
| |
| {|
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Owners</u>:
| |
| | ?
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Code</u>:
| |
| | https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/tests/
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Dashboard</u>:
| |
| | ''<link to results dashboard>''
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Documentation</u>:
| |
| | Matt Howell [https://docs.google.com/document/d/1KFtZgd2zbolo7G_7lq8Nle8gnOoUr21l4r-vdwiS-G0/edit?ts=5aa2df6b explains the reason] the Update tests are disabled. Functional and Puppeteer tests need to be audited.<br />
| |
| <u>Local Testrun - Linux x86_64 - 2018-03-21</u>:
| |
| <pre>
| |
| SUMMARY
| |
| -------
| |
| 7:22.40 INFO passed: 86
| |
| 7:22.40 INFO failed: 1
| |
| 7:22.40 INFO todo: 3 (skipped: 1)
| |
| 7:22.40 INFO
| |
| FAILED TESTS
| |
| -------
| |
| 7:22.40 INFO test_initial_download.py test_initial_download.TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download
| |
| 7:22.40 SUITE_END
| |
| </pre>
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Next Steps</u>:
| |
| | ☐ Set up a local environment to run the tests locally and see what happens<br />☐ Analyze the result of the tests
| |
| |}
| |
| | |
| ===== Benchmarking =====
| |
| {| | |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Owners</u>:
| |
| | Alin Rus (reporting to Milan)
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Code</u>:
| |
| | https://github.com/Top5Softvision/Top5_Benchmark
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Dashboard</u>:
| |
| | https://docs.google.com/spreadsheets/d/1LxEcm8Fz1tRU9s9HhIOlHdktO_a5UEEtw6bemfjq0h4/edit?usp=sharing
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Documentation</u>:
| |
| | https://docs.google.com/document/d/1UT8GTcVUQDYqCySNNhSoRBnIOiJa7TjN0GAkDyFLEFA/edit
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Next Steps</u>:
| |
| | ☐ Finish writing tests against Youtube<br />☐ Begin work on automated dashboard
| |
| |}
| |
| | |
| ===== Metrics-Graphics-GFX =====
| |
| {|
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Owners</u>:
| |
| |
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Code</u>:
| |
| |
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Dashboard</u>:
| |
| |
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Documentation</u>:
| |
| |
| |
| |-
| |
| | style="text-align:right; vertical-align:top" | <u>Next Steps</u>:
| |
| | ☐ Anthony to review project status with Milan<br />
| |
| |}
| |
| | |
| == 2017 ==
| |
| === 2017 Q1 ===
| |
| * ☐ something to do
| |
| * ☑ something done
| |
| * ✓ something done
| |
| * ✔ something done
| |
| | |
| ☐ something to do <br/>
| |
| ☑ something done <br/>
| |
| ✓ something done <br/>
| |
| ✔ something done <br/>
| |
| | |
| == 2016 ==
| |
| === 2016 Q4 ===
| |
| {|
| |
| |-
| |
| | '''1. Prototype a dashboard of long-term Graphics stability trends'''
| |
| * {{done|}} architect a system to automate updating chart data periodically beyond the 6-month Socorro API limit
| |
| ** {{done|}} create a python script to clone crash data from Socorro
| |
| ** {{done|}} set up python script on a 24-hour cron job
| |
| * {{done|}} prototype a series of long-term charts to track:
| |
| ** {{done|}} WebGL success rate - rpi3/#psd-webgl
| |
| ** {{done|}} WebGL crash rate - rpi3/#psd-webgl-rate
| |
| ** {{done|}} Shutdown crashes - rpi3/#psd-shutdown
| |
| ** {{done|}} Startup crashes - rpi3/#psd-startup
| |
| ** {{done|}} Driver crashes - rpi3/#psd-driver-rate
| |
| ** {{prev|}} High-volume crashes
| |
| ** {{done|}} Overall Graphics crash rate - rpi3/#psd-gfx-rate
| |
| * {{done|}} prototype a dashboard to display all long-term graphs - See <server>/index.htm#psd-dashboard
| |
| * {{done|}} [add'l] Move infrastructure to a Mac Mini behind VPN
| |
| * {{prev|}} host the dashboard on metrics.mozilla.com/gfx (if possible)
| |
| | |
| |-
| |
| | '''2. Prototype a small-scale, automated emunittest lab to improve discoverability of WebGL regressions in Nightly'''
| |
| * {{done|}} set up a machine to be used for testing on a daily basis
| |
| * {{done|}} get a script from Jukka Jylänki's to automate publishing test results to his local server in Finland
| |
| * {{done|}} test the script on a dummy rig at my desk in Vancouver
| |
| * {{done|}} let the machine run for several days to capture long-term results - See [http://clbri.com:6932/results?uuid=9f09167b-a166-4402-a20d-7536574b5d06 clbri.com]
| |
| * {{prev|}} optionally, get Jukka to create a mobile version of his test site that we can point to WebQA's Saucelabs instance
| |
| | |
| |-
| |
| | '''3. Utilize the Graphics test plan framework to validate shipping SKIA Content on Windows to Aurora in Firefox 52'''
| |
| * document the test plan framework and telemetry experiment lessons learned
| |
| * work with Mason to document milestones and release criteria
| |
| * ensure these criteria are measured and met along the way
| |
| |-
| |
| | '''4. Maintenance tasks to ensure GFX quality does not slip
| |
| * Daily bug triage via http://ashughes1.github.io/graphicstriage/
| |
| * Find a new home for One & Done tasks (testrail?)
| |
| * Execute testing via Betabreakers for new features in Aurora
| |
| |}
| |
| | |
| ==== GPU Process ====
| |
| * Develop a test plan to qualify MVP
| |
| * Deploy a Telemetry Experiment in Nightly to A/B test stability (blocked by {{bug|1323304}})
| |
| | |
| === 2016 Q3 ===
| |
| ; Tasks
| |
| * ...
| |
| | |
| ; Bugs
| |
| <bugzilla display="list">
| |
| {
| |
| "emailreporter1":"1",
| |
| "emaillongdesc1":"1",
| |
| "emailtype1":"exact",
| |
| "email1":"anthony.s.hughes@gmail.com",
| |
| "chfield":"[Bug creation]",
| |
| "chfieldfrom":"2016-07-01",
| |
| "chfieldto":"2016-09-30",
| |
| "include_fields":"id,summary,status"
| |
| }
| |
| </bugzilla>
| |
| | |
| === 2016 Q2 ===
| |
| ; Tasks
| |
| * Prototyped a crash-stats dashboard with auto-updating charts
| |
| * Conducted a talk at London all-hands ([https://docs.google.com/presentation/d/17CFTU_WWu0tWd_kHfSghaMbtsgPrcvavEytcat4O5MI/edit#slide=id.g14538819fd_2_263 presentation])
| |
| * Conducted Animometer benchmarking ([https://drive.google.com/open?id=1xo6Yfds1YFn8FAtFdQeb5R_JKMWej-g9cVZWgfK8Rso Mac OS], [https://drive.google.com/open?id=1vCkgNv_EGncd-naj_89s-2Ep9iL7I2tuzTYYhAOV0WA Windows 7])
| |
| * Created an add-on to integrate crash-stats visualization with Bugzilla ([https://addons.mozilla.org/en-US/firefox/addon/bugzilla-socorro-lens/ v0.1])
| |
| * Executed Firefox 46 testing to support {{bug|1255281}} ([[QA/Platform/Graphics/Betabreakers/Firefox_46|testplan]])
| |
| * Executed Firefox 48 testing to support {{bug|1260507}} ([[QA/Platform/Graphics/Betabreakers/Firefox_48|testplan]])
| |
| * Set up weekly triage of Top Crashes, Cold Crashes, Cold Trackers, Untriaged, and Help Wanted bugs with a dashboard to track progress
| |
| | |
| ; Bugs
| |
| <bugzilla display="list">
| |
| {
| |
| "emailreporter1":"1",
| |
| "emaillongdesc1":"1",
| |
| "emailtype1":"exact",
| |
| "email1":"anthony.s.hughes@gmail.com",
| |
| "chfield":"[Bug creation]",
| |
| "chfieldfrom":"2016-04-01",
| |
| "chfieldto":"2016-06-30",
| |
| "include_fields":"id,summary,status"
| |
| }
| |
| </bugzilla>
| |
| | |
| === 2016 Q1 ===
| |
| * Filed a bug to enable special character searches on Socorro ({{bug|1239402}})
| |
| * Prototyped a dashboard of Socorro data
| |
| * Prototyped a dashboard of Bugzilla data
| |
| * Executed a testrun to vet Developer Edition 45 for WinXP + D3D9 + E10S re bug 1237769
| |
| | |
| ; Bugs
| |
| <bugzilla display="list">
| |
| {
| |
| "emailreporter1":"1",
| |
| "emaillongdesc1":"1",
| |
| "emailtype1":"exact",
| |
| "email1":"anthony.s.hughes@gmail.com",
| |
| "chfield":"[Bug creation]",
| |
| "chfieldfrom":"2016-01-01",
| |
| "chfieldto":"2016-03-31",
| |
| "include_fields":"id,summary,status"
| |
| }
| |
| </bugzilla>
| |
| | |
| == 2015 ==
| |
| === 2015 Q4 ===
| |
| * Drafted documentation on graphics device/driver blocklisting ''[https://developer.mozilla.org/en-US/docs/Mozilla/QA/Graphics_Blocklisting]''
| |
| * Executed testruns to sanity check [[QA/Platform/Graphics/Firefox_43|Firefox 43]] and [[QA/Platform/Graphics/Firefox_44|Firefox 44]] in Aurora
| |
| * Organized Android sanity [[QA/Platform/Graphics/Android|testing]] for SKIA update (due Dec 7, 2015)
| |
| * Proposed inclusion of GPU chipset information in Socorro, tracked via {{bug|1192351}}
| |
| * Prototyped a graphics quality [https://metrics.mozilla.com/gfx/ dashboard]
| |
| * Prototyped a periodic triage workflow to close old bugs and escalate lost issues
| |
| * Prototyped a [http://ashughes1.github.io/testday-metrics/charts/index.htm dashboard] to measure engagement during testdays
| |
| | |
| ; Bugs | |
| <bugzilla display="list">
| |
| {
| |
| "emailreporter1":"1",
| |
| "emaillongdesc1":"1",
| |
| "emailtype1":"exact",
| |
| "email1":"anthony.s.hughes@gmail.com",
| |
| "chfield":"[Bug creation]",
| |
| "chfieldfrom":"2015-10-01",
| |
| "chfieldto":"2015-12-31",
| |
| "include_fields":"id,summary,status"
| |
| }
| |
| </bugzilla>
| |
| | |
| === 2015 Q3 ===
| |
| * Developed a process to create a mission statement of and for the community ([[QA/Future|result]])
| |
| * Conducted one-to-one meetings with peers and collect feedback to inform the mission statement
| |
| * Drafted a recommendation for Hossain Al Ikram to participate in 2015 Global Gatherings
| |
| * Provided advice to Jamie Charlton about planning Firefox OS testing events
| |
| * Dogfooded Windows 10 to report blocker bugs ahead of release
| |
| * Created a process for [http://bit.ly/1JZqzMc daily sanity checks] via the lab in Toronto
| |
| * Executed outsourced sanity checking for [[QA/Platform/Graphics/Firefox_41|Firefox Aurora 41]]
| |
| * Executed outsourced sanity checking for [[QA/Platform/Graphics/Firefox_42|Firefox Aurora 42]]
| |
| * Extended the sanity checking to volunteers via One & Done ([https://docs.google.com/a/mozilla.com/forms/d/1AoaAU6__rebVxOwjb8ZOM_fgZkdqk-DK1hITr5QDzFo/edit# results form])
| |
| * Documented best practices on MDN ([https://developer.mozilla.org/en-US/docs/Mozilla/QA/Helping_the_Graphics_Team artifact])
| |
| * Developed a catalog of graphics hardware to facilitate device-specific testing ([[QA/Platform/Graphics/Inventory|artifact]])
| |
| * Created an [https://docs.google.com/spreadsheets/d/12jNc8rlZP4MDk5xOHyOWlcO2W5J39KKofzvcVZxVFJk/edit#gid=0 internal hardware catalog] as a proof-of-concept
| |
| * Migrated the internal catalog to a
| |
| * Improved readability of the inventory based on peer review
| |
|
| |
|
| ; Bugs | | {| class="wikitable mw-collapsible" style="overflow:auto; width:800px" |
| <bugzilla display="list">
| | ! style="text-align:left" | <big>NOV 19 - NOV 23, 2018</big> |
| {
| | |- |
| "id":"1112371,1130195,1145143,1173579,1176147,1180561,1187464,1187466,1189266,1189940,1191060,1194933,1198765,1201679,1201960,1202696,1202700,1202748,1202786,1204337,1206626,1207326,1208579,1209617"
| | | |
| } | | : ☑ Create a communications page in the github wiki - [https://github.com/mozilla/iris/wiki/Communicating-with-the-Iris-team] |
| </bugzilla>
| | : ☐ Develop a Mozilla Iris test case - [https://github.com/mozilla/iris/issues/106 #106] |
| | : ☑ Set up a MacOS machine to run Iris tests and vet deployment process |
| | : ☐ Work with the Release QA SV team to deploy Iris for update testing Beta builds |
| | |} |
