MozillaWiki

Ashughes

Joined 1 March 2007
User page Discussion
Revision as of 22:19, 6 April 2018 by Ashughes (talk | contribs) (→‎Bookmarks)

About

 

My name is Anthony Hughes (typically go by ashughes) and I work as a quality engineer on Mozilla's DOM team from Mozilla's office in Vancouver, Canada.

I was introduced to open source (and Mozilla) in 2006 by David Humphrey, a professor at Seneca College in Toronto, Ontario, Canada. After volunteering for a few years, I transitioned to being a paid contributor in 2010. Starting out as an add-on reviewer I became involved with QA (quality assurance), testing Firefox features, driving Firefox releases, developing strategies, and running test events for the community. I lead a QA strategy to ship the first version of Firefox Hello (WebRTC based calling in the browser) and developed experiments to test GPU Process (a foundational piece of the Quantum architecture).

I am currently a member of the Product Integrity group working on improvements in automation coverage and transitioning to a more robust tool for managing incoming requests.

When I'm not trying to make Firefox better, I'm usually hiking up some mountain or just traveling somewhere new, always with my camera in tow.

Contact Info

Contributions

2018

PI Requests v2

OKR
Drive creation of a proof-of-concept tool to replace the mailing list / spreadsheet (wiki)
Tasks
☑ Conduct user interviews
☑ Document requirements of the ideal system based on user feedback
☑ Fix low-hanging fruit within the current system
☑ Draft a workflow design encompassing the minimum viable end-to-end workflow
☑ Get feedback on workflow design from key stakeholders (Softvision, PI Management, Service Now)
☑ Revise MVP workflow design
⇒ Discuss implementation of MVP workflow design in Service Now -- Pending return of Sean Rich from PTO (2018-04-03)
☐ Review implementation of MVP workflow design in Service Now
☐ Test implementation of MVP workflow design in Service Now with real users

PI Automation

OKR: Lead project teams in developing and maintaining automated tests

What are the metrics for this project?

  • number of defects found by automation?
  • time spent on implementation of a specific test?
  • time spent on maintenance of a specific test?
  • time saved vs manual testing?
  • key deliverable such as a dashboard or audit?
WebRTC
  • Owners: Roxana Robotin (reporting to Jan-Yvar)
  • Code: mozilla-central
  • Tracking: spreadsheet
  • Dashboard: ?
  • Status:
    • 2018-04-03: 6 Open, 0 Resolved, 0% Complete
Full Query
ID Summary Priority Status
1404994 Write unit tests for the *Conduits P3 NEW
1406372 Create mochitest to verify echo cancellation on getUserMedia P3 NEW
1406376 Create mochitest to verify noise suppression on getUserMedia P3 NEW
1406377 Create mochitest to verify auto gain control on getUserMedia P3 NEW
1406910 Add a mochitest for input-only audio capture P3 ASSIGNED
1412915 Create a loopback devices sink to source for automated testing on Windows P3 NEW

6 Total; 6 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Firefox UI Tests
Owners: ?
Code: https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/tests/
Dashboard: <link to results dashboard>
Documentation: Matt Howell explains the reason the Update tests are disabled. Functional and Puppeteer tests need to be audited.

Local Testrun - Linux x86_64 - 2018-03-21: 

SUMMARY
-------
 7:22.40 INFO passed: 86
 7:22.40 INFO failed: 1
 7:22.40 INFO todo: 3 (skipped: 1)
 7:22.40 INFO 
FAILED TESTS
-------
 7:22.40 INFO test_initial_download.py test_initial_download.TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download
 7:22.40 SUITE_END
Next Steps: ☐ Set up a local environment to run the tests locally and see what happens
☐ Analyze the result of the tests
Benchmarking
Owners: Alin Rus (reporting to Milan)
Code: https://github.com/Top5Softvision/Top5_Benchmark
Dashboard: https://docs.google.com/spreadsheets/d/1LxEcm8Fz1tRU9s9HhIOlHdktO_a5UEEtw6bemfjq0h4/edit?usp=sharing
Documentation: https://docs.google.com/document/d/1UT8GTcVUQDYqCySNNhSoRBnIOiJa7TjN0GAkDyFLEFA/edit
Next Steps: ☐ Finish writing tests against Youtube
☐ Begin work on automated dashboard
Metrics-Graphics-GFX
Owners:
Code:
Dashboard:
Documentation:
Next Steps: ☐ Anthony to review project status with Milan

2017

2017 Q1

  • ☐ something to do
  • ☑ something done
  • ✓ something done
  • ✔ something done

☐ something to do
☑ something done
✓ something done
✔ something done

2016

2016 Q4

1. Prototype a dashboard of long-term Graphics stability trends
  • [DONE] architect a system to automate updating chart data periodically beyond the 6-month Socorro API limit
    • [DONE] create a python script to clone crash data from Socorro
    • [DONE] set up python script on a 24-hour cron job
  • [DONE] prototype a series of long-term charts to track:
    • [DONE] WebGL success rate - rpi3/#psd-webgl
    • [DONE] WebGL crash rate - rpi3/#psd-webgl-rate
    • [DONE] Shutdown crashes - rpi3/#psd-shutdown
    • [DONE] Startup crashes - rpi3/#psd-startup
    • [DONE] Driver crashes - rpi3/#psd-driver-rate
    • [CARRY OVER] High-volume crashes
    • [DONE] Overall Graphics crash rate - rpi3/#psd-gfx-rate
  • [DONE] prototype a dashboard to display all long-term graphs - See <server>/index.htm#psd-dashboard
  • [DONE] [add'l] Move infrastructure to a Mac Mini behind VPN
  • [CARRY OVER] host the dashboard on metrics.mozilla.com/gfx (if possible)
2. Prototype a small-scale, automated emunittest lab to improve discoverability of WebGL regressions in Nightly
  • [DONE] set up a machine to be used for testing on a daily basis
  • [DONE] get a script from Jukka Jylänki's to automate publishing test results to his local server in Finland
  • [DONE] test the script on a dummy rig at my desk in Vancouver
  • [DONE] let the machine run for several days to capture long-term results - See clbri.com
  • [CARRY OVER] optionally, get Jukka to create a mobile version of his test site that we can point to WebQA's Saucelabs instance
3. Utilize the Graphics test plan framework to validate shipping SKIA Content on Windows to Aurora in Firefox 52
  • document the test plan framework and telemetry experiment lessons learned
  • work with Mason to document milestones and release criteria
  • ensure these criteria are measured and met along the way
4. Maintenance tasks to ensure GFX quality does not slip

GPU Process

  • Develop a test plan to qualify MVP
  • Deploy a Telemetry Experiment in Nightly to A/B test stability (blocked by bug 1323304)

2016 Q3

Tasks
  • ...
Bugs


2016 Q2

Tasks
  • Prototyped a crash-stats dashboard with auto-updating charts
  • Conducted a talk at London all-hands (presentation)
  • Conducted Animometer benchmarking (Mac OS, Windows 7)
  • Created an add-on to integrate crash-stats visualization with Bugzilla (v0.1)
  • Executed Firefox 46 testing to support bug 1255281 (testplan)
  • Executed Firefox 48 testing to support bug 1260507 (testplan)
  • Set up weekly triage of Top Crashes, Cold Crashes, Cold Trackers, Untriaged, and Help Wanted bugs with a dashboard to track progress
Bugs


2016 Q1

  • Filed a bug to enable special character searches on Socorro (bug 1239402)
  • Prototyped a dashboard of Socorro data
  • Prototyped a dashboard of Bugzilla data
  • Executed a testrun to vet Developer Edition 45 for WinXP + D3D9 + E10S re bug 1237769
Bugs


2015

2015 Q4

  • Drafted documentation on graphics device/driver blocklisting [1]
  • Executed testruns to sanity check Firefox 43 and Firefox 44 in Aurora
  • Organized Android sanity testing for SKIA update (due Dec 7, 2015)
  • Proposed inclusion of GPU chipset information in Socorro, tracked via bug 1192351
  • Prototyped a graphics quality dashboard
  • Prototyped a periodic triage workflow to close old bugs and escalate lost issues
  • Prototyped a dashboard to measure engagement during testdays
Bugs


2015 Q3

  • Developed a process to create a mission statement of and for the community (result)
  • Conducted one-to-one meetings with peers and collect feedback to inform the mission statement
  • Drafted a recommendation for Hossain Al Ikram to participate in 2015 Global Gatherings
  • Provided advice to Jamie Charlton about planning Firefox OS testing events
  • Dogfooded Windows 10 to report blocker bugs ahead of release
  • Created a process for daily sanity checks via the lab in Toronto
  • Executed outsourced sanity checking for Firefox Aurora 41
  • Executed outsourced sanity checking for Firefox Aurora 42
  • Extended the sanity checking to volunteers via One & Done (results form)
  • Documented best practices on MDN (artifact)
  • Developed a catalog of graphics hardware to facilitate device-specific testing (artifact)
  • Created an internal hardware catalog as a proof-of-concept
  • Migrated the internal catalog to a
  • Improved readability of the inventory based on peer review
Bugs
Retrieved from "https://wiki.allizom.org/index.php?title=User:Ashughes&oldid=1191938"