User:Ashughes

From MozillaWiki
Jump to navigation Jump to search

About

Ashughes-avatar.png

My name is Anthony Hughes (typically go by ashughes) and I work as a quality engineer on Mozilla's DOM team from Mozilla's office in Vancouver, Canada.

I was introduced to open source (and Mozilla) in 2006 by David Humphrey, a professor at Seneca College in Toronto, Ontario, Canada. After volunteering for a few years, I transitioned to being a paid contributor in 2010. Starting out as an add-on reviewer I became involved with QA (quality assurance), testing Firefox features, driving Firefox releases, developing strategies, and running test events for the community. Most recently I lead a QA strategy to ship the first version of Firefox Hello (WebRTC based calling in the browser).

I currently spend most of my time developing test strategies for the DOM Graphics team and finding new ways of bringing volunteers into the fold. When I'm not trying to make Firefox better, I'm usually hiking up some mountain or just traveling somewhere new, always with my camera in tow.

Contact Info

Contributions

2018

PI Requests v2

OKR: Drive creation of a proof-of-concept tool to replace the mailing list / spreadsheet

Wiki

☑ Conduct user interviews
☑ Document requirements of the ideal system based on user feedback
☑ Fix low-hanging fruit within the current system
☑ Draft a workflow design encompassing the minimum viable end-to-end workflow
☐ Get feedback on workflow design from key stakeholders (Softvision, PI Management, Service Now)
☐ Revise MVP workflow design
☐ Discuss implementation of MVP workflow design in Service Now
☐ Review implementation of MVP workflow design in Service Now
☐ Test implementation of MVP workflow design in Service Now with real users

PI Automation

OKR: Lead project teams in developing and maintaining automated tests
Mochitests
Project Owner Source Status Notes
Activity Stream George Echim, Ioana Crisan, Paul Silaghi github [1] Github repo syncs one a week to mozilla-central
Bookmarks George Echim, Ioana Crisan, Paul Silaghi mozilla-central [2]
WebRTC Roxana Robotin mozilla-central [3]
Firefox UI Tests
Project Owner Source Status Notes
Functional DXR DISABLED
Puppeteer DXR DISABLED
Update DXR DISABLED 2018-03-12

Matt Howell explains the reason these tests are disabled.

  • they are redundant with the series of manual tests that are run on each update
  • unit tests cover the various components of updating the browser (old UI, new UI, update service, code that parses/applies patches, patch file generation scripts, server components)
  • automated tests are incapable of verifying if the browser is "usable" after an update
  • manual tests are considered preferable to the automated tests

For these reasons the manual test coverage is actually preferable to the Firefox UI update tests. If we're to enable the Firefox UI update tests again we need to drop the dependency on the wizard UI and should make it easier to run these tests locally.

Next Steps

☐ Set up a local environment to run the tests locally and see what happens
☐ Analyze the result of the tests

Benchmarking
Project Owner Source Status Notes
... ... ... ... ...
Dashboard
  • Dashboard(s) for automated tests
  • Dashboard(s) for benchmarking
  • Dashboard(s) for metrics-graphics-gfx

2017

2017 Q1

  • ☐ something to do
  • ☑ something done
  • ✓ something done
  • ✔ something done

☐ something to do
☑ something done
✓ something done
✔ something done

2016

2016 Q4

1. Prototype a dashboard of long-term Graphics stability trends
  • [DONE] architect a system to automate updating chart data periodically beyond the 6-month Socorro API limit
    • [DONE] create a python script to clone crash data from Socorro
    • [DONE] set up python script on a 24-hour cron job
  • [DONE] prototype a series of long-term charts to track:
    • [DONE] WebGL success rate - rpi3/#psd-webgl
    • [DONE] WebGL crash rate - rpi3/#psd-webgl-rate
    • [DONE] Shutdown crashes - rpi3/#psd-shutdown
    • [DONE] Startup crashes - rpi3/#psd-startup
    • [DONE] Driver crashes - rpi3/#psd-driver-rate
    • [CARRY OVER] High-volume crashes
    • [DONE] Overall Graphics crash rate - rpi3/#psd-gfx-rate
  • [DONE] prototype a dashboard to display all long-term graphs - See <server>/index.htm#psd-dashboard
  • [DONE] [add'l] Move infrastructure to a Mac Mini behind VPN
  • [CARRY OVER] host the dashboard on metrics.mozilla.com/gfx (if possible)
2. Prototype a small-scale, automated emunittest lab to improve discoverability of WebGL regressions in Nightly
  • [DONE] set up a machine to be used for testing on a daily basis
  • [DONE] get a script from Jukka Jylänki's to automate publishing test results to his local server in Finland
  • [DONE] test the script on a dummy rig at my desk in Vancouver
  • [DONE] let the machine run for several days to capture long-term results - See clbri.com
  • [CARRY OVER] optionally, get Jukka to create a mobile version of his test site that we can point to WebQA's Saucelabs instance
3. Utilize the Graphics test plan framework to validate shipping SKIA Content on Windows to Aurora in Firefox 52
  • document the test plan framework and telemetry experiment lessons learned
  • work with Mason to document milestones and release criteria
  • ensure these criteria are measured and met along the way
4. Maintenance tasks to ensure GFX quality does not slip

GPU Process

  • Develop a test plan to qualify MVP
  • Deploy a Telemetry Experiment in Nightly to A/B test stability (blocked by bug 1323304)

2016 Q3

Tasks
  • ...
Bugs


2016 Q2

Tasks
  • Prototyped a crash-stats dashboard with auto-updating charts
  • Conducted a talk at London all-hands (presentation)
  • Conducted Animometer benchmarking (Mac OS, Windows 7)
  • Created an add-on to integrate crash-stats visualization with Bugzilla (v0.1)
  • Executed Firefox 46 testing to support bug 1255281 (testplan)
  • Executed Firefox 48 testing to support bug 1260507 (testplan)
  • Set up weekly triage of Top Crashes, Cold Crashes, Cold Trackers, Untriaged, and Help Wanted bugs with a dashboard to track progress
Bugs


2016 Q1

  • Filed a bug to enable special character searches on Socorro (bug 1239402)
  • Prototyped a dashboard of Socorro data
  • Prototyped a dashboard of Bugzilla data
  • Executed a testrun to vet Developer Edition 45 for WinXP + D3D9 + E10S re bug 1237769
Bugs


2015

2015 Q4

  • Drafted documentation on graphics device/driver blocklisting [4]
  • Executed testruns to sanity check Firefox 43 and Firefox 44 in Aurora
  • Organized Android sanity testing for SKIA update (due Dec 7, 2015)
  • Proposed inclusion of GPU chipset information in Socorro, tracked via bug 1192351
  • Prototyped a graphics quality dashboard
  • Prototyped a periodic triage workflow to close old bugs and escalate lost issues
  • Prototyped a dashboard to measure engagement during testdays
Bugs


2015 Q3

  • Developed a process to create a mission statement of and for the community (result)
  • Conducted one-to-one meetings with peers and collect feedback to inform the mission statement
  • Drafted a recommendation for Hossain Al Ikram to participate in 2015 Global Gatherings
  • Provided advice to Jamie Charlton about planning Firefox OS testing events
  • Dogfooded Windows 10 to report blocker bugs ahead of release
  • Created a process for daily sanity checks via the lab in Toronto
  • Executed outsourced sanity checking for Firefox Aurora 41
  • Executed outsourced sanity checking for Firefox Aurora 42
  • Extended the sanity checking to volunteers via One & Done (results form)
  • Documented best practices on MDN (artifact)
  • Developed a catalog of graphics hardware to facilitate device-specific testing (artifact)
  • Created an internal hardware catalog as a proof-of-concept
  • Migrated the internal catalog to a
  • Improved readability of the inventory based on peer review
Bugs
Retrieved from "https://wiki.allizom.org/index.php?title=User:Ashughes&oldid=1190814"