Firefox/AddOns/Status/20150929: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 1: Line 1:
= Status for September 29, 2015 =
= Status for December 24th, 2015 =


== Overall Project Health - YELLOW ==  
== Overall Project Health - YELLOW ==  


* The great news is that we've now got a dedicated developer team, developer relations and community team, all in one place (Firefox) working on Add-ons for Firefox. We're yellow because these folks are hitting the ground running, already trying to catch up for the time when we were not as well resourced.  
* While we have a dedicated developer team, developer relations and community team hitting the ground running (YAY!) - we are still hiring and on-boarding new folks to focus full-time on these projects.


* Per discussion in the last month, there's concern around what the impact to end-users will be once add-on signing is required by Firefox. Consensus is that we're not quite ready to enforce it, and we're going to continue developer outreach and burning down the existing review queues to ensure we have as many addons as possible updated and signed. Add-on signing will move to enforcement in Firefox 43. 
* Signing went out Dec 15th (with option to turn off with configuration setting).  No unfixable bugs (patched) - but definitely more excitement than ideal.  We are looking into how it could have been smoother and adjusting accordingly.  Also evaluating user sentiment data from several sources: ex: twitter, SUMO, social media.  Currently targeting to make mandatory (no pref to turn off) in Firefox 47.
 
* Review-Queues:  We are working on a dashboard for better visibility into what's happening, so we know when we are making improvements.  The goal is a Dashboard with more info than just "average review time". For example - it's interesting when we looked at a snapshot of how long it takes to review 50% of the Nominated, Pending, and Preliminary add-ons.
 
The review team has been doing fantastic work keeping the unlisted queue moving, in the New Year we are going to look at the "quick hits" to improve along with this being a primary dedicated focus area.  
Time range: 2015-11-18 to 2015-11-25
 
Listed:
 
nominated:
Average waiting time: 35 days
50% of all waiting add-ons reviewed in less than 12 days
75% of all waiting add-ons reviewed in less than 66 days
90% of all waiting add-ons reviewed in less than 107 days
 
pending:
Average waiting time: 9 days
50% of all waiting add-ons reviewed in less than 1 day
75% of all waiting add-ons reviewed in less than 2 days
90% of all waiting add-ons reviewed in less than 31 days
 
preliminary:
Average waiting time: 26 days
50% of all waiting add-ons reviewed in less than 2 days
75% of all waiting add-ons reviewed in less than 38 days
90% of all waiting add-ons reviewed in less than 80 days


* The review team has been doing fantastic work keeping the unlisted queue moving, and we don't want to take our foot off the gas in reducing all the queues. We do need to have a better picture of what the release channel profiles looks like, and we're working with the metrics team to get a regular picture of what addons remain unsigned, and what the likely impact to users will be.


* We're also looking at how we can deliver a developer mode without compromising the goals of signing, as well as whether we can improve the user experience when addons are disabled to help users who lose functionality with unsigned addons. The earliest we will add this is Firefox 44.
* We're also looking at how we can deliver a developer mode without compromising the goals of signing, as well as whether we can improve the user experience when addons are disabled to help users who lose functionality with unsigned addons. The earliest we will add this is Firefox 44.

Revision as of 18:37, 24 December 2015

Status for December 24th, 2015

Overall Project Health - YELLOW

  • While we have a dedicated developer team, developer relations and community team hitting the ground running (YAY!) - we are still hiring and on-boarding new folks to focus full-time on these projects.
  • Signing went out Dec 15th (with option to turn off with configuration setting). No unfixable bugs (patched) - but definitely more excitement than ideal. We are looking into how it could have been smoother and adjusting accordingly. Also evaluating user sentiment data from several sources: ex: twitter, SUMO, social media. Currently targeting to make mandatory (no pref to turn off) in Firefox 47.
  • Review-Queues: We are working on a dashboard for better visibility into what's happening, so we know when we are making improvements. The goal is a Dashboard with more info than just "average review time". For example - it's interesting when we looked at a snapshot of how long it takes to review 50% of the Nominated, Pending, and Preliminary add-ons.

The review team has been doing fantastic work keeping the unlisted queue moving, in the New Year we are going to look at the "quick hits" to improve along with this being a primary dedicated focus area. Time range: 2015-11-18 to 2015-11-25

Listed:

nominated: Average waiting time: 35 days 50% of all waiting add-ons reviewed in less than 12 days 75% of all waiting add-ons reviewed in less than 66 days 90% of all waiting add-ons reviewed in less than 107 days

pending: Average waiting time: 9 days 50% of all waiting add-ons reviewed in less than 1 day 75% of all waiting add-ons reviewed in less than 2 days 90% of all waiting add-ons reviewed in less than 31 days

preliminary: Average waiting time: 26 days 50% of all waiting add-ons reviewed in less than 2 days 75% of all waiting add-ons reviewed in less than 38 days 90% of all waiting add-ons reviewed in less than 80 days


  • We're also looking at how we can deliver a developer mode without compromising the goals of signing, as well as whether we can improve the user experience when addons are disabled to help users who lose functionality with unsigned addons. The earliest we will add this is Firefox 44.

Exec Summary

  • Overall add-ons strategy development in progress - team met in Toronto to discuss.
  • Development team now in place led by Andy McKay, Dev Relations and Community team also now reports to Nick Nguyen
  • One of two budgeted add-on review contractors was onboarded this week, second hire in progress.
  • FirefoxOS Add-ons
  • Add-on Signing: is now live in 41 beta, but progress to release is now extended to 43.
  • e10s: e10s will ship in 42 Aurora. Add-on compatibility testing efforts continue to be maintained at [1] Earliest e10s will ship in release is 43.
  • WebExtensions: Extension API landed on Nightly. the WebExtensions Wiki has details. Development is now being supported by Andy McKay's team
  • AMO further roadmap development is now moving forward actively.
  • Full details on each work iteration (three iterations per Firefox release) can be found on our new canonical Add-ons wiki
  • Add-ons for FxOS are progressing, status can be found at the FxOS add-ons Mana page (LDAP login required)

Risks/Issues

Initiative Risks/Issues Owner Mitigation/Approach
Addon Signing Unlisted add-ons considered critical are being reviewed as soon as they are submitted (unlisted queue is under control) but many unlisted add-on developers have not (or are presumed to have not) yet submitted for review. We are at over 50% of "need to have" add-ons signed as per https://docs.google.com/spreadsheets/d/1xLnFtXLcIfIXFZodj1sFYzyDdFJ1vR3CKYNb_n-g30c/edit#gid=1874511985 but less than 25% of "nice to have". Lisa Brewster/Add-ons Review Team Reaching out to unlisted add-on devs to ensure they submit for review, focusing resources on unlisted add-on reviews, longer term we are changing how we review.
AMO Review Queue We have insufficient resources for the listed review queue which is very long and growing. We recently hired one new FTE, but also moved one reviewer over to engineering development. We do have budget for two contract reviewers, but hiring has been a challenge. Review queue challenges illustrated here: http://diox.github.io/amo-review-times/ and https://docs.google.com/spreadsheets/d/1E41MCpqf71Ayx1Im4dCspu7bxNzaMpOSYiVDQXL8dK4 Lisa/Add-on Review team Additional contract reviewers being hired now for short term mitigation, one starts Friday at 30 hrs but second hire has not yet been made. Also longer term changes to how we review are planned, SLA being established, etc.
AMO Validator the validator has several issues slowing down the queue - it is not providing very useful results to developers, and it is rejecting 4/5 add-ons. Validator team, Andy, Kev, Larissa We have plans in development, 2 Engineers assigned to improvements, and the first phase of these is nearly complete.

Accomplished for last three Weeks

  • team reorganization mostly complete - engineering team, now led by Andy McKay, and Dev/Community team, led by Lisa Brwester, now moved to firefox org and dedicating full resources to add-ons.
  • leadership team met in Toronto last week and worked on planning the future of Add-ons. the new wiki will be the place to find plans as they evolve. Iteration 44.3 is already detailed and listed out.
  • process for streamlined review signing of internal mozilla addons (experiments, system addons, lightning, etc) tlive
  • Future of Developing Add-Ons Blog Live! https://blog.mozilla.org/addons/2015/08/21/the-future-of-developing-firefox-add-ons/
  • Add-on signing shipped in 41 beta, driving considerable growth in unlisted addons submitted for review (See attached chart)and will not ride trains to release until 43 based on feedback, need for additional development and time needed for developers to get add-ons signed.
  • Social media feedback moderate, neither positive nor negative (see attached graph)
  • WebExtensions shipped in nightly
  • Slow Add-On Watcher is in early development
  • Phase one validator improvements are all done or nearly done to be live in the coming week (see: https://bugzilla.mozilla.org/buglist.cgi?cmdtype=runnamed&namedcmd=validator-p1&list_id=12471788)
  • Benchmarks for validator improvements are in progress
  • Add-on reviewer new contract hire selected, contract in progress.
  • Many unlisted addons reviewed - Andreas averaging *30 or more per day*! https://docs.google.com/spreadsheets/d/1E41MCpqf71Ayx1Im4dCspu7bxNzaMpOSYiVDQXL8dK4

Planned for Upcoming Iteration

  • dev team working on Iteration 4.1 - 3 iterations per firefox release
  • Program management continues to develop the wiki and reporting and associated documents including an up to date schedule showing the relationships of different add-ons work streams
  • Developer and Community team working on onboarding new reviewer hire, and getting add-on developers to submit for signing.

Important Links/References

Retrieved from "https://wiki.allizom.org/index.php?title=Firefox/AddOns/Status/20150929&oldid=1110615"