WebDriver/RemoteProtocol/Meetings/2019/11/22

Agenda

• puppeteer-alpha planning session (mmucci)
  • Team will select the MVP backlog based on a review of the team completion speed and forecast schedule
• Fission (ato)

Roster

Present

whimboo, AutomatedTester, mmucci, digitarald

Regrets

maja_zf

Minutes

Update on puppeteer-alpha progress

[Starts at 00'09" in the recording.]

Resources

• Dashboard
• Remote/Milestones

Introduction

• 29 bugs in the backlog, 11 are in development, 18 listed as available
• Our team’s completion date is 1.6 days per bug
  • mmucci noted this is really fast, considering we only have three engineers
• Two scenarios:
  • Fixed-day: up to 20th December (20 working days from now)
    • Whatever is left over gets thrown into Milestone 2 (M2)
    • Would mean a commitment of two more bugs to deliver before 20th December
  • Complete entire backlog: 29 bugs + 8 contingency bugs
    • Estimate would be 60 work days (around 6th March) to complete
    • Holiday time/PTO over Christmas, all-hands, &c.

Discussion

• mmucci’s preference would be scenario #2 to complete the whole backlog scope into consideration
• AutomatedTester’s preference would be scenario #1 and fixed dates and move overflowing things into an M2
  • Concerned that we think we have everything, but this project has shown a number of times that things take longer than expected due to unforeseen work
  • A flexible, feature-driven approach may be pushed further back due to unforeseen work
• ato’s doesn’t have a preference
  • Which scenario we pick isn’t going to impact how quickly we will get to what we consider to be MVP quality
  • We will get N amount of work done, question is how we want to frame it
  • Perhaps scenario #1, with a fixed end date, will give a sense of completion
• Concern that what we will have by end-of-year (20th December) won’t have the necessary features required to have something to put into the hands of users
  • General acknowledgement that there are many ‘known-unknowns’
  • CDP API is under-specified, Puppeteer has a non-exhaustive testsuite, lack of CDP conformance testsuite
  • This makes the work highly incremental, in that you don’t know you’re missing something until you add some feature only to discover there is another blocker preventing the Gutenberg testsuite from running to completion
  • It was called out that whimboo has done a great job at documenting/organising which methods and events are used by Puppeteer and Gutenberg, and that this has helped planning tremendously
  • It was suggested by AutomatedTester that we may want to shift focus of what that MVP is to the core use cases and examples, and document deficiencies and where we lack certain features
  • AutomatedTester is concerned that scope will grow and grow if we focus too much on supporting Gutenberg fully, and that the unknowns will cause estimate to slip further back than March
• AutomatedTester is writing a blog post announcing Puppeteer support in Firefox for hacks.mozilla.org
  • Concern that a blog post saying “look, we can run 5% of Gutenberg” isn’t necessarily good publicity
• Suggestion that we have a fixed end-date for puppeteer-alpha, then make the next milestone (M2) “feature-driven” and flexible
  • mmucci confirms that would not be a problem
  • This means we are committing to delivering 12 bugs before 20th December
  • 10 of these are already in development/in review, which means we have two complete an additional two
• digitarald noted that whilst a fixed end date gives us a sense of completion, it doesn’t make it any clearer what features we think we will have by end-of-year or what’s remaining
  • The exploratory nature of this project makes it hard to predict what’s remaining
  • Besides reasons given earlier, the biggest uncertainty factor are the meta bugs for APIs we haven’t begun exploring yet
    • Some of these we have found really easy to do, some requiring more work
  • Besides the known-unknowns of the remaining API surface, the biggest unknown-unknown is Fission
  • If we believe there’s significant uncertainty, we just have to be really conservative in our estimation
  • In terms of milestone planning, we are at liberty to break up work into discrete milestones, whether time-bound or feature-bound
  • Networking-related features could also be hard, because we don’t know if we can support all of the APIs in Gecko
• On the other hand, we some low-hanging fruit that we could support quite easily
  • Cookies and file uploads we know how to do from experience, and there is earlier code in Marionette we could re-use
  • Both turn up in Gutenberg test logs frequently and would likely unblock a significant number of Gutenberg tests
• mmucci points out that new bugs should now be filed with the puppeteer-alpha-reserve whiteboard entry
  • Further, please make sure you don’t change the whiteboard entry when you assign yourself to any of these bugs and make them P1, as this allows us to track the MVP commitment for 20th December along with anything in development from the reserve backlog
• ato noted that people external to the project may look at the dashboard and think of it as a metric of how soon we will have Puppeteer support in Firefox done, and due to the number of unknowns there’s a high uncertainty factor that the dashboard doesn’t account for
  • This isn’t a problem with the dashboard per-se, but with the way people perceive it
  • It is a larger engineering and product planning question, but one strategy is to create smaller deliverables that themselves communicate achievement in some area

Questions

On whiteboard entries:

whimboo

Quick question about the backlog. We have the puppeteer-alpha whiteboard entry and the puppeteer-alpha-backlog entry. What is the difference between these?

mmucci

The backlog, or the reserve bugs, is where we put bugs that it would be good to have if we have time to get to them. But we’re not comitting to completing the reserve backlog.

On the know-unknowns that make out the bulk of the CDP API surface we know we need for Gutenberg, but which has not been explored yet:

whimboo

Should we simply file implementation bugs for all the metea bugs which are still in the list?

There are 19 bugs in total, so it contributes significant uncertainty to the model.

ato

We keep coming back to the same problem that some of these bugs might be possible to achieve within the scope of a single bug, others might need to be split up in more bugs.

mmucci

Just keep doing what you’ve been doing up to now.

As you discover bugs, just file them. That’s it.

The velocity tracking takes into account a certain amount of uncertainty already, so you don’t have to do the double work of tracking these separately in temporary bugs.

Planning for puppeteer-alpha

[Starts at 36'03" in the recording.]

• Due to the low number of remaining bugs we can commit to do, it forces us to be more deliberate about what features we want done in the next 21 working days
• The main qualifier for which those two bugs are should be, according to AutomatedTester, those “we consider blockers for getting anything running”, for example those that from a product-point of view, can demonstrate bigger impact
• Some candidates:
  • Page.frameAttached, Page.frameStoppedLoading, missing properties on some of these
  • Some of the Network domain events seem easy, in the sense that they map on to Gecko concepts we already know we can support, and then there are some uncertain ones
    • Fetch.continueRequest and friends could be tricky, but it shouldn’t prevent us from doing the rest that are easy
  • File upload and file chooser picker events
  • Device screen size override
  • Runtime.evaluate and Runtime.callFunctionOn are mostly done, just missing a few properties
  • There’s also a bug maja_zf filed about returned arrays that we need to figure out how to fix, due to the weird way CDP avoids encoding arrays as complex remote object types
• Some of these are already implemented in Juggler and can be ported over, and others are low-hanging fruit

Action: ato will start document gathering potential candidates for the remaining two must-have bugs we commit to deliver.

Changes to the dashboard

[Starts at 52'23" in the recording.]

• Dashboard is now set up and ready, and you can keep track of the progress until 20th December using weekly milestones
  • These have completion goals, like how many bugs we need to complete to stay on track
• Keep adding bugs as normal into the backlog by setting their priority to P3 and tagging them with the puppeteer-alpha-reserve whiteboard entry

Puppeteer upstream changes

• Movement in the Puppeteer project to move the GitHub repository to a different organisation
  • We were asked to give input on a design document
  • Should not impact open PRs or issues, but will affect the Mozilla Puppeteer fork
  • Besides PR 5137, adding launch support for multiple browsers, getting some more attention we have no concerns

Actions

• ato will start document gathering potential candidates for the remaining two must-have bugs we commit to deliver.

Status of Milestone 1

• Last week: 58 Total; 49 Open (84.48%); 9 Resolved (15.52%); 0 Verified (0%)
• This week: 63 Total; 47 Open (74.6%); 16 Resolved (25.4%); 0 Verified (0%)

Changelog

TBA

Work

Milestones

Development status of Puppeteer alpha

Puppeteer alpha dashboard

Bugzilla queries

All project work currently in development

Available MVP work

Completed MVP work

Bug overviews

Gutenberg dependency tree

Puppeteer examples dependency tree

Complete Puppeteer dependency tree

All ze boogs

PTO (🍂)

• whimboo away Wednedsay 20th November for public holiday
• ato PTO half-day on Thursday 21st and 22nd November