33
edits
Compatibility/Meetings/Sync w Honza: Difference between revisions
Jump to navigation
Jump to search
Compatibility/Meetings/Sync w Honza (view source)
Revision as of 13:10, 14 November 2023
, 14 November 2023October 31 2023
(meeting minutes October 17) |
(October 31 2023) |
||
| Line 1: | Line 1: | ||
== October 31 2023 == | |||
=== QA Triage Trends === | |||
Since our last meeting we've started using the new format on how we submit the QA Triage Trends. | |||
We are looking for a feedback: https://github.com/mozilla/webcompat-team-okrs/issues/275#issuecomment-1772768903 | |||
Regarding the trend metrics, we are still working on the document. | |||
We were wondering if the total number of issues (per label) received each month regardless if it's reproducible or not would be enough for the metrics. | |||
Paul: Should we go this deep, or is it enough to mention a link and a number of total issues per milestone, instead of copying the link for each individual issue? | |||
Honza: I am also thinking about the new system. We can search for individual reports by label. An important thing to add, if we are not sure about a label, it is best to not label the issue. We should use categories where we are certain. To answer your question, the numbers are more important, is that trend growing or not. Higher management wants to have some kind of metricts to see the impact the platform is making, we are trying to understand all the user reports, estimate trends, which issues have the most impact and how to prioritize them, so that we can give all the revelant info to the platform team. Whatever numbers we haev should help the higher management to see if the platform team is going into the right direction. | |||
Paul: How do we measure the impact? | |||
Honza: That is the big question. They need to know that the actions we are taking are making things better or worse. We can not base this metric on the number of reports, getting more reports vs getting less reports, the goal of the system is to have as much data as possible. It is in our interest to have more reports. We might want to identify things like Firefox not supported, and we can somehow follow that trend. We would not collect the number of reports, but the domains regarding this. We can measure how quickly the platform fixes issues. We can due issue scoring, like the State of Webcompat report, top 20 issues, that we think the platform should fix. Or the popularity of sites, using Firefox Telemetry and compared it with Chrome Telemetry. Like are there sites used by a lot of people just in Chrome. We should come up with same data to see if we are in the right direction. The trends, the numbers, is part of that goal. Are we able to spot some trends, and trust the numbers? | |||
The overall numbers indicating the trend, what is our main focus. | |||
Paul: So will show the numbers based on links, and show them by month. | |||
Honza: yes. | |||
=== Gathering info for the New Reporting System (Honza) ==== | |||
Honza: For the new system, I am interested on the triage process of the QA. That means that you would assign labels to issues? Right now you are using github label, that would likely remaing the same. | |||
Paul: Every bug tracker has a way to separate things, and labels are the way to go, free to create and easier to use. | |||
Honza: When you split the work, as per your email that Dennis asked for info, what if there was one more person? | |||
Raul: We would have a batch each day and split the issues in 3. | |||
Paul: Or we could assign the issues directly to the QA member. | |||
Raul: It would help us if we could mass assign a batch directly to a member without assigning each issue manually. | |||
Paul: Right now we are using keywords for specific OKRs. I'm not sure if its feasible to add a label for each issue (for e.g. counting number of issues triaged in a week, for now we use "[qa_44/2023]" or "[inv_44/2023]" for investigated, 44 stands for week 44). The new dashboard should have something in order to count the issues received each week. | |||
Raul: These keywords help us a lot when counting our issues in different metrics/reports. | |||
Honza: The current system is based on the reports received on github. But in the new system it will be harder to filter that out because more reports would be received. Maybe we would have to mass close an issue that has the same domain. | |||
Paul: The bot also closes issues that he finds inappropriate or irrelevant. | |||
Honza: Right, we want to measure only the work that has been done by the triage team. | |||
The issues will be grouped by the domain. There might be some tooling that groups issues that has similar description. Once we have more data, we will start learning how to make the Triage process much better. | |||
== October 17 2023 == | == October 17 2023 == | ||