Auto-tools/Projects/Signal From Noise/Meetings/2012-05-17: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 29: Line 29:
** Next steps:  
** Next steps:  


* Metrics for determining page-centric "did my test pass or not" for at least one suite (and that suite *won't* be tp5)
* Metrics for determining page-centric "did my test pass or not" for at least one suite dromaeo and ts_paint
** Owner: ??
** Owner: ??
** Next steps:  
** Next steps:  
Line 41: Line 41:
   {'result':'PASS', 'tests': {'passed': 98, 'failed': 0, 'maybe': 2}, 'details': [{'k0s.org': {'actual': 2.71, 'expected': [2.75, 3.21]}, {'askjeads.com': {'actual': 99, 'expected': [88, 98]}]}  
   {'result':'PASS', 'tests': {'passed': 98, 'failed': 0, 'maybe': 2}, 'details': [{'k0s.org': {'actual': 2.71, 'expected': [2.75, 3.21]}, {'askjeads.com': {'actual': 99, 'expected': [88, 98]}]}  


* We will have fixed the pages accessing web in tp5 and re-release the tp5 pageset with the new non-web touching pages (i.e. we are *not* waiting for google and their page set - we should start this now - this is fixed and checked-in, and on releng's list for deployment in Q3).
* We will have fixed the pages accessing web in tp5 and re-release the tp5 pageset with the new non-web touching pages  
** Owner: jmaher
** Owner: jmaher
** Next steps: write a cleanup script, run webpages through a cleanup script, verify the pages are clean, talk to releng
** Next steps: write a cleanup script, run webpages through a cleanup script, verify the pages are clean, talk to releng


* Convert all page cycle extension based tests to row-major style. (Stop analysis on row vs. column, and just make the call to go with row. This should be largely deployed by end of Q2)
* Convert all page cycle extension based tests to row-major style.
** Owner: jmaher
** Owner: jmaher
** Next steps: run tdhtml, tsvg, tsvg_opacity, tsspider, tscroll, a11y in staging, create graph server definitions for new test types, start side by side staging.
** Next steps: run tdhtml, tsvg, tsvg_opacity, tsspider, tscroll, a11y in staging, create graph server definitions for new test types, start side by side staging.


= Action Items =
= Action Items =

Revision as of 18:04, 16 May 2012

Previous Action Items

  • find what pages we can consider reliable
    • define what "reliable" is
  • triangulate
  • [christina] - give jeads/ctalbert/jmaher a set of pages that are just too noisy and operating systems
  • local TBPL (contact: edmorley)

Metric Calculations

Datazilla

Page Specific Views

Compare Talos Functionality

Round Table

ctalbert's official list of goals for SfN this quarter

  • a production system slurping up data from talos deployed (staging env is fine since we know we won't get sec review done in time)
    • Owner: ??
    • Next steps:
  • ability on this production system to accept data from non-talos projects
    • Owner: ??
    • Next steps:
  • the UI we largely have now
    • Owner: ??
    • Next steps:
  • Metrics for determining page-centric "did my test pass or not" for at least one suite dromaeo and ts_paint
    • Owner: ??
    • Next steps:
  • a stage tbpl system mocking the Yes/No/Maybe interaction
    • Owner: ??
    • Next steps: assert we can mock up tbpl to gather data post test completion, will mozilla be ok with that?
      • tbpl can do this:
 POST {'changeset': 31415926535, 'testname': 'dromaeo', 'platform': 'win7'} http://10.8.73.29/views/api/test_results
      • and recieve this:
 {'result':'PASS', 'tests': {'passed': 98, 'failed': 0, 'maybe': 2}, 'details': [{'k0s.org': {'actual': 2.71, 'expected': [2.75, 3.21]}, {'askjeads.com': {'actual': 99, 'expected': [88, 98]}]}
  • We will have fixed the pages accessing web in tp5 and re-release the tp5 pageset with the new non-web touching pages
    • Owner: jmaher
    • Next steps: write a cleanup script, run webpages through a cleanup script, verify the pages are clean, talk to releng
  • Convert all page cycle extension based tests to row-major style.
    • Owner: jmaher
    • Next steps: run tdhtml, tsvg, tsvg_opacity, tsspider, tscroll, a11y in staging, create graph server definitions for new test types, start side by side staging.

Action Items

Retrieved from "https://wiki.allizom.org/index.php?title=Auto-tools/Projects/Signal_From_Noise/Meetings/2012-05-17&oldid=431590"