Confirmed users
3,376
edits
Buildbot/Talos/Misc: Difference between revisions
Jump to navigation
Jump to search
→Steps to add a test
| Line 5: | Line 5: | ||
== Steps to add a test == | == Steps to add a test == | ||
* file a bug to add tests to [http://hg.mozilla.org/build/talos/file/tip/talos/test.py talos] [[https://bugzilla.mozilla.org/show_bug.cgi?id=893071 example]]. | |||
* File a bug to add appropriate rows to [http://hg.mozilla.org/graphs/file/tip/sql/data.sql graphserver] [[https://bugzilla.mozilla.org/show_bug.cgi?id=907876 example]]. | * File a bug to add appropriate rows to [http://hg.mozilla.org/graphs/file/tip/sql/data.sql graphserver] [[https://bugzilla.mozilla.org/show_bug.cgi?id=907876 example]]. | ||
** File an additional IT bug to deploy the sql changes on the production and staging graph servers [[https://bugzilla.mozilla.org/show_bug.cgi?id=907255 example]]. | ** File an additional IT bug to deploy the sql changes on the production and staging graph servers [[https://bugzilla.mozilla.org/show_bug.cgi?id=907255 example]]. | ||
** If this is adding new pages, ensure these sql changes include [http://hg.mozilla.org/graphs/file/5dd6f4fb9c27/sql/pages_table.sql page definitions]. NOTE: this one detail is usually forgotten since it is so rare | ** If this is adding new pages, ensure these sql changes include [http://hg.mozilla.org/graphs/file/5dd6f4fb9c27/sql/pages_table.sql page definitions]. NOTE: this one detail is usually forgotten since it is so rare | ||
** MOBILE ONLY: if we are changing/adding pages which mobile tests use or need, you need to update bm-remote [[https://bugzilla.mozilla.org/show_bug.cgi?id=883896 example]] | ** MOBILE ONLY: if we are changing/adding pages which mobile tests use or need, you need to update bm-remote [[https://bugzilla.mozilla.org/show_bug.cgi?id=883896 example]] | ||
* create a talos.zip file and file a releng bug [[https://bugzilla.mozilla.org/show_bug.cgi?id=873180 example]] to upload it to the build network | * create a talos.zip file and file a releng bug [[https://bugzilla.mozilla.org/show_bug.cgi?id=873180 example]] to upload it to the build network | ||
** NOTE: for aurora/central on desktop only - we update testing/talos/talos.json with a revision to the talos tree [[https://bugzilla.mozilla.org/show_bug.cgi?id=900913 example]] | ** NOTE: for aurora/central on desktop only - we update testing/talos/talos.json with a revision to the talos tree [[https://bugzilla.mozilla.org/show_bug.cgi?id=900913 example]] | ||
** NOTE: for other branches or mobile - we still need an updated talos.zip, many times we need to update both [[https://bugzilla.mozilla.org/show_bug.cgi?id=906825 example]] | ** NOTE: for other branches or mobile - we still need an updated talos.zip, many times we need to update both [[https://bugzilla.mozilla.org/show_bug.cgi?id=906825 example]] | ||
* | |||
** | * add the test to the running configs (as seen on tbpl) | ||
** NOTE: one exception is adding an entirely new suite- this requires a buildbot patch to add the new scheduled job [[example]] | ** for aurora/central on desktop only - we update testing/talos/talos.json with a revision to the talos tree [[https://bugzilla.mozilla.org/show_bug.cgi?id=903681 example]] | ||
** | ** NOTE: one exception is adding an entirely new suite- this requires a buildbot patch to add the new scheduled job [[https://bugzilla.mozilla.org/show_bug.cgi?id=886288 example]] | ||
** for other branches or mobile - we still need a buildbot patch, many times we need to update both buildbot and talos.json | |||
* | |||
* ensure the test follows the trains | |||
** if we are dealing with buildbot configs, then we will need bugs for each merge cycle to turn on/off appropriate tests | |||
** if we are just updating talos.json, this will be automatic | |||
* if this is an update to an existing test that could change the numbers, this needs to be treated as a new test and run side by side for a week to get a new baseline for the numbers. | * if this is an update to an existing test that could change the numbers, this needs to be treated as a new test and run side by side for a week to get a new baseline for the numbers. | ||
* file a bug to get | |||
* file a bug to get TBPL updated with a new letter to track this test [[https://bugzilla.mozilla.org/show_bug.cgi?id=897415 example]] | |||
* document the test on [[Buildbot/Talos]] and [[Buildbot/Talos/Tests]] | * document the test on [[Buildbot/Talos]] and [[Buildbot/Talos/Tests]] | ||
** NOTE: this should be updated every time the branch moves closer to release. Once on release we don't need to update | |||
While that is a laundry list of items to do, if you are developer of a component just talk to the a*team (jhammel or jmaher) and they will handle the majority of the steps above. | While that is a laundry list of items to do, if you are developer of a component just talk to the a*team (jhammel or jmaher) and they will handle the majority of the steps above. | ||