Buildbot/Talos/Sheriffing/GraphServer FAQ

What is graph server

[GraphServer] is a tool that takes uploaded data points and graphs them over time. Primarily this is used for performance data from [Talos].

The code for GraphServer can be found [here]. The graph server has a API to look at other information, it can be found [here].

Viewing details on a graph

When viewing graph server details, in many cases it is obvious where the regression is. If you mouse over the data points (not click on them) you can see some raw data values.

While looking for the specific changeset that caused the regression, you have to determine where the values changed. By moving the mouse over the values you can easily determine the high/low values historically to determine the normal 'range'. When you see values change, it should be obvious that the high/low values have a different 'range'.

If this is hard to see, it helps to zoom in to reduce the 'y' axis. Also zooming into the 'x' axis for a smaller range of revisions yields less data points, but an easier way to see the regression.

Once you find the regression point, you can click on the data point and it will lock the information as a popup. Then you can click on the revision to investigate the raw changes which were part of that.

Keep in mind, graph server doesn't show if there is missing data or a range of changesets.

Zooming

Graph server has the ability adjust the date range from a drop down box. We default to 7 days, but we can change it to 30/90/365 days from the UI drop down.

It is usually a good idea to zoom out to a 30 day view on integration branches. This allows us to see recent history as well as what the longer term trend is.

There are two parts in the graph server graph, the top box with the trendline and the bottom viewing area with the raw data points. If you select an area in the trendline box, it will zoom to that. This is useful for adjusting the Y-axis.

Adding additional Data Points

One feature of graph server is the ability to add up to 7 sets of data points at once and compare them on the same graph. In fact in alert manager we add multiple branches at once.

While looking at a graph, it is a good idea to look at that test/platform across multiple branches to see where the regression originally started at and to see if it is affected on different branches.

For all regressions it is good to see:

• original branch
• fx-team-non-pgo
• mozilla-inbound-non-pgo

The other common use case for adding data is to look at the given test across all platforms. This is important to know how widespread a regression is. The process is like this:

• find the original branch (i.e. fx-team-non-pgo)
• remove all other data sources from the graph server
• then add all platforms for fx-team-non-pgo:
  • Ubuntu 12.04
  • Ubuntu 12.04 x64
  • WINNT 5.1 (ix)
  • WINNT 6.1 (ix)
  • WINNT 6.2 X64
  • OSX 10.6 (this needs to be on fx-team branch- not the -non-pgo branch)
  • OSX 10.8 (this needs to be on fx-team branch- not the -non-pgo branch)
  • Android 4.0.4 (this needs to be on fx-team branch- not the -non-pgo branch)

Zooming in and verifying which platforms have the alerts is a common use case so we can accurately report what the scope of the regression is.

Muting additional data points

Once you become familiar with graph server it is a common use case to have >1 data point on the graph at a time. This results in a lot of confusing data points if you are trying to zoom in and investigating the values for a given data point.

Luckily there is a neat feature in graph server that allows you to mute a data point. There is an 'eye' icon (on by default) that lives in the left hand side bar where you have the list of data series. If you click this 'eye' icon, it will must that series.

Common practice is to load up a bunch of related series, and mute/unmute to verify revisions, dates, etc. for a visible regression.