Confirmed users
35
edits
QA/Automation/Projects/WebRTC/Crashtest Guide: Difference between revisions
QA/Automation/Projects/WebRTC/Crashtest Guide (view source)
Revision as of 12:37, 13 December 2012
, 13 December 2012→Step by Step Guide to WebRTC Crashtests
(Created page with "=Step by Step Guide to WebRTC Crashtests= ==Setup your system for WebRTC== There are two ways to get ready for webrtc testing. The first one is most likely what you want to do ...") |
|||
| Line 50: | Line 50: | ||
mozdownload --type=daily –date=2012-10-19 | mozdownload --type=daily –date=2012-10-19 | ||
from the command line. This should download a file called 2012-10-19-03-05-51-mozilla-central-firefox-19.0a1.en-US.linux-x86_64.tar.bz2. | from the command line. This should download a file called ''2012-10-19-03-05-51-mozilla-central-firefox-19.0a1.en-US.linux-x86_64.tar.bz2''. | ||
Unpack this version of firefox and start it from command line with | Unpack this version of firefox and start it from command line with | ||
home:~/[path]/firefox-2012-10-19$ ./firefox | |||
home:~/[path]/firefox-2012-10-19$ ./firefox | |||
Make sure, it is the nightly build. (Most common mistake to open it, while another non-nightly instance of firefox is already open). | Make sure, it is the nightly build. (Most common mistake to open it, while another non-nightly instance of firefox is already open). | ||
| Line 62: | Line 60: | ||
Now access the previously created testcase file from this instance of firefox and check if the file crashes. In this case, the nightly becomes unresponsive but does not crash. If any of this does not happen, then download a build earlier in date and repeat until you can reproduce the error. | Now access the previously created testcase file from this instance of firefox and check if the file crashes. In this case, the nightly becomes unresponsive but does not crash. If any of this does not happen, then download a build earlier in date and repeat until you can reproduce the error. | ||
SIDE NOTE: | {| border="1" cellpadding="5" | ||
|'''SIDE NOTE:'''<br> | |||
If it crashes straight away, it is possible to check the stacktrace by starting the nightly build and going on the page about:crashes | If it crashes straight away, it is possible to check the stacktrace by starting the nightly build and going on the page about:crashes | ||
Also, please make sure that media.navigator.enabled and media.peerconnection.enabled in about:config is set to true before testing for the crash. If these are not enabled, nothing will happen. | Also, please make sure that ''media.navigator.enabled'' and ''media.peerconnection.enabled'' in ''about:config'' is set to '''true''' before testing for the crash. If these are not enabled, nothing will happen. | ||
|} | |||
== | ==Recreate the testcase in a firefox-debug build and compare to the callstack== | ||
Recreate the testcase in a firefox-debug build and compare to the callstack | |||
Building the debug | ===Building the debug=== | ||
We now need to test this scenario in a full debug build. In this case, firefox only froze, so we need to make sure it crashes in order to compare the given stacktrace (in the bug) and the one obtained. | We now need to test this scenario in a full debug build. In this case, firefox only froze, so we need to make sure it crashes in order to compare the given stacktrace (in the bug) and the one obtained. | ||
| Line 107: | Line 104: | ||
https://developer.mozilla.org/en-US/docs/Developer_Guide/mach | https://developer.mozilla.org/en-US/docs/Developer_Guide/mach | ||
Check the testcase in the debug | ===Check the testcase in the debug=== | ||
Here we have two possibilities: If the testcase crashes the browser, then we can retrieve the callstack from about:crashes in the browser. In this case, follow to the next subheading “Compare the callstack”. It needs to crash, a frozen screen like in our case is no good. | Here we have two possibilities: If the testcase crashes the browser, then we can retrieve the callstack from about:crashes in the browser. In this case, follow to the next subheading “Compare the callstack”. It needs to crash, a frozen screen like in our case is no good. | ||
| Line 152: | Line 148: | ||
and observe the output. This leads us then to the next section. | and observe the output. This leads us then to the next section. | ||
Compare the output of the crash to the callstack given in the bug | ===Compare the output of the crash to the callstack given in the bug=== | ||
The output obtained in this via gdb, should look something like this (NOTE: If you arrived here via a crash, then consult about:crashes, find the report and scroll down. You should find the same output in pretty tabular form) | The output obtained in this via gdb, should look something like this (NOTE: If you arrived here via a crash, then consult about:crashes, find the report and scroll down. You should find the same output in pretty tabular form) | ||
| Line 190: | Line 185: | ||
This output should be similar to the output given in the callstack of the bug (There may be some slight differences in the numbers). You have now successfully re-created the bug in question. Time to create a crashtest for this bug. | This output should be similar to the output given in the callstack of the bug (There may be some slight differences in the numbers). You have now successfully re-created the bug in question. Time to create a crashtest for this bug. | ||
== | ==Creating a crashtest== | ||
Creating a crashtest | |||
Minimising the testcase | ===Minimising the testcase=== | ||
Now it is time to create the crashtest. The first thing to do, is to minimise the testcase.html to a minimal example in which the error still occurs. Of course, this is different for each testcase and has to be assessed on a case by case basis. | Now it is time to create the crashtest. The first thing to do, is to minimise the testcase.html to a minimal example in which the error still occurs. Of course, this is different for each testcase and has to be assessed on a case by case basis. | ||
| Line 245: | Line 237: | ||
This should set everything up to test your crashtest. | This should set everything up to test your crashtest. | ||
Testing your crashtest | ===Testing your crashtest=== | ||
Now is the time to test your crashtest in a real case scenario. In order to do this, add the following line to crastest.list in the $HOME/mozilla-central/dom/media/tests/crashtests directory. | Now is the time to test your crashtest in a real case scenario. In order to do this, add the following line to crastest.list in the $HOME/mozilla-central/dom/media/tests/crashtests directory. | ||
pref(media.peerconnection.enabled,true) load %filename% | pref(media.peerconnection.enabled,true) load %filename% | ||
| Line 282: | Line 274: | ||
As you can see from the highlighted section, the error was raised again and we successfully created a crashtest! Well done!!! | As you can see from the highlighted section, the error was raised again and we successfully created a crashtest! Well done!!! | ||
Testing on the current version | ===Testing on the current version=== | ||
Now all that is left to be done, is to test the crashtest on the current version of firefox and if everything is ok, create a patch file which can be submitted. | Now all that is left to be done, is to test the crashtest on the current version of firefox and if everything is ok, create a patch file which can be submitted. | ||
| Line 297: | Line 288: | ||
------------------------------------------------------------ | ------------------------------------------------------------ | ||
Create a patch | ===Create a patch=== | ||
Now, all that is left is to create a patch and submit it to the bug report to be implemented. | Now, all that is left is to create a patch and submit it to the bug report to be implemented. | ||
How to find a good bug to write a crashtest for? | |||
==How to find a good bug to write a crashtest for?== | |||
These are only guidelines and there is no easy rule to obey here. | These are only guidelines and there is no easy rule to obey here. | ||
*Find a bug that crashes straight away. If you have to do reloads or anything like this to re-create the crash, then it is probably not good for a start. | |||
*If the bug has the keywords 'crash' or 'in-testsuite', that is a good sign (in-testsuite may also mean mochitest, so be careful there). | |||
*If you are unsure, you can always ask ;) | |||