Auto-tools/Projects/Mozregression

mozregression

Introduction

mozregression is an interactive regression range finder for Mozilla nightly and inbound builds. It uses a binary search algorithm for quickly determining a changeset range corresponding to when a problem was introduced. It is widely use by Mozilla developers and other community members to help find regressions.

To get a feel for how mozregression works, see the video on codefirefox.com: http://codefirefox.com/video/mozregression

Homepage: http://mozilla.github.io/mozregression/

Active contributors

These people are actively working on mozregression and good people to ask questions of (or for review):

• William Lachance
• Sam Garrett
• Julien Pagès

How to setup for development

Assuming you have python, git, and virtualenv installed check out the mozregression repository and install it locally inside there. From there, you should be able to modify the code and test your changes. On Linux and MacOS X, this looks like:

 virtualenv mozr
 cd mozr
 source bin/activate
 git clone https://github.com/mozilla/mozregression.git mozregression
 cd mozregression
 pip install -e .

(Windows instructions should be pretty similar, the only difference should be the activation step)

Check style/bad code issues with flake8:

 pip install flake8
 flake8 mozregression tests

To run unit tests:

 pip install mock # not stricly required, but recommended to keep a clean dir
 python setup.py test

You could also use the coverage module to view the lines covered by tests:

 pip install coverage mock
 coverage run setup.py test
 coverage html # generate html results
 firefox htmlcov/index.html # see them in your favorite browser

Issue / feature tracking

We track mozregression bugs and feature requests inside bugzilla, in the Testing / mozregression component.

Full Query

ID	Summary	Priority	Status
997395	Allow installing addons from a nightly location	--	NEW
1084189	Cannot use IME on Nightly builds which are launched by mozregression	--	NEW
1192488	mozregression GUI installer needs admin privileges	--	NEW
1197106	Add some non regression tests for mozregression	--	NEW
1210046	Android builds should always behave the same way when doing A/B testing	--	NEW
1217443	using the skip feature is quite confusing	--	NEW
1231950	Ship (or use Desktop Fx) Android ADB with mozregression	--	NEW
1232037	Output results in machine-readable format (json?)	--	NEW
1232877	Option to check a date range prior to a certain build	--	NEW
1233339	We may need a panel to show which build has been downloaded	--	NEW
1233909	mozregression should go back further than one year for some 64-bit debug js shells	--	NEW
1234029	use debug builds from archives.m.o	P3	NEW
1235020	[gui] final result log output worthless	--	NEW
1235127	[gui] Mozregression sometimes fails to continue bisection in mozilla-inbound builds	--	UNCONFIRMED
1235260	[gui] Mozregression incorrectly sets day if I paste date string after selecting text from end to beginning	--	REOPENED
1236101	[meta] Bugs that block autoBisect from using mozregression	--	NEW
1237057	Have a link to the mozregression-gui.exe binary on the mozregression homepage	P3	NEW
1239150	Support dynamic extend reproduce range	--	UNCONFIRMED
1241358	[gui] remember "bits" choose, or take the default value ​​from the configuration file	--	NEW
1241365	[gui] show ~* steps left on interface	--	NEW
1241376	[gui] Progress list should scroll to the last item when it overflows	--	NEW
1241380	[GUI] be able to load and display a previous run	--	NEW
1241919	HTTP proxy support	--	NEW
1245283	mozregression and mozregression-gui aren't using the requested profile on windows 10	--	NEW
1245587	Make Android API changes simpler for mozregression to handle	--	NEW
1246086	[gui] Preselect profiles folder when you click on 'browse'	--	NEW
1246142	The --pref command line parameter goes silently ignored in some cases	--	NEW
1246383	[gui] Mark downloaded package on Build verdict window ('skip' action)	--	NEW
1246891	[gui] Allow set days of ApproxPersistChooser in Run a single build	--	NEW
1246903	Add a 'go to build...' action to other menu	--	UNCONFIRMED
1247061	Reusing connections to improve performance	P5	UNCONFIRMED
1248138	[gui] Allow redownload if download is interrupted or "ERROR : Unable to install"	--	NEW
1248239	[GUI] allow to check the builds of the initial range, as we do in the CLI	--	NEW
1248636	[gui] Options description is hard to find in Global preferences	--	NEW
1248690	[gui] View the parameters in "Bisecting on" details	--	UNCONFIRMED
1248767	[gui] Current build closes after 3 minutes if I restart browser (in different e10s state or with add-ons disabled/enabled)	--	NEW
1249737	Mozregression lost when a build fails to start	--	NEW
1256317	Error while trying to bisect fennec builds	--	NEW
1256379	Automatic bug number selection chose the wrong bug from an inbound range with a backout	--	NEW
1256445	mozregression does not allow to execute other mach commands	--	NEW
1258166	[gui] Support save as default and clear button to custom preferences and custom addons	--	UNCONFIRMED
1258688	[gui] Mozregression doesn't store values of some fields ("profile", "search for fix") when I switch back and forth between forms	--	NEW
1258831	[gui] Mozregression displays crash error when I try to close single build (clone-first) using X button	--	NEW
1263361	Exception in thread when hitting error ResourceNotFound 404	--	REOPENED
1270893	mozregression "To resume, run..." command goes nuts if it finds nested double/single quotes	--	NEW
1275499	TypeError: unsupported operand type(s) for -: 'datetime.date' and 'datetime.datetime'	--	NEW
1277888	mozregression --build-type fennec doesn't work	--	NEW
1281790	mozregression shouldn't output bug numbers when there are too many possibilities	--	NEW
1306661	doesn't recover from temporary interruptions to Internet connection	--	UNCONFIRMED
1308028	out of disk space not handled well (ambiguous error, no way to recover or retry)	--	UNCONFIRMED
1308417	Mozregression drops bookend integration revisions on the floor	--	NEW
1311474	Mozregression fails to run with pkg_resources.DistributionNotFound: colorama	--	NEW
1320322	mozregression should know about earliest Thunderbird build	--	NEW
1323445	Mozregression should not identify a specific commit if there are multiple commits in the final range	--	NEW
1327383	Mozregression GUI doesn't withstand disabling internet connection with background downloads enabled	--	NEW
1327595	Mozregression GUI somehow can't subtract date from buildID during bisection	--	NEW
1327668	[gui] If I doubleclick on "Bad" button in Mozregression - it marks BOTH builds as bad	--	NEW
1327669	[gui] It's possible to accidentally start background download after quick cancellation	--	NEW
1334853	mozregression --launch BUILDID used near-build instead of exact	--	UNCONFIRMED
1334856	mozregression can't handle restarted firefox	--	UNCONFIRMED
1339492	Mozregression fails to launch OSX builds older than 2016-07-12	P3	NEW
1350527	DownloadInterrupt should be processed	--	NEW
1350530	'Retest' button on past items	--	NEW
1351290	Bisecting mozilla-release with dates 2016-03-19 -> 2017-03-19 doesn't seem to work on (at least) win32	--	NEW
1362127	Add option for just getting commit range for two dates without downloading / testing builds	--	NEW
1365479	[gui] List recent used downloaded builds for Run a single build	--	NEW
1381319	[gui] Unreadable text depending on system theme	--	UNCONFIRMED
1394911	Add option to not launch Firefox	--	NEW
1394919	Add option to download binaries from final release	--	NEW
1395905	Allow reusing builds which were used recently	--	NEW
1398903	Provide standalone APK version of mozregression GUI and mozregression for Android	--	NEW
1407089	Add support for reading prefs from a JS file	--	NEW
1407430	Unable to find regression range between two specific date	--	NEW
1408696	mozregression can't verify the revisions it itself produced	--	NEW
1427371	mozregression just ran entirely the wrong build for me!	--	NEW
1439887	On small initial window, input verification is skipped	--	NEW
1458834	Retrying a build takes ages	--	NEW
1460596	[gui] Ability to save bisection session & log	--	NEW
1465145	bisecting API 11 fennec on API15 Android devices isn't working	--	UNCONFIRMED
1470495	Clean up obsolete branches and add some other ones	--	NEW
1477781	Incorrect regression range points to wrong bug	--	NEW
1477784	"Bisection informations" should say "Bisection information"	--	NEW
1478217	Bisection stuck in infinite loop	--	NEW
1484531	mozregression cannot find/use OpenSSL (libssl.so.1.0.0)	--	UNCONFIRMED
1484533	Provide virtualenv for cli	--	UNCONFIRMED
1485981	Mozregression for x86 devices	--	NEW
1487262	mozregression only outputs the topmost commit in a push.	--	NEW
1488907	Unexplained long pause before launching a Firefox instance	--	NEW
1499595	add option for downloading in background	--	NEW
1502956	Test changeset for regression	--	NEW
1503358	mozregression-gui crash: LookupError: no codec search functions registered: can't find encoding	--	NEW
1504259	Trying to use mozregression with Fennec results in "Permission denied" error	--	NEW
1519138	mozregression-gui: revise the last decision	--	UNCONFIRMED
1520789	Binary path is not getting escaped	--	NEW
1524066	--profile-persistence={clone-first,reuse} do not work on Android	--	NEW
1525283	Please make it possible to resume failed downloads	--	UNCONFIRMED
1536324	Errors running mozregression on Windows	P3	NEW
1544735	Firefox is crashing with builds from around the 2017-07-19	P3	NEW
1548014	mozgregression --arg doesn't work with --app=jsshell	P3	NEW
1550806	"Unable to start the application" when attempting a bisection when parent.lock in profile	P3	REOPENED
1558462	jsshell doesn't support testing functions	P3	NEW
1568086	Provide switch to pull only nightly builds/official signed builds	--	NEW
1570650	mozregression should work with esr branches	--	NEW
1580625	`mach mozregression` should install mozregression into its own dedicated virtualenv	P3	NEW
1604963	Attach mozregression log to related bug	--	NEW
1604965	Allow mozregression to select the range based on a bug in Bugzilla	--	NEW
1604967	Mozregression could set the 'Regressed By' field if a specific commit was found	--	NEW
1604969	mozregression could run test cases added to bugs	--	NEW
1610427	Mozregression should not look for taskcluster builds more than 1 year old	P3	NEW
1615667	mozregression buries its actionable output behind pages of warning-spew, for ranges that are too old for intermediate taskcluster builds	P3	NEW
1622958	Add a way to specify environment variables on Android	--	NEW
1631163	Updated notification is wrong when mozregression runs from repo or has pre-releases	P3	UNCONFIRMED
1631167	mozregression GUI does not sanity check user-specified good/bad builds	P4	UNCONFIRMED
1635680	No option to disable blinking cursor in Mozregression GUI for Mac	--	NEW
1639571	Prevent selecting date before 2004-12	P4	NEW
1647533	mozregression-gui.exe detected as Trojan:Win32/Zpevdo.B	--	NEW
1648260	mozregression should provide a way to intelligently prune my downloaded-builds-cache	--	NEW
1666469	mozregression fails with: OSError: [Errno 30] Read-only file system: '$HOME' on macOS	--	NEW
1680160	Mozregression won't copy my profile	P3	NEW
1681265	moregression gui does not work firefox-l10n ja build	P3	NEW
1689850	mozregression incorrectly refuses to use profile where lock file exists	--	NEW
1701318	Mozregression GUI: Fontconfig and other issues	--	UNCONFIRMED
1701671	DPI problems with mozregression on Win10	--	NEW
1713539	mozregression --addon startpage causes: error: Expecting property name enclosed in double quotes	P3	UNCONFIRMED
1714246	Mozregression rejects user-selected dates without warning	P3	UNCONFIRMED
1714682	regression range incorrectly narrowed due to cached builds colliding	P3	NEW
1718280	Release mozregression-gui binary segfaults on start	P3	UNCONFIRMED
1718288	Use uic/rcc instead of pyside2-uic/rcc	--	UNCONFIRMED
1720277	Improve the documentation how to pass multiple prefs	--	NEW
1720580	Mozregression does not open builds from 2014 or 2015	--	UNCONFIRMED
1731648	Versions are off-by-one	P4	NEW
1731835	mozregression-gui does show icons for minimize, maximize, and close buttons on Linux Ubuntu 20.04	P3	NEW
1736614	regression range incorrectly narrowed to later nightly on same day	P3	UNCONFIRMED
1737729	mozregression fails to used cached build that's not available for download anymore	P4	NEW
1738298	mozregression can't close Firefox on my machine | ERROR: Unable to stop the application (error: kill() got an unexpected keyword argument 'timeout')	--	NEW
1746919	Investigate suspiciously low mach numbers for Linux and Windows	--	NEW
1748456	Support --arg for --app gve	--	UNCONFIRMED
1749215	mozregression problem on Ubuntu 21.10	P3	NEW
1758171	Release list is breaking chronological order	P3	NEW
1759587	mozregression-gui is broken with Qt 5.15.3	P3	UNCONFIRMED
1760467	[mozregression] --addon has no effect for WebExtensions without applications.gecko.id in manifest.json	P3	UNCONFIRMED
1763188	Make mozregression work with snap builds	--	NEW
1771121	mozregression cannot launch older Beta builds	P3	NEW
1771664	[mozregression] "ERROR: Unable to install None" after retry	P3	UNCONFIRMED
1773343	Consider using a fallback mechanism to determine source of files	--	NEW
1777554	Add an option to control whether update is disabled	P3	NEW
1780012	dependabot version updates not guessing Python version correctly	P3	NEW
1780781	Add an option to bisect preferences	--	NEW
1781813	Investigate using alternate method to modify policies before launch	P3	NEW
1784493	Raspberry Pi4: Mozregression tries to download linux64-opt and fails (linux64-aarch64-opt would be correct)	P3	NEW
1785084	add helper commands to set up local dev environments	--	NEW
1785095	Improve matrices in CI tests	--	NEW
1786602	mozregression reports stable releases as newer than pre-releases (e.g. 4.0.19 is "newer" than 5.0.0.rc1)	P3	NEW
1786883	Add `lang` parameter to usage metrics	--	NEW
1787502	github-bugzilla-pr-linker being decommissioned on september 2nd, 2022	--	NEW
1791093	mozregression fails to find the push log	P3	NEW
1797537	mozregression-gui.exe - system error - missing file: api-ms-win-core-path-l1-1-0.dll	P3	UNCONFIRMED
1799851	mozregression GUI buttons/menus/strings are not visible in high contrast mode	--	NEW
1801963	Limit date range for `mozregression -n fennec` to prevent 404s	P3	NEW
1816823	Mozregression on Windows does not have version information	P3	NEW
1818142	Consider using elfshaker to reduce the size of binaries downloaded and stored	--	NEW
1818484	mozregression should clean up the files that Firefox puts in the update directory	P3	NEW
1819026	mozregression hangs (probably due to some remote service)	P3	NEW
1834765	Write a documentation for GUI tool?	P3	NEW
1835168	error message "Unable to find enough data to bisect." appears after successful bisection	P3	NEW
1836252	Replace `BaseClass.__init__` with `super().__init__` wherever possible	--	NEW
1838631	Create manually triggered workflow to build dmg with signed/notarized app bundle	--	NEW
1845801	Remove unused modules from app bundle	--	NEW
1846171	Confusing and time consuming startup behaviour on macOS since version 5.7.0	--	REOPENED
1848741	Mozregression-gui is not ignoring formatting changes done by ottland leading to a broader regression range	P3	NEW
1849403	Linux deployment matrix should support different install requirements for each entry	P3	NEW
1849643	Upgrade actions/checkout and actions/setup-python	--	NEW
1853461	Bisection wizard's window titlebar is not rendered correctly on Windows	P3	NEW
1855808	Bundle macOS app with one dir executable	--	NEW
1857026	Print a mozilla-central pushlog for Fenix regression windows	--	NEW
1857759	Bisecting stops before the end without errors	P3	NEW
1857840	mozregression permission error cert9.db file in use	P3	UNCONFIRMED
1857846	Mozregression Log - selecting text is buggy	P3	UNCONFIRMED
1858658	ERROR: Unable to install None (error: expected str, bytes or os.PathLike object, not NoneType)	P3	NEW
1863804	On Linux, Mozregression ignores accessibility settings	P3	UNCONFIRMED
1868011	auto-adjust bisection point bias based on the good/bad tendency	--	UNCONFIRMED
1868582	Glean testGetValue API returns `null` for all metrics when run in mozregression (because `datareporting.healthreport.uploadEnabled` is set to false)	P3	NEW
1876780	mozregression -g X checks X+1 build first	P3	NEW
1878937	Documentation does not state the need for having adb	--	UNCONFIRMED
1883340	mozregression-gui on Mac cannot capture camera or microphone	P3	NEW
1884938	mozregression is slow on macOS because pre-downloading doesn't pre-mount the DMG	--	NEW
1885707	UnicodeEncodeError: 'ascii' codec can't encode character '\xe5' in position 27: ordinal not in range(128)	P3	UNCONFIRMED
1886801	Mozregression tool crashes on macOS 13	--	NEW
1887037	mozregression seems to assume maturin is installed and in $PATH when uniffi-bindgen is installed even though that isn't necessarily the case	--	UNCONFIRMED
1897014	mozregression GUI error: IndexError: list index out of range in "mozregui\bisection.py", line 93	P3	UNCONFIRMED
1899515	inform user of user namespace ubuntu 24.04 required changes	P2	NEW
1902350	Virtual environment is created using the wrong Python binary on macOS (GA workflows)	P3	NEW
1902856	UnicodeEncodeError: 'ascii' codec can't encode characters in position 19-26: ordinal not in range(128)	--	UNCONFIRMED
1902890	mozregression-gui.exe v6.1.0 detected as malware by some security vendors	P3	UNCONFIRMED
1912938	Allow selecting aarch64 builds too	--	NEW
1914897	Add Python 3.13 to CI	--	NEW
1921615	When opening a preferences.ini file, GUI version of mozregression gives an error	P3	NEW
1927710	mozregression uses ~/.mozilla and should support xdg config	--	NEW
1928417	Date handling issue in GUI: Bisection starts in April instead of October	--	NEW
1930980	Tcl/Tk hooks deprecated in new PyInstaller	P2	NEW
1932901	mozregression unable to install Fenix builds using mozregression-gui 6.3.0	--	NEW
1936685	"(sway) Wayland protocol error: interface 'wl_pointer' has no event 9" when using mouse wheel	--	NEW
1937958	mozregression-gui should verify good and bad builds before running regression	--	NEW
1939786	Downloads from m-c have become very slow from 1Nov2024 to the point that doing biusection is not really possible	--	NEW
1941215	Browser crashes when scrolling a web page. This crash occurs only when the browser is launched from mozregression-gui on Ubuntu 24.04	--	REOPENED
1951335	Provide FileVersion and ProductVersion of mozregression for Windows	--	NEW
1957147	mozregression should warn & ask user for confirmation before installing fenix (since doing so is destructive and clobbers profile data)	--	NEW
1959047	no longer possible to bisect autoland builds ("Bad CPU type in executable")	--	NEW
1961359	Mozregression should stop looking for builds when it realises that there are none due to TaskCluster only keeping builds for one year	--	UNCONFIRMED
1966183	New mozregression profiles should turn off the "Send daily usage ping to Mozilla" setting	--	NEW
1969327	mozregression for Fenix does not try autoland builds after it's done bisecting Nightlies	--	NEW

211 Total; 211 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Submitting patches

If a bug is not open already corresponding to the bug you want to fix (or the feature you want to add), please file one. After that, please open a github pull request with your code change, and set a flag in bugzilla for it to be reviewed, as described here:

http://ateam-bootcamp.readthedocs.org/en/latest/guide/development_process.html#git-and-github

Please make sure your patch conforms to PEP8 before submitting it.