119
edits
Media/WebRTC/libwebrtc Update Process: Difference between revisions
Media/WebRTC/libwebrtc Update Process (view source)
Revision as of 18:46, 20 March 2024
, 20 March 2024add note about ability to stop during the rebase operation
(clarify the commit hygiene section to remove the ordering specific info and specifically call out which types of changes belong in the different commit notations) |
(add note about ability to stop during the rebase operation) |
||
| (9 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
This page describes the process for updating the in tree, vendored copy of libwebrtc using the fast-forward scripts. | This page describes the process for updating the in tree, vendored copy of libwebrtc using the fast-forward scripts. | ||
== | == Prerequisites == | ||
* NOTE: all steps below assume the user is in the bash shell. | |||
* Create a new bug for the fast-forward update. It should depend on the previous fast-forward bug. [https://bugzilla.mozilla.org/show_bug.cgi?id=1800920 Bug 1800920] is an example. | * Create a new bug for the fast-forward update. It should depend on the previous fast-forward bug. [https://bugzilla.mozilla.org/show_bug.cgi?id=1800920 Bug 1800920] is an example. | ||
* Before each new merge cycle elm needs to be reset to Central. | * Before each new merge cycle elm needs to be reset to Central. | ||
| Line 14: | Line 14: | ||
* Starting with a fresh clone of elm is recommended for each new chromium milestone. | * Starting with a fresh clone of elm is recommended for each new chromium milestone. | ||
hg clone --stream ssh://hg.mozilla.org/projects/elm | hg clone --stream ssh://hg.mozilla.org/projects/elm | ||
* Verify the most recent commit is the .arcconfig update. If the most recent commit doesn't look like "Bug xxx - FLOAT - REPO-elm - update .arcconfig repo callsign", run the following commands: | |||
./mach python dom/media/webrtc/third_party_build/restore_elm_arcconfig.py | |||
hg push -r tip | |||
== Update steps == | |||
* Start from the elm checkout | |||
cd elm | cd elm | ||
* Update the default config file for the next upstream milestone | |||
bash dom/media/webrtc/third_party_build/update_default_config.sh | bash dom/media/webrtc/third_party_build/update_default_config.sh | ||
* Prepare the git repository - Running <code>prep-repo.sh</code> will display instructions on how to add recent moz-central changes made in <code>third_party/libwebrtc</code> to the top of the patch stack in our git repository. This is expected and necessary for successfully vendoring changes into <code>third_party/libwebrtc</code>. | |||
bash dom/media/webrtc/third_party_build/prep_repo.sh | bash dom/media/webrtc/third_party_build/prep_repo.sh | ||
* Prepare no-op tracking files for cherry-pick commits | |||
bash dom/media/webrtc/third_party_build/build_no_op_commits.sh | bash dom/media/webrtc/third_party_build/build_no_op_commits.sh | ||
* Save the newly updated patch-stack - run this in a bash shell | |||
(source dom/media/webrtc/third_party_build/use_config_env.sh ; \ | (source dom/media/webrtc/third_party_build/use_config_env.sh ; \ | ||
./mach python dom/media/webrtc/third_party_build/save_patch_stack.py \ | ./mach python dom/media/webrtc/third_party_build/save_patch_stack.py \ | ||
| Line 35: | Line 38: | ||
--separate-commit-bug-number $MOZ_FASTFORWARD_BUG | --separate-commit-bug-number $MOZ_FASTFORWARD_BUG | ||
) | ) | ||
* bootstrap and sanity build | |||
./mach --no-interactive bootstrap --application-choice=browser && ./mach build | ./mach --no-interactive bootstrap --application-choice=browser && ./mach build | ||
* Run loop-ff.sh | |||
bash dom/media/webrtc/third_party_build/loop-ff.sh | bash dom/media/webrtc/third_party_build/loop-ff.sh | ||
* Push to elm - after the loop-ff.sh script completes with the message <code>===loop-ff=== Completed fast-foward to {short-sha}</code>, run the following step to push the new version of libwebrtc to elm: | |||
hg push -r tip | |||
== Operational Notes == | |||
=== Reacting to loop-ff.sh === | === Reacting to loop-ff.sh === | ||
Three main types of errors will cause <code>loop-ff.sh</code> to exit: the first is a git rebase conflict, the second is a failure to regenerate build files (due to BUILD.gn changes), and the third is a build error due to API changes in upstream code. | Three main types of errors will cause <code>loop-ff.sh</code> to exit: the first is a git rebase conflict, the second is a failure to regenerate build files (most often due to BUILD.gn changes or missing/added files), and the third is a build error due to API changes in upstream code. | ||
* When fixing rebase errors or making other changes to fix build issues in moz-libwebrtc (any changes under third_party/libwebrtc), mercurial commit messages follow the pattern: | |||
Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description} | Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description} | ||
| Line 57: | Line 62: | ||
--repo-path $MOZ_LIBWEBRTC_SRC \ | --repo-path $MOZ_LIBWEBRTC_SRC \ | ||
--target-branch-head $MOZ_TARGET_UPSTREAM_BRANCH_HEAD ) | --target-branch-head $MOZ_TARGET_UPSTREAM_BRANCH_HEAD ) | ||
* If instructed to run <code>commit-build-file-changes.sh</code>, its auto-generated mercurial commit messages follow the pattern: | |||
Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description} - moz.build file updates | Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description} - moz.build file updates | ||
* When making changes to fix Mozilla code (code outside of third_party/libwebrtc) in response to changes upstream, mercurial commit messages follow the pattern: | |||
Bug {current-fast-forward-bug-number} (MOZ) - {description} | Bug {current-fast-forward-bug-number} (MOZ) - {description} | ||
If multiple kinds of changes are needed, commit hygiene is a must: (MOZ) patches contain changes to files outside of third_party/libwebrtc, (fix-xxx) commits contain changes to third_party/libwebrtc files, and finally "(fix-xxx) ... - moz.build file updates" contain only third_party/libwebrtc moz.build changes. | If multiple kinds of changes are needed to fix build errors, commit hygiene is a must: (MOZ) patches contain changes to files outside of third_party/libwebrtc, (fix-xxx) commits contain changes to third_party/libwebrtc files, and finally "(fix-xxx) ... - moz.build file updates" contain only third_party/libwebrtc moz.build changes. | ||
=== | === General tips === | ||
* Stopping the loop-ff.sh manually is best done when in the <code>Test build</code> phase using Ctrl-c. | |||
* Rather than waiting until the end of the process update process, you may push to elm to confirm build fixes, or for others to follow along. It is best to push a commit that successfully builds, but this isn't a hard and fast rule. Sometimes, it may be helpful to push a commit that is broken to get help from others, or when the fixes are complicated enough that you feel a review is necessary. | |||
* If loop-ff.sh reveals an issue that is best dealt with in a followup, one can commit a temporary patch marked with <code>REPO-elm</code> in the first line of its description, i.e. the "title". The temporary commit allows loop-ff.sh to continue, whereas <code>REPO-elm</code> makes sure the commit cannot be pushed to any other mozilla repo than elm (and try). | * If loop-ff.sh reveals an issue that is best dealt with in a followup, one can commit a temporary patch marked with <code>REPO-elm</code> in the first line of its description, i.e. the "title". The temporary commit allows loop-ff.sh to continue, whereas <code>REPO-elm</code> makes sure the commit cannot be pushed to any other mozilla repo than elm (and try). | ||
* While loop-ff.sh is running, in a separate terminal window it may be helpful to see the overall progress by running: | * While loop-ff.sh is running, in a separate terminal window it may be helpful to see the overall progress by running: | ||
tail -f .moz-fast-forward/logs/log-loop-ff.txt | grep loop-ff | tail -f .moz-fast-forward/logs/log-loop-ff.txt | grep loop-ff | ||
* [[libwebrtc GYP generated Java files|GYP generated Java files]] | * [[libwebrtc GYP generated Java files|GYP generated Java files]] | ||
* If instructed to run [https://searchfox.org/mozilla-central/rev/3b707c8fd7e978eebf24279ee51ccf07895cfbcb/dom/media/webrtc/third_party_build/extract-for-git.py#9-12 extract-for-git.py], note that <code>loop-ff.sh</code> has more context than <code>verify_vendoring.sh</code> and offers information that is relevant: Specifically, the | * If instructed to run [https://searchfox.org/mozilla-central/rev/3b707c8fd7e978eebf24279ee51ccf07895cfbcb/dom/media/webrtc/third_party_build/extract-for-git.py#9-12 extract-for-git.py], note that <code>loop-ff.sh</code> has more context than <code>verify_vendoring.sh</code> and offers information that is relevant: Specifically, the guidance about <code>tip::tip</code>: | ||
If you have previously run loop-ff.sh successfully, there may be a new | If you have previously run loop-ff.sh successfully, there may be a new | ||
change to third_party/libwebrtc that should be extracted and added to | change to third_party/libwebrtc that should be extracted and added to | ||
| Line 78: | Line 85: | ||
git am mailbox.patch) | git am mailbox.patch) | ||
:Replace <code>tip</code> here with your (fix-xxx) mercurial patch, if it is not at tip (e.g. if it is followed by (fix-xxx) - moz.build file updates). | |||
=== Recovering from errors === | |||
* If a particularly complicated rebase conflict is encountered, it is possible to start over the most recent vendoring step by resetting some state and restoring the patch stack. | * If a particularly complicated rebase conflict is encountered, it is possible to start over the most recent vendoring step by resetting some state and restoring the patch stack. Run these commands in a bash shell: | ||
rm .moz-fast-forward/*.resume # resets the resume logic in loop-ff.sh | rm .moz-fast-forward/*.resume # resets the resume logic in loop-ff.sh | ||
hg revert third_party/libwebrtc && hg purge third_party/libwebrtc | hg revert third_party/libwebrtc && hg purge third_party/libwebrtc | ||
| Line 89: | Line 94: | ||
=== Review === | === Review === | ||
* Simple changes don't require individual review, but if you'd feel better about getting reviewed, push the vendored commit to elm (which may break all the builds). Then put the patch up for review. After r+, land it using Lando to elm. | |||
== Waiting to land == | |||
== | |||
=== Daily rebase on elm (after completing loop-ff process) === | === Daily rebase on elm (after completing loop-ff process) === | ||
Periodically, ideally daily to minimize surprises from mozilla-central, rebasing elm onto the latest mozilla-central is recommended after completing the update process. Rebases should be completed until the request for merging elm to moz-central is made. | Periodically, ideally daily to minimize surprises from mozilla-central, rebasing elm onto the latest mozilla-central is recommended after completing the update process. Rebases should be completed until the request for merging elm to moz-central is made. | ||
| Line 108: | Line 102: | ||
Historical note: we use a script to rebase because 1) it regenerates the moz.build files on the commits where those are changed. This avoids the typically massive rebase conflict if anyone has touched the build files. 2) it exports/imports each commit individually to limit rebase conflicts on individual patches, making it easier to deal with issues. | Historical note: we use a script to rebase because 1) it regenerates the moz.build files on the commits where those are changed. This avoids the typically massive rebase conflict if anyone has touched the build files. 2) it exports/imports each commit individually to limit rebase conflicts on individual patches, making it easier to deal with issues. | ||
bash dom/media/webrtc/third_party_build/elm_rebase.sh | bash dom/media/webrtc/third_party_build/elm_rebase.sh | ||
If one would like to make changes to the patch ordering during the rebase operation (say, for example, a fix-commit was necessary but is separated from the initial upstream patch), use the following command line: | |||
STOP_FOR_REORDER=1 bash dom/media/webrtc/third_party_build/elm_rebase.sh | |||
=== Requesting merge from elm to mozilla-central === | === Requesting merge from elm to mozilla-central === | ||