Media/WebRTC/libwebrtc Update Process: Difference between revisions

← Older edit

Media/WebRTC/libwebrtc Update Process (view source)

Revision as of 18:46, 20 March 2024

1,850 bytes added , 20 March 2024

add note about ability to stop during the rebase operation

Mfroman

119

edits

@@ Line 1: / Line 1: @@
 This page describes the process for updating the in tree, vendored copy of libwebrtc using the fast-forward scripts.
-== Operation Checklist ==
+== Prerequisites ==
-=== Prerequisites ===
+* NOTE: all steps below assume the user is in the bash shell.
 * Create a new bug for the fast-forward update.  It should depend on the previous fast-forward bug.  [https://bugzilla.mozilla.org/show_bug.cgi?id=1800920 Bug 1800920] is an example.
 * Before each new merge cycle elm needs to be reset to Central.
@@ Line 14: / Line 14: @@
 * Starting with a fresh clone of elm is recommended for each new chromium milestone.
   hg clone --stream ssh://hg.mozilla.org/projects/elm
+* Verify the most recent commit is the .arcconfig update.  If the most recent commit doesn't look like "Bug xxx - FLOAT - REPO-elm - update .arcconfig repo callsign", run the following commands:
+  ./mach python dom/media/webrtc/third_party_build/restore_elm_arcconfig.py
+  hg push -r tip
-=== Update steps ===
+== Update steps ==
- # Start from the elm checkout
+* Start from the elm checkout
   cd elm
- # Update the default config file for the next upstream milestone
+* Update the default config file for the next upstream milestone
   bash dom/media/webrtc/third_party_build/update_default_config.sh
- # Prepare the github repo
+* Prepare the git repository - Running <code>prep-repo.sh</code> will display instructions on how to add recent moz-central changes made in <code>third_party/libwebrtc</code> to the top of the patch stack in our git repository.  This is expected and necessary for successfully vendoring changes into <code>third_party/libwebrtc</code>.
   bash dom/media/webrtc/third_party_build/prep_repo.sh
- # Prepare no-op tracking files for cherry-pick commits
+* Prepare no-op tracking files for cherry-pick commits
   bash dom/media/webrtc/third_party_build/build_no_op_commits.sh
- # Save the newly updated patch-stack - run this in a bash shell
+* Save the newly updated patch-stack - run this in a bash shell
   (source dom/media/webrtc/third_party_build/use_config_env.sh ; \
    ./mach python dom/media/webrtc/third_party_build/save_patch_stack.py \
@@ Line 35: / Line 38: @@
    --separate-commit-bug-number $MOZ_FASTFORWARD_BUG
   )
- # bootstrap and sanity build
+* bootstrap and sanity build
   ./mach --no-interactive bootstrap --application-choice=browser && ./mach build
- # Run loop-ff.sh
+* Run loop-ff.sh
   bash dom/media/webrtc/third_party_build/loop-ff.sh
-=== Push to elm ===
+* Push to elm - after the loop-ff.sh script completes with the message <code>===loop-ff=== Completed fast-foward to {short-sha}</code>, run the following step to push the new version of libwebrtc to elm:
- # after the loop-ff.sh script completes, run the following steps
- # to push the new version of libwebrtc to elm
- ./mach bootstrap --application=browser --no-system-changes
- ./mach build
   hg push -r tip
-=== Daily rebase on elm (after completing loop-ff process) ===
+== Operational Notes ==
-Periodically, ideally daily to minimize surprises from mozilla-central, rebasing elm onto the latest mozilla-central is recommended after completing the update process.  Rebases should be completed until the request for merging elm to moz-central is made.
+=== Reacting to loop-ff.sh ===
+Three main types of errors will cause <code>loop-ff.sh</code> to exit: the first is a git rebase conflict, the second is a failure to regenerate build files (most often due to BUILD.gn changes or missing/added files), and the third is a build error due to API changes in upstream code.
-Historical note: we use a script to rebase because 1) it regenerates the moz.build files on the commits where those are changed.  This avoids the typically massive rebase conflict if anyone has touched the build files.  2) it exports/imports each commit individually to limit rebase conflicts on individual patches, making it easier to deal with issues.
- bash dom/media/webrtc/third_party_build/elm_rebase.sh
-=== Requesting merge from elm to mozilla-central ===
-Typically, immediately after the code-freeze has lifted, a request is made for the sheriffs to merge elm to mozilla-central.  This done via email, recently to Sebastian Hengst.
-=== Post-merge Steps ===
-* After the merge from elm to mozilla-central is complete, the moz-libwebrtc github repo should be updated with the new branch of the commit stack for the new release.  There is script that will make the proper branch name and do the push:
- bash dom/media/webrtc/third_party_build/push_official_branch.sh
-=== Operational notes ===
+* When fixing rebase errors or making other changes to fix build issues in moz-libwebrtc (any changes under third_party/libwebrtc), mercurial commit messages follow the pattern:
-* Running <code>prep-repo.sh</code> with a fresh github clone of moz-libwebrtc will likely display instructions on how to add recent moz-central changes made in <code>third_party/libwebrtc</code> to the top of the patch stack in github.  This is expected and necessary for successfully vendoring changes into <code>third_party/libwebrtc</code>.
-* Two main types of errors will cause <code>loop-ff.sh</code> to exit; the first is a rebase conflict, the second is a build error due to api changes in upstream code.
-* When making changes to fix build issues in moz-libwebrtc (any changes under third_party/libwebrtc), mercurial commit messages follow the pattern:
   Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description}
@@ Line 75: / Line 62: @@
     --repo-path $MOZ_LIBWEBRTC_SRC \
     --target-branch-head $MOZ_TARGET_UPSTREAM_BRANCH_HEAD )
+* If instructed to run <code>commit-build-file-changes.sh</code>, its auto-generated mercurial commit messages follow the pattern:
+ Bug {current-fast-forward-bug-number} - (fix-{upstream-sha}) {description} - moz.build file updates
 * When making changes to fix Mozilla code (code outside of third_party/libwebrtc) in response to changes upstream, mercurial commit messages follow the pattern:
   Bug {current-fast-forward-bug-number} (MOZ) - {description}
+If multiple kinds of changes are needed to fix build errors, commit hygiene is a must: (MOZ) patches contain changes to files outside of third_party/libwebrtc, (fix-xxx) commits contain changes to third_party/libwebrtc files, and finally "(fix-xxx) ... - moz.build file updates" contain only third_party/libwebrtc moz.build changes.
+=== General tips ===
+* Stopping the loop-ff.sh manually is best done when in the <code>Test build</code> phase using Ctrl-c.
+* Rather than waiting until the end of the process update process, you may push to elm to confirm build fixes, or for others to follow along. It is best to push a commit that successfully builds, but this isn't a hard and fast rule.  Sometimes, it may be helpful to push a commit that is broken to get help from others, or when the fixes are complicated enough that you feel a review is necessary.
 * If loop-ff.sh reveals an issue that is best dealt with in a followup, one can commit a temporary patch marked with <code>REPO-elm</code> in the first line of its description, i.e. the "title". The temporary commit allows loop-ff.sh to continue, whereas <code>REPO-elm</code> makes sure the commit cannot be pushed to any other mozilla repo than elm (and try).
 * While loop-ff.sh is running, in a separate terminal window it may be helpful to see the overall progress by running:
   tail -f .moz-fast-forward/logs/log-loop-ff.txt | grep loop-ff
 * [[libwebrtc GYP generated Java files|GYP generated Java files]]
-* If you are requested to run <code>extract-for-git.py</code>, note that <code>loop-ff.sh</code> offers some information in initial vendoring that is is useful later, but not repeated in <code>verify_vendoring.sh</code>. Specifically, the tip about <code>tip::tip</code>:
+* If instructed to run [https://searchfox.org/mozilla-central/rev/3b707c8fd7e978eebf24279ee51ccf07895cfbcb/dom/media/webrtc/third_party_build/extract-for-git.py#9-12 extract-for-git.py], note that <code>loop-ff.sh</code> has more context than <code>verify_vendoring.sh</code> and offers information that is relevant: Specifically, the guidance about <code>tip::tip</code>:
      If you have previously run loop-ff.sh successfully, there may be a new
      change to third_party/libwebrtc that should be extracted and added to
@@ Line 90: / Line 85: @@
           git am mailbox.patch)
-* It may not always be <code>tip::tip</code> depending on what was necessary to fix the issue (say a BUILD.gn change, followed by moz.build generation, and those moz.build files committed in a separate commit).
+:Replace <code>tip</code> here with your (fix-xxx) mercurial patch, if it is not at tip (e.g. if it is followed by (fix-xxx) - moz.build file updates).
-* Simple changes don't require individual review, but if you'd feel better about getting reviewed, push the commit prior to elm (which will break all the builds). Then put the patch up for review. After r+, land it using Lando to elm.
-=== Backing out ===
+=== Recovering from errors ===
+* If a particularly complicated rebase conflict is encountered, it is possible to start over the most recent vendoring step by resetting some state and restoring the patch stack.  Run these commands in a bash shell:
-* Stopping the loop-ff.sh manually is best done when in the <code>Test build</code> phase using Ctrl-c.
-* If a particularly complicated rebase conflict is encountered, it is possible to start over the most recent vendoring step by resetting some state and restoring the patch stack.
   rm .moz-fast-forward/*.resume # resets the resume logic in loop-ff.sh
   hg revert third_party/libwebrtc && hg purge third_party/libwebrtc
   ./mach python dom/media/webrtc/third_party_build/restore_patch_stack.py --repo-path .moz-fast-forward/moz-libwebrtc
+=== Review ===
+* Simple changes don't require individual review, but if you'd feel better about getting reviewed, push the vendored commit to elm (which may break all the builds). Then put the patch up for review. After r+, land it using Lando to elm.
+== Waiting to land ==
+=== Daily rebase on elm (after completing loop-ff process) ===
+Periodically, ideally daily to minimize surprises from mozilla-central, rebasing elm onto the latest mozilla-central is recommended after completing the update process.  Rebases should be completed until the request for merging elm to moz-central is made.
+Historical note: we use a script to rebase because 1) it regenerates the moz.build files on the commits where those are changed.  This avoids the typically massive rebase conflict if anyone has touched the build files.  2) it exports/imports each commit individually to limit rebase conflicts on individual patches, making it easier to deal with issues.
+ bash dom/media/webrtc/third_party_build/elm_rebase.sh
+If one would like to make changes to the patch ordering during the rebase operation (say, for example, a fix-commit was necessary but is separated from the initial upstream patch), use the following command line:
+ STOP_FOR_REORDER=1 bash dom/media/webrtc/third_party_build/elm_rebase.sh
+=== Requesting merge from elm to mozilla-central ===
+Typically, immediately after the code-freeze has lifted, a request is made for the sheriffs to merge elm to mozilla-central.  This done via email, recently to Sebastian Hengst.
+=== Post-merge Steps ===
+* After the merge from elm to mozilla-central is complete, the moz-libwebrtc github repo should be updated with the new branch of the commit stack for the new release.  There is script that will make the proper branch name and do the push:
+ bash dom/media/webrtc/third_party_build/push_official_branch.sh