MozillaWiki

Packaging Android host utilities: Difference between revisions

Page Discussion

Packaging Android host utilities (view source)

Revision as of 22:52, 9 January 2019

1,522 bytes added ,  9 January 2019
Major update
(Major update)
Line 1: Line 1:
== Packaging Android host utilities ==
== Packaging Android host utilities ==


The host utilities are executable files that run on a *host* machine. The utilities provide services to an Android *target* device, including a web server and certificate authority. For historical reason, the host utilities include an essentially complete version of Firefox.  Here's how to package new versions of the host utilities.
Host utilities are executable files that run on a *host* machine. These utilities provide services to an Android *target* device, such as running a web server or a certificate authority. Host utilities must be periodically updated with a good build from mozilla-central or mozilla-inbound due to addition of new tests targeted at Android and/or changing of component behavior.


First, identify the target build.  Generally, prefer Beta channel builds to Nightly channel builds.
This page will provide instruction on how to package new versions of the host utilities.


For the version 60 update, host-utils was based on builds from https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=bb369804a51e7665c0b44d3778681ca132cb1c2c. See bug 1433279.
=== Generate new archive ===


=== Linux ===
On Treeherder, identify a target build, preferably from mozilla-central or mozilla-inbound. Check the test output to ensure a reasonably good build (one without too many failures).


==== Generating new archive ====
example: for the version 66.0 update, [https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c9fa642e9e3aecaa8852f5c647904eecddaa4450&selectedJob=216676591 this build] was selected. For more details, please see [https://bugzilla.mozilla.org/show_bug.cgi?id=1514075 bug 1514075].


Follow these steps to locate and download the necessary files.
==== Linux ====


# From the selected treeherder build, identify "Linux opt" and "Linux x64 opt".  
For Linux, the steps are fairly straightforward. Follow these steps to locate and download the necessary files.
# Click on the B icon, which will bring up a pane below.  
 
# Identify and click on the hash on the left hand pane called "Task".
# From the selected treeherder build, identify '''Linux x64 opt'''.
# In the newly opened TaskCluster tab, click on tab named "Run Artifacts".
# Click on the green B icon, which will bring up a pane covering bottom third of the window.
# Download target.common.tests.tar.gz & target.tar.bz2.
# On the bottom left pane is a header named '''Task'''. Click on the hash.
# A new tab will open and load Taskcluster. Click on tab named "Run Artifacts".
# Download target.common.tests.tar.gz.
# Download target.tar.bz2.
# Follow the contents of the script below:
# Follow the contents of the script below:


<pre>
<pre>
tar xvf target.tar.bz2
tar xvf target.tar.bz2
tar xvf target.common.tests.tar.gz "bin/*"
tar xvf target.common.tests.tar.gz -C 'temp_common'
rm firefox/firefox*
rm firefox/firefox*
rm -r firefox/browser
rm -r firefox/browser
mv bin/* firefox
mv 'temp_common'/bin/* firefox
mv firefox host-utils-60.0a1.en-US.linux-x86_64
mv firefox host-utils-66.0a1.en-US.linux-x86_64
tar cvf host-utils-66.0a1.en-US.linux-x86_64.tar host-utils-66.0a1.en-US.linux-x86_64
gzip host-utils-66.0a1.en-US.linux-x86_64.tar
</pre>
 
Repeat for Linux 32bit, substituting x86 for x86_64 where necessary.
 
==== macOS ====
 
Preparing for macOS is not quite so simple. Due to requirements of macOS, it's best to do this on a Mac OS X machine so that you can codesign the host utility binaries. If binaries are unsigned, users will be prompted at every invocation to "allow network connections". For more information on signing macOS binaries, see [http://apple.stackexchange.com/a/150711 here].
 
# Navigate to [https://ftp.mozilla.org/pub/firefox/nightly/ Mozilla FTP].
# Drill down to the appropriate year, month. Select a date that is not l10n.
# Ensure files in the directory are prefixed with the version of Firefox that is desired.
# Download firefox-66.0a1.en-US.mac.common.tests.tar.gz.
# Download firefox-66.0a1.en-US.mac.dmg.
# Follow the contents of the script below:
 
<pre>
tar xvf firefox-66.0a1.en-US.mac.common.tests.tar.gz 'bin/*'
open firefox-66.0a1.en-US.mac.dmg
cp -R /Volumes/Firefox\ Nightly/Firefox\ Nightly.app/Contents/MacOS/* bin
cp -R /Volumes/Firefox\ Nightly/Firefox\ Nightly.app/Contents/Resources/* bin
find bin -type f -perm +111 -print | grep -v \\. | xargs sudo codesign --force --deep --sign -
mv bin host-utils-66.0a1.en-US.mac
tar cvf host-utils-66.0a1.en-US.mac.tar host-utils-66.0a1.en-US.mac/*
gzip host-utils-66.0a1.en-US.mac.tar
</pre>
</pre>


==== Uploading to ToolTool ====
==== Uploading to ToolTool ====


1. Prepare new archive for upload:
# Compare contents of current archive to the new archive. For instructions on existing archive, see the section below.
 
# Ensure uploading user has a valid token at [https://mozilla-releng.net/tokens/ Mozilla Releng].
# Upload the archive:
<pre>
<pre>
tar cvf host-utils-60.0a1.en-US.linux-x86_64.tar host-utils-60.0a1.en-US.linux-x86_64
python tooltool.py upload [name_of_archive] --authentication-file=[token_location] --message [commit_message]
gzip host-utils-60.0a1.en-US.linux-x86_64.tar
</pre>
</pre>
# Update the manifest in testing/config/tooltool-manifests/linux64/hostutils.manifest.


''replace the version numbers as appropriate.''
If updating host utilities for Linux, repeat using 32bit/x86 archives.


2. Compare contents of current archive to the new archive. For instructions on existing archive, see the section below.
=== Verification ===


3. Ensure uploading user has a valid token at [https://mozilla-releng.net/tokens/ Mozilla Releng].
ToolTool will return an updated manifest once the archives are uploaded and processed. This new manifest must first be tested, follow the steps below:


4. Upload the archive:
==== Linux ====
 
Changes to host utilities for Linux can be tested on the try server.
 
# Update to current tip by hg pull and hg update -r tip -C
# Copy and paste the updated manifest to the hostutils.manifest file for Linux 64 and Linux 32.
# Initiate a try run of all Android tests:


<pre>
<pre>
python tooltool.py upload [name_of_archive] --authentication-file=[token_location] --message [commit_message]
./mach try fuzzy -q=android-em-*
</pre>
</pre>


5. Update the manifest in testing/config/tooltool-manifests/linux64/hostutils.manifest.
==== macOS ====
 
6. Repeat, using "Linux opt" archives, for 32 bit.


=== Mac OS X ===
Changes to host utilities for macOS cannot be tested on try server, therefore the following steps are taken:
 
Preparing for Mac OS X is not quite so simple.  Due to changes in the way that Mac OS X codesigns binaries, it's best to do this on a Mac OS X machine so that you can codesign the host utility binaries.  If you don't codesign them, you will be prompted every invocation to "allow network connections".  See http://apple.stackexchange.com/a/150711 for a discussion of the issue and the codesigning invocation used below.  You will be prompted for your password (by <tt>sudo</tt>), but it's possible super user permissions are not needed.


# Ensure the currently selected MOZCONFIG designates an Android build target.
# Sanitize and rebuild with the following:
<pre>
<pre>
wget http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/firefox-37.0a2.en-US.mac.tests.zip
hg pull
unzip firefox-37.0a2.en-US.linux-x86_64.tests.zip "bin/*"
hg update -r tip -C
wget http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/firefox-37.0a2.en-US.mac.dmg
./mach clobber
open firefox-37.0a2.en-US.mac.dmg
./mach build
cp -R /Volumes/FirefoxDeveloperEdition/FirefoxDeveloperEdition.app/Contents/MacOS/* bin
cp -R /Volumes/FirefoxDeveloperEdition/FirefoxDeveloperEdition.app/Contents/Resources/* bin
find bin -type f -perm +111 -print | grep -v \\. | xargs sudo codesign --force --deep --sign -
mv bin host-utils-37.0a2.en-US.mac
</pre>
</pre>
# Once build is complete, package the apk using ./mach package
# Install on device with ./mach install
# Run a short test with ./mach robocop testAboutPage
# Accept offer to update host utilities.


=== Download existing archive ===
If tests complete without issues, the generated host utilities are deemed to be good.
 
== Download existing archive ==


It is possible to download existing host utilities.
It is possible to download existing host utilities.


1. locate the existing manifest file using [https://searchfox.org/mozilla-central/source/testing/config/tooltool-manifests/linux64/hostutils.manifest SearchFox] for both 32bit and 64bit.
# Locate the existing manifest file using [https://searchfox.org/mozilla-central/search?q=hostutils.manifest&path=SearchFox].
 
# Using ToolTool, download the host utilities:
2. using ToolTool, download the host utilities:
<pre>
<pre>
python tooltool.py fetch -m old_hostutils_manifest.tt
python tooltool.py fetch -m old_hostutils_manifest.tt
</pre>
</pre>
Egao
74

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1206036"