Mobile/Fennec/Android: Difference between revisions

Revision as of 20:48, 14 June 2012

Setup a Build Environment

Below are instructions for setting up a build environment on Ubuntu. See the build docs here for instructions on setting up a build environment on other OSs.

Note: It is not currently possible to compile Fennec on Windows. If you have a Windows PC, see these posts for a Linux virtual machine to compile Fennec:

Quick Script

# ensure that the "partner" repositories are enabled in /etc/apt/sources.list, or sun-java6-jdk won't be found
# Ubuntu 11.10 Java has been removed from partner, can be downloaded from Oracle
sudo apt-get update
sudo apt-get install sun-java6-jdk mercurial ccache
sudo apt-get build-dep firefox
wget http://dl.google.com/android/ndk/android-ndk-r5c-linux-x86.tar.bz2
tar -xjf android-ndk-r5c-linux-x86.tar.bz2
wget http://dl.google.com/android/android-sdk_r15-linux.tgz
tar -xzf android-sdk_r15-linux.tgz
# go get lunch, this will take a while
./android-sdk-linux/tools/android update sdk -u
./android-sdk-linux/tools/android update adb

If you're using a 64-bit Ubuntu install, you'll need ia32-libs to allow the toolchain binaries to run.

 sudo apt-get install ia32-libs

If you're using 64-bit Fedora install

 yum install glibc.i686 ncurses-libs.i686 libstdc++.i686 zlib.i686

Explained

Install Java

First install the Sun Java jdk6, which the Android SDK depends on. If you're on Ubuntu (pre-11.10), you'll need to enable the partners repo to get it. Java 7 does not work.

# Ubuntu pre-11.10
sudo apt-get update
sudo apt-get install sun-java6-jdk
sudo update-java-alternatives -s java-6-sun

# Ubuntu 11.10 and after
Download Java from Oracle
sudo mkdir /opt/java
sudo mv ~/Downloads/jdk-6u29-linux-x64.bin
sudo chmod +x ./jdk-6u29-linux-x64.bin
sudo ./jdk-6u29-linux-x64.bin
ln -f -s /opt/java/jdk1.6.0_29/bin/* /usr/local/bin/
# something like sudo update-alternatives --install /usr/local/bin/java java /opt/java/jdk1.6.0_29/bin/ 1 is close to the distro way of registering java

Install Gecko Requirements

Then install the usual stuff needed for a firefox build, you probably already have it

sudo apt-get install mercurial ccache
sudo apt-get build-dep firefox

Install Android NDK

Download and extract the Android NDK. NDK revs 4-7 have been tested and are known to work. Builders currently use NDKr5c.

wget http://dl.google.com/android/ndk/android-ndk-r5c-linux-x86.tar.bz2
tar -xjf android-ndk-r5c-linux-x86.tar.bz2

or the same thing with http://dl.google.com/android/ndk/android-ndk-r5c-darwin-x86.tar.bz2 on Mac.

Install Android SDK

You should just install the latest Android SDK, we set the API level in our manifest files. The sdk download will take a while, make sure you have a decent internet connection and go get coffee, or maybe lunch.

You will need SDK version at least 14.

wget http://dl.google.com/android/android-sdk_r14-linux_x86.tgz
tar -xzf android-sdk_r14-linux_x86.tgz
./android-sdk-linux_x86/tools/android update sdk --no-ui
./android-sdk-linux_x86/tools/android update adb

The Mac SDK is at http://dl.google.com/android/android-sdk_r16-macosx.zip

You will probably want to add the SDK's "tools" and "platform-tools" directory to the PATH environment variable in your shell, so that you can run adb and other tools easily. For example, if you installed the SDK in $HOME/opt, you could add the following line to the end of your .bashrc:

 export PATH=$PATH:$HOME/opt/android-sdk-linux_x86/platform-tools:$HOME/opt/android-sdk-linux_x86/tools

Increase linking speed (AKA using gold)

On some systems, linking libxul can takes several minutes. Using gold instead of ld can reduce this time (from around 7 minutes to a bit more than 1 minute on my machine ; YMMV, on another machine, it gets from 25 seconds to 10, or from 2 minutes to 1 minute with all system caches dropped).

To use gold, you have to build it yourself from binutils source. You can run the following commands to build it.

sudo apt-get install bison flex (or equivalent for your distribution)
mkdir ~/gold; pushd ~/gold
wget http://ftp.gnu.org/gnu/binutils/binutils-2.22.tar.bz2
tar xfj binutils-2.22.tar.bz2
mkdir binutils-build; pushd binutils-build
../binutils-2.22/configure --target=arm-linux-androideabi --prefix=$HOME/gold/arm-linux-androideabi --enable-gold --disable-werror
make
make install
popd

This will create a $HOME/gold/arm-linux-androideabi/bin/arm-linux-androideabi-ld.gold binary. Verify that this is correct and runs by running it with --version:

$ ./arm-linux-androideabi-ld.gold --version
GNU gold (GNU Binutils 2.22) 1.11
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

You can then copy it on top the ones in your ndk at <ndk>/toolchains/arm-linux-androideabi-*/prebuilt/<platform>/bin/arm-linux-androideabi-ld and <ndk>/toolchains/arm-linux-androideabi-*/prebuilt/<platform>/arm-linux-androideabi/bin/ld (be sure to back up the old ones first, in case something goes awry!).

Using mozilla-repackaged NDKs

NOTE: This section deals with the r7 variant of the NDK, which may or may not work correctly (see https://bugzilla.mozilla.org/show_bug.cgi?id=749184#c1). This is not currently what our shipping builds of Firefox are built against. Work is underway to make this the default NDK used on mozilla-central.

We have built custom NDK packages for Linux and Mac OS X that contain the Gold linker by default. These are posted to FTP. Mac OS X and Linux x86 are currently available. The Linux NDK is the exact bits we build on the buildbot CI machines.

To use either of these NDKs you'll need to set --with-android-ndk and --with-android-toolchain in your mozconfig.

An example mozconfig snippet that uses the Mac OS NDK follows:

ac_add_options --with-android-ndk="/Users/jhford/android-ndk-r7c"
ac_add_options --with-android-sdk="/Users/jhford/android-sdk-macosx/platforms/android-14"
ac_add_options --with-android-toolchain="/Users/jhford/android-ndk-r7c/toolchains/arm-linux-androideabi-4.6.3/prebuilt/darwin-x86_64"

Setup Fennec mozconfig

You build as you normally would (make -f client.mk), just with a different mozconfig.

 hg clone http://hg.mozilla.org/mozilla-central/ src
 cd src
 vi mozconfig-droid
 export MOZCONFIG=~/src/mozconfig-droid

Here's an example mozconfig:

# Add the correct paths here:
ac_add_options --with-android-ndk="$HOME/android-ndk-r5c"
ac_add_options --with-android-sdk="$HOME/android-sdk-linux_x86/platforms/android-14"
ac_add_options --with-android-version=5

# android options
ac_add_options --enable-application=mobile/android
ac_add_options --target=arm-linux-androideabi
ac_add_options --with-ccache
ac_add_options --enable-tests

mk_add_options MOZ_OBJDIR=./objdir-droid
mk_add_options MOZ_MAKE_FLAGS="-j9 -s"

Build

Instructions for getting Mozilla source code are here.

This will build (compile+link), package into apk, and deploy to the Android device. Therefore, before you start building connect your Android device and enable USB debugging

 hg clone http://hg.mozilla.org/mozilla-central/ src
 cd src
 make -f client.mk build_and_deploy

The package name will be like "objdir-droid/dist/fennec-14.0a1.en-US.android-arm.apk". The installed app name will be "Fennec".

Partial Builds: Beware the startup cache!

Developers who frequently update .js files sometimes like to manually only re-build the module they have updated. Partial builds for updated Javascript files may not work correctly if installed with adb install -r. This section describes the issue. For instance:

 make -C mobile/android && make package

This should result in the APK being properly updated with the updated Javascript, BUT the change may not be reflected on the device because of complications arising from the startup cache. If the startup cache from a previous run of Fennec exists on the device and contains an old version of the recently updated Javascript, Fennec will likely use the old version.

The startup cache is located in the profile, so deleting the profile ensures a new startup cache:

 adb uninstall org.mozilla.fennec
 adb install dist/fennec*.apk

Note that:

 adb install -r dist/fennec*.apk

retains the profile (and startup cache) -- so that's still a problem.

None of this is a problem with full builds:

 make -f client.mk && make package 
 adb install -r dist/fennec*.apk

works just fine, because the startup cache respects the buildid: If the buildid found in an APK is different from the buildid used when the startup cache was last updated, the startup cache is automatically deleted. A full build correctly updates the buildid -- a partial build may not.

A possible way (***I have not conclusively tested this***) to do a partial build, retain an old profile, and ensure the buildid is updated and the startup cache is subsequently deleted is:

 make export && make -C mobile/android && make -C toolkit/xre && make package
 adb install -r dist/fennec*.apk

Note that make export is necessary to update the buildid and make -C toolkit/xre is required to generate application.ini.h, trigger a rebuild of nsAndroidStartup.cpp, and update libxul.so.

Multilocale builds

Create a directory, clone mozharness, copy the config file for easy editing/usage:

mkdir multilocale
cd multilocale
hg clone http://hg.mozilla.org/build/mozharness
cp mozharness/configs/multi_locale/standalone_mozilla-central.py myconfig.py

Edit myconfig.py
- currently will check out m-c into a directory named 'mozilla-central' in this directory
- currently assumes your mozconfig is in this directory and named 'mozconfig'
- currently assumes your mozconfig sets your objdir name to 'objdir-droid'

pull mozilla-central

mozharness/scripts/multil10n.py --cfg myconfig.py --pull-build-source
# Alternately, you can hg clone http://hg.mozilla.org/mozilla-central

Run the script, which will create a multilocale apk

mozharness/scripts/multil10n.py --cfg myconfig.py

And you're done.

If you want to recompile or re-run the script, restore your objdir to en-US first!

mozharness/scripts/multil10n.py --cfg myconfig.py --restore-objdir

Also see this blog post for more information.

Testing

Device Managers

Most test suites - mochitests, reftests, xpcshell tests, and others - use a "device manager" module to communicate with the remote device. There are two device manager implementations: ADB and SUT.

The ADB device manager uses the adb command from the Android SDK to communicate with the remote device. To use the ADB device manager:

ensure the adb command is in your shell's PATH
set environment variable DM_TRANS=adb

The SUT device manager uses TCP to communicate with a remote agent, which must be installed on the device. To use the SUT device manager:

ensure TCP connectivity between the local host and the remote device: check that they are on the same network and you can ping each from the other
ensure the SUT agent is installed and started on the remote device
- the SUT agent APK is built alongside Fennec; just install <objdir-droid>/build/mobile/sutagent/android/sutAgentAndroid.apk
- The agent should be configured to start automatically with your phone when it boots. To start it immediately from an adb shell do: `am start -n com.mozilla.SUTAgentAndroid/.SUTAgentAndroid -a android.intent.action.MAIN`
set environment variable DM_TRANS=sut
set environment variable TEST_DEVICE=<ip address of remote device -- displayed by SUT agent>

Host Builds (MOZ_HOST_BIN)

Android mochitests and reftests require a parallel host build -- a build for the local host (desktop) environment. An environment variable, MOZ_HOST_BIN, must be set to point to that host build. MOZ_HOST_BIN is used to run xpcshell (for instance, to provide a simple web server), so the MOZ_HOST_BIN directory must contain xpcshell and all the shared libraries required by xpcshell. You can patch together these files from other sources, but the easiest way to get these is to download a xulrunner SDK build from here: http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-trunk/

You can also build desktop Firefox with a mozconfig might be as simple as:

 ac_add_options --enable-application=browser
 mk_add_options MOZ_OBJDIR=./objdir-x86

Then execute:

 export MOZCONFIG=mozconfig.x86 
 make -f client.mk
 MOZ_HOST_BIN=objdir-x86/dist/bin
 ls -l $MOZ_HOST_BIN/xpcshell

On Linux, the path to that build may also need to be in your LD_LIBRARY_PATH, unless your LD_LIBRARY_PATH contains ".":

 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.

Test Directory

We currently make the assumption that the device has a mounted sdcard when running tests (see bug 683895). If, and only if, you can't mount an sdcard for whatever reason run:

 mkdir /data/local/tests

Reftests

 MOZ_HOST_BIN="<abspath-to-objdir-x86>/dist/bin/" TEST_PATH=<path> make -C <objdir-droid> reftest-remote

For devices with screens too small you will see the error |can't drawWindow remote content|. You can ignore that resolution requirement by using the |ignore-window-size| options. NOTE: This may lead to false negative/positives.

 EXTRA_TEST_ARGS="--ignore-window-size"

Example TEST_PATH:

 TEST_PATH=layout/reftests/reftest-sanity/reftest.list

Notes:

If TEST_PATH is not specified, *all* reftests will be attempted: This usually fails/hangs and is not recommended. Specify a TEST_PATH or use the --total-chunks and --this-chunk arguments to reduce the number of tests executed.
sut recommended. Test results displayed and saved to reftest.log and reftest-remote.log.
adb works, but reports errors. Test results are not displayed but saved to reftest.log (additional diagnostics in reftest-remote.log).

Mochitests

 MOZ_HOST_BIN="<abspath-to-objdir-x86>/dist/bin/" TEST_PATH=<path> make -C <objdir-droid> mochitest-remote

Note that as of this writing (September 2011) many mochitests will not complete successfully. Try setting your TEST_PATH to "dom/tests/mochitest/dom-level1-core" or "content/smil/test" if you want to restrict yourself to a subset of tests that are known to pass.

TEST_PATH can be:

content/smil/test
content/xml/document/test
content/xslt/tests/mochitest
dom/src/json/test
dom/src/jsurl/test
dom/tests/mochitest/dom-level0
dom/tests/mochitest/dom-level1-core
dom/tests/mochitest/dom-level2-core
dom/tests/mochitest/ajax/mochikit
dom/tests/mochitest/ajax/scriptaculous
dom/tests/mochitest/ajax/jquery
dom/tests/mochitest/dom-level2-html
Harness_sanity
editor/composer/test
intl/uconv/tests
dom/tests/mochitest/orientation
dom/tests/mochitest/storageevent
layout/xul/test
modules/libjar/test/mochitest
layout/inspector/tests 
toolkit/xre/test
toolkit/components/microformats/tests
MochiKit-1.4.2/tests
parser/htmlparser/tests/mochitest
js

Notes:

A rooted device MAY be required (under investigation).
ADB or SUT device manager may be used.

Robotium

The Robotium / Robocop test suite verifies UI behavior in native Fennec.

Build and install native Fennec, then execute the mochitest-robotium make target.

 make -f client.mk
 cd <objdir>
 make package
 adb install -r dist/fennec-12*.apk
 MOZ_HOST_BIN="<abspath-to-objdir-x86>/dist/bin/" make mochitest-robotium

More info at https://wiki.mozilla.org/Auto-tools/Projects/Robocop.

Notes:

A rooted device is NOT required.
ADB device manager is used by default; SUT is also supported.
MOZ_HOST_BIN is used to launch xpcshell on the desktop to provide a web server.
Use TEST_PATH=<test-name> to run just one test at a time.
/mnt/sdcard must exist and be writable.

xpcshell

To prepare your device for xpcshell tests:

 adb shell mkdir /data/local/tests

Also be sure you have successfully built Fennec and generated an APK, as described above.

To run all tests referenced by the master xpcshell manifest:

 cd <objdir-droid>
 make xpcshell-tests-remote

To run a subset of tests in the specified directory:

 make -C <test-directory> xpcshell-tests-remote

Once either of the xpcshell-tests-remote commands has completed successfully, all test files have been copied to device, and it is then possible to run a single test quickly, without setup:

 make SOLO_FILE=<test-file> -C <test-directory> check-one-remote

Notes:

A rooted device IS required.
ADB device manager is used by default; SUT is also supported.
The test root directory cannot be on /sdcard -- it must be /data/local so that +x permissions can be set on xpcshell.

browser-chrome

Before you run tests, you will need to make sure you have packaged the tests in your object dir:

 make -C <objdir-droid> package-tests

There is currently no special make command to build and run browser-chrome tests, but it should be possible to make them run by calling:

 cd <objdir-droid>/_tests/testing/mochitest
 python runtestsremote.py --dm_trans=adb --test-path=mobile --browser-chrome --deviceIP=1.2.3.4 
                          --app=org.mozilla.fennec_$USER --xre-path=<objdir_x86>/dist/bin/

talos

NOTE: this requires python 2.5 or greater (tested up to 2.7)

See https://wiki.mozilla.org/Buildbot/Talos

This is 100% out of band from mozilla-central (same for desktop also).

 hg clone http://hg.mozilla.org/build/talos talos
 cd talos
 python INSTALL.py
 . bin/activate
 cd talos

Run these commands to check out the talos dependencies (assuming you are already in the "talos/talos" directory):

 cd page_load_test
 wget http://people.mozilla.org/~jmaher/mobile_tp4.zip
 unzip mobile_tp4.zip

Configure a talos profile. You can do this via adb or SUTAgent - if you want to use adb make sure you have rooted your device such that "adb shell" goes directly into a root shell. If you want to use adb to communicate with the device remotely, do something like this:

 remotePerfConfigurator -v -e org.mozilla.fennec \
                        --develop \
                        --activeTests tprovider \
                        --output trobo.yml \
                        --remotePort -1 \
                        --remoteDevice 1.2.3.4

If you want to use SUTAgent to communicate with the device, do something like this:

 remotePerfConfigurator -v -e org.mozilla.fennec --develop --activeTests tsvg --output tsvg.yml --results_url file://${PWD}/tsvg.txt --noChrome --remoteDevice <ip of your sutagent>

SUTAgent will not be able to read the application.ini on the device, so you will need to copy it. We highly recommend pulling the application.ini from the .apk and copying it to the talos/talos/remoteapp.ini file.

Unless two applications are signed with the same key they cannot read each others' /data/data directories, so SUTAgent cannot read org.mozilla.fennec's data directory. You can extract it from the fennec .apk or get it from adb from e.g. /data/data/org.mozilla.fennec/application.ini

 shell@android:/ $ su
 shell@android:/ # cat /data/data/org.mozilla.fennec/application.ini
 [App]
 Vendor=Mozilla
 Name=Fennec
 Version=10.0a1
 BuildID=20111031031100
 SourceRepository=http://hg.mozilla.org/mozilla-central
 SourceStamp=04b4ea333800
 ID={a23983c0-fd0e-11dc-95ff-0800200c9a66}
 
 [Gecko]
 MinVersion=1.9.2b5pre
 MaxVersion=10.0a1
 
 [XRE]
 EnableExtensionManager=1
 
 [Crash Reporter]
 Enabled=1
 ServerURL=https://crash-reports.mozilla.com/submit

Finally, run talos:

 python run_tests.py -d -n tsvg.yml

Note: As of this writing (Sept 27, 2011), the tgfx pageset does not currently work (it isn't run on the desktop either). Don't try it. :)

Note: When running robocop-based talos tests (tcheck, etc), be aware that some robocop-required files are not updated by Talos. For example, robocop.apk is not installed, and robotium.config is not re-generated and copied to device. Your best bet is to run mochitest-robotium first to make sure everything is up to date.

Aside: For a quick-and-dirty hacky way to run robocop-talos tests locally, see Mobile/Fennec/Android/LocalRoboTalos

S1/S2 Automation

These tests start Fennec with a URL and measure the time to throbber start, time to throbber stop, and drawing end times.

S1/S2 graphs can be viewed at: http://mrcote.info/phonedash/#/

Manual run instructions can be found at: https://etherpad.mozilla.org/fennec-perf-ts-take2

Eideticker

Eideticker measures perceived Firefox performance by video capturing automated browser interactions.

Eideticker graphs can be viewed at: http://wrla.ch/eideticker/dashboard/#/canvas

Trouble-shooting testing problems

Does your mozconfig contain "ac_add_options --disable-tests"?
- If so, you will see something like:

make: *** No rule to make target <your-test-target>.  Stop.

Is adb in your $PATH?
Is your device connected? Does it appear in the output from "adb devices"?
Can you run adb shell?
If running xpcshell, did you create /data/local/tests?
If running "make check-one-remote", did you first setup the device with "make xpcshell-tests-remote"?

Debugging

Using logcat

logcat is a tool that is going to show you some logs prompted by the device. It might be a good help if you don't want to or can't run gdb. You can use it by running this command:

 adb logcat

You can make things appear in logcat using printf_stderr. With debug builds, NS_WARNING, NS_ERROR and NS_ASSERTIONS will show up in logcat. A good idea is to actually run

 adb logcat -v time

so that the log includes timestamps for events. If you're trying to debug something, you may wish to pipe the logcat output through grep to filter out irrelevant things (most Fennec-related output will be viewable by "adb logcat | grep Gecko"), but remember that when attaching log output to a bug you should include unfiltered output as there may be relevant log entries under other tags. Additional tips on how to use adb logcat can be found at http://aaronmt.com/?p=503

Using aLogCat

If you don't have the Android SDK installed, you can still extract logs using an application called aLogCat. Install it from the Android Market. Use it to capture logs and attach the logs to bugs.

https://market.android.com/details?id=org.jtb.alogcat&hl=en

Once you have alogcat installed, just use Fennec as you would normally. Upon encountering a bug or issue, start the aLogcat app (as soon as possible after seeing the Fennec issue) and select "Share" or "Save" from the menu to send it via email or save it to the SD card. The log can then be attached to a bug or sent to a developer. As with adb logcat, it is better to have a log with timestamps than without timestamps. To enable timestamps in the log, select "Preferences" from the aLogcat menu, and change the "Format?" option to "Time".

If you need to, you can search for some kinds of Fennec-related output by using the "Filter" menu item and entering "Gecko". However, when submitting logs for bug reports, please make sure you clear the filter and include all of the available log data.

JavaScript dump()

To use the dump() function in JavaScript to write to the log:

Go to about:config and set browser.dom.window.dump.enabled to "true"
Run the following ADB commands:

adb shell stop
adb shell setprop log.redirect-stdio true
adb shell start

Using nVidia GDB

Nvidia's gdb is better: http://developer.download.nvidia.com/tegra/files/tegra-gdb-20100902.zip You probably want to use JimDB instead of Nvidia's gdb

Using JimDB

jchen has been working on a better Android gdb ('jimdb'). source, and build instructions. For best results, rebuild everything.
Prebuilt binaries are available but problems with them have been reported.
- The link to the prebuilt binaries seems broken, as of March 30 2012 (bjacob).
JimDB:
- has most fixes from NDK gdb
- integrates Android libthread_db (fixed dougt's problem debugging threads)
- has Python scripting support
- couple of existing Python scripts (or if using prebuilt binaries above, just run 'git pull' under the 'utils' directory)
  - feninit is a tool to initialize the GDB environment (support for multiple devices, multiple object directories; launches Fennec, pulls libraries, sets paths, and attaches gdbserver automatically)
  - tracebt is a WIP stack unwinder that traces instructions to find stack frames
  - idea for another tool to parse and integrate logcat into gdb
  - other ideas welcome!
see Mobile/Fennec/Android/GDB for installation instructions

Once JimDB is installed, just run its gdb executable. This is will take care of everything (it will start gdbserver on the device, start Fennec, and start gdb on your computer):

 ./moz-gdb/bin/gdb

Using Debug Intent

Note: this is not useful with JimDB. If you want to use JimDB, just start it.

In order to attach before things get running, launch with (replace "unofficial" with your login name):

  adb shell am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec_foobar/.App

(Replace foobar by your username)

and just click launch once gdb is attached. If you need to debug a crash that happens before XRE_Main is called, the patch on bug 572247 may be useful.

this script [1] will attach gdbserver for you

Getting dalvik java stack dumps using gdb

(gdb) call handleSigQuit()

this will dump a stack trace to gDvm.stackTraceFile which defaults to "/data/anr/traces.txt"

Note: this will only work if you have symbols for dalvik. (It may be possible to get the address for handleSigQuit by reading the symbol table using sigaction.)

Note: deleting /data/anr/traces.txt will cause this method to stop working. You can fix it by recreating the file with $ echo "" > traces.txt

Debugging with jdb

Inspired by http://asantoso.wordpress.com/2009/09/26/using-jdb-with-adb-to-debugging-of-android-app-on-a-real-device/

After staring fennec with jimdb the following seems to work at least a little.

 $ adb jdwp # lists the pids of processes hosting jdwp
 5543
 $ adb -d forward tcp:8686 jdwp:5543
 $ jdb -J-Duser.home=. -connect com.sun.jdi.SocketAttach:hostname=localhost,port=8686  -sourcepath ~/mozilla/maple/mobile/android/base/

You can also use eclipse for debugging in a similar way by setting up for debuggin "Remote Java application"

This doesn't seem to work for me in Ubuntu 12.04 anymore

Debugging with eclipse

You need to find the PID of your fennec process. Forward it to a local TCP socket as in "Debugging with jdb."

In Eclipse, Run > Debug configurations... Remote Java Application. Change the port to the TCP port you specified in your adb command. Under Source, navigate to your checkout, then into mobile/android.

Eclipse looks for source code in a specific location. You need to create the directory hierarchy: mobile

 /android
   /org
     mozilla/
       gecko -> ../../base

That is, in mozilla-central/mobile/android, create org/mozilla, and put the symlink gecko pointing to mozilla-central/mobile/android/base.

You may also want to add more debugging information and can do that like this:

diff --git a/config/android-common.mk b/config/android-common.mk
index 4591239..a47726a 100644
--- a/config/android-common.mk
+++ b/config/android-common.mk
@@ -70,6 +70,6 @@ JAVAC_FLAGS = \
   -classpath $(JAVA_CLASSPATH) \
   -bootclasspath $(JAVA_BOOTCLASSPATH) \
   -encoding UTF8 \
-  -g:source,lines \
+  -g:source,lines,vars \
   -Werror \
   $(NULL)

Arguments and Environment Variables

If you need to set an environment variable at run time, append --es env# VAR=VAL to your activity manager command where # is the ordered number of variables for example:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_<user>/.App --es env0 VAR=val --es env1 FOO=bar

If you need to pass arguments at run time, append --es args "<your-args>" to your activity manager command. For example, to launch with a specific profile:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_<user>/.App --es args "--profile /mnt/sdcard/myprofile"

To launch with a specific URL, use the am -d option to set the intent's data URI:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_<user>/.App -d 'http://www.mozilla.org'

PR Logging

You can use the env vars as described above to enable NSPR logging:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_<user>/.App --es env0 NSPR_LOG_MODULES=all:5 --es env1 NSPR_LOG_FILE=/mnt/sdcard/log.txt

If no file is specified, logging is directed to the android logs:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_<user>/.App --es env0 NSPR_LOG_MODULES=all:5

Look for lines marked "PRLog" in the adb logcat output.

debugging without rooting

with Froyo you can debug without rooting your phone. Instructions are below. See also Fennec/Android/GDBNoRoot for another guide on how to do this.

First thing, to make this work with the nvidia gdb (which I found more reliable than the android r3 gdb) you need to modify install.sh and debug.sh.

first, change the location where install.sh copies gdbserver to somewhere writable by a non-root process. I used /data/local. Be sure to update that both in the push command and the chmod command.

second, update debug.sh with the new location of gdbserver.

finally, you'll need to add run-as $2 to the adb shell command that launches gdbserver. In the end you should have: install.sh:

#!/bin/sh
mkdir lib
adb push prebuilt/gdbserver /data/local
adb shell chmod 755 /data/local/gdbserver
for file in $(adb shell ls /system/lib | tr "\n" " " | tr "\r" " "); do
  adb pull /system/lib/$file lib
done
adb pull /system/bin/app_process lib

The above will miss some vendor drivers, you can use busybox to find all shared libraries

for file in $(adb shell /data/local/busybox find / -name "*.so" | tr "\n" " " | tr "\r" " "); do  adb pull /$file lib_unlock_nexus/; done

debug.sh:

#!/bin/sh
if [ $# -ne 2 ]
then
  echo "usage: $0 /path/to/your/library.so packagename.of.your.activity"
  echo "for example:"
  echo "  $0 /code/mydemo/libs/armeabi/libmydemo.so com.nvidia.devtech.mydemo"
  exit
fi

if [ ! -f $1 ]
then
  echo "ERROR: That library file doesn't exist"
  exit
fi

cp $1 lib

p=`adb shell ps | grep $2 | awk '{print $2}'`
if [ "$p" = "" ];
then
  echo "ERROR: That doesn't seem to be a running process. Please make sure your"
  echo "application has been started and that you are using the correct"
  echo "namespace argument."
  exit
fi

adb forward tcp:12345 tcp:12345
adb shell run-as $2 /data/local/gdbserver --attach :12345 $p

Attaching GDB

Assuming you have the nvidia gdb at the top of your home directory and the app_process binary in the current working directory.

 ~/nvidia-gdb/prebuilt/linux-x86/arm-eabi-gdb ./app_process

Reading back the framebuffer

If you need to verify what is in the back buffer at a particular time, you can cleverly call functions from gdb to allocate memory, read back, and write that to disk.

You need to know the size of your framebuffer a priori; in this case, it's 480x699.

You also need to know what the GL enum values are, because unless you compile with -ggdb, you don't have #defines available to you in the debugger. Very helpful information: GL_RGBA = 0x1908, GL_UNSIGNED_BYTE = 0x1401. The rest you can find in gfx/gl/GLDefs.h.

You should be able to call glReadPixels directly, but in my experience that causes Fennec to crash. However, if you have a GLContext* lying around, as you often do, you can work around that problem.

(gdb) set $m = (int*)malloc(480*699*4)
(gdb) call aManager->mGLContext.mRawPtr->fReadPixels(0, 0, 480, 699, 0x1908, 0x1401, (void*)$m)
(gdb) set $f = fopen("/sdcard/outputfile", "wb+")
(gdb) call fwrite($m, 1, 480*699*4, $f)
$7 = 1342080
(gdb) call fclose($f)
$8 = 0

Now there is a file called /sdcard/outputfile that you can adb pull. But since it's just raw RGBA values, you need to be able to wrap that in PNG headers to display it. Jeff Muizelaar wrote a header called minpng.h that you can use to do so.

Get Jeff's minpng.h, and put it in a directory along with a driver c program:

#include "minpng.h"

int main(int argc, char* argv[])
{
	FILE* f = fopen(argv[1], "rb");
	int w = atoi(argv[2]);
	int h = atoi(argv[3]);
	char* d = (char*) malloc(w * h * 4);
	fread(d, w * h * 4, 1, f);
	fclose(f);
	write_png(argv[4], d, w, h);
}

Compile and run:

$ gcc -o minpng minpng.c
$ ./minpng outputfile 480 699 output.png

Using Rendertrace (Maple)

Rendertrace is a utility that will dump layer position and timing information (such as drawing, upload) to the console. This information can pasted into the rendertrace web front end to visualize the layer position and event timeline. This will let you understand where you're gecko is spending its time and why were checkerboarding.

To enable go in 'gfx/layers/RenderTrace.h' and uncomment '#define MOZ_RENDERTRACE'. Rebuild and run 'adb logcat | grep RENDERTRACE', paste the result in http://people.mozilla.org/~bgirard/rendertrace.html and hit 'reload'. For details talk to BenWa.

Using apitrace

Apitrace is a tool for tracing GL/EGL calls for debugging purposes. It basically uses an interim shared library called libapitrace that contains shadow gl* and egl* functions, which then get logged and then passed through to the real driver.

Use apitrace from https://github.com/apitrace/apitrace to build for desktop and android.

apt-get install libegl1-mesa-dev libgles1-mesa-dev libgles2-mesa-dev libqt4-dev cmake
git clone https://github.com/gw280/apitrace.git
cd apitrace

# Build for Android
export ANDROID_NDK=/path/to/your/ndk
cmake -DCMAKE_TOOLCHAIN_FILE=android/android.toolchain.cmake -DANDROID_API_LEVEL=9 -Bbuild-android -H.
make -C build-android -j8

# Build for desktop
cmake -H. -Bbuild
make -C build -j8

export EGL_SOFTWARE=true
./build/eglretrace -v /path/to/your/apitrace_log.trace

The Android build will create egltrace.so in build-android/wrappers, which you can then push to your device to /data/local:

adb push build-android/wrappers/egltrace.so /data/local

Restarting Fennec will cause it to load the apitrace library and the apitrace log will be saved to /data/data/org.mozilla.fennec_username/firefox.trace

You can then adb pull /data/data/org.mozilla.fennec_username/firefox.trace and analyse it on your desktop.

You can also use qapitrace as a GUI to inspect your trace files. (be sure to switch qapitrace to the EGL api using the options dialog)

These instructions provide a trace that does not include the Java GL code. To get traces including java code is more complicated. You need to use the patch from this bug https://bugzilla.mozilla.org/show_bug.cgi?id=749859 and this version of https://github.com/ideak/apitrace/tree/dev. Further, you'll need to build your own image/modify the current one to replace /init.rc. You also need to disable hardware acceleration of the UI (https://bug746703.bugzilla.mozilla.org/attachment.cgi?id=619009) Ask jrmuizel for more information if you want to do this.

Profiling

See https://wiki.mozilla.org/Mobile/Fennec/Android/Profiling.

Other useful tips and tricks

Tweaking UI prefs

By default, all of these prefs are set to "-1" in Fennec, meaning they take the values listed below, which are maintained in Axis.java.

Fractional values are specified in 1/1000th of a value; to specify a value of 0.3, write 300.

Note: You need to restart Fennec after changing these values.

Pref	Default value	Description
ui.scrolling.friction_slow	850	This fraction in 1000ths of velocity remains after every animation frame when the velocity is low.
ui.scrolling.friction_fast	970	This fraction in 1000ths of velocity remains after every animation frame when the velocity is high.
ui.scrolling.velocity_threshold	10	Below this velocity (in pixels per frame), the friction changes from friction_fast to friction_slow.
ui.scrolling.max_event_acceleration	12	The maximum velocity change factor between events, per ms, in 1000ths.
ui.scrolling.overscroll_decel_rate	40	The rate of deceleration when the surface has overscrolled, in 1000ths.
ui.scrolling.overscroll_snap_limit	300	The fraction of the surface which can be overscrolled before it must snap back, in 1000ths.
ui.scrolling.min_scrollable_distance	500	The minimum amount of space that must be present for an axis to be considered scrollable, in 1/1000ths of pixels.
gfx.displayport.strategy	1	The strategy we use to determine how display ports are calculated. 0 = fixed margin, 1 = velocity bias, 2 = dynamic resolution, 3 = no margins
gfx.displayport.strategy_fm.multiplier	1500	When gfx.displayport.strategy = 0 (fixed margin), the 1000th of each dimension of the viewport the displayport is sized to.
gfx.displayport.strategy_fm.danger_x	100	When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the width of the viewport the horizontal danger zone is set to. Danger zone is defined as the space at the edge of the viewport at which the viewport (and hence displayport) starts being changed.
gfx.displayport.strategy_fm.danger_y	200	When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the height of the viewport the vertical danger zone is set to.
gfx.displayport.strategy_vb.multiplier	1500	When gfx.displayport.strategy = 1 (velocity bias), the 1000th of each dimension of the viewport the displayport is sized to.
gfx.displayport.strategy_vb.threshold	32	When gfx.displayport.strategy = 1 (velocity bias), the threshold for velocity, in pixels/frame, when multiplied by the screen DPI.
gfx.displayport.strategy_vb.reverse_buffer	200	When gfx.displayport.strategy = 1 (velocity bias), the fraction of the buffer (in 1000ths) to be kept in the direction opposite the direction of the scroll.

Refresh the JS cache

XUL and JavaScript files (like browser.js) are cached for fast startup. If you change one of these files in your development build, the new file might not be picked up unless you also touch toolkit/xre/nsAndroidStartup.cpp and rebuild libxul. (See bug 695145 for details.)

killer script

#!/bin/sh
if [ $# -ne 1 ]
then
    echo "usage: $0  packagename.of.your.activity"
    echo "for example:"
    echo "  $0 org.mozilla.fennec"
    exit
fi

p=`adb shell ps | grep $1 | awk '{print $2}'`
if [ "$p" = "" ];
then
    echo "ERROR: That doesn't seem to be a running process. Please make sure your"
    echo "application has been started and that you are using the correct"
    echo "namespace argument."
    exit
fi

adb shell run-as $1 kill $p

.gdbinit

This is an example .gdbinit that uses the symbols from a locally built rom and automatically attaches to gdbserver. Note that putting a .gdbinit file inside a directory will make gdb load it thus you will not pollute your regular gdb init with those configurations.

set solib-search-path /home/blassey/android/system/out/target/product/passion/symbols/system/bin:/home/blassey/android/system/out/target/product/passion/symbols/system/lib/:/home/blassey/src/ndk5-m-c/objdir-droid-dbg/dist/bin
set solib-absolute-prefix /home/blassey/android/system/out/target/product/passion/symbols/system/lib/
target remote localhost:12345

Connecting Eclipse to Android Build tree

Clone scripts and templates:

 hg clone http://hg.mozilla.org/users/romaxa_gmail.com/eclipse_mobile
 cd eclipse_mobile

Modify mozconfig_values file, according to your build environment add absolute path to obj-build-dir and source dir, Ex:

 MOZOBJDIR=/home/romaxa/mozdev/mozillahg/mozilla-birch/objdir-droid
 MOZSRCDIR=/home/romaxa/mozdev/mozillahg/mozilla-birch

Generate project content (build tree must be fully compiled and make -C objdir package commands performed):

 run ./create_projects.pl
 eclipse project will be created in current folder

Create new project in Eclipse

 File->New->Project
 Android Project
 Next, Create project from existing source
 Select current folder as Location, Next
 Select Build Target "Android 4.0 / API 14"
 Next and Finish

Press Run App button in order to perform first build,

 On first run, some ADT plugin fnctionality removing bin/App.apk and resource.ap_)
 Don't know how to teach eclipse don't do that, but it breaks installable package.
 So in order to fix that problem
 after first Run App, execute in project folder
   ./fixup_links.pl
 It will update *.apk and *.ap_ symlinks

Press Run App button again

try to setup breakpoint in onCreate() and press Debug App button.

Rooting Android devices

See Rooting Android Devices.

Mobile/Fennec/Android: Difference between revisions

Revision as of 20:48, 14 June 2012

Setup a Build Environment

Quick Script

Explained

Install Java

Install Gecko Requirements

Install Android NDK

Install Android SDK

Increase linking speed (AKA using gold)

Using mozilla-repackaged NDKs

Setup Fennec mozconfig

Build

Partial Builds: Beware the startup cache!

Multilocale builds

Testing

Device Managers

Host Builds (MOZ_HOST_BIN)

Test Directory

Reftests

Mochitests

Robotium

xpcshell

browser-chrome

talos

S1/S2 Automation

Eideticker

Trouble-shooting testing problems

Debugging

Using logcat

Using aLogCat

JavaScript dump()

Using nVidia GDB

Using JimDB

Using Debug Intent

Getting dalvik java stack dumps using gdb

Debugging with jdb

Debugging with eclipse

Arguments and Environment Variables

PR Logging

debugging without rooting

Attaching GDB

Reading back the framebuffer

Using Rendertrace (Maple)

Using apitrace

Profiling

Other useful tips and tricks

Tweaking UI prefs

Refresh the JS cache

killer script

.gdbinit

Connecting Eclipse to Android Build tree

Rooting Android devices

Navigation menu

Search