Auto-tools/Projects/Mozbase: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Development Practices: section for helper scripts)
 
(82 intermediate revisions by 5 users not shown)
Line 3: Line 3:
Mozbase is a set of easy-to-use Python packages forming a supplemental standard library for Mozilla. It provides consistency and reduces redundancy in automation and other system-level software. All of Mozilla's test harnesses use mozbase to some degree, including Talos, mochitest, reftest, Autophone, and Eideticker.
Mozbase is a set of easy-to-use Python packages forming a supplemental standard library for Mozilla. It provides consistency and reduces redundancy in automation and other system-level software. All of Mozilla's test harnesses use mozbase to some degree, including Talos, mochitest, reftest, Autophone, and Eideticker.


'''Repository''': https://github.com/mozilla/mozbase
This is the project page for mozbase development, outlining current development practices, methodologies, and protocol. Other resources related to mozbase:


'''Documentation''': http://mozbase.readthedocs.org
'''Repository''': https://hg.mozilla.org/mozilla-central under testing/mozbase
 
'''Documentation''': https://firefox-source-docs.mozilla.org/mozbase/


'''Bugs''':
'''Bugs''':
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&component=Mozbase&product=Testing bugs in the Mozbase component]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&component=Mozbase&product=Testing bugs in the Mozbase component]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozbase other mozbase bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozbase bugs tracked with whiteboard: mozbase]


Please file bugs against https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase
Please file bugs against https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase


The roadmap of getting mozbase on mozilla-central is detailed at https://wiki.mozilla.org/Auto-tools/Projects/MozBase/Roadmap
The roadmap of getting mozilla-central python to use mozbase is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Roadmap . The effort to this effect is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Automation-Refactor


''Mozbase requires python 2.5, 2.6, or 2.7''
Mozbase requires python 2.7 or greater
'''however, Talos still requires compatibility with python 2.4 ({{bug|734466}}), so mozbase dependencies in http://hg.mozilla.org/build/talos/file/tip/setup.py *must* be kept compatible with python 2.4'''


== Development Practices ==
== Development Practices ==
Line 24: Line 25:
=== Getting Help with Mozbase ===
=== Getting Help with Mozbase ===


Mozbase is developed by the [https://wiki.mozilla.org/Auto-tools Automation and Tools team].  It is under the governorship of [https://mozillians.org/en-US/wlach Will Lachance (:wlach)] and [https://mozillians.org/en-US/jhammel Jeff Hammel (:jhammel)].  Please feel free to stop by [irc://irc.mozilla.org/#ateam #ateam] on IRC with any questions and we will be happy to help you!  Or if something is wrong, [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase&cc=:jhammel please file a bug] and we'll look at it as soon as possible.
Mozbase is developed by the [https://wiki.mozilla.org/Auto-tools Automation and Tools team].  It is under the governorship of [https://mozillians.org/en-US/wlach Will Lachance (:wlach)] and [https://mozillians.org/en-US/u/ahal Andrew Halberstadt (:ahal)].  Please feel free to stop by [irc://irc.mozilla.org/#ateam #ateam] on IRC with any questions and we will be happy to help you!  Or if something is wrong, [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase please file a bug] and we'll look at it as soon as possible.


=== Installing Mozbase for Development ===
=== Installing Mozbase for Development ===
Line 30: Line 31:
==== Initial Setup ====
==== Initial Setup ====


(Some of these steps may not be necessary if you've already set up your computer for development.)
* Install [http://www.pip-installer.org/en/latest/installing.html python-pip].
 
* Install [https://developer.mozilla.org/en/Python/Virtualenv virtualenv] and [https://bitbucket.org/dhellmann/virtualenvwrapper virtualenvwrapper]:
* Install [https://developer.mozilla.org/En/Simple_Firefox_build#Install_build_prerequisites Mozilla build prerequisites], as well as a copy of [http://git-scm.com git].
    $ sudo pip install virtualenv
 
    $ sudo pip install virtualenvwrapper
* (''windows-only'') Launch the 'start-l10n.bat', to get a mozilla-build environment.
    $ echo "source `which virtualenvwrapper.sh`" >> ~/.bashrc
 
    $ source ~/.bashrc
* Install [https://developer.mozilla.org/en/Python/Virtualenv virtualenv], if it's not on your system already. If on Windows, be sure to do this from inside the shell launched via 'start-l10n.bat'.
 
==== Installing Mozbase ====
 
* Create a [https://developer.mozilla.org/en/Python/Virtualenv virtualenv]:
 
  virtualenv mozbase
 
* Activate the virtualenv:
 
  cd mozbase; . bin/activate
 
* Create a source directory:
 
  mkdir src


* Clone mozbase in the source directory:
* Grab a copy of [https://developer.mozilla.org/en-US/docs/Developer_Guide/Source_Code/Mercurial mozilla-central].
* Create a new virtualenv and install mozbase into it:
    $ cd mozilla-central/testing/mozbase
    $ mkvirtualenv mozbase
    $ python setup_development.py


  cd src; git clone git://github.com/mozilla/mozbase.git
* Run the mozbase tests as a sanity check:
 
    $ python test.py
* Install packages in the virtualenv
 
  cd mozbase; python setup_development.py


=== Working on Mozbase and Contributing Patches ===
=== Working on Mozbase and Contributing Patches ===


Changes to mozbase require peer review of a properly filed bug. Here's a workflow that will help you make changes to mozbase in isolation.
Working on mozbase is the same as working on any other part of the mozilla-central source tree. See the following links for tips on getting set up with Mercurial and how to submit a patch:


* If there isn't one already on file, [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase&cc=:jhammel create a bug] corresponding to your issue in the mozbase bugzilla component.
* https://developer.mozilla.org/en-US/docs/Mercurial
* https://developer.mozilla.org/en-US/docs/Installing_Mercurial
* https://developer.mozilla.org/en-US/docs/Mercurial_Queues
* https://developer.mozilla.org/en-US/docs/Mercurial_FAQ
* https://developer.mozilla.org/en-US/docs/Developer_Guide/How_to_Submit_a_Patch


* Inside the mozbase checkout, create a local branch corresponding to your bug:
==== Design Principles ====


  git checkout -b bug-xxxxxxx
* if we're extending the standard library, we should extend via inheritance
** we are free to extend and add new methods, but we should support the existing methods when possible


* ''Develop your patch''
==== Adding a New Mozbase Package ====


* If people commit work to the master branch while you're working on your patch, you can make things up to date by issuing the following commands from your checked out branch:
A new mozbase package should start with version '0.0'. It should be bumped to a real version when released to pypi
 
  git pull origin master
  git rebase master
 
* When you're happy with your work, commit your changes, giving them a descriptive title from the bug summary:
 
  Bug 706981 - Check for DistributionNotFound error
 
* Generate a patch for your bug
 
  git show --format="From: %an <%ae>%n%s%n%b" > bug-xxxx.patch
 
* Attach the patch to the bug and ask the appropriate person for review. If further changes are required, you can amend changes to your patch by using 'git commit --amend'. When you're all done, ask someone with mozbase commit privileges to merge your changes into the mozilla mozbase repository.
 
This is a recommended way of working with mozbase.  In general the important things are:
* having an bug filed in [https://bugzilla.mozilla.org bugzilla]
* having a patch on bugzilla that applies against [https://github.com/mozilla/mozbase the mozbase github repository] master 'HEAD'
 
What '''NOT''' to do:
* don't use github issues. We track our development with bugzilla, so if you file a github issue that just makes more work for us to file a corresponding bugzilla issue and have double the issues to close out
* don't use pull requests. This again requires us to do more work duplicating the pull request and associated patch in bugzilla
 
 
=== Design Principles ===
 
* if we're extending the standard library, we should extend via inheritance
** we are free to extend and add new methods, but we should support the existing methods when possible


=== Scripts ===
=== Scripts ===
Line 105: Line 70:
In addition to Mozbase's python packages, there are several scripts written to aid Mozbase developers in pushing the workflow along:
In addition to Mozbase's python packages, there are several scripts written to aid Mozbase developers in pushing the workflow along:


* [https://github.com/mozilla/mozbase/blob/master/setup_development.py setup_development.py] : sets up mozbase packages for in development mode; see [[#Installing_Mozbase]]
* [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/setup_development.py setup_development.py] : sets up mozbase packages for in development mode; see [[#Initial_Setup]]
* [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/test.py test.py] : test runner for Mozbase [http://docs.python.org/2/library/unittest.html unittests]; see [[#Tests]]


* [https://github.com/mozilla/mozbase/blob/master/versionbump.py versionbump.py] : bump mozbase versions and release to [http://pypi.python.org/ pypi]; see [[#Using_versionbump.py]]
More about these scripts can be found from running `$SCRIPT --help` or by consulting the documentation and source.
 
* [https://github.com/mozilla/mozbase/blob/master/test.py test.py] : test runner for Mozbase [http://docs.python.org/2/library/unittest.html unittests]; see [[#Tests]]
 
* [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/generate_diff.py generate_diff.py] : generate a diff of released mozbase packages vs [http://hg.mozilla.org/mozilla-central mozilla-central]; see [[#Mirroring]]
 
More about these scripts can be found from running `$SCRIPT --help` or by consulting the linked to documentation and source.
 
 
=== Adding a New Mozbase Package ===
 
A new mozbase package should start with version '0.0'. It should be bumped to a real version when released to pypi


=== Tests ===
=== Tests ===
Line 124: Line 79:
Mozbase packages are accompanied by tests in order to ensure and illustrate proper functionality.
Mozbase packages are accompanied by tests in order to ensure and illustrate proper functionality.


* each mozbase package should have tests in a 'tests' subdirectory of the root  
* each mozbase package should have tests in a 'tests' subdirectory of its root  
**''Example'': https://github.com/mozilla/mozbase/tree/master/mozprofile/tests
**''Example'': http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/tests/
* these should be self-executing [http://docs.python.org/library/unittest.html python unittests]
* these tests should be contained in a manifest; the master [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/test-manifest.ini test manifest], is executed via [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/test.py test.py]
**''Example'': https://github.com/mozilla/mozbase/blob/master/mozprofile/tests/testprofile.py
* these tests should be contained in a manifest; the master test manifest, https://github.com/mozilla/mozbase/blob/master/test-manifest.ini , is executed via [https://github.com/mozilla/mozbase/blob/master/test.py test.py]
 
Continuous integration via [http://k0s.org/mozilla/hg/autobot autobot] is at http://k0s.org:8010/


==== Running the tests ====
==== Running the tests ====
Line 136: Line 87:
You will need [http://www.gnu.org/software/make/manual/make.html make] and [http://gcc.gnu.org/ gcc] in order to run the tests.  If you're running windows, this is available in [https://developer.mozilla.org/en/Windows_Build_Prerequisites#MozillaBuild_.2F_Pymake MozillaBuild].
You will need [http://www.gnu.org/software/make/manual/make.html make] and [http://gcc.gnu.org/ gcc] in order to run the tests.  If you're running windows, this is available in [https://developer.mozilla.org/en/Windows_Build_Prerequisites#MozillaBuild_.2F_Pymake MozillaBuild].


Ensure that all mozbase packages are installed as the tests require them.
Then, from the root of mozilla-central, run:


Then run the [https://github.com/mozilla/mozbase/blob/master/test.py test.py] test runner from the root of the git repository:
    $ ./mach python-test testing/mozbase


    python test.py
This will run all tests from every mozbase module. You can run individual tests by passing in that directory specifically, e.g:


This will run all tests from [https://github.com/mozilla/mozbase/blob/master/test-manifest.ini test-manifest.ini] which is parsed with [https://github.com/mozilla/mozbase/tree/master/manifestdestiny ManifestDestiny].  On success 'test.py' should print the number of tests run and 'OK':
    $ ./mach python-test testing/mozbase/mozprocess


    ----------------------------------------------------------------------
==== Mozbase Testing Framework ====
    Ran 56 tests in 114.807s
   
    OK


=== Check-in policy ===
''TODO: document how testing works in mozbase (test.py, etc)''


==== Review Policy ====
==== Mozbase Testing Strategy ====


'''All''' changes should be reviewed before landing.  The one exception is version bumps.  See https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Versioning for how these are done.
''TODO''


If you have admin permissions to the upstream [https://github.com/mozilla/mozbase mozbase repository], you can land the reviewed patch on your own. Otherwise you should ask someone else to get the patch landed.
==== Tests with Python Dependencies ====


==== Landing the patch ====
Tests for a mozbase package may require dependencies beyond which those listed in the package's 'setup.py' 'install_requires'.  Tests may utilize other mozbase packages even if they are not required by the package under test.


'''Please never use the Github auto-merge feature!'''
Third-party packages -- those not in python's stdlib -- should be introduced only if they provide considerable value to testing (e.g. ability to test what is otherwise not possible, considerable savings in code complexity/manhours, etc.).  In general, adding a test-only dependency should be taken (nearly) as seriously as adding a mozbase package dependency via ''install_requires''. Required packages should, if at all possible, be pure python and of high quality.  Preferably, test-required packages should be ad hoc standards if possible; for instance, if the package is slated to be included in the stdlib or is in a future python version, it is a good candidate.


To land the patch on the target branch make use of the '''git merge --squash''' command, which will land the patch as a single commit. After running this command you will have to issue a new commit message before being able to push the changes. Make sure that you add the right author if the patch wasn't created by yourself. Also use the right commit message format as shown below which includes the bug number and the list of reviewers.
If a third-party package is added for testing, it should be added as part of ''extra_packages'' in ''setup_development.py'' (Ref: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/setup_development.py ), to the ''python/'' directory and to ''build/virtualenv/packages.txt'' (Ref: [https://developer.mozilla.org/en-US/docs/Python#Python_in_Mozilla-Central] ) in mozilla-central, please [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase file a bug] to get the dependency added).


Here an example how to land a patch from the '''feature''' branch onto '''master''':
'''Third-party Packages Used in Mozbase Testing:'''
* mock


* git checkout master
In general, additional requirements add overhead to code complexity, new contributor learning curve, maintenance cost, and portability, so the value added should justify the costs. Candidate packages should be discussed with Mozbase stakeholders preferably in a bug so that a record is kept.
* git checkout -b feature
* git pull remote feature # or git am patch
* '''TEST''' the patch by [https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Running_the_tests running existent tests]
* git merge --squash feature
* git commit -a -m "Bug XYZ - Add new feature to mozbase; r=reviewer" --author="foo <foo@bar>"
* git push git@github.com:mozilla/mozbase.git master


=== Versioning ===
''We're always looking for ways to make this easier; please weigh in on {{bug|885342}} with your ideas!''


* NOTE: you need to have a ~/.pypirc file defined (username/password are account credentials for pypi.python.org):
==== Continuous Integration ====
* [server-login]
* username:username
* password:password


The major version should be bumped when the API changes in a non-backwards-compatible way or for other large conceptual changes. Otherwise, the minor version may be bumped. The version doesn't have to be bumped if there is no new release planned for PyPI.
We currently run tests automatically on every mozilla-central commit. They are run as part of make check in the build job, but {{bug|1003417}} tracks pulling them out into a separate job.


When you bump a version of any mozbase package, it should be released to [http://pypi.python.org/pypi pypi]:
=== Versioning ===


  python setup.py egg_info -RDb "" sdist register upload
NOTE: you need to have a [http://docs.python.org/2/distutils/packageindex.html ~/.pypirc] file filled out in order for python packaging scripts to upload packages to the python package index. The right hand side below of ''username''/''password'' are your account credentials for [http://pypi.python.org/pypi pypi.python.org]:


and the github repository should be tagged a la 'mozrunner-1.2'.  If multiple package versions are bumped, then the same changeset should be tagged for each of the package versions bumped.
  [server-login]
  username:username
  password:password


Tagging the version on github:
The major version should be bumped when the API changes in a non-backwards-compatible way or for other large conceptual changes. Otherwise, the minor version may be bumped. The version doesn't have to be bumped if there is no new release planned for PyPI.


git pull --tags mozilla master
When you bump a version of any mozbase package, it should be released to [http://pypi.python.org/pypi pypi]:
git tag mozrunner-1.5.5
    python setup.py sdist upload
git push --tags mozilla master


==== Using versionbump.py ====
Please be careful when doing this that there are no unwanted files in the package directory. If there are, they will be uploaded to pypi.


A script, versionbump.py, located at
Note, if you are uploading a new package, it needs to be registered first:
https://github.com/mozilla/mozbase/blob/master/versionbump.py
    python setup.py sdist register
has been written to perform the steps for
[https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Versioning mozbase versioning]


* make sure you are on the master branch of the mozbase repository. If you aren't the script will bail out.
==== PyPI ====


* make sure you don't have any uncommitted changes.  If you do, the script will bail out.
Releases of mozbase [https://developer.mozilla.org/en/Python#Python_packaging packages] are uploaded to the [http://pypi.python.org/pypi Python Package Index (pypi)] so that they may be easily consumed by [http://pypi.python.org/pypi/setuptools setuptools] and other installers.
 
* you should probably pull from git@github.com:mozilla/mozbase.git prior to running the script. If not, the script will attempt to do this for you, though it will not attempt to resolve conflicts
 
* as per usual, running <pre>versionbump.py --help</pre> will display usage information and help about CLI options
 
* <pre>versionbump.py --info</pre> will display the package versions and their dependencies and exit
 
In order to bump versions, versionbump.py is used like:
 
  versionbump.py mozrunner=5.8 mozprofile=0.5 -m 'bug 123456 - bump mozprofile to 0.5'
 
You should pass in all the packages that need bumping.  If you specify
a package which is pegged as an exact dependency of another
package (e.g. mozrunner requires mozprofile == 0.5), you must also
bump all packages which exactly depend on this package.  As an
example:
 
  versionbump.py mozprofile=0.5
 
would not work as mozrunner requires mozprofile==0.4 (as of this
writing) and all packages in the repository HEAD must be compatible
with all other packages in the repository.  In addition, packages
pegged with '>=' also follow this rule unless --strict is passed.
 
* running <pre>versionbump.py --diff path/to/file</pre> will output the resultant diff to the file and revert the repository.  The repository will not be tagged nor will anything be uploaded to pypi
 
* -m or --message should be passed to the command line in order to finalize the commit, tag the repository, and upload to pypi. If you do not pass -m or --message, the changes will be made to your working copy but not committed. The repository will not be tagged nor will packages be uploaded to pypi
 
* --dry-run will print out what versionbump.py will do but no files will be changed or commands actually called
 
It is encouraged to use --diff and/or --dry-run before actually doing
the version bump to make sure that it will do what you expect it to
do.  In the case where a known error is encountered, versionbump.py
will revert to the original (clean) repository state.  In the case
where an unexpected error is encountered, the repository state is not
reverted (so that it is easier to debug what happened).
 
Limitations of versionbump.py:
 
* it is expected that setup.py versions are specified as <pre>PACKAGE_VERSION = '1.2.3'</pre> (or "1.2.3"). They currently all are.
 
* versionbump.py can only handle simple dependencies.  That is for install_requires, foo, foo == 1.2.3, bar >= 4.5.6 are fine, but foo == 1.1, == 1.2 is not.  Mozbase currently only has simple dependencies.
 
=== Mirroring ===
 
A copy of mozbase is mirrored to mozilla-central for use by software
there: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/ .
 
All commits should be made against the github mozbase repository, not
mozilla-central.
 
Mozbase packages are mirrored from released versions:
https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Versioning . A
bug should be filed to mirror the appropriate package stating the
reason the package should be mirrored.  When appropriate, mozbase
packages should be mirrored on version bump.
 
Since several packages in mozbase are dependent on other mozbase
packages, care must be taken to ensure that the versions of the
packages on m-c are kept compatible with each other.  All packages
dependent on the package to be mirrored should also version-bumped and
mirrored.  Similarly, if any of the packages that the package to be
mirrored depends on have not already been mirrored to m-c, these
should also be mirrored.


A script has been written to aid with package mirrroring:
=== Mozbase use in Mozilla-Central ===
[http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/generate_diff.py generate_diff.py] .
This script:


* ensures that your copy of mozilla-central is clean. This means no outstanding changes and no untracked files in [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/ testing/mozbase]. Because `generate_diff.py` manipulates the repository using `hg`, this is necessary.
The mozbase packages are installed into [https://developer.mozilla.org/en-US/docs/Python/Virtualenv#The_Mozilla-Central_Virtualenv the mozilla-central virtualenv] instance. The mozbase [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/packages.txt packages.txt] file is included from the master [http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/packages.txt packages.txt] file and includes all mozbase packages (really, directories) in mozilla-central via [http://docs.python.org/2/library/site.html .pth files].
* clones a fresh copy of the [https://github.com/mozilla/mozbase mozbase repository]
* ensures that the tags associated with the specified packages and associated versions exist
* checks that the versions specified are internally consistent with what is in m-c


And if everything looks good...
The various mozbase packages are used in mozilla-central python code; see https://developer.mozilla.org/en-US/docs/Python#Python_in_Mozilla-Central . Mozbase is used extensively throughout mozilla-central, including test harnesses, tools and the build system itself.


* generates a diff that will upgrade the specified packages to the specified versions. Unless specified with `-o`, the output will be in the current working directory named after the hash tag of the git repo
==== Adding new Packages ====


'''Usage:'''
If you are adding a new package to mozbase, there are unfortunately several files you'll need to update in addition to your added files. Those are:


  generate_diff.py [options] package1[=version1] <package2=version2> <...>
* http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/packages.txt
* http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/Makefile.in
* http://mxr.mozilla.org/mozilla-central/source/build/mach_bootstrap.py#27
* http://mxr.mozilla.org/mozilla-central/source/testing/config/mozbase_requirements.txt


''(as displayed with: `generate_diff.py --help`)''
=== Uploading to Puppetagain ===


Example:
Mozharness uses packages from http://puppetagain.pub.build.mozilla.org/data/python/packages/ in production.  If a test harness invoke by mozharness requires updated mozbase packages.


  │./generate_diff.py mozcrash=0.3
In order to mirror, [https://bugzilla.mozilla.org/enter_bug.cgi?product=mozilla.org&component=Release%20Engineering%3A%20Automation%20%28General%29 file a bug] with a URL of the package(s) of interest.
  Cloning into 'mozbase'...
  remote: Counting objects: 3883, done.
  remote: Compressing objects: 100% (1746/1746), done.
  remote: Total 3883 (delta 2198), reused 3773 (delta 2105)
  Receiving objects: 100% (3883/3883), 1.16 MiB | 241 KiB/s, done.
  Resolving deltas: 100% (2198/2198), done.
  Note: checking out 'mozcrash-0.3'.
 
  You are in 'detached HEAD' state. You can look around, make experimental
  changes and commit them, and you can discard any commits you make in this
  state without impacting any branches by performing another checkout.
 
  If you want to create a new branch to retain commits you create, you may
  do so (now or later) by using -b with the checkout command again. Example:
 
    git checkout -b new_branch_name
 
  HEAD is now at 1634023... Bug 813132 - support tbpl friendly output from mozcrash and python 2.4
  HEAD is now at 1634023... Bug 813132 - support tbpl friendly output from mozcrash and python 2.4
  removing testing/mozbase/mozcrash/README.md
  undeleting testing/mozbase/mozcrash/README.md
  reverting testing/mozbase/mozcrash/mozcrash/__init__.py
  reverting testing/mozbase/mozcrash/mozcrash/mozcrash.py
  reverting testing/mozbase/mozcrash/setup.py
  Diff at /home/jhammel/mozilla/src/mozilla-central/testing/mozbase/1634023250af1cb9bce9ccbf9dfe620e4a95c970.diff


=== Licensing ===
=== Licensing ===


Mozbase code will be [https://wiki.mozilla.org/Auto-tools/Projects/MozBase#Mirroring checked into mozilla-central].  Therefore, code should be appropriately licensed with MPL2. See https://www.mozilla.org/MPL/headers/ to find a copy.
Mozbase should be appropriately licensed with MPL2. See https://www.mozilla.org/MPL/headers/ to find a copy.


=== Documentation ===
=== Documentation ===
* full docs are found in the [https://github.com/mozilla/mozbase/tree/master/docs top-level docs directory] and hosted at [http://mozbase.readthedocs.org http://mozbase.readthedocs.org]. We're always looking for help [[Auto-tools/Projects/MozBase/WritingDocs|writing docs]].
* full docs are found in the [https://github.com/mozilla/mozbase/tree/master/docs top-level docs directory] and hosted at [http://mozbase.readthedocs.org http://mozbase.readthedocs.org]. We're always looking for help [[Auto-tools/Projects/MozBase/WritingDocs|writing docs]].
 
* [https://wiki.mozilla.org/Auto-tools/Projects/Mozbase this page] is the mozbase project page that gives information for core mozbase developers and about the project itself.  Documentation may move to and from this page and the various READMEs and full docs.
* [https://wiki.mozilla.org/Auto-tools/Projects/MozBase this page] is the mozbase project page that gives information for core mozbase developers and about the project itself.  Documentation may move to and from this page and the various READMEs and full docs.


=== Directory Structure and Imports ===
=== Directory Structure and Imports ===
Line 346: Line 192:


* this allows packages to be installed with the 'setup.py' script
* this allows packages to be installed with the 'setup.py' script
* if packages cannot be installed (for deployment reasons), altering 'sys.path' (or '$PYTHONPATH') can be used to utilize
* if packages cannot be installed (for deployment reasons), altering 'sys.path' (or '$PYTHONPATH') can be used to utilized
* with single-module packages, you get a sensibly named file that is imported in '__init__.py' and can be mirrored easily and it conforms to how all of the other packages work
* with single-module packages, you get a sensibly named file that is imported in '__init__.py' and can be mirrored easily and it conforms to how all of the other packages work
=== Downstream packages ===
Many mozbase packages depend on other mozbase packages.  The packages and their versions at
https://github.com/mozilla/mozbase should be kept compatible with one another.
Other packages also depend on mozbase: http://k0s.org/mozilla/mozbase/dependencies.html
Releases and version changes should be tested against downstream packages to ensure compatibility
=== PyPI ===
Releases of mozbase [https://developer.mozilla.org/en/Python#Python_packaging packages] are uploaded to the [http://pypi.python.org/pypi Python Package Index (pypi)] so that they may be easily consumed by [http://pypi.python.org/pypi/setuptools setuptools] and other installers.
Package owners:
* ahal
* ctalbert
* davehunt
* David.Burns
* elvis314
* jgriffin
* k0s
* markrcote
* whimboo
* wlach


== Integration Notes ==
== Integration Notes ==
Line 385: Line 205:
mozharness docs: http://escapewindow.com/mozharness/
mozharness docs: http://escapewindow.com/mozharness/


Production use of mozharness will use mozbase packages from an internal pypi:
Production use of mozharness will use mozbase packages from an internal pypi: http://puppetagain.pub.build.mozilla.org/data/python/packages/
http://puppetagain.pub.build.mozilla.org/data/python/packages/
(from https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Python )
(from https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Python )
* mozprocess (basic low level process management)
** subclass popen
*** open
*** poll
*** kill
*** wait
*** stdout
*** pid
** readWithTimeout
* mozfile (os file i/o managemenet)
** copyfile
** removedir
** makedir
* mozlog (log file management)
** write
** init
=== unique to harnesses ===
* mozenv
** addvars
** createenv
* mozprofile
** addExtension(s)
** addPref(s)
** writeProfile
** copyProfile
** renameProfile
** deleteProfile

Latest revision as of 16:24, 22 November 2017

Overview

Mozbase is a set of easy-to-use Python packages forming a supplemental standard library for Mozilla. It provides consistency and reduces redundancy in automation and other system-level software. All of Mozilla's test harnesses use mozbase to some degree, including Talos, mochitest, reftest, Autophone, and Eideticker.

This is the project page for mozbase development, outlining current development practices, methodologies, and protocol. Other resources related to mozbase:

Repository: https://hg.mozilla.org/mozilla-central under testing/mozbase

Documentation: https://firefox-source-docs.mozilla.org/mozbase/

Bugs:

Please file bugs against https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase

The roadmap of getting mozilla-central python to use mozbase is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Roadmap . The effort to this effect is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Automation-Refactor

Mozbase requires python 2.7 or greater

Development Practices

If you're developing mozbase code, you're in luck! You've found the right place to read about our development practices. Please do :)

Getting Help with Mozbase

Mozbase is developed by the Automation and Tools team. It is under the governorship of Will Lachance (:wlach) and Andrew Halberstadt (:ahal). Please feel free to stop by #ateam on IRC with any questions and we will be happy to help you! Or if something is wrong, please file a bug and we'll look at it as soon as possible.

Installing Mozbase for Development

Initial Setup

   $ sudo pip install virtualenv
   $ sudo pip install virtualenvwrapper
   $ echo "source `which virtualenvwrapper.sh`" >> ~/.bashrc
   $ source ~/.bashrc
  • Grab a copy of mozilla-central.
  • Create a new virtualenv and install mozbase into it:
   $ cd mozilla-central/testing/mozbase
   $ mkvirtualenv mozbase
   $ python setup_development.py
  • Run the mozbase tests as a sanity check:
   $ python test.py

Working on Mozbase and Contributing Patches

Working on mozbase is the same as working on any other part of the mozilla-central source tree. See the following links for tips on getting set up with Mercurial and how to submit a patch:

Design Principles

  • if we're extending the standard library, we should extend via inheritance
    • we are free to extend and add new methods, but we should support the existing methods when possible

Adding a New Mozbase Package

A new mozbase package should start with version '0.0'. It should be bumped to a real version when released to pypi

Scripts

In addition to Mozbase's python packages, there are several scripts written to aid Mozbase developers in pushing the workflow along:

More about these scripts can be found from running `$SCRIPT --help` or by consulting the documentation and source.

Tests

Mozbase packages are accompanied by tests in order to ensure and illustrate proper functionality.

Running the tests

You will need make and gcc in order to run the tests. If you're running windows, this is available in MozillaBuild.

Then, from the root of mozilla-central, run:

   $ ./mach python-test testing/mozbase

This will run all tests from every mozbase module. You can run individual tests by passing in that directory specifically, e.g:

   $ ./mach python-test testing/mozbase/mozprocess

Mozbase Testing Framework

TODO: document how testing works in mozbase (test.py, etc)

Mozbase Testing Strategy

TODO

Tests with Python Dependencies

Tests for a mozbase package may require dependencies beyond which those listed in the package's 'setup.py' 'install_requires'. Tests may utilize other mozbase packages even if they are not required by the package under test.

Third-party packages -- those not in python's stdlib -- should be introduced only if they provide considerable value to testing (e.g. ability to test what is otherwise not possible, considerable savings in code complexity/manhours, etc.). In general, adding a test-only dependency should be taken (nearly) as seriously as adding a mozbase package dependency via install_requires. Required packages should, if at all possible, be pure python and of high quality. Preferably, test-required packages should be ad hoc standards if possible; for instance, if the package is slated to be included in the stdlib or is in a future python version, it is a good candidate.

If a third-party package is added for testing, it should be added as part of extra_packages in setup_development.py (Ref: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/setup_development.py ), to the python/ directory and to build/virtualenv/packages.txt (Ref: [1] ) in mozilla-central, please file a bug to get the dependency added).

Third-party Packages Used in Mozbase Testing:

  • mock

In general, additional requirements add overhead to code complexity, new contributor learning curve, maintenance cost, and portability, so the value added should justify the costs. Candidate packages should be discussed with Mozbase stakeholders preferably in a bug so that a record is kept.

We're always looking for ways to make this easier; please weigh in on bug 885342 with your ideas!

Continuous Integration

We currently run tests automatically on every mozilla-central commit. They are run as part of make check in the build job, but bug 1003417 tracks pulling them out into a separate job.

Versioning

NOTE: you need to have a ~/.pypirc file filled out in order for python packaging scripts to upload packages to the python package index. The right hand side below of username/password are your account credentials for pypi.python.org:

 [server-login]
 username:username
 password:password

The major version should be bumped when the API changes in a non-backwards-compatible way or for other large conceptual changes. Otherwise, the minor version may be bumped. The version doesn't have to be bumped if there is no new release planned for PyPI.

When you bump a version of any mozbase package, it should be released to pypi:

   python setup.py sdist upload

Please be careful when doing this that there are no unwanted files in the package directory. If there are, they will be uploaded to pypi.

Note, if you are uploading a new package, it needs to be registered first:

   python setup.py sdist register

PyPI

Releases of mozbase packages are uploaded to the Python Package Index (pypi) so that they may be easily consumed by setuptools and other installers.

Mozbase use in Mozilla-Central

The mozbase packages are installed into the mozilla-central virtualenv instance. The mozbase packages.txt file is included from the master packages.txt file and includes all mozbase packages (really, directories) in mozilla-central via .pth files.

The various mozbase packages are used in mozilla-central python code; see https://developer.mozilla.org/en-US/docs/Python#Python_in_Mozilla-Central . Mozbase is used extensively throughout mozilla-central, including test harnesses, tools and the build system itself.

Adding new Packages

If you are adding a new package to mozbase, there are unfortunately several files you'll need to update in addition to your added files. Those are:

Uploading to Puppetagain

Mozharness uses packages from http://puppetagain.pub.build.mozilla.org/data/python/packages/ in production. If a test harness invoke by mozharness requires updated mozbase packages.

In order to mirror, file a bug with a URL of the package(s) of interest.

Licensing

Mozbase should be appropriately licensed with MPL2. See https://www.mozilla.org/MPL/headers/ to find a copy.

Documentation

Directory Structure and Imports

The mozbase repository contains several packages. Each package should have a directory structure like:

   ${PACKAGE}
   ${PACKAGE}/setup.py
   ${PACKAGE}/README.md
   ${PACKAGE}/${PACKAGE}
   ${PACKAGE}/${PACKAGE}/__init__.py

Even if the package consists of a single module, this structure should be used (vs. py_modules) and '__init__.py' set to

   from ${MODULE} import *

(with a sensible '__all__' defined in the module).

This structure serves several purposes:

  • this allows packages to be installed with the 'setup.py' script
  • if packages cannot be installed (for deployment reasons), altering 'sys.path' (or '$PYTHONPATH') can be used to utilized
  • with single-module packages, you get a sensibly named file that is imported in '__init__.py' and can be mirrored easily and it conforms to how all of the other packages work

Integration Notes

Notes on integration with existing test harnesses.

Mozbase and Mozharness

Mozbase and mozharness overlap intent with respect to making testing software and tools reusable and easy to write and extend.

mozharness docs: http://escapewindow.com/mozharness/

Production use of mozharness will use mozbase packages from an internal pypi: http://puppetagain.pub.build.mozilla.org/data/python/packages/ (from https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Python )