Software Update:MAR

Mozilla ARchive

This document describes the Mozilla ARchive (MAR) format used by the update system to deliver update packages. It is basically a series of files with an index tacked on to the end.

Details

The file structure in a nutshell is a header (HEADER), followed by a (SIGNATURES) block, followed by a list of variable length files, and finally ending with an index of the files (INDEX). The index is a list of variable length entries (INDEX_ENTRY). The signatures block is a list of variable length entries (SIGNATURE_ENTRY).

HEADER

 4 bytes : MARID - "MAR1"
 4 bytes : OffsetToIndex - offset to INDEX in bytes relative to the start of MAR file

SIGNATURES

 8 bytes : FileSize - size in bytes of the entire MAR file
 4 bytes : NumSignatures - Number of signatures
 NumSignatures SIGNATURE_ENTRY elements

SIGNATURE_ENTRY (Only present if SIGNATURES.NumSignatures is more than 0)

 4 bytes : SignatureAlgorithmID - ID representing the type of signature algorithm.
 4 bytes : SignatureSize - Size in bytes of the signature that follows
 N bytes : Signature - The signature of type SIGNATURE_ENTRY.SignatureAlgorithmID and size N = SIGNATURE_ENTRY.SignatureSize bytes

ADDITIONAL_SECTIONS (Optional)

 4 bytes: NumAdditionalSections - Number of additional sections
 NumAdditionalSections ADDITIONAL_SECTION_ENTRY elements

ADDITIONAL_SECTION_ENTRY (Only present if NumAdditionalSections is more than 0)

 4 bytes: BlockSize - (Z)
 4 bytes: BlockIdentifier - Used to identify the additional section, 1 for a Product Information Block.
 (Z-8) bytes: The rest of the block.

CONTENT

 Y bytes : content data interpreted as per the INDEX_ENTRY elements. Where Y is the summation of all INDEX_ENTRY.ContentSize values.

INDEX

 4 bytes : IndexSize - Size of INDEX in bytes
 variable number of INDEX_ENTRY elements

INDEX_ENTRY

 4 bytes : OffsetToContent - Offset in bytes relative to start of the MAR file
 4 bytes : ContentSize - Size in bytes of the content
 4 bytes : Flags - File permission bits (in standard unix-style format).
 M bytes : FileName - File name (byte array)
 1 byte  : null terminator

Some old MAR files will not contain the SIGNATURE block. Old parsers simply skip over these fields because they ignore everything between the MAR header and the offset given in the HEADER.OffsetToIndex field.

SIGNATURE blocks

Zero or more SIGNATURE_ENTRYs can be specified. The signatures must be composed of all bytes of the MAR file excluding the SIGNATURE_ENTRY.Signature fields. Each contained SIGNATURE_ENTRY.Signature must be of type SIGNATURE_ENTRY.SignatureAlgorithmID.

SIGNATURE_ENTRY.SignatureAlgorithmID

 1: RSA-PKCS1-SHA1 (2048 bits / 256 bytes)
 2: RSA-PKCS1-SHA384 (4096 bits / 512 bytes)

The updater will only accept the MAR file if at least one of the signatures verify. Some versions of the updater may not apply a MAR file unless a valid signature of a particular SIGNATURE_ENTRY.SignatureAlgorithmID is included in the MAR file.

From Firefox 10-56, only RSA-PKCS1-SHA1 signatures are accepted. Bug 1105689 adds support for SHA-384 and is slated for Firefox 56. There is no indicator in the MAR file for which operating system the MAR is for.

Byte Ordering

All fields are in big-endian format. The signatures are in NSS / OpenSSL / big-endian order and not CryptoAPI order. If CryptoAPI is used to check a signature, the bytes of the signature must be reversed before verifying the signature using CryptVerifySignature.

Constraints

To protect against invalid inputs the following constraints are in place:

• There are at most 8 signatures.
• The file size of the MAR file is at most 500MB.
• No signature is more than 2048 bytes long.

Additional sections

Additional MAR file sections may be defined in the future. If at least one additional section exists, the ADDITIONAL_SECTIONS header entry must be present.

Each additional block must be in the format described above in ADDITIONAL_SECTION_ENTRY.

The Block identifier is a value used to uniquely identify additional blocks so that they may be optionally added in any order.

Product Information Block

The product information block identifies the product and channel this MAR file applies to. It also includes the new version number to avoid downgrades. The product and channel are combined in the MARChannelName field.

PRODUCT INFORMATION BLOCK

 4 bytes: BlockSize - The size of the product information block.
 4 bytes: BlockIdentifier - The identifier of the product information block, with a value of 1
 <64 bytes: MARChannelName used to identify the product and channel (such as from MAR_CHANNEL_ID) Example: mozilla-central
 1 byte: null terminator
 <32 bytes: ProductVersion string (such as from MOZ_APP_VERSION) Examples: 12.0.1.5371, 12.0a1
 1 byte: null terminator
 * bytes: Optional unused data adding up to BlockSize.

Source Code

The source code can be found under modules/libmar.

See bug 296303 and bug 699700

Why not use ZIP or some other standard file format?

This question was given a fair amount of consideration. Ultimately, we decided to go with a custom file format because using libjar would have required a fair bit of hacking. Writing custom code was a simpler option, and it resulted in less code (mar_read.c is less than 300 lines of code). Moreover, the update system does not need a standard file format. The elements stored in the archive are bzip2 compressed binary diffs, generated using a variation of bsdiff. So, being able to unpack the archive file using standard tools wouldn't be very useful in and of itself.

MAR extraction tools

There’s 2 different python scripts:

• http://hg.mozilla.org/build/tools/file/b77b517eab66/buildfarm/utils/mar.py
• https://github.com/mozilla/build-mar

Note:

• The latter is more recent, but has more dependencies.
• The former works just fine for extracting files
  • to use the former: cd tmp_dir; mar.py -x ../foo.mar
  • for bzip2'ed files such as b2g mar files: try 'mar.py -x -j fota-flame-update.mar'
  • or 'mv update.zip update.zip.bz2; bunzip2 update.zip.bz2'