AMO/SigningService

From MozillaWiki
< AMO
Revision as of 17:05, 28 January 2015 by Magopian (talk | contribs) (Add a "usage" section)
Jump to navigation Jump to search

Add-on Signature System

Hi. This project is the AMO piece of the larger Add-on Signature System. Please read that document so the rest of this wiki page makes sense.

We will need to modify several pieces of AMO and its libraries in order to accommodate this new system. Those changes are roughly laid out below and divided up into phases. See the diagram below (which compares to Marketplace for a reference) for a high level view:

Add-on Signing - Main Flow.png

Notes:

  • The flows are very similar. Apps load a certificate from disk and get signed. For add-ons we'll load a certificate from disk, but we'll generate a new certificate as well, use the loaded certificate to sign the new certificate, and then sign the add-on with the new certificate.
  • When we're done signing we discard the new certificate.
  • Any pre-existing signatures on the add-on are replaced. This includes developers who have signed the add-on with a valid AMO certificate already. The expected flow for self-hosting in addition to AMO hosting is to download the signed add-on from AMO, not do any manual signing.
  • We don't do any signing before the add-on is approved for the public. Reviewers are expected to use developer versions or debug versions of Firefox so won't need to have the add-ons signed to test them.
  • Should not need any API changes since this is all after add-on review
  • We're looking at high 10s to low 100s of signed packages per day
  • There are three users we're targeting:
    • One: Regular Jane. She writes add-ons and distributes them via AMO. This is the vast majority of developers and their workflow should be largely unchanged (assuming they already develop add-ons with a non-release version of Fx).
    • Two: Particular Joe. He writes add-ons, and is happy to show the code to Mozilla but prefers not to distribute the add-on on AMO. We'll modify AMO to allow him to upload the add-on as usual, but not show the add-on on any public pages.
    • Three: Unusual Quinn. He writes add-ons but refuses to show the code to Mozilla (eg. he may be under an NDA or similar). Hopefully this is a very rare case and a process is still being documented. (Expect something close to: file a security bug, explain circumstances, enter into a legally binding contract with Mozilla, be issued a signing key that you are responsible to keep safe, sign your own add-on)

Roadmap

Phase 1: Signing with Trunion

Current Status Bugs are being worked on!
Owners Ryan Tilder, Wil Clouser

Summary:

We currently don't sign add-ons at all on AMO, but we do sign apps on the Marketplace using trunion and could use the same system (with modifications). A rough summary of the Trunion overview is as follows:

 1. This CA will be entirely automated and self contained
     1. The CA's root certificate will be hard coded into Firefox/Fennec
        in a similar manner to the privileged FxOS apps
     2. For every request to sign an addon:
          o a brand new 2048 bit or stronger RSA key pair will be
            generated by the signing service
          o the ephemeral public key will be certified by this CA
          o the ephemeral private key generated will then be used to
            sign the addon archive in Mozilla's own bastardized
            implementation of JAR signing that we know as "XPI signing"
          o the freshly certified ephemeral public key will be included
            in the addon as part of the signature chain
          o the ephemeral private key and certificate are thrown away

In practical terms, this means:

  • Modifying Trunion to accept meta-data about what it is signing (at least an add-on ID)
  • Modifying Trunion to send meta-data about what it signed (at least the certificate serial number)
  • Modifying Trunion to generate certs on-the-fly

See API Documentation

Open Bugs:

Full Query
ID Priority Status Summary
1070155 P1 RESOLVED Define and document the API for add-on signing
1076052 -- RESOLVED Add Python key & certificate generation to trunion
1123915 P1 RESOLVED Where should we put "Preliminary" in the cert?
1124321 -- RESOLVED trunion addon signing: ephemeral certs are missing all attributes but CN
1126894 -- RESOLVED Create parallel "preliminary" addon signing trunion instance for dev
1135926 -- VERIFIED Hide existing signature warning until relevant
1158467 P1 RESOLVED Signed extensions on AMO have invalid signatures (And cause to many crashes for Fx28 and older versions)
1158938 P1 RESOLVED End mozilla.sf with two newlines

8 Total; 0 Open (0%); 7 Resolved (87.5%); 1 Verified (12.5%);


Phase 2: Initial AMO Support

Current Status Closing bugs!

This will implement the initial AMO support, signing apps after being approved by reviewers. Trunion API Documentation

Open Questions:

  • How do we push updates to all existing add-ons after they are signed? Updating to a new version number may be necessary.

Open Bugs:

Full Query
ID Priority Status Summary
1070188 P3 VERIFIED Modify the validator to warn about pre-existing signatures
1070189 P1 RESOLVED Call the Trunion API to sign add-ons when approved
1070190 P1 RESOLVED Record the certificate serial number
1070191 P2 RESOLVED Write a management command to sign specific add-ons
1070192 P2 RESOLVED Add documentation around add-on signing to MDN
1070227 P2 RESOLVED Write a script to sign all existing add-ons
1123611 -- RESOLVED sign_addons management command should be able to sign all addons
1126898 -- VERIFIED Add support for separate "preliminary" signing endpoint URL
1135139 -- RESOLVED Increment the versions in install.rdf when we sign the add-on
1142044 -- RESOLVED Make the "sign all addons" management command runnable multiple times
1148478 P1 VERIFIED Signed add-ons won't install
1152101 P1 RESOLVED Modify the add-on signing script to skip add-ons we've signed before
1152102 P1 VERIFIED Rename the .rsa files when we sign add-ons
1155786 -- RESOLVED Skip signing for Firefox and Thunderbird hot fix add-ons
1156217 -- RESOLVED Adding a .1 when bulk signing all addons might lead to version clashes
1156221 -- RESOLVED Only sign already reviewed files when bulk signing
1156333 -- VERIFIED Don't review a file if its signing failed
1156768 -- RESOLVED Using bulk sign script, sign all files of a version, and don't bump version if not signing files
1157349 P3 VERIFIED Add a visibility notice to status box and submissions page
1157444 -- RESOLVED Only bulk sign versions that are compatible with recent versions of Firefox
1158738 -- RESOLVED When auto-signing files are cached by cdn and it results in hash mismatch
1158739 -- RESOLVED When auto-signing files the Version.version_int isn't bumped
1161486 -- VERIFIED Display an information about the signed status of a file on the public pages
1162001 -- RESOLVED "Author not verified" message is displayed when installing a signed add-on
1162058 -- RESOLVED Bulk signing fails on non-ascii filenames
1162458 -- VERIFIED Add-ons which support both Thunderbird and android are signed after approval
1169463 P3 RESOLVED Use a better compression library when repackaging add-ons
1169537 -- RESOLVED some signed extensions on AMO fail to install (addon corrupted)
1169574 -- RESOLVED some signed extensions on AMO fail to install (signature not recognized)
1170266 -- VERIFIED Unable to change add-on authors for an unlisted add-on without being forced to select a license
1172030 P3 RESOLVED Improve the display of validation results for automated signing validation
1172035 -- RESOLVED Give rejected beta versions option to submit to unlisted preliminary review queue
1172037 -- VERIFIED Standalone /validate page should have option to run unlisted add-on validation
1172041 -- RESOLVED Beta versions that don't support Firefox shouldn't have signing requirements enforced
1172554 -- VERIFIED Adjust wording on themes submission page
1172555 -- RESOLVED Adjust the add-on submission text
1172690 -- RESOLVED Admins can't override failed validation for betas
1172696 -- VERIFIED Extensions contained in signed add-ons with type=32 aren't properly signed
1174012 -- RESOLVED Open the bugzilla link in a new tab
1185789 P1 RESOLVED Enable extension signatures for Firefox for Android
1190704 P2 RESOLVED Show a warning, or error, for multi-package XPIs with inner extensions not signed by Mozilla
1198425 P1 RESOLVED Sign locale packs
1201176 P2 RESOLVED Some add-ons have broken signatures due to missing ID in the CN field
1202016 -- VERIFIED Prevent submission of add-ons with GUIDs longer than 64 chars
1202063 -- RESOLVED [tracking] Improve Validator for Signing
1202880 -- RESOLVED Add-on signing of large CNs
1203915 -- VERIFIED Re-allow submitting add-ons with GUIDs longer than 64 chars
1204019 -- VERIFIED Update signing messages to refer to Firefox 43
1205823 -- RESOLVED Unsign multi-package XPIs that were signed
1206696 -- RESOLVED Adding Firefox as a supported appversion does not sign previously unsigned add-ons

50 Total; 0 Open (0%); 35 Resolved (70%); 15 Verified (30%);


Phase 3: Blocklist

Current Status On Hold

On Hold: On hold pending further info. Since add-ons are signed and can't change their IDs, and we can already block on IDs this is of questionable value. This would be a blocker if we give partners their own keys to sign their add-ons.

Summary: This system expects to use the blocklist until a more official cert revocation system can be implemented. Blocklist administration will use the administrative scaffolding it currently uses. A new top level item will be added called <certificates>. An example entry is below:

 <certificates>
   <certificate blockID="100">
     <issuer>mozilla.com</issuer>
     <serialnumber>00245EADBDE07F113DEE</serialnumber>
   </certificate>
 </certificates>

Open Bugs:

Full Query
ID Priority Status Resolution Summary
1070218 P1 RESOLVED WONTFIX Add 'certificates' blocklist item
1070219 P1 RESOLVED WONTFIX Modify the blocklist to include certificates
1070220 P2 RESOLVED WONTFIX Add certificates to the /blocked/ list
1070221 P1 RESOLVED WONTFIX Add certificate block details pages

4 Total; 0 Open (0%); 4 Resolved (100%); 0 Verified (0%);


Phase 4: Non-AMO hosted add-ons

Current Status Ready to start

Notes:

Open Bugs:

Full Query
ID Priority Status Summary
1121159 -- RESOLVED Build a place for developers to request a cert to sign their own files
1121161 -- RESOLVED Add option to get add-ons signed without uploading to AMO
1121236 -- VERIFIED Add option to not list add-ons on AMO
1121237 -- VERIFIED Add review queues for unlisted add-ons
1121238 -- VERIFIED Adjust emails sent to developers from the Unlisted Queues
1121239 -- VERIFIED Add Addons:ReviewUnlisted permission to the Senior Add-on Reviewers group
1121240 -- RESOLVED Adjust wording on Step 6 to support unlisted add-ons
1121241 -- VERIFIED Don't link intermediate steps on Step 6 for unlisted add-ons
1121242 -- VERIFIED Adjust wording on Step 7 to support unlisted add-ons
1121243 -- RESOLVED What kind of automatic validation should we perform on non-hosted add-ons?
1121244 -- VERIFIED Ensure unlisted add-ons are not accessible by the public
1122107 P2 RESOLVED Allow switching an add-on from Unlisted to Listed
1122108 P2 RESOLVED Allow switching an add-on from Listed to Unlisted
1122109 -- VERIFIED Build new Status UI
1125890 -- VERIFIED Allow some users to specify an <updateURL> in their install.rdf
1148437 P1 VERIFIED Wording changes on Step 1
1148438 P1 VERIFIED Update the Developer Agreement for signed add-ons
1148439 P1 VERIFIED Wording changes on Step 2
1148440 P1 VERIFIED Wording changes on Step 7
1148446 P1 VERIFIED Add checkbox to step 2 for side-loading
1148447 P1 VERIFIED Adjust flow to be steps 1 -> 2 -> 3ish -> 7
1148448 P1 VERIFIED Remove skipped steps from the navbar
1148449 P1 VERIFIED Add a check and prompt for validation passing on step 2
1152457 P1 RESOLVED Which add-ons should we sign at first?
1153231 P1 VERIFIED Allow some validation messages/warnings for the automatic unlisted addons signature
1157215 -- RESOLVED Add a waffle flag for the automatic validation
1159679 -- VERIFIED can't display devhub page for unlisted addon once submitted
1159690 -- VERIFIED The devhub whiteboard doesn't work for unlisted add-ons
1159708 -- VERIFIED An editor should not be able to grant prelim review to a sideload add-on
1159710 -- VERIFIED Fix the breadcrumbs on the review page of an unlisted add-on in the editor tools
1159712 -- VERIFIED When granting a review to an unlisted add-on, redirect to the unlisted queues
1159715 -- VERIFIED Fix the wording for unlisted add-ons in the devhub edit page
1159720 -- VERIFIED Don't automatically set prelim review queue for unlisted add-on on failed validation
1160292 -- RESOLVED Add-ons which are specifically for Firefox for Android are getting signed
1160593 -- RESOLVED Sign beta versions of add-ons
1160627 -- VERIFIED Switching from hidden -> unlisted doesn't update the add-on status
1160654 -- VERIFIED Make sure that unlisted add-ons cannot be beta add-ons
1160969 -- VERIFIED Sign multi-package XPIs
1161536 -- VERIFIED Add a waffle flag for the unlisted addons
1164390 -- RESOLVED Signing of a version fails if one of its files is missing on the filesystem
1164488 -- VERIFIED Automatic validation and signing should also do everything a manual review does
1164494 -- RESOLVED Only automatically review/sign an unlisted addon after step 3 is done
1167185 -- VERIFIED Unlisted addons that didn't complete step 3 should be seen as incomplete
1173607 -- VERIFIED Unlisted add-ons should not enforce unique constraint on names

44 Total; 0 Open (0%); 12 Resolved (27.27%); 32 Verified (72.73%);


Usage

There's two management commands to manually sign add-ons:

sign_addons

This can be used to sign a list of add-ons, by providing their IDs:

python manage.py sign_addons 123 124 125

process_addons

This is a more general management command that can run tasks on every add-on:

python manage.py process_addons --task sign_addons

Running this will sign each add-on. Celery tasks will be used.