Account confirmers, Anti-spam team, Confirmed users, Bureaucrats and Sysops emeriti
4,925
edits
CA/Symantec Issues: Difference between revisions
Jump to navigation
Jump to search
Updated based on latest Symantec communication
(Update Issue Y) |
(Updated based on latest Symantec communication) |
||
| Line 1: | Line 1: | ||
This page lists all confirmed or suspected issues involving the CA "Symantec". It | This page lists all confirmed or suspected issues involving the CA "Symantec". It may be further updated by Mozilla as more information becomes available, although the deadline for comments from Symantec has now passed. Please do not edit this page yourself; if you have proposed changes, email [mailto:gerv@mozilla.org Gerv]. Information here is correct to the best of Mozilla's knowledge and belief. | ||
Information here is correct to the best of Mozilla's knowledge and belief | |||
The issues are in broadly chronological order by end date. | The issues are in broadly chronological order by end date. | ||
| Line 33: | Line 31: | ||
==Issue D: Test Certificate Misissuance (April 2009 - September 2015)== | ==Issue D: Test Certificate Misissuance (April 2009 - September 2015)== | ||
Between 2009 and 2015, Symantec issued a large number of test certificates in their publicly trusted hierarchies. These contained domains that Symantec did not own or control, and for which domain validation was not performed. Some of these domains were unregistered, and others were owned by other organizations. | Between 2009 and 2015, Symantec issued a large number of test certificates in their publicly trusted hierarchies. These contained domains that Symantec did not own or control, and for which domain validation was not performed. Some of these domains were unregistered, and others were owned by other organizations. Issuing test certificates for unregistered domains was not a BR violation before 3rd April 2014 (ballot 112), because they counted as Internal Server Names, but they continued the practice even after that date. The registered domains used included those belonging to Google and Opera Software. Given the numbers involved, this sort of test certificate issuance appears to have been common practice at Symantec. Some of the test certificates (including one for www.google.com) left Symantec's network because they were logged in CT. (Symantec claim that no certificates left their network; however, it's not clear how this can be true, and clarification is being sought.) However, Symantec personnel would have had access to the public and private keys of the certs. | ||
Some details of this incident are recorded in {{bug|1214321}}. | Some details of this incident are recorded in {{bug|1214321}}. | ||
| Line 48: | Line 46: | ||
Symantec took a number of remediation steps, as outlined in the report. | Symantec took a number of remediation steps, as outlined in the report. | ||
===Further Comments and Conclusion=== | |||
Symantec later [https://bug1334377.bmoattachments.org/attachment.cgi?id=8860216 revealed] that there was a further small (6 certificate) incident of this type in March 2016, "involving approved domains in a test account". | |||
==Issue E: Domain Validation Vulnerability (October 2015)== | ==Issue E: Domain Validation Vulnerability (October 2015)== | ||
| Line 59: | Line 61: | ||
===Further Comments and Conclusion=== | ===Further Comments and Conclusion=== | ||
The set of circumstances which would have allowed this issue to be exploited (+ or = character in WHOIS, domain where arbitrary email address registration by 3rd parties is possible, necessary email address still available to register) are relatively rare, and Symantec fixed the issue quickly and performed appropriate remediation. | The set of circumstances which would have allowed this issue to be exploited (+ or = character in WHOIS, domain where arbitrary email address registration by 3rd parties is possible, necessary email address still available to register) are relatively rare, and Symantec fixed the issue quickly and performed appropriate remediation. Software has bugs. I do not consider this issue to be particularly severe. | ||
==Issue F: Symantec Audit Issues 2015 (December 2014 - November 2015)== | ==Issue F: Symantec Audit Issues 2015 (December 2014 - November 2015)== | ||
| Line 199: | Line 201: | ||
As detailed in their [https://www.symantec.com/content/en/us/about/media/repository/Cover_Letter_for_WebTrust_Audit.pdf covering letter], the audits for the second period, June 16th to November 30th 2016, are mostly unqualified. The BR audits have a total of five qualifications, two of which relate to previously disclosed incidents which are not of concern, and three other qualifications which seem to be only of minor concern. | As detailed in their [https://www.symantec.com/content/en/us/about/media/repository/Cover_Letter_for_WebTrust_Audit.pdf covering letter], the audits for the second period, June 16th to November 30th 2016, are mostly unqualified. The BR audits have a total of five qualifications, two of which relate to previously disclosed incidents which are not of concern, and three other qualifications which seem to be only of minor concern. | ||
The audits for the first period contain many or all of the same issues as the 2015 audits (Issue F). One can surmise that Symantec chose to split the audits in order that they would not have a qualified audit covering the entirety of 2016. This | The audits for the first period contain many or all of the same issues as the 2015 audits (Issue F). One can surmise that Symantec chose to split the audits in order that they would not have a qualified audit covering the entirety of 2016. This raised questions about how long the issues which led to these qualifications persisted. | ||
None of the audits contain any qualification related to the audit status of the GeoRoot or RA program partners. | |||
===Symantec Response=== | ===Symantec Response=== | ||
| Line 205: | Line 209: | ||
Each of the documents contains, in a following table, Symantec's comments on the qualifications and what they have done or are doing to remedy them. | Each of the documents contains, in a following table, Symantec's comments on the qualifications and what they have done or are doing to remedy them. | ||
Symantec recently [https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/0dOt9JDsE0k stated]: "In our 2014-2015 audits, certain issues were identified that we promptly took action on, including addressing the test certificate incident. We continued these efforts until the Point in Time audit was conducted." | Symantec recently [https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/0dOt9JDsE0k stated]: "In our 2014-2015 audits, certain issues were identified that we promptly took action on, including addressing the test certificate incident. We continued these efforts until the Point in Time audit was conducted." | ||
When asked about the amount of time taken to remediate, Symantec's [https://bug1334377.bmoattachments.org/attachment.cgi?id=8860216 response] stated that the issues were discovered late in 2015 or early in 2016, and some took several months to remediate if you include the need for revalidations and HR team reorganizations, which is why they persisted into the 2016 audit period. Specifically: | |||
* Test certs to registered domains: Discovered and fixed in Sep 2015, but "while inappropriate use of registered domains for testing stopped during the course of our 2014-2015 audits, we did not complete the ID and revocation of all certificates until Mar 2016, and so the finding remained in our first-half Dec 1, 2015-Jun 15, 2016 audits." | |||
* Test certs to un-registered domains: Discovered and fixed in October 2015, but there was a "discovery of additional instance involving approved domains in a test account resulting in 6 additional issued certificates in Approximately Mar 2016." (Mozilla isn't aware that Symantec has previously made disclosure of this mis-issuance.) | |||
* Failure to maintain physical security records for 7 years: discovered and fixed in January 2016. | |||
* Failure to review application and system logs: discovered and fixed in January 2016. | |||
* The failure to refresh background checks every 5 years: discovered in February 2016, fixed in June 2016 (required an internal reorganization). | |||
In regard to the lack of qualifications related to GeoRoot and the RA program, Symantec said: "We acknowledge there were deficiencies in audits for both the GeoRoot and RA programs. The plan for the GeoRoot deficiencies was communicated in the cover letter accompanying our Point in Time audit (see issue V) and for the RA program, in the cover letter to browsers with our 2015-2016 audits." | |||
===Additional Comments and Conclusion=== | |||
The time for further discussion has passed, but it remains unclear how the Management Assertions for the 2014-2015 audits can contain information about problems [https://bug1334377.bmoattachments.org/attachment.cgi?id=8860216 apparently only discovered] in January or February 2016 ("Failure to maintain physical security records for 7 years", "Failure to review application and system logs" and "failure to refresh background checks every 5 years") during the audit process. Is a CA allowed to rewrite its management assertions during the audit process so as to include as "known" any problems found? Would this make the difference between failing and passing an audit? | |||
==Issue R: Insecure Issuance API (2013 or earlier - November 2016)== | It is also not clear whether the disclosure in the cover letters excuse the absence of qualifications related to GeoRoot and the RA program in these audits. | ||
==STRUCK: <strike>Issue R: Insecure Issuance API (2013 or earlier - November 2016)</strike>== | |||
According to [https://www.facebook.com/cbyrneiv/posts/10155129935452436 a report], it is alleged that for several years Symantec operated an issuance API which was insufficiently secure, such that URL parameter substitution attacks would allow one customer to view, reissue, revoke and otherwise control certificates (including non-server certs) belonging to another customer. It is further alleged that, when made aware of these issues, Symantec took a very long time to fix them, and they may not have been fully fixed at the time Symantec terminated its RA program entirely. | According to [https://www.facebook.com/cbyrneiv/posts/10155129935452436 a report], it is alleged that for several years Symantec operated an issuance API which was insufficiently secure, such that URL parameter substitution attacks would allow one customer to view, reissue, revoke and otherwise control certificates (including non-server certs) belonging to another customer. It is further alleged that, when made aware of these issues, Symantec took a very long time to fix them, and they may not have been fully fixed at the time Symantec terminated its RA program entirely. | ||
| Line 239: | Line 259: | ||
Some of these misissuance were caused by employees of CrossCert overriding compliance flags in Symantec's issuance system. Symantec had no process in place to review the logs of overridden flags. For some of the certs, they contained domains neither Symantec nor CrossCert own or control, and CrossCert did not complete the appropriate domain validations for them. | Some of these misissuance were caused by employees of CrossCert overriding compliance flags in Symantec's issuance system. Symantec had no process in place to review the logs of overridden flags. For some of the certs, they contained domains neither Symantec nor CrossCert own or control, and CrossCert did not complete the appropriate domain validations for them. | ||
Lastly, it later emerged that for a number of Certs, CrossCert has incompletely logged records of their telephone validations and so it could not be confirmed that the validation work had been performed correctly. | |||
This incident is recorded in {{bug|1334377}}. | This incident is recorded in {{bug|1334377}}. | ||
| Line 246: | Line 268: | ||
Symantec made a number of comments on this issue - [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831038 0], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831933 1], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831933 2], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8838825 3], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8843448 4]. | Symantec made a number of comments on this issue - [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831038 0], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831933 1], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8831933 2], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8838825 3], [https://bug1334377.bmoattachments.org/attachment.cgi?id=8843448 4]. | ||
The Baseline Requirements, in section 4.9.1.1 item 9, state that the CA SHALL revoke a certificate if "The CA is made aware that the Certificate was not issued in accordance with these Requirements or the CA’s Certificate | The Baseline Requirements, in section 4.9.1.1 item 9, state that the CA SHALL revoke a certificate if "The CA is made aware that the Certificate was not issued in accordance with these Requirements or the CA’s Certificate Policy or Certification Practice Statement". However, Symantec did not revoke all the certificates. | ||
Instead, Symantec decided to shut down the RA program entirely and re-assess every certificate issued under it. Symantec committed to revalidating all of the CrossCert-issued certificates (10,000+) and any of the 20,000+ certificates issued by their other RAs if deficient validation was discovered. However, the determination of deficient validation was made based on the RAs own logs of activity, which may themselves be suspect given some of the audit deficiencies found at these RAs. | Instead, Symantec decided to shut down the RA program entirely and re-assess every certificate issued under it. Symantec committed to revalidating all of the CrossCert-issued certificates (10,000+) and any of the 20,000+ certificates issued by their other RAs if deficient validation was discovered. However, the determination of deficient validation was made based on the RAs own logs of activity, which may themselves be suspect given some of the audit deficiencies found at these RAs. | ||
| Line 254: | Line 276: | ||
===Further Comments and Conclusion=== | ===Further Comments and Conclusion=== | ||
Symantec's reaction to the discovery of these problems was | Symantec's reaction to the discovery of these problems was, to mind, swift and comprehensive - the entire RA program was shut down in fairly short order. | ||
Their case is that WebTrust audit monitoring should have been sufficient, but that they were let down by their auditor, who failed to notice some of the problems, or in other cases it just so happened that the issues were either a long time ago or too recent to be caught by audit. This case is undermined by Issue W - CrossCert and many of the other RA partners had significantly deficient audits and/or audits with serious qualifications. | |||
I conclude that the reason that the situation at CrossCert was not replicated elsewhere was a matter of luck, and that Symantec's monitoring regime for its RA partners - who had full powers of issuance in the WebPKI - was not sufficiently robust. | |||
==Issue V: GeoRoot Program Audit Issues (2013 or earlier - January 2017)== | ==Issue V: GeoRoot Program Audit Issues (2013 or earlier - January 2017)== | ||
| Line 275: | Line 301: | ||
If Symantec did indeed notify us of this situation and we made no comment, that is a relevant fact. | If Symantec did indeed notify us of this situation and we made no comment, that is a relevant fact. | ||
=== | Symantec have also [https://bug1334377.bmoattachments.org/attachment.cgi?id=8860216 stated] that, as of April 21st, 2017, the "Intel, Aetna, and Unicredit CAs have all expired or been revoked." This leaves Google and Apple as the only participants in the GeoRoot program. They also say that: "We agree that getting audits for Aetna and Unicredit took too long. After many discussions, requests, and delays, they finally produced the reports that they did. This experience informed our decision to transition them to alternative solutions." | ||
On the subject of NTT DoCoMo, Symantec [https://bug1334377.bmoattachments.org/attachment.cgi?id=8860216 write]: "All authentication performed for the NTT CA’s has been completed by Symantec personnel. The authentication applications used have historically been fully within the scope of our WTCA/WTBR audits. The infrastructure on which this specific enterprise PKI application and those CAs operate was not covered under audit until these CAs were added to the 2015-2016 audits." | |||
===Further Comments and Conclusion=== | |||
I am following up on the open questions above with Kathleen. | |||
It seems that the NTT DoCoMo infrastructure did fall through the cracks audit-wise until 2015-2016. | |||
Given the power which those organizations held, Symantec did not pursue Aetna and UniCredit for proper audits and appropriate compliance with sufficient alacrity (on UniCredit, see Issue P). | |||
==Issue W: RA Program Audit Issues (2013 or earlier - January 2017)== | ==Issue W: RA Program Audit Issues (2013 or earlier - January 2017)== | ||
| Line 289: | Line 323: | ||
* the network was an insecure mess; and | * the network was an insecure mess; and | ||
* non-trusted staff had access to issuance. | * non-trusted staff had access to issuance. | ||
Prior to 2016, Certsuperior was providing WebTrust audits from an unlicensed auditor. | |||
[https://cert.webtrust.org/SealFile?seal=2168&file=pdf CrossCert's audit] does not list or cover the full number of Symantec roots under which they had issuance capability. Symantec did not notice this mismatch until their recent investigations, when they discovered that CrossCert had the scope of the audit reduced for cost reasons. | [https://cert.webtrust.org/SealFile?seal=2168&file=pdf CrossCert's audit] does not list or cover the full number of Symantec roots under which they had issuance capability. Symantec did not notice this mismatch until their recent investigations, when they discovered that CrossCert had the scope of the audit reduced for cost reasons. | ||
[https://cert.webtrust.org/SealFile?seal=2067&file=pdf Certisur's audit] is only a WebTrust for CAs audits - they do not appear to have a Baseline Requirements audit. According to Symantec's records, this has been true ever since BR audits became required. Mozilla policy requires "CA operations and issuance of certificates to be used for SSL-enabled servers" to conform to the Baseline Requirements. As Symantec has stated that audit was their only mechanism for monitoring their RAs, they can have had no assurance that RAs without a BR audit were actually following the BRs. | |||
[https://bug1334377.bmoattachments.org/attachment.cgi?id=8831929 Certisign's audits] appear to be in order, although their 2016 audit is past due. | |||
===Symantec Response=== | ===Symantec Response=== | ||
| Line 298: | Line 336: | ||
Symantec required the issues at CertSuperior to be fixed and a 90-day action plan was executed to fix them. However, until they decided to shut down the RA program, no certificates issued during the period of suspect operations were checked to see if the poor practice had caused misissuance. They have requested that their next audit include both WebTrust for CAs and WebTrust Baseline. | Symantec required the issues at CertSuperior to be fixed and a 90-day action plan was executed to fix them. However, until they decided to shut down the RA program, no certificates issued during the period of suspect operations were checked to see if the poor practice had caused misissuance. They have requested that their next audit include both WebTrust for CAs and WebTrust Baseline. | ||
Symantec appears to have taken no action to deal with that fact that | Symantec appears to have taken no action to deal with that fact that Certisur did not have a BR audit until recently, when they have requested that their next audit include both WebTrust for CAs and WebTrust Baseline. | ||
Symantec did not notice that CrossCert's audits did not cover all the relevant roots until they did the RA investigation in early 2017. | Symantec did not notice that CrossCert's audits did not cover all the relevant roots until they did the RA investigation in early 2017. | ||
| Line 306: | Line 344: | ||
Despite the clear warning signs shown on the Certsuperior audit, Symantec did not put in place any monitoring of their RAs, other than audit, to check that they were correctly performing the tasks delegated to them under the BRs. (There were some - overridable - technical checks on certificate issuance.) | Despite the clear warning signs shown on the Certsuperior audit, Symantec did not put in place any monitoring of their RAs, other than audit, to check that they were correctly performing the tasks delegated to them under the BRs. (There were some - overridable - technical checks on certificate issuance.) | ||
Symantec's compliance department [https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/Ga1bfOiJr70 | Symantec's compliance department [https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/Ga1bfOiJr70 did not notice] many or any of these audit scope problems until 2016 - in particular those from Certisur and CrossCert. | ||
Symantec have asserted that their oversight of these RA partners was primarily based on reviewing audits. However, the audits had a number of irregularities of various kinds, including being of the wrong type and so not covering the BRs at all, or not covering all issuance, which were not noted until recently. This claim of oversight therefore rings fairly hollow. | |||
==Issue X: Incomplete RA Program Remediation (February - March 2017)== | ==STRUCK: <strike>Issue X: Incomplete RA Program Remediation (February - March 2017)</strike>== | ||
At the time Symantec shut down their RA program, they had four RAs - CrossCert, Certisign, Certsuperior, and Certisur. Symantec committed to revalidating all certificates issued by those RAs. Independent of the rightness or otherwise of this course of action, it should have been applied consistently. However, the program seems to have previously had additional RAs, and Symantec has as yet taken no action to revalidate the certificates that they issued, despite some still being valid. | At the time Symantec shut down their RA program, they had four RAs - CrossCert, Certisign, Certsuperior, and Certisur. Symantec committed to revalidating all certificates issued by those RAs. Independent of the rightness or otherwise of this course of action, it should have been applied consistently. However, the program seems to have previously had additional RAs, and Symantec has as yet taken no action to revalidate the certificates that they issued, despite some still being valid. | ||
| Line 324: | Line 364: | ||
</blockquote> | </blockquote> | ||
Symantec agree that the audits are not clear about what specific CAs are used for, but assert that these organizations are not in a position to issue SSL/TLS certificates. | |||
===Further Comments and Conclusion=== | |||
In the absence of evidence that these organizations have the ability to issue SSL/TLS certificates, we must accept Symantec's assertion. | |||
==Issue Y: Unaudited Unconstrained Intermediates (December 2015 - April 2017)== | ==STRUCK: <strike>Issue Y: Unaudited Unconstrained Intermediates (December 2015 - April 2017)</strike>== | ||
Two intermediate CAs, which are subordinates of or cross-certified by VeriSign Universal Root Certification Authority, appear not to be covered by any of Symantec's audits as listed in their document repository: | Two intermediate CAs, which are subordinates of or cross-certified by VeriSign Universal Root Certification Authority, appear not to be covered by any of Symantec's audits as listed in their document repository: | ||
| Line 339: | Line 383: | ||
===Symantec Response=== | ===Symantec Response=== | ||
Symantec has not | Symantec say that these intermediates "are covered under Symantec’s Non-Fed SSP audits, and the latest unqualified audits that we just received are being published." And indeed such an audit [https://www.symantec.com/content/en/us/about/media/repository/Symantec-NFSSP-WTCA_11-30-2016.pdf has now appeared], albeit significantly late. | ||
===Further Comments and Conclusions=== | |||
These intermediates do appear to be covered by audits, albeit tardily-submitted ones. They are constrained by policy OID, although such constrains are not recognised by the Web PKI, and so they remain capable of issuing SSL certs trusted by Mozilla and need to continue to be under appropriate control and audit. | |||