DRAFT blog post: Beware of Applications Misusing Root Stores

257 views
Skip to first unread message

Kathleen Wilson

unread,
May 4, 2021, 11:59:06 AM5/4/21
to dev-secur...@mozilla.org
All,

I will greatly appreciate your feedback on the following post that I am drafting for Mozilla's Security Blog. I am hoping to publish it on Monday, May 10. The target audience is application developers, especially those using Mozilla's root store either directly or via NSS or Linux.


Thanks,
Kathleen



Ryan Sleevi

unread,
May 4, 2021, 12:35:51 PM5/4/21
to Kathleen Wilson, dev-secur...@mozilla.org
Kathleen,

One thing that stands out is Mozilla's approach to removing trust, which has changed somewhat in the past several years. Typically, once a CA has some large set of non-compliance issues, but not yet to the point of a public discussion of how to address these issues, the CA may decide to say "We've stopped issuing TLS certificates" or "We're starting over with new roots, and have stopped using the old ones", at which point, Mozilla decides to distrust new certificates while continuing to trust old certificates (based on the notBefore).

I raise it, mostly because I didn't see any links to https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F or https://wiki.mozilla.org/CA/Additional_Trust_Changes in the doc, both of which seem relevant. For example, if a non-NSS application uses the Mozilla root store, and encounters a CA that is sunset, should the behaviour be to accept the certificate (fail insecure) or reject it (fail secure)? If the recommended answer is "fail insecure", then it seems it may be 1-2 years after Mozilla has removed trust before these consumers will remove trust.

The same example also applies for Microsoft, which has a similar mechanism for code signing ("notBefore"ing), so it seems some highlighting of these issues may be warranted?

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/9603f3c0-af0c-44ee-bb40-589444d542acn%40mozilla.org.

Brian Smith

unread,
May 4, 2021, 12:40:02 PM5/4/21
to Ryan Sleevi, Kathleen Wilson, dev-secur...@mozilla.org
Ryan Sleevi <ry...@sleevi.com> wrote:
One thing that stands out is Mozilla's approach to removing trust, which has changed somewhat in the past several years. Typically, once a CA has some large set of non-compliance issues, but not yet to the point of a public discussion of how to address these issues, the CA may decide to say "We've stopped issuing TLS certificates" or "We're starting over with new roots, and have stopped using the old ones", at which point, Mozilla decides to distrust new certificates while continuing to trust old certificates (based on the notBefore).

Kathleen,

Thanks for sharing this.

Besides the "notBefore" issue Ryan mentioned, are there any other constraints that users are supposed to enforce on the roots? I think it's worth calling out name constraints if that mechanism is still being used. I think it would be useful to hear from the people working on PSM regarding restrictions that are placed on trust anchors that aren't encoded in the NSS data.

Cheers,
Brian

Rob Stradling

unread,
May 4, 2021, 12:46:52 PM5/4/21
to Kathleen Wilson, dev-secur...@mozilla.org
Hi Kathleen.

> Additionally, some application developers make the mistake of using OpenSSL to directly parse a file in Mozilla’s source code management system called certdata.txt, in which Mozilla’s root store is maintained in a form that is convenient for NSS to build from.

I don't think OpenSSL provides any function to directly parse certdata.txt, so I suggest striking the OpenSSL reference.  Also, "to directly parse" isn't necessarily "a mistake"...if the application developer does know what they're doing!

> The problem with the scripts that directly parse this file is that some of the certificates in this file are not trusted but rather explicitly distrusted, so scripts that do not take the trust records into account may be trusting root certificates, such as the DigiNotar certificates, which Mozilla explicitly distrusts.

Is it your intent to:
  (1) Dissuade application developers from using certdata.txt directly, because they'll almost certainly do it wrong; or
  (2) Help application developers to use certdata.txt correctly, because a "Here be dragons!" disclaimer probably won't stop them from using it
?

If (2), might it be useful to provide example code, perhaps by pointing to one or more projects that do parse certdata.txt correctly?


From: dev-secur...@mozilla.org <dev-secur...@mozilla.org> on behalf of Kathleen Wilson <kwi...@mozilla.com>
Sent: 04 May 2021 16:59
To: dev-secur...@mozilla.org <dev-secur...@mozilla.org>
Subject: DRAFT blog post: Beware of Applications Misusing Root Stores
 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Rob Stradling

unread,
May 4, 2021, 12:56:40 PM5/4/21
to Kathleen Wilson, dev-secur...@mozilla.org
Relatedly, I just rediscovered https://github.com/mozilla/pkipolicy/issues/97.  If Mozilla wants certdata.txt to be regarded as a proprietary internal file format with no backward compatibility promise, then dissuading application developers from using certdata.txt directly would seem sensible.


From: dev-secur...@mozilla.org <dev-secur...@mozilla.org> on behalf of Rob Stradling <r...@sectigo.com>
Sent: 04 May 2021 17:46
To: Kathleen Wilson <kwi...@mozilla.com>; dev-secur...@mozilla.org <dev-secur...@mozilla.org>
Subject: Re: DRAFT blog post: Beware of Applications Misusing Root Stores
 

Kathleen Wilson

unread,
May 4, 2021, 1:13:31 PM5/4/21
to dev-secur...@mozilla.org
I have updated the document to address all of the input so far (greatly appreciated!). Please let me know if I did not fully address anything.

Note that I added a link to https://wiki.mozilla.org/CA/Additional_Trust_Changes which talks about distrust-after and name constraints.

Thanks!
Kathleen

On Tuesday, May 4, 2021 at 9:56:40 AM UTC-7 r...@sectigo.com wrote:
Relatedly, I just rediscovered https://github.com/mozilla/pkipolicy/issues/97.  If Mozilla wants certdata.txt to be regarded as a proprietary internal file format with no backward compatibility promise, then dissuading application developers from using certdata.txt directly would seem sensible.

Sent: 04 May 2021 17:46

Subject: Re: DRAFT blog post: Beware of Applications Misusing Root Stores

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hi Kathleen.

> Additionally, some application developers make the mistake of using OpenSSL to directly parse a file in Mozilla’s source code management system called certdata.txt, in which Mozilla’s root store is maintained in a form that is convenient for NSS to build from.

I don't think OpenSSL provides any function to directly parse certdata.txt, so I suggest striking the OpenSSL reference.  Also, "to directly parse" isn't necessarily "a mistake"...if the application developer does know what they're doing!

> The problem with the scripts that directly parse this file is that some of the certificates in this file are not trusted but rather explicitly distrusted, so scripts that do not take the trust records into account may be trusting root certificates, such as the DigiNotar certificates, which Mozilla explicitly distrusts.

Is it your intent to:
  (1) Dissuade application developers from using certdata.txt directly, because they'll almost certainly do it wrong; or
  (2) Help application developers to use certdata.txt correctly, because a "Here be dragons!" disclaimer probably won't stop them from using it
?

If (2), might it be useful to provide example code, perhaps by pointing to one or more projects that do parse certdata.txt correctly?


Sent: 04 May 2021 16:59

Subject: DRAFT blog post: Beware of Applications Misusing Root Stores

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

All,

I will greatly appreciate your feedback on the following post that I am drafting for Mozilla's Security Blog. I am hoping to publish it on Monday, May 10. The target audience is application developers, especially those using Mozilla's root store either directly or via NSS or Linux.


Thanks,
Kathleen



--
You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsub...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsub...@mozilla.org.

Rob Stradling

unread,
May 4, 2021, 3:11:17 PM5/4/21
to Kathleen Wilson, dev-secur...@mozilla.org
Thanks Kathleen.

> "Application developers who continue to parse the certdata.txt file should use a script that correctly takes the trust records into account"

I'd be tempted to emphasize here the insufficiency of only considering the trust records (CKA_TRUST_SERVER_AUTH and CKA_TRUST_EMAIL_PROTECTION).  If an application developer is going to use certdata.txt directly, then why wouldn't they also be advised to consider the distrust records (CKA_NSS_SERVER_DISTRUST_AFTER and CKA_NSS_EMAIL_DISTRUST_AFTER) that are also present in certdata.txt ?
(Are we perhaps presuming that application developers want to have a simple bundle of PEM root certs with no associated metadata?)

I see you've now linked to https://github.com/agl/extract-nss-root-certs, which was last updated 8 years ago and doesn't currently consider any of the items mentioned on https://wiki.mozilla.org/CA/Additional_Trust_Changes, including the distrust records.  Also, AGL's code only considers CKA_TRUST_SERVER_AUTH, not CKA_TRUST_EMAIL_PROTECTION.

In case it's useful, https://github.com/crtsh/root_programs/tree/master/mozilla_certdata is a fork of https://github.com/agl/extract-nss-root-certs that considers all of the trust and distrust records.  Its output is an SQL script that updates the trust store data on crt.sh; obviously this isn't likely to be a useful output format for any other project to consume, but as example code that might not matter.

> We recommend that you use the certificate lists provided on the CCADB Resources page rather than directly parsing the certdata.txt file.

I notice that these CCADB certificate lists contain the certificates but not the distrust fields.  Have you considered adding the CKA_NSS_SERVER_DISTRUST_AFTER information to the "PEM of Root Certificates in Mozilla’s Root Store with the Websites (TLS/SSL) Trust Bit Enabled (CSV)" list and the CKA_NSS_EMAIL_DISTRUST_AFTER information to the "PEM of Root Certificates in Mozilla’s Root Store with the Email (S/MIME) Trust Bit Enabled (CSV)" list?

Are these CCADB certificate lists always guaranteed to be in sync with the latest (mozilla-central) certdata.txt?
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/28b35167-5d39-41ed-a0db-c869cbc8ede8n%40mozilla.org.

Kathleen Wilson

unread,
May 4, 2021, 3:28:05 PM5/4/21
to dev-secur...@mozilla.org
On Tuesday, May 4, 2021 at 12:11:17 PM UTC-7 r...@sectigo.com wrote:
Thanks Kathleen.

> "Application developers who continue to parse the certdata.txt file should use a script that correctly takes the trust records into account"

I'd be tempted to emphasize here the insufficiency of only considering the trust records (CKA_TRUST_SERVER_AUTH and CKA_TRUST_EMAIL_PROTECTION).  If an application developer is going to use certdata.txt directly, then why wouldn't they also be advised to consider the distrust records (CKA_NSS_SERVER_DISTRUST_AFTER and CKA_NSS_EMAIL_DISTRUST_AFTER) that are also present in certdata.txt ?
(Are we perhaps presuming that application developers want to have a simple bundle of PEM root certs with no associated metadata?)

I see you've now linked to https://github.com/agl/extract-nss-root-certs, which was last updated 8 years ago and doesn't currently consider any of the items mentioned on https://wiki.mozilla.org/CA/Additional_Trust_Changes, including the distrust records.  Also, AGL's code only considers CKA_TRUST_SERVER_AUTH, not CKA_TRUST_EMAIL_PROTECTION.

In case it's useful, https://github.com/crtsh/root_programs/tree/master/mozilla_certdata is a fork of https://github.com/agl/extract-nss-root-certs that considers all of the trust and distrust records.  Its output is an SQL script that updates the trust store data on crt.sh; obviously this isn't likely to be a useful output format for any other project to consume, but as example code that might not matter.



Could you add a README to tell app developers more about it?
 

> We recommend that you use the certificate lists provided on the CCADB Resources page rather than directly parsing the certdata.txt file.

I notice that these CCADB certificate lists contain the certificates but not the distrust fields.  Have you considered adding the CKA_NSS_SERVER_DISTRUST_AFTER information to the "PEM of Root Certificates in Mozilla’s Root Store with the Websites (TLS/SSL) Trust Bit Enabled (CSV)" list and the CKA_NSS_EMAIL_DISTRUST_AFTER information to the "PEM of Root Certificates in Mozilla’s Root Store with the Email (S/MIME) Trust Bit Enabled (CSV)" list?

Yes, I can add "Distrust for TLS After Date" / "Distrust for S/MIME After Date"  and "Mozilla Applied Constraints" columns to those CSV reports. Would that be helpful?


 

Are these CCADB certificate lists always guaranteed to be in sync with the latest (mozilla-central) certdata.txt?

 I have a data-integrity check that I run whenever the Firefox release train moves to Beta, to make sure CCADB and certdata.txt are kept in sync.

There are no guarantees for anyone scraping certdata.txt or using reports published from the CCADB: https://www.ccadb.org/rootstores/usage#ccadb-data-usage-terms




Kathleen Wilson

unread,
May 10, 2021, 12:46:41 PM5/10/21
to dev-secur...@mozilla.org
Thanks to all of you for your very helpful input.

I will proceed with publishing the blog post today. The final draft is here:

Also, I will be updating the "PEM of Root Certificates in Mozilla’s Root Store with the Email (S/MIME) Trust Bit Enabled (CSV)"  and "PEM of Root Certificates in Mozilla’s Root Store with the Websites (TLS/SSL) Trust Bit Enabled (CSV)" lists to add columns for distrust-after and Mozilla applied constraints.

Thanks,
Kathleen

Kathleen Wilson

unread,
May 10, 2021, 1:28:42 PM5/10/21
to dev-secur...@mozilla.org

Phillip Hallam-Baker

unread,
May 10, 2021, 4:06:10 PM5/10/21
to Kathleen Wilson, dev-secur...@mozilla.org
Reading through it, it seems to slightly miss the mark. Especially given recent queries I have received regarding the use of WebPKI certs.

As far as I am concerned, 'client' and 'server' miss the point. Every communication must be started by one party. So one party is going to be the initiator and the other a responder. Applied to a Web browser context, the initiator is a person and the responder (typically) an organization. The WebPKI is designed to authenticate organizations and so it works for TLS server auth in Web browsing situations.

The WebPKI does not work at all for TLS client auth when it is a person being authenticated. And I don't see the S/MIME pki as being much better either. There is no good way to get strong authentication for Alice when it is tied to the validation of the name 'al...@example.com'. Doesn't work for S/MIME and it doesn't work for OpenPGP either. Not unless Alice is using a DNS name she owns and thanks to the Network Solutions lawyers being shortsighted idiots in the sex.com case, DNS names are not owned, only rented.

But what of the very common Web Services case where a query by user Alice to service B is based on information B fetches from C? Can we use the WebPKI to authenticate B to C or can we only authenticate C to B? There are two defensible answers here: Both and neither. The fact that B is a client to C makes no difference to the authentication.

What does make a difference is that we have stepped out of the domain for which the certificates are issued and the roots of trust are being curated. And that is true regardless of whether we are using TLS to secure the transport or WS-Security, RUD or anything else. 


I am not trying to nit pick here for the sake of it. I think the biggest mistake we made with the WebPKI was that we allowed the scope to creep far beyond the original purpose of giving people a justified feeling of confidence when they make transactions over the Web. We should have redesigned the system when it became clear that TLS had become an all purpose confidentiality solution. Building on the DNS and X.509 made sense in 1995 for the limited brief that VeriSign class 3 was designed to meet.

For most IoT applications, there is absolutely no reason we should need to rely on third party trust to enable communications.



--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Kathleen Wilson

unread,
May 10, 2021, 5:57:51 PM5/10/21
to dev-secur...@mozilla.org
The ccadb.org Resources tab has been updated with new links for the "PEM of Root Certificates in Mozilla’s Root Store with the Email (S/MIME) Trust Bit Enabled (CSV)"  and "PEM of Root Certificates in Mozilla’s Root Store with the Websites (TLS/SSL) Trust Bit Enabled (CSV)" lists -- to add columns for distrust-after and Mozilla applied constraints.

Thanks,
Kathleen

Rob Stradling

unread,
May 11, 2021, 8:54:08 AM5/11/21
to Kathleen Wilson, dev-secur...@mozilla.org
> I updated the draft to link to  https://github.com/crtsh/root_programs/tree/master/mozilla_certdata
>
> Could you add a README to tell app developers more about it?

Done.


Sent: 04 May 2021 20:28

To: dev-secur...@mozilla.org <dev-secur...@mozilla.org>
Subject: Re: DRAFT blog post: Beware of Applications Misusing Root Stores

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
Reply all
Reply to author
Forward
0 new messages