X.509 certificate parsing error in v41.0.0, not observed in v38.0.4, v40.0.2

[ThorodanBrom]

Hello,

Using cryptography version 41.0.0, I get a parsing error when trying to decode a particular certificate. The error message is pretty cryptic to me, so I'm not sure why exactly.

I did not get the any error when using versions 38.0.4 or 40.0.2

Reproducer

from cryptography.x509 import load_pem_x509_certificate

# the cert isn't sensitive
CERT = '''
-----BEGIN CERTIFICATE-----\nMIIBmjCCAT+gAwIBAgIENly8LzAMBggqhkjOPQQDAgUAMEIxCTAHBgNVBAYTADEJMAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNVBAoTADEJMAcGA1UECxMAMQkwBwYDVQQDEwAwHhcNMjIwNjA3MDYzNDA4WhcNMjMwNjA3MDYzNDA4WjBCMQkwBwYDVQQGEwAxCTAHBgNVBAgTADEJMAcGA1UEBxMAMQkwBwYDVQQKEwAxCTAHBgNVBAsTADEJMAcGA1UEAxMAMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE51twjnVX0FuF+8krofQrvhF6TiTwOWI2BYO4vK3zrImy4EyhpuMJL86//EmuXmJkt6Omd3YrKkKVZkiN4zyZV6MhMB8wHQYDVR0OBBYEFFozVBqs1HcRVwMXDiAOyciW7BOtMAwGCCqGSM49BAMCBQADRwAwRAIgepMfKTURqtkh/4L4kxei6fc972othjG62JAsbyZ1YwICID2Ma+l20vVyWtCbRc7JlHawoYg8OfXfQxIxuUKVw4gG\n-----END CERTIFICATE-----
'''

cert_obj = load_pem_x509_certificate(bytes(CERT, 'utf-8'))
print(cert_obj)

Running with cryptography version 41.0.0

• Wheel name - cryptography-41.0.0-cp37-abi3-manylinux_2_28_x86_64
• cffi - 1.15.1
• pycparser - 2.21

The error I get is:

File "...", line 7, in <module>
    cert_obj = load_pem_x509_certificate(bytes(CERT, 'utf-8'))
  File ".../env/lib/python3.10/site-packages/cryptography/x509/base.py", line 583, in load_pem_x509_certificate
    return rust_x509.load_pem_x509_certificate(data)
ValueError: error parsing asn1 value: ParseError { kind: ExtraData, location: ["Certificate::tbs_cert", "TbsCertificate::signature_alg"] }

Running with cryptography version 38.0.4, 40.0.2

• cffi - 1.15.1
• pycparser - 2.21

It works with these versions, I get a Certificate object

Certificate

The certificate (the same one in the reproducer):

-----BEGIN CERTIFICATE-----
MIIBmjCCAT+gAwIBAgIENly8LzAMBggqhkjOPQQDAgUAMEIxCTAHBgNVBAYTADEJ
MAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNVBAoTADEJMAcGA1UECxMAMQkwBwYD
VQQDEwAwHhcNMjIwNjA3MDYzNDA4WhcNMjMwNjA3MDYzNDA4WjBCMQkwBwYDVQQG
EwAxCTAHBgNVBAgTADEJMAcGA1UEBxMAMQkwBwYDVQQKEwAxCTAHBgNVBAsTADEJ
MAcGA1UEAxMAMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE51twjnVX0FuF+8kr
ofQrvhF6TiTwOWI2BYO4vK3zrImy4EyhpuMJL86//EmuXmJkt6Omd3YrKkKVZkiN
4zyZV6MhMB8wHQYDVR0OBBYEFFozVBqs1HcRVwMXDiAOyciW7BOtMAwGCCqGSM49
BAMCBQADRwAwRAIgepMfKTURqtkh/4L4kxei6fc972othjG62JAsbyZ1YwICID2M
a+l20vVyWtCbRc7JlHawoYg8OfXfQxIxuUKVw4gG
-----END CERTIFICATE-----

The decoded certificate:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 912047151 (0x365cbc2f)
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = , ST = , L = , O = , OU = , CN = 
        Validity
            Not Before: Jun  7 06:34:08 2022 GMT
            Not After : Jun  7 06:34:08 2023 GMT
        Subject: C = , ST = , L = , O = , OU = , CN = 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:e7:5b:70:8e:75:57:d0:5b:85:fb:c9:2b:a1:f4:
                    2b:be:11:7a:4e:24:f0:39:62:36:05:83:b8:bc:ad:
                    f3:ac:89:b2:e0:4c:a1:a6:e3:09:2f:ce:bf:fc:49:
                    ae:5e:62:64:b7:a3:a6:77:76:2b:2a:42:95:66:48:
                    8d:e3:3c:99:57
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                5A:33:54:1A:AC:D4:77:11:57:03:17:0E:20:0E:C9:C8:96:EC:13:AD
    Signature Algorithm: ecdsa-with-SHA256
    Signature Value:
        30:44:02:20:7a:93:1f:29:35:11:aa:d9:21:ff:82:f8:93:17:
        a2:e9:f7:3d:ef:6a:2d:86:31:ba:d8:90:2c:6f:26:75:63:02:
        02:20:3d:8c:6b:e9:76:d2:f5:72:5a:d0:9b:45:ce:c9:94:76:
        b0:a1:88:3c:39:f5:df:43:12:31:b9:42:95:c3:88:06

(I'm not sure if 41.0.0 is complaining because the Issuer and Subject information is blank)

[alex]

It's complaining because of the way the signature algorithm is encoded. Specifically this is encoded with a parameters of NULL, even though per the RFC they should be omitted (from RFC 5758):

When the ecdsa-with-SHA224, ecdsa-with-SHA256, ecdsa-with-SHA384, or
ecdsa-with-SHA512 algorithm identifier appears in the algorithm field
as an AlgorithmIdentifier, the encoding MUST omit the parameters
field.

Can you share more about how this certificate was issued?

[ThorodanBrom]

Hello @alex,

Thanks for the info. The certificate was issued using keytool, a tool for generating Java keystores. I can go over the steps to generate it.

The main keytool command to generate a keystore containing a SHA256-ECDSA private key is:

keytool -genkeypair -keystore <keystore-name>.jks -storetype jks -storepass <store-password> -keyalg EC -alias <alias-name> -keypass <key-password> -sigalg SHA256withECDSA -dname "CN=,OU=,O=,L=,ST=,C=" -validity 360 -deststoretype pkcs12

There are 2 ways to get the certificate out:

With Java

Our code loads the keystore and uses a Java method to pull out the certificate/public key associated with the private key in said keystore. The method is Keystore.getCertificate.

This is how our code does it.

Using keytool CLI

You can also use keytool to fetch the certificate.

keytool -exportcert -keystore <keystore-name> -alias <alias-name> -rfc > cert.pem

---

In the reproducer, the certificate was obtained with the Java code method - I was a little suspicious if we were doing something wrong there, like calling the wrong method. So I used the keytool command to get a certificate out from a keystore I just created today. This also did not work with cryptography v41.0.0 and I get the same error. It works with v38.0.4, v40.0.2.

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 800508053 (0x2fb6c895)
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = , ST = , L = , O = , OU = , CN = 
        Validity
            Not Before: May 31 11:49:25 2023 GMT
            Not After : May 25 11:49:25 2024 GMT
        Subject: C = , ST = , L = , O = , OU = , CN = 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:eb:59:0e:f4:cd:f8:83:ce:ba:a2:e5:f4:c2:ce:
                    ef:61:6f:cd:68:c4:fd:75:a9:e7:5d:a3:36:b8:f8:
                    d3:ba:07:a1:68:70:39:dc:bb:59:3e:38:fb:1d:f4:
                    be:b7:1a:f5:d0:5f:8b:9c:e6:84:47:4c:1d:b6:a4:
                    db:5e:82:1b:72
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                B9:33:C4:9A:B0:6B:E6:A0:3D:C4:7A:85:C4:AE:EE:5F:0C:9E:D2:AB
    Signature Algorithm: ecdsa-with-SHA256
    Signature Value:
        30:45:02:21:00:bd:95:f6:48:e4:08:f0:49:b9:a2:d6:84:fb:
        f3:2b:93:e6:1d:22:14:4f:6c:36:d5:e0:79:a3:66:ae:f7:3e:
        1c:02:20:46:bf:14:de:4c:c8:27:b7:52:4b:5c:69:b2:52:6a:
        e7:51:41:7e:71:66:dd:fe:a0:c2:2b:ce:73:50:0e:0a:82
-----BEGIN CERTIFICATE-----
MIIBmzCCAT+gAwIBAgIEL7bIlTAMBggqhkjOPQQDAgUAMEIxCTAHBgNVBAYTADEJ
MAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNVBAoTADEJMAcGA1UECxMAMQkwBwYD
VQQDEwAwHhcNMjMwNTMxMTE0OTI1WhcNMjQwNTI1MTE0OTI1WjBCMQkwBwYDVQQG
EwAxCTAHBgNVBAgTADEJMAcGA1UEBxMAMQkwBwYDVQQKEwAxCTAHBgNVBAsTADEJ
MAcGA1UEAxMAMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE61kO9M34g866ouX0
ws7vYW/NaMT9dannXaM2uPjTugehaHA53LtZPjj7HfS+txr10F+LnOaER0wdtqTb
XoIbcqMhMB8wHQYDVR0OBBYEFLkzxJqwa+agPcR6hcSu7l8MntKrMAwGCCqGSM49
BAMCBQADSAAwRQIhAL2V9kjkCPBJuaLWhPvzK5PmHSIUT2w21eB5o2au9z4cAiBG
vxTeTMgnt1JLXGmyUmrnUUF+cWbd/qDCK85zUA4Kgg==
-----END CERTIFICATE-----

If there's any more information or test data that you'd like me to provide, I'm happy to help

[alex]

This seems highly likely to be a keytool bug. Sadly for us (though I'm sure you'll be glad :-)) this likely means we'll have to add a workaround, as it means it'll be widely impacting many people.

However, before we do this, I'm dispatching @reaperhulk to go interface with the Java people and find out what the deal is, and get them to fix the issue.

[reaperhulk]

@ThorodanBrom I just tested this using JDK17 and it appears to generate the proper ASN.1 for the certificate -- what version of the JDK are you using?

[ThorodanBrom]

Hello @reaperhulk, unfortunately we're still using JDK 11 (the OpenJDK variant). If it helps, the version is openjdk 11.0.19 2023-04-18

[reaperhulk]

Thanks @ThorodanBrom I've confirmed it on the latest OpenJDK 11 (11.0.19 as you noted). I've reached out to some Java folks to talk about what a fix might look like.

Assuming we get a commitment for a fix then we'll look at putting in a workaround on our end to tolerate this non-compliant certificate structure.

[ThorodanBrom]

Thanks @reaperhulk and @alex

[seanjmullan]

This issue was fixed in JDK 16, so if you use keytool from JDK 16 or later version, it should work. I'll request that this be backported and fixed in earlier JDK versions.

[alex]

Thanks @seanjmullan. We'll figure out our plans in terms of a temporary workaround.