ECDSA/sigVer/sigGen wrong XOF length used with SHAKE-128/SHAKE-256

[sandor-szendro-i4p]

environment
Demo

testSessionId
462560

vsId
2006870

Algorithm registration
"algorithm":"ECDSA",
"mode":"sigVer",
"revision":"FIPS186-5",
"componentTest":false,
"capabilities":[
{
"curve":[
"P-224",
"P-256",
"P-384",
"P-521"
],
"hashAlg":[
"SHA2-224",
"SHA2-256",
"SHA2-384",
"SHA2-512",
"SHA3-224",
"SHA3-256",
"SHA3-384",
"SHA3-512",
"SHAKE-128",
"SHAKE-256"
]
}

Endpoint in which the error is experienced
https://demo.acvts.nist.gov/acvp/v1/testSessions GET

Expected behavior
For ECDSA/sigVer when hashAlg is SHAKE-128 / SHAKE-256 the expected results returned by ACVP server is calculated with the XOF length of 16 / 32 bytes instead of 32 and 64 bytes.
This also applies to ECDSA/sigGen as well.

For example :
"tgId": 9,
"testType": "AFT",
"componentTest": false,
"curve": "P-224",
"hashAlg": "SHAKE-128",

{
"tcId": 62,
"message": "CF9D6EA7D7B7703552B3B37F3DBD8BAF29CE59F5B2CCFD3ECC41AF14570CFA1DC48252BC823A9C2581253B38CAB2A0C79617F0DC8A668E225AF0E274DCF15A8EEB7E31DE8521C61AAAE35C41EF572FAEB58B06B29BE59BF560D8DE48544608DD672B8FFA6056539CC98CC2265CEFFB383937BBE5CFC2B97A789234D9BE90A66C",
"qx": "6BC5DF751BA4B84F003BE8B5D3040AEE25AC5B7160EE9BA478484627",
"qy": "E99462FA46ABC8BA83CBECDABC9C52BD383DB911D17EE8F28CC4FD2B",
"r": "8D97A315D570CE28C76C65A970E8CAAEB90DD479F31B3AFF7595BBE9",
"s": "F62FB523CDF04994F72869B8284F0579A83259048C8B9F4348891633"
},

The expectation says this signature is valid.
This vector will only be valid with the XOF len = 16 bytes, but XOF len of 32 bytes should be used.

Additional context
According to FIPS 186-6.4 ECDSA Digital Signature Generation and Verification
When SHAKE128 or SHAKE256 is used as an XOF in Sections 6.4.1 and 6.4.2 below, its output
length shall be 256 or 512 bits, respectively.
It seem like on the ACVP Server 16 / 32 bytes are used.
This issue seems similar to #277

[szendros]

Can you please confirm or reject this?
This is a blocker for us in the FIPS certification process.

[livebe01]

Hi @szendros @sandor-szendro-i4p, just a note that we're actively looking at and working on this issue.

[livebe01]

The fix for this is on Demo in release v1.1.0.34.

[szendros]

Hi @livebe01 and thanks for the notification

However I did a test run on the Demo environment today and the result was the same:
SHAKE128/SHAKE256 was used with the output length of 16/32 bytes

Are you sure this fix is on the Demo server?

the test run's testSessionID was 500004

[jbrock24]

@szendros - Yes, I will look into it, ty.

[dghgit]

We're seeing this as well - the following from a SIGVer test with "vsId": 2287576, SHAKE-128 for P224.

        "tcId": 1,
        "message": "E42662C6E9F2778B4BE22BD3AC8A719280E0AC29FA0F26CB10E6AFD9422AE4AEBEA5C627FEBF890CE9926023D0F85B052697A2CE625CD1AB4E618573FE611EAC05FBCB0890972923AF0F85CBBA752D9659AE3A60D6EAAA89AE56BE28E3391B6E5E1B98FE8FB5B460F779B90DE67F020C8A69AC1ACF7CC89DFA5ABD44F7219D68",
        "qx": "919078798FF821A72A19FE8350536DA0CCADE4F34B811740027992B2",
        "qy": "4A3625D0B9248BE115B62FF945BAB8A85127068F49A62EBBD22498DD",
        "r": "138031F835963F565EB6A20E6134A244413814BD374036D100D45F76",
        "s": "74BE946A8FA3EA070C4804F31B4CF0C7B05240F0883131B8AD5DDBC3"

This does pass (which appears to be expected) if the SHAKE output is truncated to 16 bytes instead of being 32.

[livebe01]

Hi @szendros, @dghgit, we have the fix for this prepared and it will be included in the next hotfix for Demo. We'd like to deploy the hotfix near the end of this week or the beginning of next.

[livebe01]

The fix for this issue is now on Demo as of today's hotfix deployment.

[slontis]

Would it also be possible to update the test vectors? e.g.
https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigVer-FIPS186-5

[szendros]

I can confirm the fix in the Demo environment.
Thank you, this issue can be closed.

[livebe01]

Would it also be possible to update the test vectors? e.g. https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigVer-FIPS186-5

Hi @slontis, this issue is for ECDSA/sigVer/sigGen. Are you saying that the sample test vectors for ECDSA/sigVer/sigGen need to be updated? Or is it that the RSA sigVer samples need to be updated? Thank you.

[slontis]

Sorry, I meant the ECDSA test vectors. I assume they are incorrect if they were generated without this fix.

[jbrock24]

Good call, I'll update them today, appreciated.

[livebe01]

Sorry, I meant the ECDSA test vectors. I assume they are incorrect if they were generated without this fix.

Hi @slontis, I can confirm that the test vectors were updated/generated with this fix. Unfortunately, the internal commit history seems to get lost when we push the code/files from our internally hosted code repository to GitHub, but you can see that the prompt, expectedResults, and internalProjection files were updated as of the hotfix for v1.1.0.34 that was deployed to Demo last week. See here: https://github.com/usnistgov/ACVP-Server/tree/master/gen-val/json-files/ECDSA-SigGen-FIPS186-5.

[livebe01]

The fix for this is now on Prod as part of the v1.1.0.34 release.