bpo-43279: Update code taken from Keccak Code Package

[illia-v]

There were some updates to the Keccak package since it was integrated in CPython initially.

History can be found in https://github.com/XKCP/XKCP.

XKCP's contributors did some refactoring. In particular, they replaced UINT64 and UINT8 with uint64_t and uint8_t.
Also, they added support for 64-bit big-endian platforms.
The changes are reflected in sha3module.c.

I replaced files in the kcp folder with ones generated with generic64lc/libXKCP.a.pack and generic32lc/libXKCP.a.pack targets.
And removed PlSnP-Fallback.inc because it is not used.

https://bugs.python.org/issue43279

[tiran]

Thanks for your patch, but I'm going to remove the entire _sha3 code after PEP 644 has landed. With OpenSSL 1.1.1 we wont need a copy of Keccak any more.

[illia-v]

Thanks for your patch, but I'm going to remove the entire _sha3 code after PEP 644 has landed. With OpenSSL 1.1.1 we wont need a copy of Keccak any more.

Nice.
Would you mind leaving this pull request open for merging it in case PEP 644 is postponed for some reason?

BTW, it looks like a title of the PEP misses one digit in the OpenSSL version, it is "1.1 or newer" instead of "1.1.1 or newer" as mentioned later in the proposal.

Also, I noticed a weird behavior related to SHA-3 in Python 3.8 that uses OpenSSL 1.1.1:

>>> import hashlib
>>> hashlib.new("sha3_224")
<_sha3.sha3_224 object at 0x7f5a57e2ca30>
>>> hashlib.new("sha3-224")
<sha3_224 HASH object @ 0x7f5a57dc6c10>

In the first case, it uses the built-in implementation. And in the second one, it uses an implementation from OpenSSL. I am not sure whether this is a feature or a bug that should be fixed 🙂.
Let me know if I should submit a patch.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[illia-v]

@tiran congrats on landing PEP 644! 🎉

I can see that the _sha3 module is still present in the source code.
Are you going to remove the code before the 3.10 feature freeze?
If not, could you please consider the updates?

[tiran]

@pablogsal What's your opinion on this patch? Ot updates the internal code base of _sha3 module. The code is only used when Python is compiled without OpenSSL support. I initially planned to remove the internal _sha3 module, but I won't have time to finish the task until tomorrow.

I'm a bit worried to land the change. It's a fairly large change. The current code works and as (AFAIK) no problems.

[pablogsal]

@pablogsal What's your opinion on this patch? Ot updates the internal code base of _sha3 module. The code is only used when Python is compiled without OpenSSL support. I initially planned to remove the internal _sha3 module, but I won't have time to finish the task until tomorrow.

I'm a bit worried to land the change. It's a fairly large change. The current code works and as (AFAIK) no problems.

Unless is strictly necessary I would recommend to not do it, specially given the size of the diff (unless someone can review it thoroughly). We are still before beta so this could theoretically be merged but if you are going to remove the _sha3 module this is going to cause a bunch of merge conflicts.

Ultimately is up to you although my recommendation is to at least do it after you remove the _sha3 module.

[illia-v]

but if you are going to remove the _sha3 module this is going to cause a bunch of merge conflicts.

Ultimately is up to you although my recommendation is to at least do it after you remove the _sha3 module.

The update won't be needed when the _sha3 module is removed :)

[illia-v]

I initially planned to remove the internal _sha3 module, but I won't have time to finish the task until tomorrow.

Do you think the module can be removed at the current stage?

[tiran]

I have removed the KCP and replaced it with a much smaller fallback implementation. See bpo-47098 GH-32060