CircleCI tests failing on main

[SWilson4]

CircleCI is failing on the current main, 8b3d460. However, it passed on the same ref a couple of days ago. Possible causes: merge of open-quantum-safe/liboqs#1650? CircleCI bug?

[SWilson4]

This is due to issue #1815. However, it's not readily apparent because of a bug in our CI logic: the "build" step appeared to have passed: https://app.circleci.com/pipelines/github/open-quantum-safe/oqs-provider/1444/workflows/c02d99f6-46ed-4e91-9384-0b740596d184/jobs/6805.

[baentsch]

This is not just one but two bugs -- 1) CCI build failure must not be considered a pass 2) stateful hash based API include order is not OK; see also open-quantum-safe/oqs-demos#281

[cothan]

The current approach is when stateful Keygen and Sign operations are not defined, we fallback to assign OQS_SIG_STFL to OQS_SIG, which is not an equivalent object.

I think it's better if we define the OQS_SIG_STFL struct according to define flag.
So for OQS_SIG_STFL, Keygen and Sign are disable, thus it becomes:

From

#ifndef OQS_ALLOW_STFL_KEY_AND_SIG_GEN
#define OQS_SIG_STFL OQS_SIG

to (move the #ifdef to include/exclude Keygen and Sign)

/**
 * Stateful signature scheme object
 */
typedef struct OQS_SIG_STFL {

	/**
	 * A local ordinal representing the LMS/XMSS OID parameter of the signature scheme.
	 * This OID is unrelated to ASN.1 OID, it's only for LMS/XMSS internal usage.
	 */
	uint32_t oid;

	/** Printable string representing the name of the signature scheme. */
	const char *method_name;

	/**
	 * Printable string representing the version of the cryptographic algorithm.
	 *
	 * Implementations with the same method_name and same alg_version will be interoperable.
	 * See README.md for information about algorithm compatibility.
	 */
	const char *alg_version;

	/** Whether the signature offers EUF-CMA security (TRUE) or not (FALSE). */
	bool euf_cma;

	/** The (maximum) length, in bytes, of public keys for this signature scheme. */
	size_t length_public_key;
	/** The (maximum) length, in bytes, of secret keys for this signature scheme. */
	size_t length_secret_key;
	/** The (maximum) length, in bytes, of signatures for this signature scheme. */
	size_t length_signature;

#ifdef OQS_ALLOW_STFL_KEY_AND_SIG_GEN
	/**
	 * Keypair generation algorithm.
	 *
	 * Caller is responsible for allocating sufficient memory for `public_key`
	 * based on the `length_*` members in this object or the per-scheme
	 * compile-time macros `OQS_SIG_STFL_*_length_*`.
	 *
	 * @param[out] public_key The public key is represented as a byte string.
	 * @param[out] secret_key The secret key object
	 * @return OQS_SUCCESS or OQS_ERROR
	 */
	OQS_STATUS (*keypair)(uint8_t *public_key, OQS_SIG_STFL_SECRET_KEY *secret_key);

	/**
	 * Signature generation algorithm.
	 *
	 * For stateful signatures, there is always a limited number of signatures that can be used,
	 * The private key signature counter is increased by one once a signature is successfully generated,
	 * When the signature counter reaches the maximum number of available signatures, the signature generation always fails.
	 *
	 * Caller is responsible for allocating sufficient memory for `signature`,
	 * based on the `length_*` members in this object or the per-scheme
	 * compile-time macros `OQS_SIG_STFL_*_length_*`.
	 *
	 * @param[out] signature The signature on the message is represented as a byte string.
	 * @param[out] signature_len The length of the signature.
	 * @param[in] message The message to sign is represented as a byte string.
	 * @param[in] message_len The length of the message to sign.
	 * @param[in] secret_key The secret key object pointer.
	 * @return OQS_SUCCESS or OQS_ERROR
	 *
	 * @note Internally, if `lock/unlock` functions and `mutex` are set, it will attempt to lock the private key and unlock
	 *       the private key after the Signing operation is completed.
	 */
	OQS_STATUS (*sign)(uint8_t *signature, size_t *signature_len, const uint8_t *message, size_t message_len, OQS_SIG_STFL_SECRET_KEY *secret_key);
#endif
	/**
	 * Signature verification algorithm.
	 *
	 * @param[in] message The message is represented as a byte string.
	 * @param[in] message_len The length of the message.
	 * @param[in] signature The signature on the message is represented as a byte string.
	 * @param[in] signature_len The length of the signature.
	 * @param[in] public_key The public key is represented as a byte string.
	 * @return OQS_SUCCESS or OQS_ERROR
	 */
	OQS_STATUS (*verify)(const uint8_t *message, size_t message_len, const uint8_t *signature, size_t signature_len, const uint8_t *public_key);

	/**
	 * Query the number of remaining signatures.
	 *
	 * The remaining signatures are the number of signatures available before the private key runs out of its total signature and expires.
	 *
	 * @param[out] remain The number of remaining signatures
	 * @param[in] secret_key The secret key object pointer.
	 * @return OQS_SUCCESS or OQS_ERROR
	 */
	OQS_STATUS (*sigs_remaining)(unsigned long long *remain, const OQS_SIG_STFL_SECRET_KEY *secret_key);

	/**
	 * Query the total number of signatures.
	 *
	 * The total number of signatures is the constant number present in how many signatures can be generated from a private key.
	 *
	 * @param[out] total The total number of signatures
	 * @param[in] secret_key The secret key key object pointer.
	 * @return OQS_SUCCESS or OQS_ERROR
	 */
	OQS_STATUS (*sigs_total)(unsigned long long *total, const OQS_SIG_STFL_SECRET_KEY *secret_key);

} OQS_SIG_STFL;

What do you think? @ashman-p @dstebila @baentsch @SWilson4 ?

[baentsch]

The drawback of this proposal is that oqsprovider then needs new code to handle OQS_SIG_STFL. The original idea was that that isn't necessary (for SHBS-verify-only liboqs).
But whatever, merging #429 would detach oqsprovider from liboqs "main" branch such as to get CI to pass again until this is resolved for good. This of course should not become a permanent "solution" as this basically disconnects the two projects.

[SWilson4]

What about simply forward-declaring OQS_SIG in sig_stfl.h?

#ifndef OQS_ALLOW_STFL_KEY_AND_SIG_GEN
typedef struct OQS_SIG OQS_SIG;
#define OQS_SIG_STFL OQS_SIG
// ...
#endif //OQS_ALLOW_STFL_KEY_AND_SIG_GEN

[baentsch]

What about simply forward-declaring OQS_SIG in sig_stfl.h?

Could work. But why is the existing include of oqs.h (including sig.h in turn, declaring OQS_SIG, if I'm not mistaken) in sig_stfl.h insufficient?

[SWilson4]

What about simply forward-declaring OQS_SIG in sig_stfl.h?

Could work. But why is the existing include of oqs.h (including sig.h in turn, declaring OQS_SIG, if I'm not mistaken) in sig_stfl.h insufficient?

When the application includes sig.h, the OQS_SIG_H include guard prevents oqs.h from actually including the contents of sig.h, which means that sig_stfl.h and its reference to OQS_SIG is actually included before the body of sig.h.

[baentsch]

the OQS_SIG_H include guard prevents

So would undef'ing this in sig_stfl.h be improving things or cause complete havoc? Anyway, the forward declare should be a solution.

[SWilson4]

So would undef'ing this in sig_stfl.h be improving things or cause complete havoc? Anyway, the forward declare should be a solution.

I suspect the latter, and local experimentation seems to confirm this 😜

[SWilson4]

Resolved by open-quantum-safe/liboqs#1820: CI on oqs-provider main is green again.