Repository navigation
Feature/x25519 - #186
Feature/x25519#186ounsworth wants to merge 59 commits into
Conversation
|
In bc-java we've found the formulation completely different - I don't think there's any common code between the Edwards and the Weierstrass form - the basis chosen is usually based on efficiency and simplicity as well, so while it's technically possible, it's unlikely to be worth it. I think we have a Weierstrass representation of Curve 25519 somewhere, but I think it's only their for compatibility (usage wise) with someone else's implementation... |
|
Can you do me a favor and add perf tests? |
| type SharedSecretType: AsRef<[u8]>; | ||
|
|
||
| /// Perform a key agreement | ||
| fn key_agreement(pk: &PK, sk: SK) -> Result<Self::SharedSecretType, KeyAgreementError>; |
There was a problem hiding this comment.
sk in this context may vary - in the sense it could be unavailable to the software layer and hardware only. I'd suggest only passing pk, and having the key agreement constructed with sk, and what ever other parameters might be required (for what we're doing it might be useful to be able to apply KDFs as well).
This might actually be a good place for using KeyMaterial as well - ideally we'd make someone do extra work if they want to use a 128 bit secure secret to create a 256 bit shared secret.
There was a problem hiding this comment.
Yes, I know that you want to suggest a refactor like that, and we'll get to that, but for now I'm making the new KeyAgreement trait match the existing KEMEncapsulator trait.
Do you mean having key_agreement(..) -> KeyMaterial instead of returning a DH public key?
I tried this, but:
- A lot of things really do assume that the output of
key_agreementis another public key. For example, the iterated tests in RFC 7748 section 5.2 work that way. Also, I think that group protocols (and maybe Signal / MLS) chain DH's together using the property that the output of DH is a usable public key. - If you are going to hash an X25519PublicKey to get a usable shared KeyMaterial, then which KDF do you use? And what do you include in the KDF? RFC7748 suggests all three values: K, K_A, K_B. But since DH is symmetric, the crypto library doesn't know who is Alice and who is Bob, so it's going to get the order of K_A, K_B wrong.
- To do this robustly, you really need to wrap it in a standardized KEM, both to establish direction, to stardardize the inputs to the KDF, and which KDF to use. That's why I'm also adding a new crate
crypto/hpkeand using that to implKEMEncryptor/KEMDecryptoron the X25519 object.
Does that make sense?
Also, I'm still actively building this. I only put up a PR so that I could see the diff. While I appreciate the enthusiasm, maybe everyone should hold off on review until I've finished thinking this through and it's ready to be reviewed.
There was a problem hiding this comment.
No, it doesn't really make sense.
Concerning the suggestion, yes, 6 times, now 7. We've already gotten to that re-factor so I don't really understand why something would be getting designed with the idea that we haven't - PR #170 does this and we even spent the afternoon reviewing it in Prague.
Also, ECCDH only returns one coordinate, at least as defined in FIPS, strictly speaking so does XDH, although the RFC does hint more is possible. It's possible to create multi-party agreement, but not the way that's suggested, it needs a phased approach so you can tell when the right number of parties have come through the door. For most things, such as HPKE, 2 party is enough. I would only start returning public keys if I was trying to implement multi-party up front, which is something, at the moment at least, I would definitely not suggest we do, but if I was we would need an API which reflected the need to provide phases - it's the reason the Java KeyAgreement API has phases.
Concerning KeyMaterial, I noted that seemed to be the first thing being produced in the doc code. The security strength was correct, but we'd be able to take the guess work out if we simply returned it.
Yes that's on my TODO list! That'll be in place before I mark this PR ready for review. UPDATE: done Everything is about the same (basically a single x25519 scalar mult), except the KEM Encaps, which is a KeyGen + a KeyAgreement, so is about 1/2. The DHKEM's also include a round of HKDF to derive the output shared secret. Neat that that is not really noticeable next to the core x25519 op. |
…o/hpke/dhkem implementation. Also started crate docs.
…from ~1600 to ~1200 bytes.
The release's KDF description (4ce9853) makes KdfParams borrow a KdfDescription rather than a running KDF, so the X25519 HKDF-SHA256 pairing takes KdfParams<'p, HkdfDescription<'s, SHA256>, LEN> as eccdh's P-256 pairing does: X25519KdfAgreement carries the salt's lifetime and builds the engine from the description for each derivation; bouncycastle-ed depends on bouncycastle-hkdf for the description type; the crate example and the tests build an HkdfDescription in place of an HKDF_SHA256 instance, the by-hand check deriving through its engine. No other conflict. Verified: the workspace suite (2910 passed), the all-targets build, rustdoc with warnings denied on ed and hpke, nightly rustfmt, and the x25519 CLI shell suite (14 cases). Assisted-by: Claude:claude-mythos-5-1
…ent's wipe The crate docs' per-call stack table was from before the move onto the key agreement roles and before the review fixes (the field element's Drop wipe, keys and K built in place). Re-measured as the deepest frame chain below each measure closure of mem_usage_benches' x25519 harness (release build, x86-64 Linux, rustc 1.97.0-nightly, frames from -C remark=prologepilog and the call graph from objdump, 8 bytes of return address per hop, the from_bytes row rooted at the harness's key-load helper): keygen_from_seed 1784 B, from_bytes 1704 B, agree 1816 B, encaps 2792 B, decaps 2264 B. The three ladder entry points run through scalar_mult_into (920 B), the ladder (600 B) and a field multiplication; the KEM figures through the DHKEM, HMAC and SHA-256 compression. Assisted-by: Claude:claude-mythos-5-1
The derivation a software pairing runs over its shared secret (zeroize the caller's buffer, build the engine from the description, derive_key_out, check the length) was written out in eccdh's kdf.rs and in ed's X25519 HKDF pairing, the same dozen lines in two crates. KdfParams::derive_out is that derivation once, on the parameters it reads: a KDFError, the engine's or the derivation's, or GenericError if the KDF writes anything but LEN bytes, with `out` left zeroized on every error. The length checks stay with the pairings, which own the KDF's MAX_LEN. Assisted-by: Claude:claude-mythos-5-1
derive keeps the pairing's length check and hands the derivation to the parameters; a wrong-length write now comes back as KeyAgreementError::KDFError over the KDF's GenericError rather than the agreement's own GenericError. The module doc's link to KDF is spelled by path, since the import went with the inlined body. Assisted-by: Claude:claude-mythos-5-1
… doc nits X25519KdfAgreement::agree_out hands K to KdfParams::derive_out, the same lines eccdh's P-256 pairing runs, so the two cannot drift. A test runs the pairing under a salted HkdfDescription: the framework suite, both sides agreeing, the key equal to HKDF-SHA256 extract-and-expand by hand over K with that salt, the absent salt and another info deriving different keys, and a salt typed Unknown refused at the description. Doc nits from the review of 2b4b29e: normalize's doc opened with the two-pass paragraph it replaced, now one summary and the four-pass argument with the RFC 7748 Section 5 sentence folded in; raw_shared_secret and clamped_into each carried their pre-_out summary above the current one; sqr_n said "Declared const" and is not; four links named scalar_mult, now scalar_mult_into; the TODO above inv cited a P64 constant that no longer exists. Assisted-by: Claude:claude-mythos-5-1
…ummary each Each carried its by-value predecessor's summary and "# Errors" heading above the _out form's; the three docs are folded to one summary, keeping the key type and strength the old lines stated. Assisted-by: Claude:claude-mythos-5-1
…HA256>, LEN> The X25519 and ECC CDH entries still named KdfParams<HKDF_SHA256, LEN>, the type from before the KDF description (4ce9853). Assisted-by: Claude:claude-mythos-5-1
Merge review follow-ups, five commits from 626e00b to 3e3b532The review of the merge 7afeed1 found it sound (0 Bug, 0 Hardening, 4 Notes), and the re-check of the fixes at 2b4b29e left doc nits. N1 to N3 and the nits are addressed here, one commit per crate; N4 is the HKDF residue already accepted as outside this PR.
The ML-KEM pairing in mlkem-common has a third copy of the same lines, with a strength step on top; it is left for its own change rather than widening this one. Verified: nightly rustfmt, the all-targets build, rustdoc with warnings denied on core, eccdh, ed and hpke, the workspace suite (2911 passed, 0 failed, 9 ignored), and the x25519 (14), eccdh (5) and hkdf (17) CLI shell suites. Not in these commits: the two residue remainders from the re-check, H1 (the inversion's two field elements moved out of Assisted-by: Claude:claude-mythos-5-1 |
pow_p_sub5_div8 returned (x^3, x^((p-5)/8)) by value, and a move out of a frame runs no Drop, so the two field elements, which together give x^-1 and so the ladder's final z2, stayed on the dead stack after every scalar multiplication (the H1 remainder of the review at 2b4b29e). It now writes them into two buffers inv holds, which inv's frame wipes when they drop; the chain's intermediates were temporaries already, wiped as they drop. The doc's reference to a sqrt_ratio_var that is not in the tree goes with it. The exponent arithmetic is unchanged and the inversion tests pin both powers. Assisted-by: Claude:claude-mythos-5-1
…e caller holds keygen and keygen_from_seed built the key in a local and returned Ok((sk.pk, sk)): two moves, into the pair and into the Result, and a move wipes nothing, so two images of the scalar stayed on the dead stack (the H3 remainder of the review at 2b4b29e, which measured two where the Security Considerations promised one). keygen_out(rng, &mut sk) and keygen_from_seed_out(seed, &mut sk) write into a key the caller holds, made by X25519PrivateKey::zeroized(): the RNG draws straight into the key's own Secret, the key is zeroized first and left so on a refused RNG or seed, and the public key comes back by value. The by-value forms are wrappers over them and say what a by-value return leaves; the Security Considerations drops the count and names the out forms. A test overwrites a stale key through each out form, checks the result against the by-value form for the same RNG output or seed, and checks the key is all zero after a refusal. Assisted-by: Claude:claude-mythos-5-1
keygen_out into a zeroized key in place of the by-value keygen, so the command stages no copy of the scalar on its way to the encoding Secret. Assisted-by: Claude:claude-mythos-5-1
The two residue remainders, three commits from aab8e0b to 5015f1fThe re-check of the fixes at 2b4b29e left two items from the residue probe. Both are in the ed crate and both are the same mechanism: a value returned by value is moved out of the frame that built it, and a move runs no
Not changed: the trait's Verified: nightly rustfmt, the all-targets build, rustdoc with warnings denied on ed including private items, the ed and hpke suites in debug and release, the cli suite, the workspace suite (2912 passed, 0 failed, 9 ignored) and the x25519 CLI shell suite (14 cases). Mutants on ed are being re-run and the residue probe has been asked for; both numbers will follow here. Assisted-by: Claude:claude-mythos-5-1 |
The one real miss of the ed mutants run at 5015f1f (416 mutants: 332 caught, 68 unviable, 3 timeouts in sqr_n's loop, 13 missed): the constant's product replaced by a sum derived a 287-byte key just as well. The rest are the equivalents recorded before: the Drop and zeroize bodies, OR for XOR on disjoint bit ranges in decode64_255 and encode64_255, zero for Default, the debug-only range check, and the KeyMaterialError arm that keygen never returns. Assisted-by: Claude:claude-mythos-5-1
|
Mutants on ed at 5015f1f, on apollo: 416 mutants, 332 caught, 68 unviable, 3 timeouts (the Assisted-by: Claude:claude-mythos-5-1 |
Brings in TDES (fa7789f), OFB (edec585), the cipher test-toys include (52f0221) and the in-place finals of the bcgit#173 sweep (e7a4645, b398a3f). Three conflicts, each kept as both sides: the CLI module list and the stack-harness manifest and module list, where this branch adds X25519 and the release adds TDES. One adaptation: hmac's hazmat finish_out now borrows the inner hash, so HPKE's labeled extract and expand in dhkem.rs pass `&mut hmac`; the bytes are unchanged. Nothing in bouncycastle-ed needed changing. Verified: workspace tests 3110 passed, the all-targets build, rustdoc with warnings denied on ed and hpke, nightly rustfmt, the debug CLI's --help, and test_x25519.sh 14/14. No mutants run for the two changed call sites. Assisted-by: Claude:claude-mythos-5-2
… checks have doctests DHKEMParams::generate_key_pair_rng, which returned the pair by value, becomes with_generated_key_pair: the group generates the private key in its own frame, lends it to the caller and drops it there, so an encapsulation's ephemeral key is wiped where it was drawn. labeled_extract becomes labeled_extract_into, writing the PRK into a buffer the caller holds in place of a KeyMaterial returned through a Result. PARAMS_CHECK and the DERIVE_KEY_PAIR_BY_EXPAND assertion each get a compile_fail doctest with a compiling twin. Comments that named the by-value forms, called hmac's finish_out infallible or cited the review are corrected, and the crate docs' dependency list. Mutants on apollo over the changed functions with the hpke and ed tests: 23 (18 caught, 5 unviable, 0 missed). Assisted-by: Claude:claude-mythos-5-2
…r points are tested in tree
The KEM roles' algorithm is the new marker DHKEM_X25519_HKDF_SHA256 ("DHKEM(X25519, HKDF-SHA256)"), and X25519 no longer implements KEMAlgorithm, so the RFC 7748 function and the RFC 9180 KEM are told apart. A key in the zeroized() state, which the _out generators also leave on an error, makes no agreement and no decapsulator (KeyMaterialError::ActingOnZeroizedKey); it is recognised by its all-zero stored public key, which X25519(k, 9) never is for a clamped k. X25519 implements DHKEMParams::with_generated_key_pair over keygen_out. keygen's doc cites RFC 7748 Section 6.1 for the 32 random bytes in place of a misread RFC 9180 sentence. Tests: all fourteen public keys of Wycheproof's x25519_test.json with an all-zero shared secret, in the source, through agreement and through encaps and decaps; non-canonical u (p + 3, and bit 255 set) agreeing as the reduced value (RFC 7748 Section 5); the unfilled key refused; the KEM workers' name and strength. Mutants on apollo over the changed functions: 19 (7 caught, 12 unviable, 0 missed).
Assisted-by: Claude:claude-mythos-5-2
Assisted-by: Claude:claude-mythos-5-2
|
Update ahead of the squash merge: the branch is now at Release merge ( Brings in TDES, OFB and the in-place finals of the #173 sweep. Three conflicts, each kept as both sides (the CLI module list and the stack-harness manifest and module list). One adaptation: hmac's hazmat Last review No correctness problem was found in the X25519 arithmetic or the RFC behaviour; the citations and vectors checked against fresh copies of RFC 7748 and RFC 9180 match. Of the earlier review's items, two zeroization points were only partly closed and are fixed here, along with three smaller findings. Fixed in
Tests added:
Verification on the branch: workspace tests pass, the all-targets build and rustdoc with warnings denied on Left as follow-ups, not changed here
Assisted-by: Claude:claude-mythos-5-2 |
…s its key before an error exit write_secret_bytes_or_hex, a new shared helper, builds the hex form of a secret in a stack Secret, 32 bytes at a time, with each digit computed rather than looked up; the x25519 command uses it for the private key and the shared secrets, where the heap String of the general helper was freed as it stood. exit runs no destructors, so every action drops the private key before an error path can exit, decaps checks the ciphertext length before it reads the key, and the key file's bytes go out of scope before a parse failure exits. The keygen buffer is sized by SCALAR_LEN. A unit test pins the digit for all sixteen nibbles; test_x25519.sh compares agree -x and decaps -x against the RFC 7748 and RFC 9180 answers. Assisted-by: Claude:claude-mythos-5-2
inv's and normalize's docs lose their accounts of the forms they replaced, and the module's Security Considerations paragraph states what is wiped and what a by-value return leaves without counting copies. Docs only. Assisted-by: Claude:claude-mythos-5-2
The two sections QUALITY_AND_STYLE.md asks of a crate: a doctest that runs DHKEM(X25519, HKDF-SHA256) encap and decap directly, and the call's working memory. Docs only. Assisted-by: Claude:claude-mythos-5-2
…KEM(X25519, HKDF-SHA256) (RFC 9180 Section 4.1) in two new crates. bouncycastle-ed holds the X25519 function over Curve25519 on the key agreement roles: X25519PrivateKey implements AgreementKey over () for the raw shared secret (typed SharedSecret at 128 bits, an all-zero output refused as AgreementFailed, RFC 7748 Section 6.1) and over KdfParams<HkdfDescription<SHA256>, LEN> for a derived key as X25519_HKDF_SHA256; keys come from keygen(rng) / keygen_from_seed and their _out forms, which draw the scalar straight into a key the caller holds; the scalar is kept as given and clamped when used and when encoded; the field arithmetic is five 51-bit limbs with a four-pass normalize, a fixed addition chain for the inversion and a constant-time Montgomery ladder, and field elements, the decoded scalar and the shared secret are wiped where they sit. bouncycastle-hpke holds the generic DHKEM of RFC 9180 over a group that implements DHKEMParams (with_generated_key_pair lends the ephemeral key so that it is wiped where it was drawn, dh_into carries the group's validation, the PRK is written into a buffer the caller holds, and compile-time checks tie Nsecret to the KDF hash, bound Nsk for HKDF-Expand, require a hash of twice the group's strength and restrict derive_key_pair to the expand form of Section 7.1.3); the rest of HPKE is not implemented. The X25519 keys carry the KEM roles through it under the marker DHKEM_X25519_HKDF_SHA256, so the KEM is named apart from the function; a key in the zeroized() state that the _out generators write into makes no agreement and no decapsulator until it is filled. Outside the two crates: KEMError::InvalidPublicKey in core, the x25519 CLI subcommand (keygen, pk-from-sk, check-consistency, agree, encaps, decaps) with a shared write_secret_bytes_or_hex that builds the hex of a secret in a wiped buffer and with the private key dropped before any error exit, the umbrella re-exports, a stack-usage harness, and the 0.1.4 release notes. Tests: RFC 7748 Section 5.2's vectors and iterated test (the million-iteration run ignored by default) and Section 6.1; RFC 9180 Appendix A.1.1, A.1.2, A.2.1, A.2.2, A.7.1 and A.7.2 through the keys and through the generic DHKEM directly; Wycheproof's X25519 set; the fourteen public keys of that set with an all-zero shared secret, carried in the source, through agreement, encaps and decaps; non-canonical u-coordinates agreeing as their reduced value; field unit tests (multiplication, squaring, inversion against x^(p-2), the conditional swap, bounds and a normalize sweep with its edge values); the key agreement, KEM and key frameworks; compile_fail doctests with compiling twins for a DHKEM with the wrong Nsecret and for derive_key_pair on a group without the expand form; and cli/tests/test_x25519.sh against the RFC answers. By Mike Ounsworth, ported onto the release's AgreementKey and KEM key roles and through three release merges (the KDF descriptions; the in-place finals, where HPKE's HMAC halves pass the inner hash by &mut). Reviewed externally (a normalize assertion that was false for real outputs with wrong results on some wide-top-limb values, field elements, scalars and shared secrets left unwiped, the DHKEM secret copied several times, no hpke tests, a generic DHKEM compiling for wrong instantiations; all fixed) and once more before the squash (the ephemeral key and the PRK returned by value, the unfilled key usable, the KEM reporting itself as X25519, low-order points covered only through Wycheproof, stale docs, CLI secrets in an unwiped String and live at error exits, review history in public docs and the hpke crate docs' missing sections; all fixed). Left as follow-ups: the key types implement both the agreement and the KEM key traits, so encode and from_bytes are ambiguous with both in scope; labeled_expand's multi-block path is unreachable for every group derive_key_pair allows and untested; the other CLI commands still print secrets through the general hex helper; the encaps row of the stack table predates the ephemeral-key change. Verified on the final tree: the workspace suite (3120 tests), the all-targets build, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (31 files). Mutants on apollo: ed 411 (333 caught, 61 unviable, 3 timeouts, 14 missed, all equivalents) and hpke 55 (29 caught, 20 unviable, 6 missed, the private strength_bits table, pinned by a unit test) after the external review's fixes, and scoped runs over the functions changed in the last round, ed 19 (7 caught, 12 unviable, 0 missed) and hpke 23 (18 caught, 5 unviable, 0 missed); the CLI and doc changes after that had no mutants run. Assisted-by: Claude:claude-mythos-5-1 Assisted-by: Claude:claude-mythos-5-2
…one pair. core had six per-role traits with the same three methods and supertraits (KEMPrivateKey / KEMPublicKey, KeyAgreementPrivateKey / KeyAgreementPublicKey, SignaturePrivateKey / SignaturePublicKey), differing only in the error from_bytes returned, so a key that serves two roles implemented two of them and its encode, encode_out and from_bytes were ambiguous whenever both were in scope: the X25519 keys of PR #186, in key agreement and as DHKEM, and any DHKEM over the ECC CDH keys to come. They are now PrivateKey<N> and PublicKey<N>, implemented once per key type, with from_bytes returning KeyDecodingError, a message that converts into KEMError, KeyAgreementError and SignatureError as each one's DecodingError, so a function that returns a role error still decodes a key with ?. Every key type is moved onto the pair (ML-KEM and ML-DSA in both forms, ECC CDH, ECDSA, SM2, X25519), the X25519 keys lose their duplicate KEM-side impls and the fully qualified calls that worked around the ambiguity, the crate-private decode helpers of ML-KEM and ML-DSA that could only fail in decoding return KeyDecodingError, and the KEM, key agreement and signature key suites of core-test-framework, the per-crate tests, the CLI and the stack harnesses follow. No decoder accepts or refuses different input and the messages are unchanged, but the error a caller sees changes in three places: an ML-KEM private key whose stored H(ek) does not match its public key is refused by from_bytes as a KeyDecodingError (KEMError::DecodingError once converted) where it was KEMError::ConsistencyCheckFailed, sk_decode still returning the latter; and the low-memory ML-KEM and ML-DSA seed private keys report any failure of their construction as a KeyDecodingError naming the key where a key-material or key-generation error could surface. feature/rsa-sig's RSA keys need the same move when they take this tip. Verified: the workspace suite (3120 tests), the all-targets build with no warnings, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (31 files). Mutants on apollo, scoped to the decoders and their helpers: mlkem 65 (35 caught, 30 unviable), mlkem-lowmemory 22 (4 caught, 17 unviable, 1 missed), mldsa 48 (38 caught, 10 unviable), mldsa-lowmemory 10 (2 caught, 7 unviable, 1 missed), sm2 9 (3 caught, 6 unviable), and core 3, ed 7, eccdh 6 and ecdsa 6 with every mutant unviable, a decoder's return type admitting none of the replacement values; the two misses were a match arm in each seed key's decoder that could not be taken, removed after the run. Assisted-by: Claude:claude-mythos-5-2
… key encoding traits as one PrivateKey / PublicKey pair) into feature/rsa-sig No conflicts. One adaptation: the RSA signature keys and the RSASVE KEM keys implement PrivateKey<N> / PublicKey<N>, with from_bytes returning KeyDecodingError. Four decoders delegate to the crate's raw component decoders and to the KEM keys' from_rsa_key, which return a role error; they convert through two private helpers that keep a decoding failure's message, the only kind those produce. The tests that asserted SignatureError::DecodingError or KEMError::DecodingError on a key's from_bytes assert KeyDecodingError; the CLI, the benches and the stack harness follow the rename. No decoder accepts or refuses different input. Workspace tests 3383 passed, the all-targets build has no warnings, rustdoc with warnings denied over the workspace and nightly rustfmt are clean. No mutants run for the decoder changes. Assisted-by: Claude:claude-mythos-5-2
…521: RFC 9180 Section 4.1's DHKEM(P-256, HKDF-SHA256), DHKEM(P-384, HKDF-SHA384) and DHKEM(P-521, HKDF-SHA512) (kem_id 0x0010 to 0x0012, Section 7.1 Table 2) on the ECC CDH keys. It also settles PR #186's follow-up on hpke's labeled_expand, whose multi-block path no group reached: P-521's key derivation asks HKDF-SHA512 for 66 bytes and is checked against the RFC's own vectors. In bouncycastle-hpke, DeriveKeyPair has both forms of Section 7.1.3: DHKEMParams::DERIVE_KEY_PAIR_BY_EXPAND becomes DERIVE_KEY_PAIR, a DeriveKeyPairForm (Expand for X25519 and X448, Candidate with its bitmask for the NIST curves' rejection sampling), private_key_from_bytes returns an Option, None being the loop's test, and derive_key_pair runs at most 256 candidates and returns KeyGenError past that; the const assertion that refused the candidate form goes with its doctest pair. derive_key_pair no longer refuses keying material shorter than Nsk, which the RFC makes a SHOULD: it requires a seed or full-entropy key of the group's strength, so a 64-byte seed at 256 bits serves P-521 (Nsk 66); a 31-byte seed that still carries 128 bits is now accepted for X25519 too. DHKEM::decap_with_pk_out takes the recipient's public key, for a group whose private key does not hold it. In bouncycastle-eccdh, the new module dhkem makes ECCDH<C> the DHKEM group for the three curves: the DH output is the x-coordinate with the point at infinity refused (Sections 4.1 and 7.1.4), through z_bytes_from_product, the agreement's steps 2 to 4 split out so both share them, the agreement itself unchanged; public keys serialize uncompressed, and enc is accepted only in the uncompressed form with the curve decoder's validation (7.1.1, 7.1.4); a DeriveKeyPair candidate is a key when it is in [1, n-1], under the bitmask 0xFF, or 0x01 for P-521. ECCDHPublicKey<C> and ECCDHPrivateKey<C> carry the EncapsulatingKey and DecapsulatingKey roles over () for those curves, under the marker ECDHKEM<C> and its aliases DHKEM_P256_HKDF_SHA256, DHKEM_P384_HKDF_SHA384 and DHKEM_P521_HKDF_SHA512; dhkem::derive_key_pair is DeriveKeyPair; the decapsulator computes pk(skR) once, when it is made; the sealed DhkemCurve ties each curve's five lengths, its three operations hidden from the docs. The ephemeral key is generated by keygen, returned by value once inside the group's frame, as the module's Security Considerations say. bouncycastle-ed's X25519 group follows the new trait shape. The CLI's eccdh-p256, eccdh-p384 and eccdh-p521 gain encaps and decaps, shaped as the x25519 command's, with secrets through the wiped hex writer and the key dropped before an error exit; the other four curves' commands refuse the two actions. The ECC CDH stack harness and benches gain DHKEM entries. Tests: the RFC's machine-readable vectors (bc-test-data crypto/hpke.txt, the [TestVectors] of Appendix A) for kem_id 16 and 18, 32 entries each, 80 DeriveKeyPair and 16 Decap checks a curve, with Appendix A.3.1 and A.6.1 also carried in the test file so a known answer runs without bc-test-data; Encap is checked by round trip, its ephemeral key coming from the extra-random-bits generation and not from raw RNG bytes. P-384 is defined by the RFC but has no vectors in the appendix or in the cited file, so four sets from BC Java 1.84 are its known answers, from a driver that first reproduced the RFC's P-256 and P-521 vectors with BC Java (32 sets); nine encapsulations from this code, three a curve, were decapsulated by BC Java to the same secret, once, outside the suite. Also: a test group in hpke that refuses its first candidates (the loop moves to the next counter, the mask is applied, the 256th candidate is tried and no 257th), the KEM and key frameworks on the three curves, ciphertexts off the curve or not in the uncompressed form refused, DeriveKeyPair's checks on its keying material, an RNG failure during encapsulation reported as the RNG's with the outputs zeroized, the DH buffer left zeroized on the point-at-infinity path, the names and strengths, and cli/tests/test_eccdh_dhkem.sh (11 cases, known-answer decapsulation on all three curves). Reviewed before the squash against a fresh copy of RFC 9180 (no divergence found; the CLI moving the private key out of its Result, a doc comment left on the wrong function, the unpinned zeroization, the vacuous pass without bc-test-data, the trait docs overstating what a group guarantees for the ephemeral key, the plumbing methods shown as API, the Nsk length rule and the recomputed public key; all fixed). Left as follow-ups: stack figures for the three KEMs in the crate docs' Memory Usage table, and one generic KEM command for the CLI in place of the copy of x25519's. Verified on the final tree: the workspace suite (3134 tests), the all-targets build with no warnings, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (32 files). Mutants on apollo, on the final tree: hpke's derive_key_pair, labeled_expand and decap functions with the hpke, ed and eccdh tests 26 (15 caught, 11 unviable, 0 missed); eccdh's dhkem.rs 135 (16 caught, 119 unviable, 0 missed); the agreement's z functions 19 (8 caught, 11 unviable, 0 missed). Assisted-by: Claude:claude-mythos-5-2
|
After much ado and review, merged! |
…KEM work and PR #186: stack figures, one copy of the CLI's DHKEM actions, and the CLI's secret output through wiped buffers. In bouncycastle-eccdh the Memory Usage table gains encaps and decaps columns for DHKEM over P-256, P-384 and P-521 (5,216 / 4,864, 7,008 / 6,720 and 10,464 / 10,016 bytes), and its keygen and agree columns are re-measured on the same build so the table has one source; in bouncycastle-ed the five rows of the X25519 table are taken again (keygen_from_seed 1816 B, from_bytes 1800, agree 1848, encaps 2840, decaps 2152), with a from_bytes bench added to the x25519 stack harness so that row has the same root as the others. Both tables are the deepest frame chain from the harness's measure wrapper down, the wrapper's own frame counted, with rustc 1.97.0-nightly (d3cd04068) on x86-64; docs and harness only. In the CLI, helpers::kem_helpers::encaps_cmd and decaps_cmd are the DHKEM encaps and decaps actions written once over the EncapsulatingKey and DecapsulatingKey roles, called by the x25519 command and by eccdh-p256, eccdh-p384 and eccdh-p521 in place of the copy the latter carried; the only change in behaviour is that x25519's parse errors on those two actions read "a valid X25519 public key" or "private key"; the ML-KEM commands keep their own path. The CLI's secret output: write_secret_bytes_or_hex, the wiped hex writer, now also writes the shared secret of mlkem encaps and decaps and of eccdh agree and the output of the hkdf and other kdf commands; a private key is written by write_private_key, which encodes it into a Secret through PrivateKey::encode_out and not by value (keygen and the from-seed forms of mlkem, mldsa, ecdsa, eccdh and sm2, and x25519 in place of its own copy); the key-wrap commands, where one side of either direction is a key in the clear, read stdin through a reader that wipes each buffer the input outgrows, call wrap_out and unwrap_out into one buffer and wipe input and output before returning or exiting, with input still raw bytes of any length; the rng command holds its chunk in a Secret, and the hex writer's buffer grows from 32 bytes to a chunk so a long hex stream costs one write a chunk as before. Raw output and the hex text are unchanged throughout. Not reached: keygen in those commands still returns the key pair by value, which a keygen_out in each crate would change and is left for now; decrypted plaintext, which streams. Tests: test_rng.sh gains a 5000-byte hex stream checked against the hex of the same raw stream. Verified on the final tree: the workspace suite (3134 tests), the CLI shell suite (32 files), the all-targets build with no warnings, rustdoc with warnings denied over the workspace and nightly rustfmt. No mutants run: no library code changes. Assisted-by: Claude:claude-mythos-5-2
…efuse a private key that has not been filled. X25519PrivateKey::zeroized(), the key keygen_out and keygen_from_seed_out write into and leave on an error, already made no agreement and no decapsulator (ActingOnZeroizedKey). The group's DH hook, DHKEMParams::dh_into, did not check, so the generic DHKEM functions, which take the key without its role, still computed with it: the all-zero scalar clamps to a value anyone can compute. The hook now refuses the same way and leaves its output zeroized, so DHKEM::decap_out and auth_decap_out on such a key are DecapsulationFailed and auth_encap_rng_out with it as the sender's key is refused. The encoding cannot refuse, PrivateKey::encode having no error to give; zeroized()'s docs say so under Security Considerations: do not store or send the encoding of a key whose generation failed. From the residue re-review of PR #186 (its R2; its R1, the ephemeral key's copy, was already fixed by generating that key in place). Tests: the dhkem test for the unfilled key covers the generic decapsulation, the hook, and the generic authenticated forms on both sides, each with its outputs zeroized. Reviewed before the squash with feature/hpke (correct; the heading, the missing tests of the authenticated forms and the misleading message for a refused sender key, the last fixed in feature/hpke, were its findings). Verified: the ed, dhkem and hpke suites (97 tests), the all-targets build with no warnings, rustdoc with warnings denied and nightly rustfmt. Mutants on apollo over dh_into and is_filled: 7 (5 caught, 2 unviable, 0 missed). Assisted-by: Claude:claude-mythos-5-2
State at the current head (2026-11-14)
X25519 (RFC 7748) and DHKEM(X25519, HKDF-SHA256) (RFC 9180 Section 4.1) in the new crates
bouncycastle-edandbouncycastle-hpke, on the release's key agreement and KEM roles, with the CLIx25519command and the stack harness. Merged up torelease/0.1.4alphaf213c9e (the ECCurvetrait, generic ECDSA and ECC CDH, and the KDF description). What follows is the record of the two rounds since Mike's original description below.Onto the release's key agreement API (2026-11-12)
The branch's own
KeyAgreementtrait went in favour of the roles PR #193 put incore:AgreementKey<Params, SS_LEN>on the private key, theKeyAgreement<PK, SS_LEN>worker withagree_out,KeyAgreementPrivateKey/KeyAgreementPublicKey,KeyAgreementErrorand the framework's agreement suite;KEMError::InvalidPublicKeyfrom this branch stays, for the DHKEM's refusal of a small-order recipient key.X25519PrivateKeyimplementsAgreementKey<(), 32>throughX25519Agreement<'k>, writingKinto the caller's buffer typedSharedSecretat 128 bits (zeroized first, left so on error) and refusing the all-zeroKof a small-order peer key asAgreementFailed; andAgreementKey<KdfParams<'p, HkdfDescription<'s, SHA256>, LEN>, LEN>asX25519_HKDF_SHA256,Kthrough HKDF-SHA256 with the caller's info (RFC 7748 Section 6.1's KDF overKand both public keys), the engine built from the description for each derivation as the release's ECC CDH pairing does.keygen(rng)takes the caller's RNG;keygen_from_seedstays. Gone, with no twin: the OS-seededkeygen(),keypair_consistency_check/consistency_check(sk.pk() == pk),X25519DerivedKey, and the three-party chaining example.x25519 agreegoes through the role and writes the raw shared secret for a KDF;check-consistencycompares the derived public key.The review round (2026-11-12, commits fc84c80, c9f5e71, 4c04b86, 88ec18d)
A combined review of a5b5faa found the mathematics sound (about 480,000 differential cases, all 518 Wycheproof vectors, RFC 7748's 1,000,000-iteration test, the twelve X25519 sets of RFC 9180 Appendix A) and raised one bug, eleven hardening items and twelve notes. All are addressed:
normalizedomain:normalizeis total,reduce(0).reduce(0).reduce(1).reduce(-1)with its domain argued in its doc; thedebug_assertthat fired on a valid agreement is gone, with the reproducing input as a regression test in debug and release.FieldElement25519zeroizes itself on drop;scalar_mult_intowrites into the caller's array, soKnever travels in the public key type; keys are built in place, the scalar going straight into the key's ownSecretand the RNG drawing into it; the DHKEM hasencap_rng_out/decap_out/extract_and_expand_intowith the by-value forms as wrappers; oneHmacKeyper expand. What the trait signatures'Resultmoves leave is disclosed in the crate docs' Security Considerations.crypto/hpke/tests/dhkem_x25519_tests.rs; direct field arithmetic tests (mul, sqr, mul_u32, inversion againstx^(p-2), cswap, bounds, anormalizesweep); RFC 9180 A.2.1, A.2.2, A.7.1 and A.7.2 beside A.1; the framework's bit-flipping on.constcheck that the KDF's hash is at least twice the KEM's strength, andDERIVE_KEY_PAIR_BY_EXPANDas a const on the trait with a const assert.Secret; the help texts say how hex is read.i128products on 32-bit targets, the derived key's strength at short lengths.ed_todo.mdand the commented-out Ed25519 code are deleted (Ed25519 lands as reviewed code in its own PR).from_bytes1704 B, agree 1816 B, encaps 2792 B, decaps 2264 B.Verified at the head:
cargo test --workspace(2910 passed),cargo test --workspace --all-targets --no-run, rustdoc with warnings denied on ed and hpke, nightly rustfmt,cli/tests/test_x25519.sh14 of 14.Still open from below: the crate name (
bouncycastle-ed).Creating as a draft. This is very much not ready for review, but this is the easiest way for me to see the diff.
Plan
I'm planning to do this feature in stages:
bouncycastle-mathcrate, or something. (in theory, this could happen in parallel any time after 1)How to review this PR
cargo doc -p bouncycastle-ed. The sample code is where an end-user would start, and probably also where a reviewer should start.crypto/core/src/traits.rs.crypto/ed/src/curve25519.rs. This is the core math. Don't worry about all the commented-out code that's waiting for ed25519; just skip it and it'll be in a future PR.crypto/ed/src/x25519.rs.x25519.rs, actually using the KeyAgreement trait to obtain a shared symmetric key is surprisingly tricky. So I wanted to impl the KEM traits over a DHKEM construction to hide the details, but then the question becomes: which DHKEM construction to use? RFC9180 HPKE is the only standardized one, so I decided to bite the bullet and add a new cratecrypto/hpkethat provides the generic DHKEM construction, and expose it by impl'ingKEMEcapsulatorandKEMDecapsulatoron theX25519struct.crypto/ed/src/ed_todo.mdthat tracks all the stuff I've done. It's also tracking the x448, ed25519, and ed448 work that I want to add as separate PRs stacked on top of this one.Open Questions
bouncycastle-edis not a great name for the crate, because technically only the signature variants ed25519 / ed448 use Twisted Edwards curves while x25519 / x448 are Montgomery curves. That said, there doesn't seem to be a generally-accepted overarching term for{x25519, x448, ed25519, ed448}. Maybexcurvesoredcurves? I don't want to split them up into separate crates just because there doesn't exist a good name.TODO before merging
crypto/ed/src/ed_todo.md.