Skip to content

Feature/x25519 - #186

Closed
ounsworth wants to merge 59 commits into
bcgit:release/0.1.4alphafrom
ounsworth:feature/x25519
Closed

ounsworth wants to merge 59 commits into
bcgit:release/0.1.4alphafrom
ounsworth:feature/x25519

Conversation

@ounsworth

@ounsworth ounsworth commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

State at the current head (2026-11-14)

X25519 (RFC 7748) and DHKEM(X25519, HKDF-SHA256) (RFC 9180 Section 4.1) in the new crates bouncycastle-ed and bouncycastle-hpke, on the release's key agreement and KEM roles, with the CLI x25519 command and the stack harness. Merged up to release/0.1.4alpha f213c9e (the EC Curve trait, generic ECDSA and ECC CDH, and the KDF description). What follows is the record of the two rounds since Mike's original description below.

Onto the release's key agreement API (2026-11-12)

The branch's own KeyAgreement trait went in favour of the roles PR #193 put in core:

  • core, framework: the release's AgreementKey<Params, SS_LEN> on the private key, the KeyAgreement<PK, SS_LEN> worker with agree_out, KeyAgreementPrivateKey / KeyAgreementPublicKey, KeyAgreementError and the framework's agreement suite; KEMError::InvalidPublicKey from this branch stays, for the DHKEM's refusal of a small-order recipient key.
  • ed: X25519PrivateKey implements AgreementKey<(), 32> through X25519Agreement<'k>, writing K into the caller's buffer typed SharedSecret at 128 bits (zeroized first, left so on error) and refusing the all-zero K of a small-order peer key as AgreementFailed; and AgreementKey<KdfParams<'p, HkdfDescription<'s, SHA256>, LEN>, LEN> as X25519_HKDF_SHA256, K through HKDF-SHA256 with the caller's info (RFC 7748 Section 6.1's KDF over K and both public keys), the engine built from the description for each derivation as the release's ECC CDH pairing does. keygen(rng) takes the caller's RNG; keygen_from_seed stays. Gone, with no twin: the OS-seeded keygen(), keypair_consistency_check / consistency_check (sk.pk() == pk), X25519DerivedKey, and the three-party chaining example.
  • hpke: the labelled extract and expand run HMAC through its hazmat halves, as HKDF does since The MAC worker splits into MACGenerator and MACVerifier, with the tag length in the type #194.
  • cli: x25519 agree goes through the role and writes the raw shared secret for a KDF; check-consistency compares the derived public key.

The review round (2026-11-12, commits fc84c80, c9f5e71, 4c04b86, 88ec18d)

A combined review of a5b5faa found the mathematics sound (about 480,000 differential cases, all 518 Wycheproof vectors, RFC 7748's 1,000,000-iteration test, the twelve X25519 sets of RFC 9180 Appendix A) and raised one bug, eleven hardening items and twelve notes. All are addressed:

  • The bug, and the normalize domain: normalize is total, reduce(0).reduce(0).reduce(1).reduce(-1) with its domain argued in its doc; the debug_assert that fired on a valid agreement is gone, with the reproducing input as a regression test in debug and release.
  • Secrets on the stack: FieldElement25519 zeroizes itself on drop; scalar_mult_into writes into the caller's array, so K never travels in the public key type; keys are built in place, the scalar going straight into the key's own Secret and the RNG drawing into it; the DHKEM has encap_rng_out / decap_out / extract_and_expand_into with the by-value forms as wrappers; one HmacKey per expand. What the trait signatures' Result moves leave is disclosed in the crate docs' Security Considerations.
  • Tests: worker tests pinning the zeroize-first lines; crypto/hpke/tests/dhkem_x25519_tests.rs; direct field arithmetic tests (mul, sqr, mul_u32, inversion against x^(p-2), cswap, bounds, a normalize sweep); RFC 9180 A.2.1, A.2.2, A.7.1 and A.7.2 beside A.1; the framework's bit-flipping on.
  • hpke's generic DHKEM refuses instantiations it would get wrong: a const check that the KDF's hash is at least twice the KEM's strength, and DERIVE_KEY_PAIR_BY_EXPAND as a const on the trait with a const assert.
  • cli: the private key's encoding lives in a Secret; the help texts say how hex is read.
  • Docs: non-canonical public key encodings (RFC 9180 to the letter), the all-zero seed versus all-zero key bytes, the HMAC-on-zero-PRK and all-zero-output cases, the arithmetic doc errors, the i128 products on 32-bit targets, the derived key's strength at short lengths. ed_todo.md and the commented-out Ed25519 code are deleted (Ed25519 lands as reviewed code in its own PR).
  • Mutants: ed 411 (333 caught, 61 unviable, 3 timeouts, 14 missed, all equivalents after the longest-key test); hpke 55 (29 caught, 20 unviable, 6 missed, the private strength table, pinned by a unit test).
  • Stack table: re-measured on the roles after the wipe, as the deepest frame chain below each harness closure (x86-64, rustc 1.97.0-nightly): keygen 1784 B, from_bytes 1704 B, agree 1816 B, encaps 2792 B, decaps 2264 B.

Verified at the head: cargo test --workspace (2910 passed), cargo test --workspace --all-targets --no-run, rustdoc with warnings denied on ed and hpke, nightly rustfmt, cli/tests/test_x25519.sh 14 of 14.

Still open from below: the crate name (bouncycastle-ed).


Creating as a draft. This is very much not ready for review, but this is the easiest way for me to see the diff.


Plan

I'm planning to do this feature in stages:

  1. Implement x25519, including all the new traits, cli and factory stuff for KeyAgreement. (this PR)
  2. Add curve448 as a separate PR, stacked on 1.
  3. Add the signature modes Ed25519 and Ed448 as a separate PR stacked on 2.
  4. Integrate with David / Luis' ECDSA / ECDH work which I think will be a lot of de-duplicating the core elliptic curve and finite-field stuff into a bouncycastle-math crate, or something. (in theory, this could happen in parallel any time after 1)

How to review this PR

  1. Start with the crate docs -- build cargo doc -p bouncycastle-ed. The sample code is where an end-user would start, and probably also where a reviewer should start.
  2. new KeyAgreement traits in crypto/core/src/traits.rs.
  3. Then crypto/ed/src/curve25519.rs. This is the core math. Don't worry about all the commented-out code that's waiting for ed25519; just skip it and it'll be in a future PR.
  4. The public API is in crypto/ed/src/x25519.rs.
  5. As you'll see in the docs sample code in x25519.rs, actually using the KeyAgreement trait to obtain a shared symmetric key is surprisingly tricky. So I wanted to impl the KEM traits over a DHKEM construction to hide the details, but then the question becomes: which DHKEM construction to use? RFC9180 HPKE is the only standardized one, so I decided to bite the bullet and add a new crate crypto/hpke that provides the generic DHKEM construction, and expose it by impl'ing KEMEcapsulator and KEMDecapsulator on the X25519 struct.
  6. There's a file in there, crypto/ed/src/ed_todo.md that tracks all the stuff I've done. It's also tracking the x448, ed25519, and ed448 work that I want to add as separate PRs stacked on top of this one.

Open Questions

  1. bouncycastle-ed is not a great name for the crate, because technically only the signature variants ed25519 / ed448 use Twisted Edwards curves while x25519 / x448 are Montgomery curves. That said, there doesn't seem to be a generally-accepted overarching term for {x25519, x448, ed25519, ed448}. Maybe xcurves or edcurves? I don't want to split them up into separate crates just because there doesn't exist a good name.

TODO before merging

  • Delete crypto/ed/src/ed_todo.md.

@dghgit

dghgit commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

In bc-java we've found the formulation completely different - I don't think there's any common code between the Edwards and the Weierstrass form - the basis chosen is usually based on efficiency and simplicity as well, so while it's technically possible, it's unlikely to be worth it. I think we have a Weierstrass representation of Curve 25519 somewhere, but I think it's only their for compatibility (usage wise) with someone else's implementation...

@npajkovsky

Copy link
Copy Markdown
Collaborator

Can you do me a favor and add perf tests?

Comment thread crypto/core/src/traits.rs Outdated
type SharedSecretType: AsRef<[u8]>;

/// Perform a key agreement
fn key_agreement(pk: &PK, sk: SK) -> Result<Self::SharedSecretType, KeyAgreementError>;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sk in this context may vary - in the sense it could be unavailable to the software layer and hardware only. I'd suggest only passing pk, and having the key agreement constructed with sk, and what ever other parameters might be required (for what we're doing it might be useful to be able to apply KDFs as well).

This might actually be a good place for using KeyMaterial as well - ideally we'd make someone do extra work if they want to use a 128 bit secure secret to create a 256 bit shared secret.

@ounsworth ounsworth Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I know that you want to suggest a refactor like that, and we'll get to that, but for now I'm making the new KeyAgreement trait match the existing KEMEncapsulator trait.


Do you mean having key_agreement(..) -> KeyMaterial instead of returning a DH public key?
I tried this, but:

  1. A lot of things really do assume that the output of key_agreement is another public key. For example, the iterated tests in RFC 7748 section 5.2 work that way. Also, I think that group protocols (and maybe Signal / MLS) chain DH's together using the property that the output of DH is a usable public key.
  2. If you are going to hash an X25519PublicKey to get a usable shared KeyMaterial, then which KDF do you use? And what do you include in the KDF? RFC7748 suggests all three values: K, K_A, K_B. But since DH is symmetric, the crypto library doesn't know who is Alice and who is Bob, so it's going to get the order of K_A, K_B wrong.
  3. To do this robustly, you really need to wrap it in a standardized KEM, both to establish direction, to stardardize the inputs to the KDF, and which KDF to use. That's why I'm also adding a new crate crypto/hpke and using that to impl KEMEncryptor / KEMDecryptor on the X25519 object.

Does that make sense?

Also, I'm still actively building this. I only put up a PR so that I could see the diff. While I appreciate the enthusiasm, maybe everyone should hold off on review until I've finished thinking this through and it's ready to be reviewed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't really make sense.

Concerning the suggestion, yes, 6 times, now 7. We've already gotten to that re-factor so I don't really understand why something would be getting designed with the idea that we haven't - PR #170 does this and we even spent the afternoon reviewing it in Prague.

Also, ECCDH only returns one coordinate, at least as defined in FIPS, strictly speaking so does XDH, although the RFC does hint more is possible. It's possible to create multi-party agreement, but not the way that's suggested, it needs a phased approach so you can tell when the right number of parties have come through the door. For most things, such as HPKE, 2 party is enough. I would only start returning public keys if I was trying to implement multi-party up front, which is something, at the moment at least, I would definitely not suggest we do, but if I was we would need an API which reflected the need to provide phases - it's the reason the Java KeyAgreement API has phases.

Concerning KeyMaterial, I noted that seemed to be the first thing being produced in the doc code. The security strength was correct, but we'd be able to take the guess work out if we simply returned it.

@ounsworth

ounsworth commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Can you do me a favor and add perf tests?

Yes that's on my TODO list! That'll be in place before I mark this PR ready for review.


UPDATE: done

KeyGen/X25519          
                        time:   [1.2698 ms 1.2719 ms 1.2742 ms]
                        thrpt:  [25.113 Kelem/s 25.159 Kelem/s 25.200 Kelem/s]

KeyAgreement/X25519
                        time:   [1.2705 ms 1.2815 ms 1.2943 ms]
                        thrpt:  [24.724 Kelem/s 24.972 Kelem/s 25.188 Kelem/s]

Encaps/DHKEM(X25519, HKDF-SHA256)
                        time:   [2.6005 ms 2.6078 ms 2.6164 ms]
                        thrpt:  [12.231 Kelem/s 12.271 Kelem/s 12.305 Kelem/s]

Decaps/DHKEM(X25519, HKDF-SHA256)
                        time:   [1.3371 ms 1.3405 ms 1.3445 ms]
                        thrpt:  [23.800 Kelem/s 23.872 Kelem/s 23.933 Kelem/s]

Everything is about the same (basically a single x25519 scalar mult), except the KEM Encaps, which is a KeyGen + a KeyAgreement, so is about 1/2. The DHKEM's also include a round of HKDF to derive the output shared secret. Neat that that is not really noticeable next to the core x25519 op.

@ounsworth
ounsworth marked this pull request as ready for review October 31, 2026 01:01
@ounsworth ounsworth added this to the 0.1.4alpha milestone Oct 31, 2026
The release's KDF description (4ce9853) makes KdfParams borrow a KdfDescription
rather than a running KDF, so the X25519 HKDF-SHA256 pairing takes
KdfParams<'p, HkdfDescription<'s, SHA256>, LEN> as eccdh's P-256 pairing does:
X25519KdfAgreement carries the salt's lifetime and builds the engine from the
description for each derivation; bouncycastle-ed depends on bouncycastle-hkdf for
the description type; the crate example and the tests build an HkdfDescription in
place of an HKDF_SHA256 instance, the by-hand check deriving through its engine.
No other conflict. Verified: the workspace suite (2910 passed), the all-targets
build, rustdoc with warnings denied on ed and hpke, nightly rustfmt, and the x25519
CLI shell suite (14 cases).

Assisted-by: Claude:claude-mythos-5-1
…ent's wipe

The crate docs' per-call stack table was from before the move onto the key
agreement roles and before the review fixes (the field element's Drop wipe,
keys and K built in place). Re-measured as the deepest frame chain below each
measure closure of mem_usage_benches' x25519 harness (release build, x86-64
Linux, rustc 1.97.0-nightly, frames from -C remark=prologepilog and the call
graph from objdump, 8 bytes of return address per hop, the from_bytes row rooted
at the harness's key-load helper): keygen_from_seed 1784 B, from_bytes 1704 B,
agree 1816 B, encaps 2792 B, decaps 2264 B. The three ladder entry points run
through scalar_mult_into (920 B), the ladder (600 B) and a field multiplication;
the KEM figures through the DHKEM, HMAC and SHA-256 compression.

Assisted-by: Claude:claude-mythos-5-1
The derivation a software pairing runs over its shared secret (zeroize the
caller's buffer, build the engine from the description, derive_key_out, check
the length) was written out in eccdh's kdf.rs and in ed's X25519 HKDF pairing,
the same dozen lines in two crates. KdfParams::derive_out is that derivation
once, on the parameters it reads: a KDFError, the engine's or the derivation's,
or GenericError if the KDF writes anything but LEN bytes, with `out` left
zeroized on every error. The length checks stay with the pairings, which own
the KDF's MAX_LEN.

Assisted-by: Claude:claude-mythos-5-1
derive keeps the pairing's length check and hands the derivation to the
parameters; a wrong-length write now comes back as KeyAgreementError::KDFError
over the KDF's GenericError rather than the agreement's own GenericError. The
module doc's link to KDF is spelled by path, since the import went with the
inlined body.

Assisted-by: Claude:claude-mythos-5-1
… doc nits

X25519KdfAgreement::agree_out hands K to KdfParams::derive_out, the same lines
eccdh's P-256 pairing runs, so the two cannot drift. A test runs the pairing
under a salted HkdfDescription: the framework suite, both sides agreeing, the
key equal to HKDF-SHA256 extract-and-expand by hand over K with that salt, the
absent salt and another info deriving different keys, and a salt typed Unknown
refused at the description.

Doc nits from the review of 2b4b29e: normalize's doc opened with the two-pass
paragraph it replaced, now one summary and the four-pass argument with the RFC
7748 Section 5 sentence folded in; raw_shared_secret and clamped_into each
carried their pre-_out summary above the current one; sqr_n said "Declared
const" and is not; four links named scalar_mult, now scalar_mult_into; the TODO
above inv cited a P64 constant that no longer exists.

Assisted-by: Claude:claude-mythos-5-1
…ummary each

Each carried its by-value predecessor's summary and "# Errors" heading above
the _out form's; the three docs are folded to one summary, keeping the key
type and strength the old lines stated.

Assisted-by: Claude:claude-mythos-5-1
…HA256>, LEN>

The X25519 and ECC CDH entries still named KdfParams<HKDF_SHA256, LEN>, the
type from before the KDF description (4ce9853).

Assisted-by: Claude:claude-mythos-5-1
@dghgit

dghgit commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Merge review follow-ups, five commits from 626e00b to 3e3b532

The review of the merge 7afeed1 found it sound (0 Bug, 0 Hardening, 4 Notes), and the re-check of the fixes at 2b4b29e left doc nits. N1 to N3 and the nits are addressed here, one commit per crate; N4 is the HKDF residue already accepted as outside this PR.

  • 626e00b core. KdfParams::derive_out: the derivation a software pairing runs over its shared secret (zeroize the caller's buffer, build the engine from the description, derive_key_out, check the length), once, on the parameters it reads. The length checks stay with the pairings, which own the KDF's MAX_LEN.
  • ed80cb6 eccdh (N3). derive keeps its length check and calls derive_out. A wrong-length write, a defensive branch, now comes back as KeyAgreementError::KDFError over the KDF's GenericError.
  • 9839cbd ed (N3, N2, nits). The X25519 pairing calls derive_out, so it and eccdh's P-256 pairing cannot drift. A test runs the pairing under a salted HkdfDescription: the framework suite, both sides agreeing, the key checked against HKDF-SHA256 extract-and-expand by hand over K with that salt, the absent salt and another info deriving different keys, and a salt typed Unknown refused at the description. The doc nits: normalize's two-pass opening is gone, with the RFC 7748 Section 5 sentence folded into the four-pass text; raw_shared_secret and clamped_into have one summary each; sqr_n no longer says it is const; the four scalar_mult links name scalar_mult_into; inv's TODO no longer cites the deleted P64.
  • 212d8c8 hpke (nits). encap_rng_out, decap_out and extract_and_expand_into have one summary and one Errors section each.
  • 3e3b532 release notes (N1). The X25519 and ECC CDH entries say KdfParams<HkdfDescription<SHA256>, LEN>.

The ML-KEM pairing in mlkem-common has a third copy of the same lines, with a strength step on top; it is left for its own change rather than widening this one.

Verified: nightly rustfmt, the all-targets build, rustdoc with warnings denied on core, eccdh, ed and hpke, the workspace suite (2911 passed, 0 failed, 9 ignored), and the x25519 (14), eccdh (5) and hkdf (17) CLI shell suites.

Not in these commits: the two residue remainders from the re-check, H1 (the inversion's two field elements moved out of pow_p_sub5_div8's frame) and H3 (the scalar copies key generation leaves), both in the ed crate.

Assisted-by: Claude:claude-mythos-5-1

pow_p_sub5_div8 returned (x^3, x^((p-5)/8)) by value, and a move out of a
frame runs no Drop, so the two field elements, which together give x^-1 and
so the ladder's final z2, stayed on the dead stack after every scalar
multiplication (the H1 remainder of the review at 2b4b29e). It now writes
them into two buffers inv holds, which inv's frame wipes when they drop; the
chain's intermediates were temporaries already, wiped as they drop. The doc's
reference to a sqrt_ratio_var that is not in the tree goes with it. The
exponent arithmetic is unchanged and the inversion tests pin both powers.

Assisted-by: Claude:claude-mythos-5-1
…e caller holds

keygen and keygen_from_seed built the key in a local and returned
Ok((sk.pk, sk)): two moves, into the pair and into the Result, and a move
wipes nothing, so two images of the scalar stayed on the dead stack (the H3
remainder of the review at 2b4b29e, which measured two where the Security
Considerations promised one). keygen_out(rng, &mut sk) and
keygen_from_seed_out(seed, &mut sk) write into a key the caller holds, made by
X25519PrivateKey::zeroized(): the RNG draws straight into the key's own Secret,
the key is zeroized first and left so on a refused RNG or seed, and the public
key comes back by value. The by-value forms are wrappers over them and say what
a by-value return leaves; the Security Considerations drops the count and names
the out forms. A test overwrites a stale key through each out form, checks the
result against the by-value form for the same RNG output or seed, and checks
the key is all zero after a refusal.

Assisted-by: Claude:claude-mythos-5-1
keygen_out into a zeroized key in place of the by-value keygen, so the command
stages no copy of the scalar on its way to the encoding Secret.

Assisted-by: Claude:claude-mythos-5-1
@dghgit

dghgit commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

The two residue remainders, three commits from aab8e0b to 5015f1f

The re-check of the fixes at 2b4b29e left two items from the residue probe. Both are in the ed crate and both are the same mechanism: a value returned by value is moved out of the frame that built it, and a move runs no Drop, so the self-wiping types never reach the source.

  • aab8e0b ed (H1). pow_p_sub5_div8 returned (x^3, x^((p-5)/8)) as a tuple, so the two field elements, which together give x^-1 and so the ladder's final z2, stayed on the dead stack after every scalar multiplication. It now writes them into two buffers inv holds, wiped when inv's frame drops them; the chain's intermediates were temporaries already. The exponent arithmetic is unchanged and the inversion tests pin both powers. The doc's reference to a sqrt_ratio_var that is not in the tree goes with it.
  • 68a5b76 ed (H3). keygen and keygen_from_seed returned Ok((sk.pk, sk)): two moves, into the pair and into the Result, which is the two images the probe measured against the one the Security Considerations promised. keygen_out(rng, &mut sk) and keygen_from_seed_out(seed, &mut sk) write into a key the caller holds, made by X25519PrivateKey::zeroized(): the RNG draws straight into the key's own Secret, the key is zeroized first and left so on a refused RNG or seed, and the public key comes back by value. The by-value forms are wrappers over them and say what a by-value return leaves, as the DHKEM's by-value forms do; the Security Considerations drops the count and names the out forms. A test overwrites a stale key through each out form, checks the result against the by-value form for the same RNG output or seed, and checks the key is all zero after a refusal.
  • 5015f1f cli. x25519 keygen draws into a key it holds through keygen_out.

Not changed: the trait's from_bytes and the DHKEM's derive_key_pair return by value by their signatures and are covered by the corrected sentence; the partial skE on the DHKEM's encaps error path is the same mechanism inside hpke and is left for the probe to re-measure after these changes.

Verified: nightly rustfmt, the all-targets build, rustdoc with warnings denied on ed including private items, the ed and hpke suites in debug and release, the cli suite, the workspace suite (2912 passed, 0 failed, 9 ignored) and the x25519 CLI shell suite (14 cases). Mutants on ed are being re-run and the residue probe has been asked for; both numbers will follow here.

Assisted-by: Claude:claude-mythos-5-1

The one real miss of the ed mutants run at 5015f1f (416 mutants: 332 caught,
68 unviable, 3 timeouts in sqr_n's loop, 13 missed): the constant's product
replaced by a sum derived a 287-byte key just as well. The rest are the
equivalents recorded before: the Drop and zeroize bodies, OR for XOR on
disjoint bit ranges in decode64_255 and encode64_255, zero for Default, the
debug-only range check, and the KeyMaterialError arm that keygen never returns.

Assisted-by: Claude:claude-mythos-5-1
@dghgit

dghgit commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Mutants on ed at 5015f1f, on apollo: 416 mutants, 332 caught, 68 unviable, 3 timeouts (the sqr_n loop made endless, as before), 13 missed. Twelve of the misses are the equivalents recorded at 88ec18d: the Drop and zeroize bodies, OR for XOR on disjoint bit ranges in decode64_255 and encode64_255, zero for Default, the debug-only range check, and the KeyMaterialError arm that keygen never returns. The one real miss was 255 * 32 to 255 + 32 in HKDF_SHA256_MAX_LEN, which derived a 287-byte key just as well; 1110935 pins the constant in the longest-key test. The residue probe is with the reviewer.

Assisted-by: Claude:claude-mythos-5-1

Brings in TDES (fa7789f), OFB (edec585), the cipher test-toys include (52f0221)
and the in-place finals of the bcgit#173 sweep (e7a4645, b398a3f). Three conflicts,
each kept as both sides: the CLI module list and the stack-harness manifest and
module list, where this branch adds X25519 and the release adds TDES. One
adaptation: hmac's hazmat finish_out now borrows the inner hash, so HPKE's
labeled extract and expand in dhkem.rs pass `&mut hmac`; the bytes are unchanged.
Nothing in bouncycastle-ed needed changing.

Verified: workspace tests 3110 passed, the all-targets build, rustdoc with
warnings denied on ed and hpke, nightly rustfmt, the debug CLI's --help, and
test_x25519.sh 14/14. No mutants run for the two changed call sites.

Assisted-by: Claude:claude-mythos-5-2
… checks have doctests

DHKEMParams::generate_key_pair_rng, which returned the pair by value, becomes with_generated_key_pair: the group generates the private key in its own frame, lends it to the caller and drops it there, so an encapsulation's ephemeral key is wiped where it was drawn. labeled_extract becomes labeled_extract_into, writing the PRK into a buffer the caller holds in place of a KeyMaterial returned through a Result. PARAMS_CHECK and the DERIVE_KEY_PAIR_BY_EXPAND assertion each get a compile_fail doctest with a compiling twin. Comments that named the by-value forms, called hmac's finish_out infallible or cited the review are corrected, and the crate docs' dependency list. Mutants on apollo over the changed functions with the hpke and ed tests: 23 (18 caught, 5 unviable, 0 missed).

Assisted-by: Claude:claude-mythos-5-2
…r points are tested in tree

The KEM roles' algorithm is the new marker DHKEM_X25519_HKDF_SHA256 ("DHKEM(X25519, HKDF-SHA256)"), and X25519 no longer implements KEMAlgorithm, so the RFC 7748 function and the RFC 9180 KEM are told apart. A key in the zeroized() state, which the _out generators also leave on an error, makes no agreement and no decapsulator (KeyMaterialError::ActingOnZeroizedKey); it is recognised by its all-zero stored public key, which X25519(k, 9) never is for a clamped k. X25519 implements DHKEMParams::with_generated_key_pair over keygen_out. keygen's doc cites RFC 7748 Section 6.1 for the 32 random bytes in place of a misread RFC 9180 sentence. Tests: all fourteen public keys of Wycheproof's x25519_test.json with an all-zero shared secret, in the source, through agreement and through encaps and decaps; non-canonical u (p + 3, and bit 255 set) agreeing as the reduced value (RFC 7748 Section 5); the unfilled key refused; the KEM workers' name and strength. Mutants on apollo over the changed functions: 19 (7 caught, 12 unviable, 0 missed).

Assisted-by: Claude:claude-mythos-5-2
Assisted-by: Claude:claude-mythos-5-2
@dghgit

dghgit commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Update ahead of the squash merge: the branch is now at 4f5b8e72, with release/0.1.4alpha (b398a3fb) merged in and a last review round applied.

Release merge (52099d62)

Brings in TDES, OFB and the in-place finals of the #173 sweep. Three conflicts, each kept as both sides (the CLI module list and the stack-harness manifest and module list). One adaptation: hmac's hazmat finish_out now borrows the inner hash, so HPKE's labeled extract and expand pass &mut hmac.

Last review

No correctness problem was found in the X25519 arithmetic or the RFC behaviour; the citations and vectors checked against fresh copies of RFC 7748 and RFC 9180 match. Of the earlier review's items, two zeroization points were only partly closed and are fixed here, along with three smaller findings.

Fixed in 0183e8c6 (hpke) and 6cb504c9 (ed):

  • Ephemeral key in encaps. DHKEMParams::generate_key_pair_rng returned the pair by value, so copies of the ephemeral private key could be left behind by moves. It is replaced by with_generated_key_pair, which generates the key in the group's own frame, lends it to the caller and drops it there. This changes the DHKEMParams trait.
  • PRK returned by value. labeled_extract is now labeled_extract_into, writing into a buffer the caller holds.
  • An unfilled key was usable. X25519PrivateKey::zeroized(), which is also the state the _out generators leave on an error, agreed and decapsulated with a publicly known scalar. It now makes no agreement and no decapsulator (KeyMaterialError::ActingOnZeroizedKey).
  • The KEM reported itself as "X25519". The KEM roles' algorithm is a new marker, DHKEM_X25519_HKDF_SHA256, named "DHKEM(X25519, HKDF-SHA256)"; X25519 no longer implements KEMAlgorithm. This changes the Algorithm of the KEM role impls.
  • Docs. keygen's doc misread RFC 9180 on DeriveKeyPair (the code was right); it now cites RFC 7748 Section 6.1. Two out-of-date "Cannot fail" comments, a module comment and the crate docs' dependency list are corrected.

Tests added:

  • All fourteen public keys that Wycheproof's x25519_test.json lists with an all-zero shared secret, carried in the source, through agreement and through encaps and decaps. Before, only u = 0 was covered without the Wycheproof checkout.
  • Non-canonical u-coordinates (p + 3, and bit 255 set) agree as their reduced value (RFC 7748 Section 5).
  • The unfilled key is refused; the KEM workers report the DHKEM name.
  • compile_fail doctests, each with a compiling twin, for a DHKEM with the wrong Nsecret and for derive_key_pair on a group that does not declare the expand form.

Verification on the branch: workspace tests pass, the all-targets build and rustdoc with warnings denied on ed and hpke are clean, nightly rustfmt is clean, and test_x25519.sh passes 14/14. Scoped mutants over the changed functions: ed 19 (7 caught, 12 unviable, 0 missed), hpke 23 (18 caught, 5 unviable, 0 missed).

Left as follow-ups, not changed here

  • X25519PrivateKey and X25519PublicKey implement both the key-agreement and the KEM key traits, so encode, encode_out and from_bytes are ambiguous when both are in scope.
  • labeled_expand's multi-block path is unreachable for every group derive_key_pair allows, and has no test.
  • The CLI's hex output passes secrets through an unwiped String in shared helper code, and its error paths exit with a private key still live.
  • Some public docs still carry review history, and the hpke crate docs lack the "Usage Examples" and "Memory Usage" sections QUALITY_AND_STYLE.md asks for.

Assisted-by: Claude:claude-mythos-5-2

…s its key before an error exit

write_secret_bytes_or_hex, a new shared helper, builds the hex form of a secret in a stack Secret, 32 bytes at a time, with each digit computed rather than looked up; the x25519 command uses it for the private key and the shared secrets, where the heap String of the general helper was freed as it stood. exit runs no destructors, so every action drops the private key before an error path can exit, decaps checks the ciphertext length before it reads the key, and the key file's bytes go out of scope before a parse failure exits. The keygen buffer is sized by SCALAR_LEN. A unit test pins the digit for all sixteen nibbles; test_x25519.sh compares agree -x and decaps -x against the RFC 7748 and RFC 9180 answers.

Assisted-by: Claude:claude-mythos-5-2
inv's and normalize's docs lose their accounts of the forms they replaced, and the module's Security Considerations paragraph states what is wiped and what a by-value return leaves without counting copies. Docs only.

Assisted-by: Claude:claude-mythos-5-2
The two sections QUALITY_AND_STYLE.md asks of a crate: a doctest that runs DHKEM(X25519, HKDF-SHA256) encap and decap directly, and the call's working memory. Docs only.

Assisted-by: Claude:claude-mythos-5-2
dghgit pushed a commit that referenced this pull request Nov 17, 2026
…KEM(X25519, HKDF-SHA256) (RFC 9180 Section 4.1) in two new crates. bouncycastle-ed holds the X25519 function over Curve25519 on the key agreement roles: X25519PrivateKey implements AgreementKey over () for the raw shared secret (typed SharedSecret at 128 bits, an all-zero output refused as AgreementFailed, RFC 7748 Section 6.1) and over KdfParams<HkdfDescription<SHA256>, LEN> for a derived key as X25519_HKDF_SHA256; keys come from keygen(rng) / keygen_from_seed and their _out forms, which draw the scalar straight into a key the caller holds; the scalar is kept as given and clamped when used and when encoded; the field arithmetic is five 51-bit limbs with a four-pass normalize, a fixed addition chain for the inversion and a constant-time Montgomery ladder, and field elements, the decoded scalar and the shared secret are wiped where they sit. bouncycastle-hpke holds the generic DHKEM of RFC 9180 over a group that implements DHKEMParams (with_generated_key_pair lends the ephemeral key so that it is wiped where it was drawn, dh_into carries the group's validation, the PRK is written into a buffer the caller holds, and compile-time checks tie Nsecret to the KDF hash, bound Nsk for HKDF-Expand, require a hash of twice the group's strength and restrict derive_key_pair to the expand form of Section 7.1.3); the rest of HPKE is not implemented. The X25519 keys carry the KEM roles through it under the marker DHKEM_X25519_HKDF_SHA256, so the KEM is named apart from the function; a key in the zeroized() state that the _out generators write into makes no agreement and no decapsulator until it is filled. Outside the two crates: KEMError::InvalidPublicKey in core, the x25519 CLI subcommand (keygen, pk-from-sk, check-consistency, agree, encaps, decaps) with a shared write_secret_bytes_or_hex that builds the hex of a secret in a wiped buffer and with the private key dropped before any error exit, the umbrella re-exports, a stack-usage harness, and the 0.1.4 release notes. Tests: RFC 7748 Section 5.2's vectors and iterated test (the million-iteration run ignored by default) and Section 6.1; RFC 9180 Appendix A.1.1, A.1.2, A.2.1, A.2.2, A.7.1 and A.7.2 through the keys and through the generic DHKEM directly; Wycheproof's X25519 set; the fourteen public keys of that set with an all-zero shared secret, carried in the source, through agreement, encaps and decaps; non-canonical u-coordinates agreeing as their reduced value; field unit tests (multiplication, squaring, inversion against x^(p-2), the conditional swap, bounds and a normalize sweep with its edge values); the key agreement, KEM and key frameworks; compile_fail doctests with compiling twins for a DHKEM with the wrong Nsecret and for derive_key_pair on a group without the expand form; and cli/tests/test_x25519.sh against the RFC answers. By Mike Ounsworth, ported onto the release's AgreementKey and KEM key roles and through three release merges (the KDF descriptions; the in-place finals, where HPKE's HMAC halves pass the inner hash by &mut). Reviewed externally (a normalize assertion that was false for real outputs with wrong results on some wide-top-limb values, field elements, scalars and shared secrets left unwiped, the DHKEM secret copied several times, no hpke tests, a generic DHKEM compiling for wrong instantiations; all fixed) and once more before the squash (the ephemeral key and the PRK returned by value, the unfilled key usable, the KEM reporting itself as X25519, low-order points covered only through Wycheproof, stale docs, CLI secrets in an unwiped String and live at error exits, review history in public docs and the hpke crate docs' missing sections; all fixed). Left as follow-ups: the key types implement both the agreement and the KEM key traits, so encode and from_bytes are ambiguous with both in scope; labeled_expand's multi-block path is unreachable for every group derive_key_pair allows and untested; the other CLI commands still print secrets through the general hex helper; the encaps row of the stack table predates the ephemeral-key change. Verified on the final tree: the workspace suite (3120 tests), the all-targets build, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (31 files). Mutants on apollo: ed 411 (333 caught, 61 unviable, 3 timeouts, 14 missed, all equivalents) and hpke 55 (29 caught, 20 unviable, 6 missed, the private strength_bits table, pinned by a unit test) after the external review's fixes, and scoped runs over the functions changed in the last round, ed 19 (7 caught, 12 unviable, 0 missed) and hpke 23 (18 caught, 5 unviable, 0 missed); the CLI and doc changes after that had no mutants run.

Assisted-by: Claude:claude-mythos-5-1
Assisted-by: Claude:claude-mythos-5-2
dghgit added a commit that referenced this pull request Nov 17, 2026
…one pair. core had six per-role traits with the same three methods and supertraits (KEMPrivateKey / KEMPublicKey, KeyAgreementPrivateKey / KeyAgreementPublicKey, SignaturePrivateKey / SignaturePublicKey), differing only in the error from_bytes returned, so a key that serves two roles implemented two of them and its encode, encode_out and from_bytes were ambiguous whenever both were in scope: the X25519 keys of PR #186, in key agreement and as DHKEM, and any DHKEM over the ECC CDH keys to come. They are now PrivateKey<N> and PublicKey<N>, implemented once per key type, with from_bytes returning KeyDecodingError, a message that converts into KEMError, KeyAgreementError and SignatureError as each one's DecodingError, so a function that returns a role error still decodes a key with ?. Every key type is moved onto the pair (ML-KEM and ML-DSA in both forms, ECC CDH, ECDSA, SM2, X25519), the X25519 keys lose their duplicate KEM-side impls and the fully qualified calls that worked around the ambiguity, the crate-private decode helpers of ML-KEM and ML-DSA that could only fail in decoding return KeyDecodingError, and the KEM, key agreement and signature key suites of core-test-framework, the per-crate tests, the CLI and the stack harnesses follow. No decoder accepts or refuses different input and the messages are unchanged, but the error a caller sees changes in three places: an ML-KEM private key whose stored H(ek) does not match its public key is refused by from_bytes as a KeyDecodingError (KEMError::DecodingError once converted) where it was KEMError::ConsistencyCheckFailed, sk_decode still returning the latter; and the low-memory ML-KEM and ML-DSA seed private keys report any failure of their construction as a KeyDecodingError naming the key where a key-material or key-generation error could surface. feature/rsa-sig's RSA keys need the same move when they take this tip. Verified: the workspace suite (3120 tests), the all-targets build with no warnings, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (31 files). Mutants on apollo, scoped to the decoders and their helpers: mlkem 65 (35 caught, 30 unviable), mlkem-lowmemory 22 (4 caught, 17 unviable, 1 missed), mldsa 48 (38 caught, 10 unviable), mldsa-lowmemory 10 (2 caught, 7 unviable, 1 missed), sm2 9 (3 caught, 6 unviable), and core 3, ed 7, eccdh 6 and ecdsa 6 with every mutant unviable, a decoder's return type admitting none of the replacement values; the two misses were a match arm in each seed key's decoder that could not be taken, removed after the run.

Assisted-by: Claude:claude-mythos-5-2
dghgit added a commit that referenced this pull request Nov 17, 2026
… key encoding traits as one PrivateKey / PublicKey pair) into feature/rsa-sig

No conflicts. One adaptation: the RSA signature keys and the RSASVE KEM keys implement PrivateKey<N> / PublicKey<N>, with from_bytes returning KeyDecodingError. Four decoders delegate to the crate's raw component decoders and to the KEM keys' from_rsa_key, which return a role error; they convert through two private helpers that keep a decoding failure's message, the only kind those produce. The tests that asserted SignatureError::DecodingError or KEMError::DecodingError on a key's from_bytes assert KeyDecodingError; the CLI, the benches and the stack harness follow the rename. No decoder accepts or refuses different input. Workspace tests 3383 passed, the all-targets build has no warnings, rustdoc with warnings denied over the workspace and nightly rustfmt are clean. No mutants run for the decoder changes.

Assisted-by: Claude:claude-mythos-5-2
dghgit added a commit that referenced this pull request Nov 17, 2026
…521: RFC 9180 Section 4.1's DHKEM(P-256, HKDF-SHA256), DHKEM(P-384, HKDF-SHA384) and DHKEM(P-521, HKDF-SHA512) (kem_id 0x0010 to 0x0012, Section 7.1 Table 2) on the ECC CDH keys. It also settles PR #186's follow-up on hpke's labeled_expand, whose multi-block path no group reached: P-521's key derivation asks HKDF-SHA512 for 66 bytes and is checked against the RFC's own vectors. In bouncycastle-hpke, DeriveKeyPair has both forms of Section 7.1.3: DHKEMParams::DERIVE_KEY_PAIR_BY_EXPAND becomes DERIVE_KEY_PAIR, a DeriveKeyPairForm (Expand for X25519 and X448, Candidate with its bitmask for the NIST curves' rejection sampling), private_key_from_bytes returns an Option, None being the loop's test, and derive_key_pair runs at most 256 candidates and returns KeyGenError past that; the const assertion that refused the candidate form goes with its doctest pair. derive_key_pair no longer refuses keying material shorter than Nsk, which the RFC makes a SHOULD: it requires a seed or full-entropy key of the group's strength, so a 64-byte seed at 256 bits serves P-521 (Nsk 66); a 31-byte seed that still carries 128 bits is now accepted for X25519 too. DHKEM::decap_with_pk_out takes the recipient's public key, for a group whose private key does not hold it. In bouncycastle-eccdh, the new module dhkem makes ECCDH<C> the DHKEM group for the three curves: the DH output is the x-coordinate with the point at infinity refused (Sections 4.1 and 7.1.4), through z_bytes_from_product, the agreement's steps 2 to 4 split out so both share them, the agreement itself unchanged; public keys serialize uncompressed, and enc is accepted only in the uncompressed form with the curve decoder's validation (7.1.1, 7.1.4); a DeriveKeyPair candidate is a key when it is in [1, n-1], under the bitmask 0xFF, or 0x01 for P-521. ECCDHPublicKey<C> and ECCDHPrivateKey<C> carry the EncapsulatingKey and DecapsulatingKey roles over () for those curves, under the marker ECDHKEM<C> and its aliases DHKEM_P256_HKDF_SHA256, DHKEM_P384_HKDF_SHA384 and DHKEM_P521_HKDF_SHA512; dhkem::derive_key_pair is DeriveKeyPair; the decapsulator computes pk(skR) once, when it is made; the sealed DhkemCurve ties each curve's five lengths, its three operations hidden from the docs. The ephemeral key is generated by keygen, returned by value once inside the group's frame, as the module's Security Considerations say. bouncycastle-ed's X25519 group follows the new trait shape. The CLI's eccdh-p256, eccdh-p384 and eccdh-p521 gain encaps and decaps, shaped as the x25519 command's, with secrets through the wiped hex writer and the key dropped before an error exit; the other four curves' commands refuse the two actions. The ECC CDH stack harness and benches gain DHKEM entries. Tests: the RFC's machine-readable vectors (bc-test-data crypto/hpke.txt, the [TestVectors] of Appendix A) for kem_id 16 and 18, 32 entries each, 80 DeriveKeyPair and 16 Decap checks a curve, with Appendix A.3.1 and A.6.1 also carried in the test file so a known answer runs without bc-test-data; Encap is checked by round trip, its ephemeral key coming from the extra-random-bits generation and not from raw RNG bytes. P-384 is defined by the RFC but has no vectors in the appendix or in the cited file, so four sets from BC Java 1.84 are its known answers, from a driver that first reproduced the RFC's P-256 and P-521 vectors with BC Java (32 sets); nine encapsulations from this code, three a curve, were decapsulated by BC Java to the same secret, once, outside the suite. Also: a test group in hpke that refuses its first candidates (the loop moves to the next counter, the mask is applied, the 256th candidate is tried and no 257th), the KEM and key frameworks on the three curves, ciphertexts off the curve or not in the uncompressed form refused, DeriveKeyPair's checks on its keying material, an RNG failure during encapsulation reported as the RNG's with the outputs zeroized, the DH buffer left zeroized on the point-at-infinity path, the names and strengths, and cli/tests/test_eccdh_dhkem.sh (11 cases, known-answer decapsulation on all three curves). Reviewed before the squash against a fresh copy of RFC 9180 (no divergence found; the CLI moving the private key out of its Result, a doc comment left on the wrong function, the unpinned zeroization, the vacuous pass without bc-test-data, the trait docs overstating what a group guarantees for the ephemeral key, the plumbing methods shown as API, the Nsk length rule and the recomputed public key; all fixed). Left as follow-ups: stack figures for the three KEMs in the crate docs' Memory Usage table, and one generic KEM command for the CLI in place of the copy of x25519's. Verified on the final tree: the workspace suite (3134 tests), the all-targets build with no warnings, rustdoc with warnings denied over the workspace, nightly rustfmt and the CLI shell suite (32 files). Mutants on apollo, on the final tree: hpke's derive_key_pair, labeled_expand and decap functions with the hpke, ed and eccdh tests 26 (15 caught, 11 unviable, 0 missed); eccdh's dhkem.rs 135 (16 caught, 119 unviable, 0 missed); the agreement's z functions 19 (8 caught, 11 unviable, 0 missed).

Assisted-by: Claude:claude-mythos-5-2
@dghgit

dghgit commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

After much ado and review, merged!

@dghgit dghgit closed this Nov 17, 2026
dghgit added a commit that referenced this pull request Nov 17, 2026
…KEM work and PR #186: stack figures, one copy of the CLI's DHKEM actions, and the CLI's secret output through wiped buffers. In bouncycastle-eccdh the Memory Usage table gains encaps and decaps columns for DHKEM over P-256, P-384 and P-521 (5,216 / 4,864, 7,008 / 6,720 and 10,464 / 10,016 bytes), and its keygen and agree columns are re-measured on the same build so the table has one source; in bouncycastle-ed the five rows of the X25519 table are taken again (keygen_from_seed 1816 B, from_bytes 1800, agree 1848, encaps 2840, decaps 2152), with a from_bytes bench added to the x25519 stack harness so that row has the same root as the others. Both tables are the deepest frame chain from the harness's measure wrapper down, the wrapper's own frame counted, with rustc 1.97.0-nightly (d3cd04068) on x86-64; docs and harness only. In the CLI, helpers::kem_helpers::encaps_cmd and decaps_cmd are the DHKEM encaps and decaps actions written once over the EncapsulatingKey and DecapsulatingKey roles, called by the x25519 command and by eccdh-p256, eccdh-p384 and eccdh-p521 in place of the copy the latter carried; the only change in behaviour is that x25519's parse errors on those two actions read "a valid X25519 public key" or "private key"; the ML-KEM commands keep their own path. The CLI's secret output: write_secret_bytes_or_hex, the wiped hex writer, now also writes the shared secret of mlkem encaps and decaps and of eccdh agree and the output of the hkdf and other kdf commands; a private key is written by write_private_key, which encodes it into a Secret through PrivateKey::encode_out and not by value (keygen and the from-seed forms of mlkem, mldsa, ecdsa, eccdh and sm2, and x25519 in place of its own copy); the key-wrap commands, where one side of either direction is a key in the clear, read stdin through a reader that wipes each buffer the input outgrows, call wrap_out and unwrap_out into one buffer and wipe input and output before returning or exiting, with input still raw bytes of any length; the rng command holds its chunk in a Secret, and the hex writer's buffer grows from 32 bytes to a chunk so a long hex stream costs one write a chunk as before. Raw output and the hex text are unchanged throughout. Not reached: keygen in those commands still returns the key pair by value, which a keygen_out in each crate would change and is left for now; decrypted plaintext, which streams. Tests: test_rng.sh gains a 5000-byte hex stream checked against the hex of the same raw stream. Verified on the final tree: the workspace suite (3134 tests), the CLI shell suite (32 files), the all-targets build with no warnings, rustdoc with warnings denied over the workspace and nightly rustfmt. No mutants run: no library code changes.

Assisted-by: Claude:claude-mythos-5-2
dghgit added a commit that referenced this pull request Nov 17, 2026
…efuse a private key that has not been filled. X25519PrivateKey::zeroized(), the key keygen_out and keygen_from_seed_out write into and leave on an error, already made no agreement and no decapsulator (ActingOnZeroizedKey). The group's DH hook, DHKEMParams::dh_into, did not check, so the generic DHKEM functions, which take the key without its role, still computed with it: the all-zero scalar clamps to a value anyone can compute. The hook now refuses the same way and leaves its output zeroized, so DHKEM::decap_out and auth_decap_out on such a key are DecapsulationFailed and auth_encap_rng_out with it as the sender's key is refused. The encoding cannot refuse, PrivateKey::encode having no error to give; zeroized()'s docs say so under Security Considerations: do not store or send the encoding of a key whose generation failed. From the residue re-review of PR #186 (its R2; its R1, the ephemeral key's copy, was already fixed by generating that key in place). Tests: the dhkem test for the unfilled key covers the generic decapsulation, the hook, and the generic authenticated forms on both sides, each with its outputs zeroized. Reviewed before the squash with feature/hpke (correct; the heading, the missing tests of the authenticated forms and the misleading message for a refused sender key, the last fixed in feature/hpke, were its findings). Verified: the ed, dhkem and hpke suites (97 tests), the all-targets build with no warnings, rustdoc with warnings denied and nightly rustfmt. Mutants on apollo over dh_into and is_filled: 7 (5 caught, 2 unviable, 0 missed).

Assisted-by: Claude:claude-mythos-5-2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants