Skip to content

Add DataPacket sender oneof for packets sent on behalf of another participant - #1965

Open
1egoman wants to merge 1 commit into
mainfrom
data-packet-sender-identity
Open

1egoman wants to merge 1 commit into
mainfrom
data-packet-sender-identity

Conversation

@1egoman

@1egoman 1egoman commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When an agent sends a text stream on behalf of another participant (e.g. lk.transcription), the SFU leaves the header's participant_identity as the attributed participant and marks it generated, but overwrites the identity on the stream's chunks and trailer with the agent's own. A receiver can't know from the header which participant the rest of the stream should come from, so it can't check that a stream's packets all come from one sender without breaking these streams (see livekit/client-sdk-js#2215, reverted in livekit/client-sdk-js#2242).

Change

Adds a sender oneof to DataPacket:

  • string sender_identity = 19: the participant that actually sent the packet.
  • bool sender_hidden = 20: the real sender is a hidden participant, so its identity is withheld (only ever true).

Only the SFU sets it, and only when participant_identity names someone other than the real sender (today: an agent's text stream header or chat message sent on another participant's behalf). The SFU clears any value a client sends. When it's unset, participant_identity is the real sender, so receivers expect a stream's chunks and trailer from sender_identity if set, from a hidden participant if sender_hidden, and from participant_identity otherwise.

The SFU side will bump the server protocol to 20 at the same time as this is merged, which lets clients rely on the field being set when it's needed. See my comment below though, I'm not 100% sure this is a good assumption.

participant_identity keeps its current meaning, so existing clients are unaffected.

…ticipant

The SFU leaves participant_identity as the attributed participant when an
agent sends a text stream header or chat message on someone else's behalf,
but stamps the agent's identity on the stream's chunks and trailer. Receivers
can't tell which participant to expect the rest of the stream from.

The new sender oneof carries the real sender in those cases: sender_identity,
or sender_hidden when that sender is a hidden participant. Only the SFU sets
it (from server protocol 20).
@changeset-bot

changeset-bot Bot commented Nov 10, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 61b8900

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
github.com/livekit/protocol Patch
@livekit/protocol Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Comment on lines +474 to +486
// The participant that actually sent this packet, when `participant_identity` names a different
// participant (an agent sending a text stream or chat message on behalf of another participant).
// Set only by the SFU, which clears any value a client sends; unset when `participant_identity` is
// the real sender. Available from server protocol 20.
oneof sender {
// identity of the participant that actually sent this packet
string sender_identity = 19;
// the participant that actually sent this packet is hidden, so its identity is withheld; only
// ever true
bool sender_hidden = 20;
}

// NEXT_ID: 21

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for reviewers: I am assuming currently I am ok to keep this non optional because it would be paired with a SFU protocol version bump.

But I'm not 100% sure about that, is the goal that the protobufs in here should remain backwards compatible with older SFU versions as well as the latest?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine. The dependencies are updated. If we have to go back to pinning a different hash, will have to change the corresponding code.

@1egoman
1egoman marked this pull request as ready for review November 10, 2026 23:57
@1egoman
1egoman requested review from boks1971 and lukasIO November 10, 2026 23:58

@boks1971 boks1971 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to get 👀 from @lukasIO as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants