Skip to content

Twilio ConversationRelay: report a caller hanging up, and a cut-in on text spoken after the turn - #1475

Merged
yileicn merged 5 commits into
SciSharp:masterfrom
yuyixg:features/relay-caller-hangup-hook
Nov 3, 2026
Merged

yileicn merged 5 commits into
SciSharp:masterfrom
yuyixg:features/relay-caller-hangup-hook

Conversation

@yuyixg

@yuyixg yuyixg commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Two additions to the Twilio ConversationRelay plugin, for agents that need to know how a call ended and whether the caller heard a line in full.

The problems

  1. A caller hanging up reaches no hook. ITwilioSessionHook reports the agent hanging up (OnAgentHangUp) and transferring (OnAgentTransferring). When the caller hangs up, the session ends with no hook at all, so an agent can't act on the end of most calls, for example to keep the transcript once the call is over.
  2. A cut-in on text spoken after the turn goes unnoticed. Text a tool gives is spoken once the turn is over (GetUnstreamedText). When the caller cuts into it, the history has already kept it whole, and only streamed text gets the cut-off mark. So a flow that reads a line the caller must hear in full, such as a disclaimer they agree to, has to make it uninterruptible.

What this does

  • ITwilioSessionHook.OnCallerHangUp (default no-op) fires from the action callback when the session ends with the call completed and no handoff from the agent (a transfer that falls through for want of CsrAgentNumber is not reported as one).
  • Twilio closes the WebSocket and calls the action at about the same time, so the session may still be finishing the turn the caller hung up on. The sessions running on an instance are known by conversation (ConversationRelaySessions), and the callback waits for that conversation's sessions to end, which is when the turn is saved, before telling the hooks; a session waits for a turn it gave up on after AiResponseTimeoutSeconds as well. The wait is capped by ConversationRelay.CallerHangUpWaitSeconds (10 by default, under Twilio's 15-second webhook timeout); past it the hooks are told anyway and a warning is logged. A session held by another instance is not seen and not waited for.
  • The session notes the line said after a turn as it goes out (under the send lock), and an interrupt that lands on it: utteranceUntilInterrupt is the start of the line but not all of it (all of it means the caller heard it), or, when Twilio gives nothing usable, nothing was said after it. The next turn gets ConversationRelayStates.LastReplyCutOff (twilio_relay_last_reply_cut_off) set to "true", and the state is removed otherwise. The flow can then say the line again.
  • The line is kept past the next turn's start, since Twilio may still be playing it while that turn runs; a cut-in then is told to the turn after. A turn answering words received before the line went out (queued while the previous turn worked) is told too, and a turn dropped because the caller went on talking passes the flag to the turn that takes its place.

Testing

Covered downstream, where the relay session runs against a fake WebSocket. After the review fixes (bdb98eb):

  • a caller cutting into text spoken after the turn sets the flag for the next turn;
  • text played to the end leaves it unset;
  • cutting into a "say it again" prompt said after a line heard in full leaves it unset, with utteranceUntilInterrupt set to the prompt's words or empty;
  • words queued while a turn worked set it for the turn that answers them, and a cut-in while that turn runs (the line still playing) sets it for the turn after;
  • a superseded turn passes it to the turn that takes its place.

The last three fail at abfb119 and pass with the fix; the existing relay tests downstream still pass. Noting the line under the send lock has no test: nothing else holds the lock at that point, so the window could not be produced.

The caller-hangup callback (d5e8f5e) was tested downstream by calling the controller directly:

  • with a session still running for the conversation, OnCallerHangUp fires only once it ends;
  • a session running past the wait gets the hook told anyway, after the wait;
  • with no session on this instance, or only another conversation's, the hook is told at once;
  • a session that ended, even by throwing, is not waited for, and after a reconnect both sessions of the conversation are;
  • the agent hanging up does not wait.

With the wait taken out, the first two fail.

The third round of review (77882f1), tested the same way; each test fails at d5e8f5e and passes with the fix:

  • a turn still running past AiResponseTimeoutSeconds when the caller hangs up keeps the session from ending until it is done;
  • an interrupt reporting the whole line as played leaves the flag unset, and one reporting part of it still sets it;
  • a transfer handoff with no CsrAgentNumber on a completed call does not fire OnCallerHangUp.

Still not covered: after the timeout the agent hangs up (OnAgentHangUp), and that callback does not wait for the abandoned turn. Neither path has been run on a real call, and there is no test project here that covers the Twilio plugin.

🤖 Generated with Claude Code

yuyixg and others added 2 commits November 2, 2026 17:37
The relay action callback told hooks when the agent hung up or transferred, but a caller hanging up
ended the call with no hook at all, so an agent had no way to act on the end of most calls (e.g. to
keep the transcript once the call is over). OnCallerHangUp fires when the session ends with the
call completed and no handoff from the agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter the last one

Text a tool gives is spoken once the turn is over, so when the caller cuts into it the history
has already kept it whole and nothing marks what they missed. A flow that reads a line the
caller must hear in full (a disclaimer they agree to) had to keep it uninterrupted for that
reason.

The session now notes text said after a turn, and an interrupt that lands on it; the next turn
gets twilio_relay_last_reply_cut_off set to "true" (removed otherwise), so the flow can say the
line again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@qodo-code-review

qodo-code-review Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Code Review by Qodo

Grey Divider

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again manually by commenting /agentic_review on this PR.

Grey Divider

Qodo Logo

@qodo-code-review

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Report ConversationRelay caller hang-ups and interrupted post-turn speech

✨ Enhancement 🕐 20-40 Minutes

Grey Divider

AI Description

• Notify session hooks when a caller hangs up without an agent handoff.
• Tell the next turn when the caller interrupted text spoken after the previous turn.
• Clear the interruption state on turns without a post-turn cut-in.
Diagram

graph TD
  R["Twilio Relay"] --> A["Action callback"] --> H["Caller hang-up hook"]
  R --> S["Relay session"] --> C["Turn callbacks"] --> P["Conversation processor"] --> T["Conversation state"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Record post-turn playback cut-offs in conversation history
  • ➕ Keeps the interruption alongside the transcript for later readers.
  • ➖ Requires revising text already stored when playback occurs after the turn.
  • ➖ Adds history-mutation complexity to a signal the next turn needs immediately.

Recommendation: Use the proposed next-turn state for flow decisions: it avoids retroactively changing stored text, while the action callback provides call-status context for the hang-up hook. Review the timing of the playback flag against WebSocket interrupts, since that determines whether the signal accurately means the caller missed text.

Files changed (6) +58 / -1

Enhancement (6) +58 / -1
TwilioConversationRelayController.csEmit a hook for completed calls without an agent hang-up +5/-0

Emit a hook for completed calls without an agent hang-up

• The relay action callback now emits OnCallerHangUp when the call is completed and the agent-hang-up branch was not taken. Transfer and reconnect handling remain ahead of this branch.

src/Plugins/BotSharp.Plugin.Twilio/Controllers/TwilioConversationRelayController.cs

ITwilioSessionHook.csExpose an optional caller-hang-up hook +9/-0

Expose an optional caller-hang-up hook

• Adds OnCallerHangUp with a default no-op implementation, allowing existing hook implementations to remain compatible.

src/Plugins/BotSharp.Plugin.Twilio/Interfaces/ITwilioSessionHook.cs

AgentTurnCallbacks.csCarry the previous post-turn cut-off into an agent turn +6/-0

Carry the previous post-turn cut-off into an agent turn

• Adds a boolean callback property that conveys whether the caller cut into text spoken after the preceding turn.

src/Plugins/BotSharp.Plugin.Twilio/Models/AgentTurnCallbacks.cs

ConversationRelayStates.csDefine the last-reply cut-off state key +7/-0

Define the last-reply cut-off state key

• Adds twilio_relay_last_reply_cut_off for flows that need to detect missed post-turn speech.

src/Plugins/BotSharp.Plugin.Twilio/Models/ConversationRelayStates.cs

ConversationRelaySession.csTrack interruptions of text spoken after a turn +23/-1

Track interruptions of text spoken after a turn

• Records when unstreamed reply text is about to be spoken and whether a subsequent interrupt occurs. Passes that result to the next turn's callbacks, then resets the tracking flags.

src/Plugins/BotSharp.Plugin.Twilio/Services/ConversationRelaySession.cs

TwilioConversationProcessor.csExpose the cut-off marker to the next conversation turn +8/-0

Expose the cut-off marker to the next conversation turn

• Sets the application conversation state to "true" when the callback reports a post-turn cut-off; otherwise removes the state before the turn proceeds.

src/Plugins/BotSharp.Plugin.Twilio/Services/TwilioConversationProcessor.cs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Post-turn playback and cutoff tracking can lose genuine interruptions or falsely mark completed or unsent replies as cut off, causing incorrect next-turn state and repeated disclaimers.

4 open issues
What changed in this PR

Adds caller-hangup notifications and post-turn interruption reporting to the Twilio ConversationRelay plugin, allowing agents to react to call completion and repeat interrupted tool-provided text.

Changes:

  • Adds a default no-op OnCallerHangUp hook and invokes it from the session action callback.
  • Tracks interruptions of text spoken after a turn.
  • Exposes the cutoff flag through turn callbacks and conversation state.
File Description
src/​Plugins/​BotSharp.Plugin.Twilio/​Services/​TwilioConversationProcessor.cs Sets or removes the cutoff state at turn start.
src/​Plugins/​BotSharp.Plugin.Twilio/​Services/​ConversationRelaySession.cs Tracks post-turn speech and interruptions.
src/​Plugins/​BotSharp.Plugin.Twilio/​Models/​ConversationRelayStates.cs Defines the cutoff state key.
src/​Plugins/​BotSharp.Plugin.Twilio/​Models/​AgentTurnCallbacks.cs Adds the previous-reply cutoff flag.
src/​Plugins/​BotSharp.Plugin.Twilio/​Interfaces/​ITwilioSessionHook.cs Adds the caller-hangup hook.
src/​Plugins/​BotSharp.Plugin.Twilio/​Controllers/​TwilioConversationRelayController.cs Emits caller-hangup notifications when the call has ended.

Review effort: Balanced

… landed there

Review on SciSharp#1475:
- An interrupt counts for the line only when Twilio was playing it: utteranceUntilInterrupt is the start of the line,
  or, when Twilio says nothing usable, nothing was said after the line. A "say it again" prompt or a filler said after
  a line the caller heard in full is no longer taken for it.
- The line is noted once the send lock is held, as it goes out, so an interrupt of something said before it is not
  put on it.
- The line is kept past the next turn's start, since Twilio may still be playing it while that turn runs: a cut-in
  then is told to the turn after. A turn that answers words said before the line went out (queued while the last
  turn worked) is told the caller missed it.
- A turn dropped because the caller went on talking passes its cut-off on to the turn that takes its place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The hang-up callback can fire before an in-flight reply is persisted, causing transcript archival to omit the final reply unless hook delivery is coordinated with turn completion.

1 open issue
4 resolved since last review

Review effort: Balanced

…up on is saved

Review on SciSharp#1475: Twilio closes the WebSocket and calls the session-ended action at about the same time, so
OnCallerHangUp could run while the session was still finishing the caller's last turn, before that turn was saved.
A handler keeping the transcript then missed the last exchange.

The sessions running on an instance are now known by conversation (ConversationRelaySessions). When the caller hung
up, the callback waits for that conversation's sessions on this instance to end, up to
ConversationRelay.CallerHangUpWaitSeconds (10 by default, under Twilio's 15-second webhook timeout), then tells the
hooks. Past the wait it tells them anyway and logs a warning. A session held by another instance is not seen and
not waited for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Caller-hangup callbacks can misclassify agent transfers and run before in-flight replies are saved, while fully heard responses can be incorrectly marked as cut off.

2 open issues
2 previously missed issues

In code that hasn't changed since last review

Medium severity Exclude transfer handoffs from caller-hangup hook

src/​Plugins/​BotSharp.Plugin.Twilio/​Controllers/​TwilioConversationRelayController.cs:69

This branch also runs for an agent's transfer handoff when CsrAgentNumber is unset: the earlier transfer guard falls through, and a completed call then emits OnCallerHangUp. That misclassifies an agent-initiated handoff as a caller hangup, contrary to the new hook's contract. Require the handoff action to be absent before emitting the caller hook.

Medium severity Treat an exact utterance match as fully heard

src/​Plugins/​BotSharp.Plugin.Twilio/​Services/​ConversationRelaySession.cs:984

When utteranceUntilInterrupt contains the entire post-turn line, this prefix check still reports a cutoff, even though no text remains unheard. This can set LastReplyCutOff and make the flow repeat a confirmation or disclaimer already spoken in full. Treat an exact match as fully heard and only report a cutoff for a proper prefix, as the streamed-history path does when its unheard suffix is empty.

Review effort: Balanced

…d lines out of the hang-up and cut-in reports

Review on SciSharp#1475:
- A turn given up on after AiResponseTimeoutSeconds keeps running; the session now waits for it as well before it
  ends, so a caller hanging up just before the timeout no longer gets OnCallerHangUp ahead of that turn's save.
- A transfer handoff that falls through (no CsrAgentNumber) is no longer reported as the caller hanging up:
  OnCallerHangUp fires only when the agent sent no handoff.
- An interrupt whose utteranceUntilInterrupt is the whole line said after the turn means the caller heard all of
  it, so it no longer counts as a cut-in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yuyixg

yuyixg commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

On the two previously missed issues in the latest review, both fixed in 77882f1:

  • Exclude transfer handoffs from caller-hangup hook: OnCallerHangUp now fires only when the agent sent no handoff, so a transfer that falls through for want of CsrAgentNumber is not reported as the caller hanging up.
  • Treat an exact utterance match as fully heard: an interrupt whose utteranceUntilInterrupt is the whole post-turn line no longer counts as a cut-in; only a proper prefix does, as on the streamed path.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Real-call validation is needed because hangup classification and cutoff attribution depend on Twilio payloads and event ordering, while the described tests cover only downstream fake sockets and controllers.

1 open issue
1 resolved since last review

Review effort: Balanced

@yileicn

yileicn commented Nov 3, 2026

Copy link
Copy Markdown
Member

reviewed

@yileicn
yileicn merged commit c6c169f into SciSharp:master Nov 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants