You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Twilio ConversationRelay: report a caller hanging up, and a cut-in on text spoken after the turn - #1475
Two additions to the Twilio ConversationRelay plugin, for agents that need to know how a call ended and whether the caller heard a line in full.
The problems
A caller hanging up reaches no hook.ITwilioSessionHook reports the agent hanging up (OnAgentHangUp) and transferring (OnAgentTransferring). When the caller hangs up, the session ends with no hook at all, so an agent can't act on the end of most calls, for example to keep the transcript once the call is over.
A cut-in on text spoken after the turn goes unnoticed. Text a tool gives is spoken once the turn is over (GetUnstreamedText). When the caller cuts into it, the history has already kept it whole, and only streamed text gets the cut-off mark. So a flow that reads a line the caller must hear in full, such as a disclaimer they agree to, has to make it uninterruptible.
What this does
ITwilioSessionHook.OnCallerHangUp (default no-op) fires from the action callback when the session ends with the call completed and no handoff from the agent (a transfer that falls through for want of CsrAgentNumber is not reported as one).
Twilio closes the WebSocket and calls the action at about the same time, so the session may still be finishing the turn the caller hung up on. The sessions running on an instance are known by conversation (ConversationRelaySessions), and the callback waits for that conversation's sessions to end, which is when the turn is saved, before telling the hooks; a session waits for a turn it gave up on after AiResponseTimeoutSeconds as well. The wait is capped by ConversationRelay.CallerHangUpWaitSeconds (10 by default, under Twilio's 15-second webhook timeout); past it the hooks are told anyway and a warning is logged. A session held by another instance is not seen and not waited for.
The session notes the line said after a turn as it goes out (under the send lock), and an interrupt that lands on it: utteranceUntilInterrupt is the start of the line but not all of it (all of it means the caller heard it), or, when Twilio gives nothing usable, nothing was said after it. The next turn gets ConversationRelayStates.LastReplyCutOff (twilio_relay_last_reply_cut_off) set to "true", and the state is removed otherwise. The flow can then say the line again.
The line is kept past the next turn's start, since Twilio may still be playing it while that turn runs; a cut-in then is told to the turn after. A turn answering words received before the line went out (queued while the previous turn worked) is told too, and a turn dropped because the caller went on talking passes the flag to the turn that takes its place.
Testing
Covered downstream, where the relay session runs against a fake WebSocket. After the review fixes (bdb98eb):
a caller cutting into text spoken after the turn sets the flag for the next turn;
text played to the end leaves it unset;
cutting into a "say it again" prompt said after a line heard in full leaves it unset, with utteranceUntilInterrupt set to the prompt's words or empty;
words queued while a turn worked set it for the turn that answers them, and a cut-in while that turn runs (the line still playing) sets it for the turn after;
a superseded turn passes it to the turn that takes its place.
The last three fail at abfb119 and pass with the fix; the existing relay tests downstream still pass. Noting the line under the send lock has no test: nothing else holds the lock at that point, so the window could not be produced.
The caller-hangup callback (d5e8f5e) was tested downstream by calling the controller directly:
with a session still running for the conversation, OnCallerHangUp fires only once it ends;
a session running past the wait gets the hook told anyway, after the wait;
with no session on this instance, or only another conversation's, the hook is told at once;
a session that ended, even by throwing, is not waited for, and after a reconnect both sessions of the conversation are;
the agent hanging up does not wait.
With the wait taken out, the first two fail.
The third round of review (77882f1), tested the same way; each test fails at d5e8f5e and passes with the fix:
a turn still running past AiResponseTimeoutSeconds when the caller hangs up keeps the session from ending until it is done;
an interrupt reporting the whole line as played leaves the flag unset, and one reporting part of it still sets it;
a transfer handoff with no CsrAgentNumber on a completed call does not fire OnCallerHangUp.
Still not covered: after the timeout the agent hangs up (OnAgentHangUp), and that callback does not wait for the abandoned turn. Neither path has been run on a real call, and there is no test project here that covers the Twilio plugin.
The relay action callback told hooks when the agent hung up or transferred, but a caller hanging up
ended the call with no hook at all, so an agent had no way to act on the end of most calls (e.g. to
keep the transcript once the call is over). OnCallerHangUp fires when the session ends with the
call completed and no handoff from the agent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter the last one
Text a tool gives is spoken once the turn is over, so when the caller cuts into it the history
has already kept it whole and nothing marks what they missed. A flow that reads a line the
caller must hear in full (a disclaimer they agree to) had to keep it uninterrupted for that
reason.
The session now notes text said after a turn, and an interrupt that lands on it; the next turn
gets twilio_relay_last_reply_cut_off set to "true" (removed otherwise), so the flow can say the
line again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report ConversationRelay caller hang-ups and interrupted post-turn speech
✨ Enhancement🕐 20-40 Minutes
AI Description
• Notify session hooks when a caller hangs up without an agent handoff.
• Tell the next turn when the caller interrupted text spoken after the previous turn.
• Clear the interruption state on turns without a post-turn cut-in.
The following are alternative approaches to this PR:
1. Record post-turn playback cut-offs in conversation history
➕ Keeps the interruption alongside the transcript for later readers.
➖ Requires revising text already stored when playback occurs after the turn.
➖ Adds history-mutation complexity to a signal the next turn needs immediately.
Recommendation: Use the proposed next-turn state for flow decisions: it avoids retroactively changing stored text, while the action callback provides call-status context for the hang-up hook. Review the timing of the playback flag against WebSocket interrupts, since that determines whether the signal accurately means the caller missed text.
Files changed (6) +58 / -1
Enhancement (6) +58 / -1
TwilioConversationRelayController.csEmit a hook for completed calls without an agent hang-up+5/-0
Emit a hook for completed calls without an agent hang-up
• The relay action callback now emits OnCallerHangUp when the call is completed and the agent-hang-up branch was not taken. Transfer and reconnect handling remain ahead of this branch.
ConversationRelaySession.csTrack interruptions of text spoken after a turn+23/-1
Track interruptions of text spoken after a turn
• Records when unstreamed reply text is about to be spoken and whether a subsequent interrupt occurs. Passes that result to the next turn's callbacks, then resets the tracking flags.
TwilioConversationProcessor.csExpose the cut-off marker to the next conversation turn+8/-0
Expose the cut-off marker to the next conversation turn
• Sets the application conversation state to "true" when the callback reports a post-turn cut-off; otherwise removes the state before the turn proceeds.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Post-turn playback and cutoff tracking can lose genuine interruptions or falsely mark completed or unsent replies as cut off, causing incorrect next-turn state and repeated disclaimers.
Adds caller-hangup notifications and post-turn interruption reporting to the Twilio ConversationRelay plugin, allowing agents to react to call completion and repeat interrupted tool-provided text.
Changes:
Adds a default no-op OnCallerHangUp hook and invokes it from the session action callback.
Tracks interruptions of text spoken after a turn.
Exposes the cutoff flag through turn callbacks and conversation state.
… landed there
Review on SciSharp#1475:
- An interrupt counts for the line only when Twilio was playing it: utteranceUntilInterrupt is the start of the line,
or, when Twilio says nothing usable, nothing was said after the line. A "say it again" prompt or a filler said after
a line the caller heard in full is no longer taken for it.
- The line is noted once the send lock is held, as it goes out, so an interrupt of something said before it is not
put on it.
- The line is kept past the next turn's start, since Twilio may still be playing it while that turn runs: a cut-in
then is told to the turn after. A turn that answers words said before the line went out (queued while the last
turn worked) is told the caller missed it.
- A turn dropped because the caller went on talking passes its cut-off on to the turn that takes its place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The hang-up callback can fire before an in-flight reply is persisted, causing transcript archival to omit the final reply unless hook delivery is coordinated with turn completion.
…up on is saved
Review on SciSharp#1475: Twilio closes the WebSocket and calls the session-ended action at about the same time, so
OnCallerHangUp could run while the session was still finishing the caller's last turn, before that turn was saved.
A handler keeping the transcript then missed the last exchange.
The sessions running on an instance are now known by conversation (ConversationRelaySessions). When the caller hung
up, the callback waits for that conversation's sessions on this instance to end, up to
ConversationRelay.CallerHangUpWaitSeconds (10 by default, under Twilio's 15-second webhook timeout), then tells the
hooks. Past the wait it tells them anyway and logs a warning. A session held by another instance is not seen and
not waited for.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Caller-hangup callbacks can misclassify agent transfers and run before in-flight replies are saved, while fully heard responses can be incorrectly marked as cut off.
This branch also runs for an agent's transfer handoff when CsrAgentNumber is unset: the earlier transfer guard falls through, and a completed call then emits OnCallerHangUp. That misclassifies an agent-initiated handoff as a caller hangup, contrary to the new hook's contract. Require the handoff action to be absent before emitting the caller hook.
When utteranceUntilInterrupt contains the entire post-turn line, this prefix check still reports a cutoff, even though no text remains unheard. This can set LastReplyCutOff and make the flow repeat a confirmation or disclaimer already spoken in full. Treat an exact match as fully heard and only report a cutoff for a proper prefix, as the streamed-history path does when its unheard suffix is empty.
…d lines out of the hang-up and cut-in reports
Review on SciSharp#1475:
- A turn given up on after AiResponseTimeoutSeconds keeps running; the session now waits for it as well before it
ends, so a caller hanging up just before the timeout no longer gets OnCallerHangUp ahead of that turn's save.
- A transfer handoff that falls through (no CsrAgentNumber) is no longer reported as the caller hanging up:
OnCallerHangUp fires only when the agent sent no handoff.
- An interrupt whose utteranceUntilInterrupt is the whole line said after the turn means the caller heard all of
it, so it no longer counts as a cut-in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the two previously missed issues in the latest review, both fixed in 77882f1:
Exclude transfer handoffs from caller-hangup hook: OnCallerHangUp now fires only when the agent sent no handoff, so a transfer that falls through for want of CsrAgentNumber is not reported as the caller hanging up.
Treat an exact utterance match as fully heard: an interrupt whose utteranceUntilInterrupt is the whole post-turn line no longer counts as a cut-in; only a proper prefix does, as on the streamed path.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Real-call validation is needed because hangup classification and cutoff attribution depend on Twilio payloads and event ordering, while the described tests cover only downstream fake sockets and controllers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two additions to the Twilio ConversationRelay plugin, for agents that need to know how a call ended and whether the caller heard a line in full.
The problems
ITwilioSessionHookreports the agent hanging up (OnAgentHangUp) and transferring (OnAgentTransferring). When the caller hangs up, the session ends with no hook at all, so an agent can't act on the end of most calls, for example to keep the transcript once the call is over.GetUnstreamedText). When the caller cuts into it, the history has already kept it whole, and only streamed text gets the cut-off mark. So a flow that reads a line the caller must hear in full, such as a disclaimer they agree to, has to make it uninterruptible.What this does
ITwilioSessionHook.OnCallerHangUp(default no-op) fires from the action callback when the session ends with the call completed and no handoff from the agent (a transfer that falls through for want ofCsrAgentNumberis not reported as one).ConversationRelaySessions), and the callback waits for that conversation's sessions to end, which is when the turn is saved, before telling the hooks; a session waits for a turn it gave up on afterAiResponseTimeoutSecondsas well. The wait is capped byConversationRelay.CallerHangUpWaitSeconds(10 by default, under Twilio's 15-second webhook timeout); past it the hooks are told anyway and a warning is logged. A session held by another instance is not seen and not waited for.utteranceUntilInterruptis the start of the line but not all of it (all of it means the caller heard it), or, when Twilio gives nothing usable, nothing was said after it. The next turn getsConversationRelayStates.LastReplyCutOff(twilio_relay_last_reply_cut_off) set to"true", and the state is removed otherwise. The flow can then say the line again.Testing
Covered downstream, where the relay session runs against a fake WebSocket. After the review fixes (bdb98eb):
utteranceUntilInterruptset to the prompt's words or empty;The last three fail at abfb119 and pass with the fix; the existing relay tests downstream still pass. Noting the line under the send lock has no test: nothing else holds the lock at that point, so the window could not be produced.
The caller-hangup callback (d5e8f5e) was tested downstream by calling the controller directly:
OnCallerHangUpfires only once it ends;With the wait taken out, the first two fail.
The third round of review (77882f1), tested the same way; each test fails at d5e8f5e and passes with the fix:
AiResponseTimeoutSecondswhen the caller hangs up keeps the session from ending until it is done;CsrAgentNumberon a completed call does not fireOnCallerHangUp.Still not covered: after the timeout the agent hangs up (
OnAgentHangUp), and that callback does not wait for the abandoned turn. Neither path has been run on a real call, and there is no test project here that covers the Twilio plugin.🤖 Generated with Claude Code