Skip to content

fix(examples): handle interruptions and disconnects in push-to-talk - #4192

Merged
markstuart-oai merged 20 commits into
mainfrom
markstuart/realtime-push-to-talk-lifecycle-20261027
Oct 28, 2026
Merged

markstuart-oai merged 20 commits into
mainfrom
markstuart/realtime-push-to-talk-lifecycle-20261027

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Pressing Enter tried to use a button the app never mounted, and transcripts containing Rich tags could crash the session. This change displays transcripts and API errors as literal text, reports failed or incomplete responses, and closes the audio device and SDK client on exit. Q exits with code 0; an unexpected normal or abnormal disconnect exits with code 1.

When K interrupts playback, the sample cancels the active response and truncates unheard audio, including queued items after response.done. Capture starts before awaiting cancellation and truncation; microphone sends wait until those writes finish. If audio or its transcript arrives after interruption, the sample truncates each previously unknown item part to zero once. It retains the original cutoff for partly played audio. Manual mode skips commit and response.create if no audio was sent before release. Manual finalization also reserves the server input buffer before the next recording can append, so successive recordings remain separate while capture and interruption can still proceed.

A response.created event that arrives after K, before any new audio is offered, belongs to the earlier turn. The sample reserves interruption work before it waits for the send lock. Microphone appends and manual commits recheck that work after obtaining the lock and yield outside it when necessary. They cannot overtake a cancellation or truncation that was queued while they waited. It suppresses late audio and transcripts and truncates unheard parts, even after a quick release. Once new audio is offered, these events cannot identify whether a delayed response belongs to an earlier turn or to a current pause detected by server VAD. The sample preserves possible current VAD replies; it does not guarantee interruption of every old response in that interval.

All 48 committed sample regressions pass locally on Python 3.10 and 3.13 against the public SDK, and on Python 3.10 against the internal SDK. They use fake audio devices and held writes. Contended delayed cancellations reproduced wrong send ordering in both VAD and manual modes. Separate tests reproduce a runnable microphone or manual completion overtaking an interruption worker that has not started. The new regressions also verify progress when that worker is cancelled before starting. Four late-audio cases reproduced missing truncations. The tests also cover frame counts, queued playback, recording while cancellation is blocked, manual and VAD modes, transcripts and connection closure. Scoped Ruff and pinned Pyright checks pass. Eight earlier Textual scenarios exercised SDK connections to a local WebSocket peer and virtual audio devices.

Run the sample regressions explicitly after installing the sample dependencies:

python -m pytest -o addopts= --noconftest -q examples/realtime/test_push_to_talk_app.py

The default SDK test run does not collect these example tests.

The first microphone append restores recording state and immediately re-raises any failure, including asyncio.CancelledError. Narrowing that cleanup to Exception would skip restoring _awaiting_new_input when asyncio cancels an append. The committed canceled-append regression tests that no event is sent, the microphone closes, and cancellation propagates.

Playback accounting uses PortAudio DAC timestamps to exclude audio still buffered on the device. Interrupting silences callbacks and aborts playback before processing the accounting queue. It measures the last known playing time immediately before abort; PortAudio's clock keeps advancing after playback stops and does not provide an exact hardware stop timestamp. No physical-device timing precision was tested; prolonged backpressure can fill the device buffer. Unsupported audio content parts are reported without sending an invalid truncate event.

@github-actions

Copy link
Copy Markdown
Contributor

Castiron custom code

Evaluated main: 0f434e49de356e4a29867084c708157f75eca8e0.

Trusted main policy: 0f434e49de356e4a29867084c708157f75eca8e0.

✅ No new custom-code files detected.

0 mixed files remain; 0 existing customizations changed.

Compared 0f434e49de35 → 2a4b61c46a9d. Generated baselines verified.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 40527876339 --repo openai/openai-python \
  --name castiron-custom-code-40527876339-1 --dir /tmp/castiron-custom-code-40527876339-1
git apply --stat /tmp/castiron-custom-code-40527876339-1/custom-code.patch
cat /tmp/castiron-custom-code-40527876339-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 0f434e49de356e4a29867084c708157f75eca8e0 2a4b61c46a9dd0cb02607b487d5587d0db143e31
python3 scripts/castiron/custom_code_report.py report \
  --base 0f434e49de356e4a29867084c708157f75eca8e0 \
  --head 2a4b61c46a9dd0cb02607b487d5587d0db143e31 --target-branch main --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-2a4b61c46a9d
cat /tmp/castiron-custom-code-2a4b61c46a9d/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@markstuart-oai
markstuart-oai marked this pull request as ready for review October 27, 2026 17:51
@markstuart-oai
markstuart-oai requested a review from a team as a code owner October 27, 2026 17:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T17:47:11.995283Z 5801a1a New commits
🔒 Security Review ✅ Completed 2026-10-28T17:45:30.506851Z 5801a1a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a4b61c46a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Fixed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 022c50de8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml Outdated
Comment thread examples/realtime/push_to_talk_app.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5663b7c6ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Fixed
Comment thread examples/realtime/test_push_to_talk_app.py
@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 0390b4fb74

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 0390b4fb74

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0390b4fb74

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread examples/realtime/audio_util.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Outdated
Comment thread examples/realtime/push_to_talk_app.py Fixed
Comment thread examples/realtime/push_to_talk_app.py

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current sample and regression coverage with an independent second pass. Playback accounting retains scheduled DAC spans until heard, and abort happens outside the callback lock. Interruption truncates newly discovered suppressed audio/transcript items once without replacing a partial playback cutoff. Per-recording state and ordered workers keep microphone capture responsive while avoiding empty commits; cancellation cleanup re-raises. No new actionable findings. The three sample files match the companion PR; the internal-only CI setup uses pinned hashed dependencies and checks environment consistency.

Static review only; no candidate code or local tests run. Hosted CI for this new head is still running.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reserve delayed interruptions before waiting for the microphone lock. The shared lock fixes the uncontended case, but a microphone waiter can already be ahead of this cancellation. For example: K snapshots locally active response A and its cancellation write blocks holding the lock; the microphone queues; incoming response.done(A) and then response.created(B) for previously buffered speech arrive. B is classified as old while _awaiting_new_input is still true, but _interrupt_response(B, []) queues behind the microphone. When A releases the lock, the microphone clears _awaiting_new_input and appends fresh audio before B is cancelled. No simultaneous active server responses are needed.

Please register pending interruption work before waiting for the lock, and keep new appends/manual finalization behind that work. Add a regression that holds an initial interruption, queues the microphone, then delivers the delayed response before releasing the initial interruption. The current new test starts the microphone only after the delayed cancellation has acquired the lock, so it does not cover this ordering.

This is the remaining contended case of the existing cancellation-ordering feedback. Static review with an independent second pass; no candidate code or local tests run. Hosted CI still has unfinished jobs.

Comment thread examples/realtime/push_to_talk_app.py Fixed

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the interruption-ordering fix with an independent second pass. Inline receiver work reserves a pending interruption before awaiting the send lock, and registered interruption workers cover the pre-start scheduling window. Microphone appends and manual finalization yield and recheck until that work settles; finally cleanup and cancelled-before-start coverage prevent stranded reservations. The new contended and runnable-sender regressions cover the reported race. The source and test changes match between the mirrors. No remaining actionable findings.

Static review only; no candidate code or local tests run. Hosted checks passed with skipped jobs.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd098453f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread examples/realtime/audio_util.py Outdated
Comment thread examples/realtime/push_to_talk_app.py

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The manual recording boundary issue is fixed: registered finalization workers reserve the shared input buffer before scheduling, and mic appends wait through commit/create without blocking capture or interruption. Verified matching internal/public source and regression coverage with two independent reviewers. No additional findings. Hosted Python tests and OkTest remain pending.

Static review only; no candidate code or tests executed.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved the playback-stop increment after independent static review. Callbacks are silenced before abort, abort runs outside the callback lock and before accounting traversal, and the pre-abort cutoff avoids crediting silence during slow abort completion. The cutoff is intentionally conservative because the interface exposes no exact hardware stop timestamp. Active/completed responses and callback access are covered by the added regression cases; public/internal changed blobs match. No actionable findings.

Current hosted CI remains pending. No candidate code or tests executed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5801a1abe7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread examples/realtime/test_push_to_talk_app.py
@markstuart-oai
markstuart-oai added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 35fc9f3 Oct 28, 2026
26 checks passed
@markstuart-oai
markstuart-oai deleted the markstuart/realtime-push-to-talk-lifecycle-20261027 branch October 28, 2026 18:45
@openai-sdks openai-sdks Bot mentioned this pull request Oct 28, 2026
gh-actions-shared Bot pushed a commit to xf-qubit/openai-python that referenced this pull request Oct 28, 2026
Automated Release PR
---


##
[3.41.0](openai/openai-python@v3.40.0...v3.41.0)
(2026-10-28)


### Features

* **api:** accept typed references in Realtime response inputs
([openai#4189](openai#4189))
([8ecb487](openai@8ecb487))
* **api:** add response tool usage models
([openai#4201](openai#4201))
([55fc500](openai@55fc500))
* **api:** filter agent sessions by metadata
([openai#4200](openai#4200))
([77fd116](openai@77fd116))
* **websocket:** recover queued binary frames after reconnect
([openai#4191](openai#4191))
([f079868](openai@f079868))


### Bug Fixes

* **examples:** detect incomplete WebSocket responses
([openai#4187](openai#4187))
([8559299](openai@8559299))
* **examples:** handle interruptions and disconnects in push-to-talk
([openai#4192](openai#4192))
([35fc9f3](openai@35fc9f3))
* keep consumer fixtures clean under strict Pyright
([openai#4204](openai#4204))
([5012565](openai@5012565))


### Chores

* **api:** document upcoming model shutdowns
([openai#4199](openai#4199))
([5b43cce](openai@5b43cce))
* **api:** update image-model documentation
([openai#4198](openai#4198))
([fa67521](openai@fa67521))
* enforce generated-file ownership
([openai#4193](openai#4193))
([1b37b9a](openai@1b37b9a))
* migrate development type checking to ty
([openai#4202](openai#4202))
([45b9531](openai@45b9531))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: openai-sdks[bot] <284451331+openai-sdks[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants