Skip to content

fix(live): handle goAway and voice activity notifications - #1878

Open
YuqiGuo105 wants to merge 1 commit into
google:mainfrom
YuqiGuo105:fix/issue-1872-live-control-messages
Open

YuqiGuo105 wants to merge 1 commit into
google:mainfrom
YuqiGuo105:fix/issue-1872-live-control-messages

Conversation

@YuqiGuo105

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

Closes #1872.

Problem

GeminiLlmConnection treats valid Live goAway, voiceActivity, and voiceActivityDetectionSignal messages as unknown-message errors. In particular, the goAway.timeLeft warning never reaches callers. Simply recognizing these messages in the connection is insufficient: the flow also discards contentless responses unless their metadata is explicitly preserved.

Solution

  • Add optional, typed goAway and voiceActivity fields to LlmResponse and Event, following ADK Python's response fields. Preserve them through the flow, JSON round-trips, builders, and event equality.
  • Forward goAway and voiceActivity notifications without reporting an error. Recognize the lower-level voiceActivityDetectionSignal without emitting an event, since it has no ADK event representation.
  • Keep control-only events from being classified as final answers. Notifications do not complete a turn, flush buffered text, or proactively close the connection.
  • Preserve existing handling of content, usage metadata, and genuinely unknown messages, including messages carrying multiple fields.

This change exposes the warning; it does not implement automatic reconnection/session resumption or address SDK-level wire parsing issues.

Testing Plan

Automated tests

  • Added 15 regression tests across GeminiLlmConnectionTest, LlmResponseTest, EventTest, BaseLlmFlowTest, and RunnerTest.
  • Confirmed that three reproducer tests fail against the original converter.
  • Verified payload preservation, optional/null fields, JSON round-trips, non-final notification events, mixed content/notification messages, uninterrupted text aggregation, and delivery through Runner.runLive.
  • Verified with a fake transport that goAway does not close the connection or prevent later responses.

Validation on the latest fetched main (dde4b23d), using Maven 3.9.16 and Temurin 25.0.3 (Java release 17):

mvn -o -B -pl core -am clean test

Both main core test executions (default-test and basic) report 4,177 tests, 0 failures, 0 errors, and 35 skipped. The additional configured Vertex AI/Apigee/tracking-header executions also pass. These are repeated/configured executions, not distinct-test counts to be added together.

The illegal-Optional-parameter check passes for all changed production files. Running that optional profile across all core sources previously reported 12 violations in seven unchanged files; those are outside this patch.

Manual End-to-End (E2E) Tests

Not performed against a real Gemini Live endpoint; automated tests use simulated messages and a fake transport/model. To verify manually:

  1. Start a BIDI Runner.runLive session and inspect emitted events.
  2. When the server sends a goAway warning, verify event.goAway() preserves timeLeft, with no unknown-message error and no immediate client-initiated close.
  3. In a session configured to emit voice activity notifications, speak and stop speaking; verify event.voiceActivity() carries the activity type without being classified as a final answer.

Checklist

  • I have read the contribution guide.
  • This pull request contains a single commit.
  • I have reviewed the changes and formatted only changed lines with google-java-format.
  • Regression tests cover the fix, and the core test suite passes locally.
  • I have manually tested against a real Gemini Live endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Live: goAway and voiceActivity messages are reported as "Unknown server message" errors

1 participant