Repository navigation
feat: report stream events the parsers skip (DUX-14977) - #45
Conversation
The streaming parsers rescued events they could not decode and dropped them with only a debug log, so a lost event (for example part of a tool call the model really sent) never surfaced. Every skip on the streamed path now logs at warn (provider, model, site, event type, payload size; never the payload), calls the new config.on_stream_parse_error hook with a frozen RubyLLM::SkippedStreamEvent, and is counted on the final message as Message#skipped_stream_events. A hook that raises is logged and never breaks the stream. Converse now decodes a string delta once per event so a bad one counts once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Shawn Scott <shawn.scott@appfolio.com>
|
Jira Issue: https://appfolio.atlassian.net/browse/DUX-14977 |
BadassBison
left a comment
There was a problem hiding this comment.
Reviewed: every streamed-path skip now warns (no payload text), calls on_stream_parse_error (hook errors contained), and counts on the final message. CI green across the matrix.
|
🤖 Supernova Code Review — View trace — via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com) |
There was a problem hiding this comment.
Adversarial review of the skipped-stream-event reporting. Things I checked that hold up: protocol instances are built per call, so @stream_accumulator doesn't race between requests. @provider/@model/@config resolve correctly on Protocol. Hook exceptions are contained. The SSE handle_data non-Hash path now reports skips correctly.
One inline finding: in lib/ruby_llm/protocols/converse/event_stream.rb, Bedrock eventstream payloads that are valid JSON but not objects still raise and end the stream instead of being reported as :event_payload skips. The docs promise these get reported.
— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)
| @@ -134,8 +136,7 @@ def decode_event_payload(payload) | |||
| outer | |||
| end | |||
| rescue JSON::ParserError => e | |||
| RubyLLM.logger.debug { "Failed to decode Bedrock stream event payload: #{e.message}" } | |||
| nil | |||
| skip_stream_event(:event_payload, payload, event_type: event_type, error: e) | |||
| end | |||
There was a problem hiding this comment.
A payload that is valid JSON but not an object still crashes the stream instead of being reported as a skip. JSON.parse returns nil, an Integer, or an Array for null, 5, or [...]. Then outer['bytes'] raises NoMethodError or TypeError, and the rescue only catches JSON::ParserError. The same thing happens when bytes decodes to a non-object: Converse's decode_events passes it on to nest_event_under_type / build_chunk, which call event.key? / event.dig and raise. The streaming guide says :event_payload covers payloads that were "not valid JSON or not a JSON object", so the Converse path doesn't do what the docs say. The non-Hash branch added in InvokeAnthropic#decode_invoke_event only catches part of these cases. The fix is to make the guard here cover both protocols:
outer = JSON.parse(payload)
decoded = outer.is_a?(Hash) && outer['bytes'].is_a?(String) ? JSON.parse(Base64.decode64(outer['bytes'])) : outer
return decoded if decoded.is_a?(Hash)
skip_stream_event(:event_payload, payload, event_type: event_type)With that in place, the separate unless data.nil? skip in invoke_anthropic/streaming.rb can go. Add a Converse spec for a null or array payload.
What this does
Streaming parsers used to rescue events they could not decode and drop them with only a debug log. A dropped event, such as a tool call the model actually sent, never surfaced anywhere.
This PR makes every skip on the streamed path visible:
RubyLLM::SkippedStreamEvent, a value object describing an event that could not be decoded.config.on_stream_parse_error, a proc called with aSkippedStreamEventfor each skipped event, so applications can count or alert on them.warn, regardless oflog_stream_debug.Message#skipped_stream_events, viaStreamAccumulator.Type of change
Scope check
Required for new features
Quality check
overcommit --installand all hooks passbundle exec rake vcr:record[provider_name]bundle exec rspecmodels.json,aliases.json)AI-generated code
API changes
Notes for reviewers
RubyLLM::SkippedStreamEvent,Message#skipped_stream_events, andconfig.on_stream_parse_error. Other APIs are unchanged.