Skip to content

Add foxglove compressed topics - #207

Open
engnfrc wants to merge 8 commits into
RobotWebTools:ros2from
engnfrc:add_foxglove_compressed_topics
Open

engnfrc wants to merge 8 commits into
RobotWebTools:ros2from
engnfrc:add_foxglove_compressed_topics

Conversation

@engnfrc

@engnfrc engnfrc commented Nov 13, 2026

Copy link
Copy Markdown

Public API Changes

Adds support for two additional ROS 2 message types:

  • foxglove_msgs/msg/CompressedImage
  • foxglove_msgs/msg/CompressedVideo

New streamer types are available through the existing type query parameter:

  • foxglove_compressed_image
  • foxglove_compressed_video

The Foxglove compressed video viewer also supports optional client-side display parameters:

  • width=<pixels> - set the rendered width while preserving aspect ratio when height is not provided
  • height=<pixels> - set the rendered height while preserving aspect ratio when width is not provided
  • width=<pixels>&height=<pixels> - set both rendered dimensions explicitly
  • display_fps=<fps> - limit the browser-side display rate without changing the source stream or server-side encoding

Example:

/stream_viewer?topic=/camera_rear/video&type=foxglove_compressed_video&height=360&display_fps=15

Description

Adds native support for Foxglove compressed image and video message types without requiring conversion to intermediary ROS image message types.

foxglove_msgs/msg/CompressedImage messages are streamed directly using their existing compressed image payload.

foxglove_msgs/msg/CompressedVideo messages are forwarded in their existing encoded form and decoded in the client browser using the WebCodecs API. This avoids decoding, colorspace conversion, scaling, or re-encoding the video on the ROS host and keeps additional server-side CPU overhead low.

The H.264 browser path supports Annex-B input as defined by the Foxglove message format and includes handling for:

  • H.264 NAL-unit parsing
  • SPS/PPS caching across messages
  • keyframe/IDR detection
  • initialization of newly connected browser decoders
  • recovery when a browser connects between keyframes
  • browser-side scaling and display-rate limiting

The implementation is also compatible with the newer SubscriberFactoryInterface API added to the ROS 2 branch. The Foxglove streamers retain direct typed ROS subscriptions, similar to the existing compressed-image streamer, while passing the subscriber factory map through the updated streamer interfaces.

Client-side scaling and display_fps only affect browser rendering. They do not alter the ROS topic, encoded video stream, network bitrate, or server-side video processing.

WebCodecs requires a secure browser context for remote access. In testing, the viewer was served through HTTPS using a reverse proxy while web_video_server itself continued to listen over HTTP.

This change was developed and tested with H.264 video published through foxglove_msgs/msg/CompressedVideo, including a GStreamer pipeline using x264enc and h264parse.

Closes #206

AI Assistance

These changes were developed with assistance from OpenAI ChatGPT using the GPT-5.6 Sol model. ChatGPT was used to help design the implementation, generate and revise portions of the C++ and browser-side JavaScript, diagnose build and runtime issues, and adapt the implementation to upstream API changes.

The generated changes were built and tested by the contributor, with iterative validation against live ROS 2 Foxglove compressed video streams.

- Add foxglove_msgs/CompressedImage passthrough streaming
- Add foxglove_msgs/CompressedVideo browser playback support
- Forward encoded video frames without decode or re-encode
- Add browser-side WebCodecs decoding
- Handle Annex-B and AVCC H.264 payloads
- Cache SPS/PPS and initialize H.264 decoding from keyframes
- Add Foxglove streamer plugin registration and dependencies
- Add width and height query parameters to the Foxglove video viewer
- Preserve aspect ratio when only one output dimension is specified
- Add display_fps query parameter for client-side frame presentation limiting
- Continue decoding full video while reducing canvas rendering work
- Wait for a usable keyframe without aborting the stream
- Retry initial stream startup until decoder configuration succeeds
- Pass subscriber factory maps through Foxglove streamer constructors
- Update streamer factory create_streamer() signatures
- Update get_available_topics() signatures for the new upstream API
- Preserve direct typed subscriptions for Foxglove compressed image and video topics
- Restore compatibility with the latest ros2 branch subscriber plugin changes
- Apply ament_uncrustify formatting to Foxglove image streamer
- Apply ament_uncrustify formatting to Foxglove video streamer
- Resolve industrial_ci source test failures
- Update formatting for Humble uncrustify compatibility
- Add direct includes required by Rolling clang-tidy
- Remove unused includes
- Rename constexpr variables to match project naming rules
- Suppress FFmpeg parser const-correctness warning where required
- Add direct subscriber interface includes
- Add direct FFmpeg codec ID and decoder includes
- Rename frame header size constant to match project naming rules
- Resolve remaining include-cleaner warnings in Foxglove streamers
- Add direct libavcodec codec header include
- Resolve clang-tidy warning for avcodec_find_decoder()
- Keep Rolling CI compatible with strict include-cleaner checks
@engnfrc

engnfrc commented Nov 13, 2026

Copy link
Copy Markdown
Author

It appears the remaining pipeline failures at this point are isolated to industrial_ci (rolling, main) and will require a maintainers attention: "E: Unable to locate package ros-rolling-ros-environment".

@bjsowa

bjsowa commented Nov 13, 2026

Copy link
Copy Markdown
Member

It appears the remaining pipeline failures at this point are isolated to industrial_ci (rolling, main) and will require a maintainers attention: "E: Unable to locate package ros-rolling-ros-environment".

Rolling didn't had a sync to main for like 6 months now. It's expected to fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Support for the Foxglove Compressed Image/Video Message Types

3 participants