Skip to content

MediaReader cannot open RTSP sources whose SDP carries no codec parameters (analyzeduration=0) #598

Closed

Description

@594495520

MediaReader cannot open RTSP sources whose SDP carries no codec parameters (analyzeduration=0)

Summary

webrtc-java-media 1.0.1 fails to open RTSP sources whose SDP does not contain the codec parameters (resolution / sprop-parameter-sets). The failure surfaces as

Opening the decoders failed: Decoder not found

which is misleading: the decoder is present in the build — the stream's width/height are unknown at that point, so the decoder cannot be opened.

The same source opens fine with the ffmpeg CLI, which differs from the library only in that it gives FFmpeg a non-zero stream-analysis budget.

Environment

  • dev.onvoid.webrtc:webrtc-java-media:1.0.1 (Maven Central, published 2026-11-14)
  • Platforms tested: linux-x86_64 (Ubuntu 24.04) and windows-x86_64
  • Java 17+ (GraalVM 25 also tested)
  • The FFmpeg build shipped inside the webrtc-java-media native artifact (libavcodec-62, libavformat-62)

Steps to reproduce

Use any RTSP source whose SDP does not describe the video stream (no a=framesize, no a=sprop-parameter-sets), i.e. an on-camera / cheap-NVR encoder that only sends the SPS in band.

try (MediaReader reader = new MediaReader(url, Duration.ofSeconds(10))) {
    MediaInfo info = reader.getInfo();
    System.out.println(info);
    // -> video=false  codec=h264   0x0 @0.0 | audio=true  codec=pcm_alaw  8000Hz 1ch
}

or through the player:

MediaPlayer player = new MediaPlayer(reader, videoSource, audioSource);
player.play();

Observed behaviour

FFmpeg warns that it could not determine the codec parameters:

[rtsp @ 0x...] Could not find codec parameters for stream 0 (Video: hevc, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

and the Java call then fails:

java.io.IOException: Opening the decoders failed: Decoder not found

analyzeduration (0) in the message above is the giveaway: the analysis budget is zero, so FFmpeg never reads a packet, never sees the in-band SPS, and codecpar->width/height stay 0.

The same native library opens an MP4 of the very same H.264 stream without any problem, so the decoder is present:

out.mp4   video=true  codec=h264   640x480 @15.0 | audio=false

Root cause

In webrtc-media/src/main/cpp/src/media/MediaReader.cpp (tag v1.0.1):

// line 95
result = avformat_open_input(&format_context_, url.c_str(), nullptr, &options);

// line 109
result = avformat_find_stream_info(format_context_, nullptr);

avformat_open_input() is given only a protocol_whitelist entry, and avformat_find_stream_info() is called without any max_analyze_duration / probesize being set on format_context_. The AVFormatContext returned by avformat_alloc_context() has max_analyze_duration == 0, so FFmpeg performs no stream analysis at all. For a container (MP4, …) the size is in the metadata and this is invisible; for RTSP it is not.

The Java-side Duration does not help here: it is passed to the interrupt_callback as an I/O deadline (MediaReader::timeout_us_ → deadline_us_), not as an analysis budget. Raising it to 45 s changes nothing.

The resulting codecpar->width/height == 0 then reaches VideoDecoder.cpp (tag v1.0.1):

// line 77
const AVCodec * preferred = avcodec_find_decoder(id);
// ...
// line 228
result = avcodec_open2(codec_context_, codec, nullptr);

and the decoder cannot be opened with an unknown frame size, which is reported as Opening the decoders failed: Decoder not found.

Suggested fix

Give FFmpeg a stream-analysis budget before avformat_find_stream_info() in MediaReader.cpp:

    format_context_->max_analyze_duration = 5 * AV_TIME_BASE;
    format_context_->probesize = 5 * 1024 * 1024;

    result = avformat_find_stream_info(format_context_, nullptr);

An alternative that also helps API users: accept an AVDictionary of FFmpeg options from Java (e.g. analyzeduration, probesize, rtsp_transport) and pass it to avformat_open_input(), so applications can tune this per source.

Impact / scope

  • Affected: RTSP sources whose SDP does not carry the codec parameters — the SPS is only in band (many on-camera / 4G-router encoders).
  • Not affected: containers (MP4, …) and RTSP sources that do send a=sprop-parameter-sets / a=framesize, or whose size can be derived from the SDP. These open correctly today.

Related issues

Workaround

Transcode/remux through the ffmpeg CLI, which applies a default analysis budget and therefore resolves the size:

ffmpeg -rtsp_transport tcp -i "rtsp://…" -c copy -f flv -listen 1 rtsp://127.0.0.1:8554/stream

The remuxed stream carries the dimensions in its container metadata, so MediaReader opens it.

Activity

  1. devopvoid commented on Nov 19, 2026

    @devopvoid
    Owner

    Edited: corrected the cause of Decoder not found. It comes from the audio track, not from HEVC.

    Thanks for the detailed report. I dug into it, and there are a few separate things going on.

    1. Opening the decoders failed: Decoder not found. For your H.264 source, this comes from the audio, not the video. The webrtc-java-media 1.0.1 FFmpeg build doesn't include the G.711 decoders (pcm_alaw / pcm_mulaw). Opening the player stops as soon as the audio decoder for your pcm_alaw track can't be found. The H.264 decoder itself opens fine. That's a bug on our side: G.711 is what most IP cameras send over RTSP. I'll add both decoders to the build and make the error name the codec that's missing.

    2. The analysis budget isn't the cause. FFmpeg treats analyzeduration = 0 as "use the default". avformat_find_stream_info then analyzes up to 5 s, or 7 s for MPEG-TS, and probesize defaults to 5,000,000 bytes. The warning prints the raw option value, which is why it says (0). The ffmpeg CLI prints the same (0) with its defaults. So setting max_analyze_duration = 5 * AV_TIME_BASE and probesize = 5 MB would leave things exactly as they are now.

    3. H.264 stream reported as 0x0 / no video. FFmpeg analyzed for its full 5 s and never saw an SPS and keyframe. That usually means the camera's keyframe interval is longer than 5 s, which is common on 4G and NVR encoders, especially with "smart codec" modes. MediaInfo.hasVideo() needs a known size, so the stream shows up as having no video, although the player would still decode it. The CLI only looks like it works because -c copy doesn't need the frame size.

    4. HEVC needs a hardware decoder. webrtc-java-media decodes HEVC on hardware only (see the MediaPlayer javadoc). On Linux that means VA-API, Vulkan or NVDEC; on Windows, D3D11/D3D12 or NVDEC. If no hardware device can be opened, an HEVC source fails with the same Decoder not found, separately from the G.711 problem.

    These would help, but you don't need to answer them to get the G.711 fix:

    • the keyframe (GOP) interval configured on the camera,
    • which of your sources are HEVC and which are H.264,
    • whether the machines have a usable GPU. On Linux, the output of vainfo or ls /dev/dri would help, and for Docker, whether /dev/dri is passed into the container.

    As a quick check for (3), set the camera's keyframe interval to 1–2 s and see whether MediaReader then reports the size.

    Planned changes: the G.711 decoders, a clearer error naming the missing decoder, and MediaReader options to tune stream analysis (analyzeDuration, probeSize) so long-GOP sources report their size.

  2. 594495520 commented on Nov 20, 2026

    @594495520
    Author

    Thanks for the detailed analysis — you are right on both counts, and my "analysis budget" reading was wrong: I took the raw option value printed in the warning (analyzeduration (0)) for the effective value, but FFmpeg treats 0 as "use the default" (5 s / 5,000,000 bytes), and the CLI prints the same. Good to have that cleared up.

    Here is the data you asked for.

    1. Which of our sources are HEVC and which are H.264

    All of them are RTSP cameras of the same generation (one NVR, several channels). Every one of them carries G.711 (pcm_alaw) audio:

    source video codec audio codec notes
    rtsp://…:16005/LiveStream/CH0/subStream hevc pcm_alaw size reported 0x0
    rtsp://…:16005/LiveStream/CH2/subStream hevc pcm_alaw size reported 0x0
    rtsp://…:16005/LiveStream/CH8/subStream hevc pcm_alaw size reported 0x0
    rtsp://…:554/LiveStream/CH8/SubStream h264 pcm_alaw size reported 0x0
    rtsp://…:16005/LiveStream/CH1/subStream – – fails to open ("Invalid data found when processing input")
    rtsp://…:16005/LiveStream/CH3/subStream – – same

    So: 3 × HEVC, 1 × H.264, and every source has G.711 audio — none of them is "H.264 without audio", which is why the missing pcm_alaw decoder hits all of them.

    2. GPU / hardware decoding on the machine

    The box is a KVM guest without a real GPU:

    $ ls -l /dev/dri
    crw-rw---- 1 root video 226, 0 Nov  5 07:25 card0
    $ cat /sys/class/drm/card0/device/vendor            -> 0x1013   (QEMU / Cirrus standard VGA)
    $ cat /sys/class/drm/card0/device/subsystem_vendor  -> 0x1af4   (virtio)
    $ ls /usr/lib/x86_64-linux-gnu/dri/                -> No such file or directory
    $ nvidia-smi                                        -> could not communicate with the NVIDIA driver
    

    No VA-API driver is installed at all and there is no NVIDIA device, so hardware HEVC decoding is not available on this host — the three HEVC channels cannot be decoded here, whatever the library does.

    3. Keyframe (GOP) interval

    Not measured yet. We will measure it with ffprobe (-show_frames, key_frame) on the H.264 channel and report the number, and we will also try setting the camera to a 1–2 s GOP and check whether MediaReader then reports the frame size, as you suggested.

    Two extra data points that may help

    • The H.264 video decoder itself is fine: opening an H.264 MP4 (no audio track) with the same native gives video=true codec=h264 640x480 @15.0. So on our streams only the audio decoder is missing.
    • The error message does not name the missing codec: we get Opening the decoders failed: Decoder not found for a stream whose video codec is h264, which is misleading. Your planned "name the missing codec" would have saved us a lot of time — thanks for adding it.

    One question that would unblock us before the next release

    Could a missing audio decoder be made non-fatal — i.e. skip that track and still play the video? Every one of our sources has G.711 audio, so with a tolerant behaviour the H.264 channel could serve video immediately, without waiting for G.711 to be added to the build. If that is not feasible, we will of course use the new build when it lands.

    Also, for the tunable analyzeduration / probesize: it would help if they could be passed from Java (e.g. as an AVDictionary on MediaReader), so a long-GOP source can be given a larger analysis window per stream instead of relying on a global default.

    Thanks again — the G.711 decoder and the clearer error message will unblock our deployment.

  3. devopvoid commented on Nov 20, 2026

    @devopvoid
    Owner

    Thanks for the data, that settles it. Two PRs cover what came up here, and both are merged now:

    #599 adds the pcm_alaw and pcm_mulaw decoders. A missing decoder is now reported by name, e.g. Opening the decoders failed: No decoder for audio codec pcm_s24le. A hardware-only codec without a device gives No hardware decoder for hevc, which is decoded in hardware only.

    #600 fixes the 0x0 H.264 case. MediaInfo.hasVideo() now reports a video stream whenever there is one, even if its size isn't known yet. MediaFileSource creates the video source for it, and the size comes with the first keyframe. Pictures that arrive before that keyframe, joined mid-GOP, are skipped instead of failing with Invalid data. #600 also adds the analysis options you asked for.

    Non-fatal audio. In 1.0.1 there's no way around it: the player opens a decoder for every stream the reader found, even if you pass null as the audio source. Once G.711 is in the build, your sources no longer hit this, so I'd rather keep a missing decoder fatal than drop a track silently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions