diff --git a/docs/guide/client-api.md b/docs/guide/client-api.md index 3172a174b..e2ce18f19 100644 --- a/docs/guide/client-api.md +++ b/docs/guide/client-api.md @@ -860,10 +860,82 @@ peer.onStats(Duration.ofSeconds(1), stats -> System.out.printf( | `packetLoss()` | The share of received packets lost so far, 0 to 1. After a long session it barely moves. | | `recentPacketLoss()` | The share of packets lost of those expected since the previous sample, 0 to 1, and 0 if none were expected; -1 for the first sample and after the clock went back. A stream that came up or whose counters started again counts all it has. | | `outboundVideo()`, `inboundVideo()` | Codec, resolution and frame rate of each video stream, and what limits a sent one. | -| `report()` | The whole `RTCStatsReport`. | +| `inboundAudio()`, `outboundAudio()` | Audio streams ordered by statistics id: codec, jitter, concealment counters, audio level and sample counters. Unavailable fields are null; received PCM callbacks are observed only while an `onAudio` sink is attached. | +| `diagnostics()` | The current state and confirmed `QualityCondition` causes. | +| `diagnosticReport()` | A JSON snapshot with allowlisted diagnostics, media measurements, codec names, OS/architecture, Java version, library version and processor count. | +| `report()` | The whole `RTCStatsReport`, including potentially sensitive connection information. | A peer bundles its media on one transport by default. When the remote side refuses to bundle, each m-line has a transport of its own. `transport()` is then the one with a candidate pair in use that carried the most bytes, sent and received, and of transports that carried as many, the one with the lowest id. Read `transports()` to watch each connection by itself. +### Diagnosis and quality events + +```java +Subscription diagnosis = peer.onDiagnostics(Duration.ofSeconds(1), status -> { + System.out.println(status.state()); + status.conditions().forEach(cause -> System.out.println(cause.reason())); +}); + +// Export on demand, for example from a support button. File I/O belongs on +// an application worker, outside the peer's event executor. +peer.stats().thenAccept(stats -> supportExecutor.execute(() -> { + try { + Files.write(Paths.get("diagnostics.json"), + stats.diagnosticReport().getBytes(StandardCharsets.UTF_8)); + } + catch (IOException error) { + error.printStackTrace(); + } +})); +``` + +The export example uses `java.nio.file.Files`, `java.nio.file.Paths`, +`java.nio.charset.StandardCharsets` and `java.io.IOException`; `supportExecutor` +is an application-owned executor. + +`onDiagnostics` emits the initial diagnosis and then only changes to its state or +causes, including recovery. Handlers run on the peer's event executor; closing the +subscription stops sampling. All `stats`, `onStats` and `onDiagnostics` calls share +one history per peer. A captured `PeerStats` and its diagnosis stay immutable. + +There is no combined score. `OBSERVING` means there is no continuous measurement +interval yet (including after a clock rollback or a gap longer than five seconds). +`HEALTHY` means no supported problem has been confirmed; it is not a guarantee +that all media paths work. `DEGRADED` includes the following confirmed causes: + +| Condition | Evidence | +|---|---| +| `VIDEO_CPU_LIMITED` | The same outgoing video RTP stream consistently reports `qualityLimitationReason=cpu`. | +| `HIGH_PACKET_LOSS` | Recent inbound packet loss is at least 5%; recovery needs at most 2%. Values between those thresholds retain the current state and restart the confirmation window. | +| `AUDIO_PACKETS_WITHOUT_PCM` | The same received audio RTP stream gains packets while no PCM callbacks reach its attached `onAudio` sinks. This observes sink ingress, before application queues and handlers; it does not diagnose speakers or slow application handlers. | + +Each problem and its recovery need at least three distinct measurements spanning +at least three seconds. Polling once a second is recommended; cached reports do +not advance the window. A missing stream or required measurement, a changed sink +set, or a reset audio counter discards that stream's evidence instead of guessing. +Sampling gaps longer than five seconds and backwards timestamps discard temporal +evidence. Diagnostics do not add audio sinks: without an application subscription, +PCM availability is unknown. Silence (audio level zero) alone is not a fault. + +`TURN_IN_USE` is immediate information when either selected candidate on any +transport is a relay. It does not make a peer degraded. `TransportStats.relayed()` +also exposes this fact, while `candidateType()` continues to describe the local +candidate. + +`AudioStats.jitter()` is a `Duration`, `audioLevel()` is linear (0 to 1), and +`concealedSamples()`, `concealmentEvents()` and `totalSamplesReceived()` are +cumulative counters. Outbound level is read from the media source when available; +receive-only measurements on outbound audio remain null. `pcmCallbacks()` counts +sink callbacks across subscriptions, so multiple subscriptions can count the same +audio more than once. + +The JSON export includes the confirmation policy and cause explanations. It uses +schema version 1 and an explicit allowlist, including a +bounded set of codec MIME names; unknown names become `other`. It excludes raw +statistics, track/stream/transport/candidate identifiers, addresses, ports, URLs, +ICE credentials, SDP, certificates and codec parameters. Missing measurements +become JSON null. The raw `report()` and its string representation are not +sanitized diagnostic exports. + ## Closing You close what you created, and closing an owner closes what it owns: diff --git a/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/ClientApiExample.java b/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/ClientApiExample.java index 46662f3d9..00935deee 100644 --- a/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/ClientApiExample.java +++ b/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/ClientApiExample.java @@ -97,6 +97,10 @@ public static void main(String[] args) throws Exception { receive(callee); caller.onStateChange(state -> System.out.println("Caller is " + state)); + caller.onDiagnostics(Duration.ofSeconds(1), diagnosis -> { + System.out.println("Caller diagnosis: " + diagnosis.state()); + diagnosis.conditions().forEach(cause -> System.out.println(cause.reason())); + }); // Pushed media. The same tracks could be sent on any number of peers. CustomVideo video = rtc.media().customVideo("pattern"); diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioDispatcher.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioDispatcher.java index 1df942d40..9af64f380 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioDispatcher.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioDispatcher.java @@ -18,6 +18,7 @@ import java.time.Duration; import java.util.function.Consumer; +import java.util.concurrent.atomic.AtomicLong; import dev.onvoid.webrtc.internal.Timeouts; import dev.onvoid.webrtc.media.audio.AudioTrackSink; @@ -29,6 +30,12 @@ */ final class AudioDispatcher extends Dispatcher implements AudioTrackSink { + private final AtomicLong pcmCallbacks = new AtomicLong(); + + long pcmCallbacks() { + return pcmCallbacks.get(); + } + /** * @param buffer how much audio may wait for the handler. */ @@ -43,6 +50,12 @@ public void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, return; } + if (frames > 0) { + + pcmCallbacks.incrementAndGet(); + + } + // The track reuses the array for the next chunk. offer(new AudioChunk(data.clone(), sampleRate, channels, frames)); } diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioStats.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioStats.java new file mode 100644 index 000000000..e5e2aec87 --- /dev/null +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/AudioStats.java @@ -0,0 +1,104 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +import java.time.Duration; + +/** Immutable statistics of one audio RTP stream. Missing measurements are null. */ +public final class AudioStats { + private final String id; + private final String trackId; + private final String codec; + private final Long packetsReceived; + private final Duration jitter; + private final Long concealedSamples; + private final Long concealmentEvents; + private final Double audioLevel; + private final Long totalSamplesReceived; + private final Long pcmCallbacks; + final long pcmGeneration; + + AudioStats(String id, String trackId, String codec, Long packetsReceived, + Duration jitter, Long concealedSamples, Long concealmentEvents, + Double audioLevel, Long totalSamplesReceived, Long pcmCallbacks, long pcmGeneration) { + this.id = id; + this.trackId = trackId; + this.codec = codec; + this.packetsReceived = packetsReceived; + this.jitter = jitter; + this.concealedSamples = concealedSamples; + this.concealmentEvents = concealmentEvents; + this.audioLevel = audioLevel; + this.totalSamplesReceived = totalSamplesReceived; + this.pcmCallbacks = pcmCallbacks; + this.pcmGeneration = pcmGeneration; + } + + /** @return the RTP statistics id, for correlating successive measurements. */ + public String id() { + return id; + } + + /** @return the track identifier, or null. */ + public String trackId() { + return trackId; + } + + /** @return the negotiated codec MIME type, or null. */ + public String codec() { + return codec; + } + + /** @return cumulative packets received, or null for outbound or unavailable data. */ + public Long packetsReceived() { + return packetsReceived; + } + + /** @return interarrival jitter, or null if unavailable. */ + public Duration jitter() { + return jitter; + } + + /** @return cumulative samples replaced by concealment, or null. */ + public Long concealedSamples() { + return concealedSamples; + } + + /** @return cumulative concealment events, or null. */ + public Long concealmentEvents() { + return concealmentEvents; + } + + /** @return linear audio level from 0 to 1, or null. Zero can be silence. */ + public Double audioLevel() { + return audioLevel; + } + + /** @return cumulative received samples reported by WebRTC, or null. */ + public Long totalSamplesReceived() { + return totalSamplesReceived; + } + + /** + * @return cumulative PCM callbacks across this track's onAudio subscriptions, + * or null when none is attached. Counts sink ingress, before application queues. + * Several subscriptions can count the same audio more than once. + */ + public Long pcmCallbacks() { + return pcmCallbacks; + } +} diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticExport.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticExport.java new file mode 100644 index 000000000..870450c9b --- /dev/null +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticExport.java @@ -0,0 +1,156 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +import java.util.Arrays; +import java.util.Locale; + +/** An explicit export schema: never serializes raw stats or arbitrary string attributes. */ +final class DiagnosticExport { + private DiagnosticExport() { } + + static String json(PeerStats stats) { + StringBuilder out = new StringBuilder("{\"schemaVersion\":1"); + field(out, "timestamp", stats.timestamp().toString()); + field(out, "state", stats.diagnostics().state().name()); + field(out, "os", platform(System.getProperty("os.name", ""))); + field(out, "architecture", allowed(System.getProperty("os.arch", ""), + "amd64", "x86_64", "aarch64", "arm", "x86", "i386")); + String javaVersion = System.getProperty("java.specification.version", ""); + field(out, "javaVersion", javaVersion.matches("[0-9]{1,3}(\\.[0-9]{1,3})?") ? javaVersion : "other"); + String version = PeerStats.class.getPackage().getImplementationVersion(); + field(out, "libraryVersion", version != null && version.matches("[0-9]+\\.[0-9]+\\.[0-9]+(-SNAPSHOT)?") + ? version : "unversioned"); + number(out, "availableProcessors", Runtime.getRuntime().availableProcessors()); + number(out, "bytesSent", stats.bytesSent()); + number(out, "bytesReceived", stats.bytesReceived()); + number(out, "outgoingBitrate", stats.outgoingBitrate()); + number(out, "incomingBitrate", stats.incomingBitrate()); + number(out, "packetLoss", stats.packetLoss()); + number(out, "recentPacketLoss", stats.recentPacketLoss()); + out.append(",\"policy\":{\"confirmationSeconds\":").append(DiagnosticsReader.WINDOW.getSeconds()); + number(out, "minimumMeasurements", DiagnosticsReader.MIN_SAMPLES); + number(out, "lossEnter", DiagnosticsReader.LOSS_ENTER); + number(out, "lossExit", DiagnosticsReader.LOSS_EXIT); + number(out, "maximumGapSeconds", DiagnosticsReader.MAX_GAP.getSeconds()); + out.append('}'); + out.append(",\"conditions\":["); + boolean comma = false; + for (QualityCondition condition : stats.diagnostics().conditions()) { + if (comma) { + out.append(','); + } + out.append("{\"cause\":"); + Json.writeString(out, condition.name()); + field(out, "reason", condition.reason()); + out.append(",\"problem\":").append(condition.isProblem()).append('}'); + comma = true; + } + out.append("],\"transports\":["); + comma = false; + for (TransportStats transport : stats.transports()) { + if (comma) { + out.append(','); + } + out.append("{\"relayed\":").append(transport.relayed()); + field(out, "candidateType", allowed(transport.candidateType(), "host", "srflx", "prflx", "relay")); + number(out, "rttSeconds", transport.roundTripTime() == null ? null + : transport.roundTripTime().toNanos() / 1e9); + out.append('}'); + comma = true; + } + out.append("],\"video\":["); + comma = false; + for (boolean outbound : new boolean[] { false, true }) { + for (VideoStats video : outbound ? stats.outboundVideo() : stats.inboundVideo()) { + if (comma) { + out.append(','); + } + out.append("{\"outbound\":").append(outbound); + field(out, "codec", codec(video.codec())); + number(out, "width", video.width()); + number(out, "height", video.height()); + number(out, "framesPerSecond", video.framesPerSecond()); + field(out, "qualityLimitation", allowed(video.qualityLimitation(), "none", "cpu", "bandwidth", "other")); + out.append('}'); + comma = true; + } + } + out.append("],\"audio\":["); + comma = false; + for (boolean outbound : new boolean[] { false, true }) { + for (AudioStats audio : outbound ? stats.outboundAudio() : stats.inboundAudio()) { + if (comma) { + out.append(','); + } + out.append("{\"outbound\":").append(outbound); + field(out, "codec", codec(audio.codec())); + number(out, "packetsReceived", audio.packetsReceived()); + number(out, "jitterSeconds", audio.jitter() == null ? null : audio.jitter().toNanos() / 1e9); + number(out, "concealedSamples", audio.concealedSamples()); + number(out, "concealmentEvents", audio.concealmentEvents()); + number(out, "audioLevel", audio.audioLevel()); + number(out, "totalSamplesReceived", audio.totalSamplesReceived()); + number(out, "pcmCallbacks", audio.pcmCallbacks()); + out.append('}'); + comma = true; + } + } + return out.append("]}").toString(); + } + + private static String codec(String value) { + if (value == null) { + return null; + } + return allowed(value.toLowerCase(Locale.ROOT), "audio/opus", "audio/pcmu", "audio/pcma", + "audio/g722", "audio/isac", "audio/cn", "audio/telephone-event", "audio/red", + "video/vp8", "video/vp9", "video/h264", "video/h265", "video/av1", "video/rtx", + "video/red", "video/ulpfec", "video/flexfec-03"); + } + + private static String allowed(String value, String... allowed) { + return value == null ? null : Arrays.asList(allowed).contains(value) ? value : "other"; + } + + private static String platform(String name) { + if (name.startsWith("Windows")) { + return "Windows"; + } + if (name.equals("Mac OS X")) { + return "macOS"; + } + return name.equals("Linux") ? "Linux" : "other"; + } + + private static void field(StringBuilder out, String name, String value) { + out.append(','); + Json.writeString(out, name); + out.append(':'); + if (value == null) { + out.append("null"); + } + else Json.writeString(out, value); + } + + private static void number(StringBuilder out, String name, Number value) { + out.append(','); + Json.writeString(out, name); + out.append(':'); + out.append(value == null || !Double.isFinite(value.doubleValue()) ? "null" : value.toString()); + } +} diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticsReader.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticsReader.java new file mode 100644 index 000000000..69034339a --- /dev/null +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/DiagnosticsReader.java @@ -0,0 +1,138 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +import java.time.Duration; +import java.time.Instant; +import java.util.EnumSet; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import dev.onvoid.webrtc.RTCStats; +import dev.onvoid.webrtc.RTCStatsType; + +/** Temporal evidence, confined to the statistics reader's control executor. */ +final class DiagnosticsReader { + static final Duration WINDOW = Duration.ofSeconds(3); + static final Duration MAX_GAP = Duration.ofSeconds(5); + static final int MIN_SAMPLES = 3; + static final double LOSS_ENTER = 0.05; + static final double LOSS_EXIT = 0.02; + + private final Map gates = new HashMap<>(); + private Map lastAudio = new HashMap<>(); + private Instant lastTime; + private PeerDiagnostics last; + + PeerDiagnostics read(Instant time, double loss, List transports, + Map stats, List audio) { + if (time.equals(lastTime)) { + return last; + } + boolean continuous = lastTime != null && time.isAfter(lastTime) + && Duration.between(lastTime, time).compareTo(MAX_GAP) <= 0; + if (!continuous) { + gates.clear(); + lastAudio.clear(); + } + Set seen = new HashSet<>(); + Set conditions = EnumSet.noneOf(QualityCondition.class); + update("loss", QualityCondition.HIGH_PACKET_LOSS, + !continuous || !Double.isFinite(loss) || loss < 0 ? null + : loss >= LOSS_ENTER ? Boolean.TRUE : loss <= LOSS_EXIT ? Boolean.FALSE : null, + time, seen, conditions, loss > LOSS_EXIT && loss < LOSS_ENTER && continuous); + + for (RTCStats entry : stats.values()) { + Map values = entry.getAttributes(); + if (entry.getType() == RTCStatsType.OUTBOUND_RTP && "video".equals(values.get("kind"))) { + Object reason = values.get("qualityLimitationReason"); + update("cpu:" + entry.getId(), QualityCondition.VIDEO_CPU_LIMITED, + "cpu".equals(reason) ? Boolean.TRUE + : "none".equals(reason) || "bandwidth".equals(reason) || "other".equals(reason) + ? Boolean.FALSE : null, time, seen, conditions, false); + } + } + Map nextAudio = new HashMap<>(); + for (AudioStats now : audio) { + AudioStats before = lastAudio.get(now.id()); + Boolean stalled = null; + if (before != null && now.packetsReceived() != null && before.packetsReceived() != null + && now.pcmCallbacks() != null && before.pcmCallbacks() != null + && now.pcmGeneration == before.pcmGeneration + && java.util.Objects.equals(now.trackId(), before.trackId()) + && now.packetsReceived() >= before.packetsReceived() + && now.pcmCallbacks() >= before.pcmCallbacks()) { + stalled = now.packetsReceived() > before.packetsReceived() + && now.pcmCallbacks().equals(before.pcmCallbacks()); + } + update("pcm:" + now.id(), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM, + stalled, time, seen, conditions, false); + nextAudio.put(now.id(), now); + } + gates.keySet().retainAll(seen); + if (transports.stream().anyMatch(TransportStats::relayed)) { + conditions.add(QualityCondition.TURN_IN_USE); + } + lastAudio = nextAudio; + lastTime = time; + last = new PeerDiagnostics(time, continuous && !stats.isEmpty(), conditions); + return last; + } + + private void update(String key, QualityCondition condition, Boolean evidence, Instant time, + Set seen, Set conditions, boolean deadband) { + seen.add(key); + Gate gate = gates.computeIfAbsent(key, ignored -> new Gate()); + if (deadband) { + gate.since = null; + gate.count = 0; + } + else if (evidence == null) { + // Unknown data cannot confirm an old problem or count as recovery. + gate.active = false; + gate.since = null; + gate.count = 0; + } + else if (evidence == gate.active) { + gate.since = null; + gate.count = 0; + } + else { + if (gate.since == null) { + gate.since = time; + } + gate.count++; + if (gate.count >= MIN_SAMPLES && Duration.between(gate.since, time).compareTo(WINDOW) >= 0) { + gate.active = evidence; + gate.since = null; + gate.count = 0; + } + } + if (gate.active) { + conditions.add(condition); + } + } + + private static final class Gate { + boolean active; + Instant since; + int count; + } +} diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/Peer.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/Peer.java index 0c13463b7..c8952ac04 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/Peer.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/Peer.java @@ -16,9 +16,12 @@ package dev.onvoid.webrtc.client; +import static java.util.Objects.requireNonNull; + import java.time.Duration; import java.util.List; import java.util.concurrent.CompletableFuture; +import java.util.concurrent.atomic.AtomicReference; import java.util.function.Consumer; import java.util.function.Function; @@ -176,6 +179,28 @@ public interface Peer extends AutoCloseable { */ Subscription onStats(Duration interval, Consumer handler); + /** + * Samples statistics and reports the initial diagnosis and subsequent changes + * to its state or causes, on the event executor. Closing the subscription + * stops sampling. Uses the same per-peer history as stats() and onStats(). + * @param interval statistics sampling interval; one second is recommended. + * @param handler receives explainable diagnosis transitions, including recovery. + * @return the subscription. + */ + default Subscription onDiagnostics(Duration interval, Consumer handler) { + requireNonNull(handler, "handler"); + AtomicReference previous = + new AtomicReference<>(); + return onStats(interval, stats -> { + PeerDiagnostics diagnosis = stats.diagnostics(); + PeerDiagnostics before = previous.getAndSet(diagnosis); + if (before == null || before.state() != diagnosis.state() + || !before.conditions().equals(diagnosis.conditions())) { + handler.accept(diagnosis); + } + }); + } + /** * Gathers new ICE candidates and renegotiates, to recover a failed * connection. diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerDiagnostics.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerDiagnostics.java new file mode 100644 index 000000000..327239180 --- /dev/null +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerDiagnostics.java @@ -0,0 +1,74 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +import java.time.Instant; +import java.util.Collections; +import java.util.EnumSet; +import java.util.Set; + +/** + * An immutable diagnosis at a measurement time. TURN alone is informational. + *

+ * Media problems and recovery require three distinct measurements spanning at + * least three seconds. Packet loss enters at 5% and recovers at 2%; the band + * between retains the current state. Missing evidence, clock rollback and gaps + * longer than five seconds discard temporal history. Cached reports do not advance it. + */ +public final class PeerDiagnostics { + /** Observation is limited to the conditions supported by this API. */ + public enum State { + /** No continuous measurement interval is available yet. */ + OBSERVING, + /** No supported media problem has been confirmed; not a quality guarantee. */ + HEALTHY, + /** At least one supported media problem is confirmed. */ + DEGRADED + } + + private final Instant timestamp; + private final State state; + private final Set conditions; + + PeerDiagnostics(Instant timestamp, boolean measured, Set conditions) { + this.timestamp = timestamp; + this.conditions = Collections.unmodifiableSet(conditions.isEmpty() + ? EnumSet.noneOf(QualityCondition.class) : EnumSet.copyOf(conditions)); + this.state = conditions.stream().anyMatch(QualityCondition::isProblem) + ? State.DEGRADED : measured ? State.HEALTHY : State.OBSERVING; + } + + /** @return when the underlying statistics were measured. */ + public Instant timestamp() { + return timestamp; + } + + /** @return the diagnosis state, not a quality score or guarantee. */ + public State state() { + return state; + } + + /** @return the active causes; an empty set means none was confirmed. */ + public Set conditions() { + return conditions; + } + + @Override + public String toString() { + return state + " " + conditions; + } +} diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerImpl.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerImpl.java index a6c3d841b..96a341644 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerImpl.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerImpl.java @@ -22,6 +22,7 @@ import java.util.ArrayList; import java.util.Collections; import java.util.List; +import java.util.Map; import java.util.UUID; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionException; @@ -399,6 +400,10 @@ void onLocalCandidates(Consumer candidates, Runnable gathered) this.gatheredListener = gathered; } + Map pcmObservations() { + return tracks.pcmObservations(); + } + @Override public CompletableFuture stats() { return statistics.read(); diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerStats.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerStats.java index 755bca417..2d872e889 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerStats.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/PeerStats.java @@ -57,12 +57,19 @@ public final class PeerStats { private final RTCStatsReport report; + private final List inboundAudio; + private final List outboundAudio; + private final PeerDiagnostics diagnostics; + PeerStats(Instant timestamp, TransportStats transport, List transports, - long bytesSent, long bytesReceived, long outgoingBitrate, - long incomingBitrate, double packetLoss, double recentPacketLoss, - List outboundVideo, List inboundVideo, - RTCStatsReport report) { + long bytesSent, long bytesReceived, long outgoingBitrate, long incomingBitrate, + double packetLoss, double recentPacketLoss, List outboundVideo, + List inboundVideo, RTCStatsReport report, List inboundAudio, + List outboundAudio, PeerDiagnostics diagnostics) { + this.inboundAudio = inboundAudio; + this.outboundAudio = outboundAudio; + this.diagnostics = diagnostics; this.timestamp = timestamp; this.transport = transport; this.transports = transports; @@ -193,6 +200,30 @@ public RTCStatsReport report() { return report; } + /** @return immutable statistics of received audio streams, ordered by statistics id. */ + public List inboundAudio() { + return inboundAudio; + } + + /** @return immutable statistics of sent audio streams, ordered by statistics id. */ + public List outboundAudio() { + return outboundAudio; + } + + /** @return the diagnosis maintained across samples of this peer. */ + public PeerDiagnostics diagnostics() { + return diagnostics; + } + + /** + * Exports a JSON snapshot using an allowlist. Raw stats, identifiers, addresses, + * ports, URLs, credentials, SDP, certificates and codec parameters are excluded. + * @return the diagnostic report, suitable for writing to a UTF-8 file. + */ + public String diagnosticReport() { + return DiagnosticExport.json(this); + } + @Override public String toString() { return "PeerStats[candidate=" + candidateType() + ", rtt=" + roundTripTime() diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/QualityCondition.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/QualityCondition.java new file mode 100644 index 000000000..37a24c92a --- /dev/null +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/QualityCondition.java @@ -0,0 +1,43 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +/** An explainable observation, without a combined quality score. */ +public enum QualityCondition { + VIDEO_CPU_LIMITED("Video is limited by CPU", true), + HIGH_PACKET_LOSS("High packet loss over several measurements", true), + TURN_IN_USE("The connection uses TURN", false), + AUDIO_PACKETS_WITHOUT_PCM("Audio receives packets but no PCM reaches subscribed sinks", true); + + private final String reason; + private final boolean problem; + + QualityCondition(String reason, boolean problem) { + this.reason = reason; + this.problem = problem; + } + + /** @return a human-readable explanation. */ + public String reason() { + return reason; + } + + /** @return whether this condition indicates degraded media, rather than information. */ + public boolean isProblem() { + return problem; + } +} diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteAudioImpl.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteAudioImpl.java index 0d4f95a80..cd086c644 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteAudioImpl.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteAudioImpl.java @@ -39,6 +39,19 @@ final class RemoteAudioImpl extends RemoteTrackImpl implements RemoteAudio { // Confined to the peer's control executor. private final List dispatchers = new ArrayList<>(); + private long pcmGeneration; + + /** On control; generation changes whenever the observed sink set changes. */ + long[] pcmObservation() { + if (dispatchers.isEmpty()) { + return null; + } + long callbacks = 0; + for (AudioDispatcher dispatcher : dispatchers) { + callbacks += dispatcher.pcmCallbacks(); + } + return new long[] { pcmGeneration, callbacks }; + } RemoteAudioImpl(PeerImpl peer, RTCRtpReceiver receiver, AudioTrack track, @@ -69,6 +82,7 @@ public Subscription onAudio(Duration buffer, Consumer handler) { onControl(() -> { dispatchers.add(dispatcher); + pcmGeneration++; // The dispatcher copies each chunk, so it needs no new array per call. track.addSink(dispatcher, AudioDataAccess.REUSED); }); @@ -78,6 +92,7 @@ public Subscription onAudio(Duration buffer, Consumer handler) { onControl(() -> { if (dispatchers.remove(dispatcher)) { + pcmGeneration++; track.removeSink(dispatcher); } }); diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteTracks.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteTracks.java index 4befd761c..c67922387 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteTracks.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/RemoteTracks.java @@ -93,6 +93,20 @@ List current() { return Collections.unmodifiableList(new ArrayList(tracks)); } + /** Snapshot only existing application sinks; diagnostics never enables PCM delivery. */ + Map pcmObservations() { + Map result = new HashMap<>(); + for (RemoteTrackImpl track : tracks) { + if (track instanceof RemoteAudioImpl) { + long[] observation = ((RemoteAudioImpl) track).pcmObservation(); + if (observation != null) { + result.put(track.id(), observation); + } + } + } + return result; + } + Subscription onTrack(Consumer handler) { requireNonNull(handler, "handler"); diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsReader.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsReader.java index 6477b1fa4..f3c33a98a 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsReader.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsReader.java @@ -38,6 +38,8 @@ */ final class StatsReader { + private final DiagnosticsReader diagnostics = new DiagnosticsReader(); + private long lastMicros = -1; private long lastBytesSent; @@ -55,6 +57,10 @@ final class StatsReader { PeerStats read(RTCStatsReport report) { + return read(report, Collections.emptyMap()); + } + + PeerStats read(RTCStatsReport report, Map pcm) { Map stats = report.getStats(); long bytesSent = 0; @@ -65,6 +71,8 @@ PeerStats read(RTCStatsReport report) { Map streams = new HashMap<>(); List outbound = new ArrayList<>(); List inbound = new ArrayList<>(); + List inboundAudio = new ArrayList<>(); + List outboundAudio = new ArrayList<>(); for (RTCStats entry : stats.values()) { Map values = entry.getAttributes(); @@ -88,6 +96,9 @@ else if (entry.getType() == RTCStatsType.INBOUND_RTP) { if ("video".equals(values.get("kind"))) { inbound.add(video(stats, values, string(values.get("trackIdentifier")), null)); } + else if ("audio".equals(values.get("kind"))) { + inboundAudio.add(audio(stats, entry, values, pcm)); + } } else if (entry.getType() == RTCStatsType.OUTBOUND_RTP) { if ("video".equals(values.get("kind"))) { @@ -96,6 +107,15 @@ else if (entry.getType() == RTCStatsType.OUTBOUND_RTP) { outbound.add(video(stats, values, trackId, string(values.get("qualityLimitationReason")))); } + else if ("audio".equals(values.get("kind"))) { + RTCStats source = lookup(stats, values.get("mediaSourceId")); + Map audioValues = new HashMap<>(); + if (source != null) { + audioValues.putAll(source.getAttributes()); + } + audioValues.putAll(values); + outboundAudio.add(audio(stats, entry, audioValues, Collections.emptyMap())); + } } } @@ -137,11 +157,18 @@ else if (entry.getType() == RTCStatsType.OUTBOUND_RTP) { lastRecentPacketLoss = recentPacketLoss; } - return new PeerStats(Instant.EPOCH.plus(micros, ChronoUnit.MICROS), inUse(transports), + inboundAudio.sort(Comparator.comparing(AudioStats::id)); + outboundAudio.sort(Comparator.comparing(AudioStats::id)); + Instant timestamp = Instant.EPOCH.plus(micros, ChronoUnit.MICROS); + PeerDiagnostics diagnosis = diagnostics.read(timestamp, recentPacketLoss, transports, + stats, inboundAudio); + + return new PeerStats(timestamp, inUse(transports), Collections.unmodifiableList(transports), bytesSent, bytesReceived, outgoingBitrate, incomingBitrate, lossRate(packetsLost, packetsReceived), recentPacketLoss, Collections.unmodifiableList(outbound), - Collections.unmodifiableList(inbound), report); + Collections.unmodifiableList(inbound), report, + Collections.unmodifiableList(inboundAudio), Collections.unmodifiableList(outboundAudio), diagnosis); } private static TransportStats transport(Map stats, RTCStats entry) { @@ -149,6 +176,7 @@ private static TransportStats transport(Map stats, RTCStats en String pairId = string(values.get("selectedCandidatePairId")); RTCStats pair = lookup(stats, pairId); String candidateType = null; + boolean relayed = false; Duration roundTripTime = null; if (pair != null) { @@ -160,13 +188,16 @@ private static TransportStats transport(Map stats, RTCStats en RTCStats local = lookup(stats, pair.getAttributes().get("localCandidateId")); + RTCStats remote = lookup(stats, pair.getAttributes().get("remoteCandidateId")); + relayed = remote != null && "relay".equals(remote.getAttributes().get("candidateType")); if (local != null) { candidateType = string(local.getAttributes().get("candidateType")); } } return new TransportStats(entry.getId(), pairId, candidateType, roundTripTime, - longValue(values.get("bytesSent")), longValue(values.get("bytesReceived"))); + longValue(values.get("bytesSent")), longValue(values.get("bytesReceived")), + relayed || "relay".equals(candidateType)); } /** @@ -247,6 +278,38 @@ private static VideoStats video(Map stats, Map qualityLimitation); } + private static AudioStats audio(Map stats, RTCStats entry, + Map values, Map pcm) { + RTCStats codec = lookup(stats, values.get("codecId")); + String trackId = string(values.get("trackIdentifier")); + Double jitter = optionalDouble(values.get("jitter")); + Double level = optionalDouble(values.get("audioLevel")); + if (level != null && level > 1) { + level = null; + } + long[] observed = pcm.get(trackId); + return new AudioStats(entry.getId(), trackId, + codec == null ? null : string(codec.getAttributes().get("mimeType")), + optionalLong(values.get("packetsReceived")), + jitter == null ? null : Duration.ofNanos((long) (jitter * 1e9)), + optionalLong(values.get("concealedSamples")), optionalLong(values.get("concealmentEvents")), + level, optionalLong(values.get("totalSamplesReceived")), + observed == null ? null : observed[1], observed == null ? -1 : observed[0]); + } + + private static Long optionalLong(Object value) { + return value instanceof Number && ((Number) value).longValue() >= 0 + ? ((Number) value).longValue() : null; + } + + private static Double optionalDouble(Object value) { + if (!(value instanceof Number)) { + return null; + } + double number = ((Number) value).doubleValue(); + return Double.isFinite(number) && number >= 0 ? number : null; + } + private static RTCStats lookup(Map stats, Object id) { return id == null ? null : stats.get(id.toString()); } diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsSampler.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsSampler.java index 198e41a9d..bff95d536 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsSampler.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/StatsSampler.java @@ -69,7 +69,7 @@ private CompletableFuture report() { try { peer.connection().getStats(report -> peer.control.execute(() -> { try { - result.complete(reader.read(report)); + result.complete(reader.read(report, peer.pcmObservations())); } catch (Throwable e) { result.completeExceptionally(e); diff --git a/webrtc/src/main/java/dev/onvoid/webrtc/client/TransportStats.java b/webrtc/src/main/java/dev/onvoid/webrtc/client/TransportStats.java index 766069be9..de1ffa20f 100644 --- a/webrtc/src/main/java/dev/onvoid/webrtc/client/TransportStats.java +++ b/webrtc/src/main/java/dev/onvoid/webrtc/client/TransportStats.java @@ -25,6 +25,8 @@ */ public final class TransportStats { + private final boolean relayed; + private final String id; private final String selectedCandidatePairId; @@ -39,7 +41,8 @@ public final class TransportStats { TransportStats(String id, String selectedCandidatePairId, String candidateType, - Duration roundTripTime, long bytesSent, long bytesReceived) { + Duration roundTripTime, long bytesSent, long bytesReceived, boolean relayed) { + this.relayed = relayed; this.id = id; this.selectedCandidatePairId = selectedCandidatePairId; this.candidateType = candidateType; @@ -93,6 +96,11 @@ public long bytesReceived() { return bytesReceived; } + /** @return whether either selected candidate is a TURN relay. */ + public boolean relayed() { + return relayed; + } + @Override public String toString() { return "TransportStats[" + id + ", candidate=" + candidateType + ", rtt=" + roundTripTime diff --git a/webrtc/src/test/java/dev/onvoid/webrtc/client/DiagnosticsTests.java b/webrtc/src/test/java/dev/onvoid/webrtc/client/DiagnosticsTests.java new file mode 100644 index 000000000..3564d01a0 --- /dev/null +++ b/webrtc/src/test/java/dev/onvoid/webrtc/client/DiagnosticsTests.java @@ -0,0 +1,336 @@ +/* + * Copyright 2026 Alex Andres + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package dev.onvoid.webrtc.client; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.lang.reflect.Proxy; +import java.time.Duration; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.atomic.AtomicReference; +import java.util.function.Consumer; + +import dev.onvoid.webrtc.RTCStats; +import dev.onvoid.webrtc.RTCStatsReport; +import dev.onvoid.webrtc.RTCStatsType; +import org.junit.jupiter.api.Test; + +class DiagnosticsTests { + private final StatsReader reader = new StatsReader(); + + @Test + void cpuNeedsTimeAndSamples() { + assertEquals(PeerDiagnostics.State.OBSERVING, cpu(0, "cpu").diagnostics().state()); + assertFalse(has(cpu(1, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + assertFalse(has(cpu(2, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + PeerStats limited = cpu(3, "cpu"); + assertTrue(has(limited, QualityCondition.VIDEO_CPU_LIMITED)); + assertEquals(PeerDiagnostics.State.DEGRADED, limited.diagnostics().state()); + assertTrue(has(cpu(4, "none"), QualityCondition.VIDEO_CPU_LIMITED)); + assertTrue(has(cpu(5, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + cpu(6, "none"); + cpu(7, "none"); + assertTrue(has(cpu(8, "none"), QualityCondition.VIDEO_CPU_LIMITED)); + assertFalse(has(cpu(9, "none"), QualityCondition.VIDEO_CPU_LIMITED)); + assertEquals(PeerDiagnostics.State.HEALTHY, cpu(10, "none").diagnostics().state()); + assertTrue(has(limited, QualityCondition.VIDEO_CPU_LIMITED)); + assertThrows(UnsupportedOperationException.class, () -> limited.diagnostics().conditions().clear()); + } + + @Test + void duplicatesDoNotConfirm() { + cpu(0, "cpu"); + cpu(3, "cpu"); + for (int i = 0; i < 10; i++) { + assertFalse(has(cpu(3, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + } + assertTrue(has(cpu(4, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + } + + @Test + void discontinuitiesDiscardEvidence() { + cpu(0, "cpu"); + cpu(1, "cpu"); + assertFalse(has(cpu(10, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + cpu(11, "cpu"); + assertTrue(has(cpu(13, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + assertEquals(PeerDiagnostics.State.OBSERVING, cpu(2, "cpu").diagnostics().state()); + assertFalse(has(cpu(3, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + } + + @Test + void streamsNeedOwnEvidence() { + for (int second = 0; second <= 6; second++) { + Map map = new HashMap<>(); + add(map, RTCStatsType.OUTBOUND_RTP, "v1", "kind", "video", + "qualityLimitationReason", second % 2 == 0 ? "cpu" : "none"); + add(map, RTCStatsType.OUTBOUND_RTP, "v2", "kind", "video", + "qualityLimitationReason", second % 2 == 1 ? "cpu" : "none"); + assertFalse(has(read(second, map), QualityCondition.VIDEO_CPU_LIMITED)); + } + } + + @Test + void missingVideoIsUnknown() { + for (int second = 0; second <= 3; second++) cpu(second, "cpu"); + assertFalse(has(read(4, new HashMap<>()), QualityCondition.VIDEO_CPU_LIMITED)); + assertFalse(has(cpu(5, "cpu"), QualityCondition.VIDEO_CPU_LIMITED)); + } + + @Test + void lossHasHysteresis() { + loss(0, 0, 0); + for (int second = 1; second <= 3; second++) { + assertFalse(has(loss(second, second * 90, second * 10), QualityCondition.HIGH_PACKET_LOSS)); + } + PeerStats high = loss(4, 360, 40); + assertTrue(has(high, QualityCondition.HIGH_PACKET_LOSS)); + assertTrue(has(loss(5, 457, 43), QualityCondition.HIGH_PACKET_LOSS)); // 3% dead band + assertTrue(has(loss(6, 555, 45), QualityCondition.HIGH_PACKET_LOSS)); // 2% starts recovery + loss(7, 653, 47); + loss(8, 751, 49); + PeerStats recovered = loss(9, 849, 51); + assertFalse(has(recovered, QualityCondition.HIGH_PACKET_LOSS)); + assertEquals(PeerDiagnostics.State.HEALTHY, recovered.diagnostics().state()); + } + + @Test + void turnChecksBothEndsAndAllTransports() { + Map map = new HashMap<>(); + add(map, RTCStatsType.TRANSPORT, "transport", "selectedCandidatePairId", "pair"); + add(map, RTCStatsType.CANDIDATE_PAIR, "pair", "localCandidateId", "local", "remoteCandidateId", "remote"); + add(map, RTCStatsType.LOCAL_CANDIDATE, "local", "candidateType", "host"); + add(map, RTCStatsType.REMOTE_CANDIDATE, "remote", "candidateType", "relay"); + add(map, RTCStatsType.TRANSPORT, "primary", "selectedCandidatePairId", "direct-pair", "bytesSent", 10000L); + add(map, RTCStatsType.CANDIDATE_PAIR, "direct-pair", "localCandidateId", "direct"); + add(map, RTCStatsType.LOCAL_CANDIDATE, "direct", "candidateType", "host"); + PeerStats first = read(0, map); + assertFalse(first.transport().relayed()); + assertTrue(first.transports().stream().anyMatch(TransportStats::relayed)); + assertTrue(has(first, QualityCondition.TURN_IN_USE)); + assertEquals(PeerDiagnostics.State.OBSERVING, first.diagnostics().state()); + assertEquals(PeerDiagnostics.State.HEALTHY, read(1, map).diagnostics().state()); + add(map, RTCStatsType.REMOTE_CANDIDATE, "remote", "candidateType", "host"); + assertFalse(has(read(2, map), QualityCondition.TURN_IN_USE)); + add(map, RTCStatsType.LOCAL_CANDIDATE, "local", "candidateType", "relay"); + assertTrue(has(read(3, map), QualityCondition.TURN_IN_USE)); + } + + @Test + void audioIsTypedAndOptional() { + Map map = audio(20); + map.get("audio").getAttributes().putAll(attributes("jitter", 0.012, + "concealedSamples", 42L, "concealmentEvents", 2L, "audioLevel", 0.25, + "totalSamplesReceived", 48000L, "codecId", "codec")); + add(map, RTCStatsType.CODEC, "codec", "mimeType", "audio/opus"); + add(map, RTCStatsType.MEDIA_SOURCE, "source", "audioLevel", 0.5, "trackIdentifier", "microphone"); + add(map, RTCStatsType.OUTBOUND_RTP, "out", "kind", "audio", "mediaSourceId", "source", "codecId", "codec"); + PeerStats sample = read(0, map); + AudioStats audio = sample.inboundAudio().get(0); + assertEquals(Duration.ofMillis(12), audio.jitter()); + assertEquals(42L, audio.concealedSamples()); + assertEquals(2L, audio.concealmentEvents()); + assertEquals(0.25, audio.audioLevel()); + assertEquals(48000L, audio.totalSamplesReceived()); + assertEquals("audio/opus", audio.codec()); + assertNull(audio.pcmCallbacks()); + assertEquals(0.5, sample.outboundAudio().get(0).audioLevel()); + assertEquals("microphone", sample.outboundAudio().get(0).trackId()); + AudioStats missing = read(1, audio(30)).inboundAudio().get(0); + assertNull(missing.jitter()); + assertNull(missing.concealedSamples()); + assertNull(missing.audioLevel()); + assertThrows(UnsupportedOperationException.class, () -> sample.inboundAudio().clear()); + } + + @Test + void pcmRequiresObservedSinkAndPackets() { + for (int second = 0; second < 8; second++) { + assertFalse(has(read(second, audio(second * 10)), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + for (int second = 8; second <= 11; second++) { + assertFalse(has(pcm(second, second * 10, 0, 1), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + assertTrue(has(pcm(12, 120, 0, 1), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + pcm(13, 130, 1, 1); + pcm(14, 140, 2, 1); + pcm(15, 150, 3, 1); + assertFalse(has(pcm(16, 160, 4, 1), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + + @Test + void silenceAndAbsentPacketsAreNotPcmFailures() { + for (int second = 0; second <= 6; second++) { + assertFalse(has(pcm(second, 100, 0, 1), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + for (int second = 7; second <= 12; second++) { + Map map = audio(second * 100); + map.get("audio").getAttributes().put("audioLevel", 0.0); + PeerStats stats = reader.read(report(second, map), Collections.singletonMap("track", new long[] { 1, second })); + assertFalse(has(stats, QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + } + + @Test + void pcmCounterAndSinkChangesResetWindow() { + for (int second = 0; second <= 4; second++) pcm(second, 10 * second, 10, 1); + assertFalse(has(pcm(5, 50, 0, 1), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + pcm(6, 60, 0, 1); + pcm(7, 70, 0, 1); + assertFalse(has(pcm(8, 80, 0, 2), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + pcm(9, 90, 0, 2); + pcm(10, 100, 0, 2); + assertFalse(has(pcm(11, 1, 0, 2), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + assertFalse(has(read(12, audio(11)), QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); + } + + @Test + void exportExcludesSensitiveAttributes() { + Map map = audio(100); + map.get("audio").getAttributes().putAll(attributes("codecId", "codec", + "address", "192.0.2.123", "url", "turn:secret.example", "username", "private-user", + "credential", "private-password", "sdp", "private-sdp", "certificate", "private-cert")); + add(map, RTCStatsType.CODEC, "codec", "mimeType", "audio/opus", "sdpFmtpLine", "private-fmtp"); + PeerStats sample = read(0, map); + String json = sample.diagnosticReport(); + assertInstanceOf(Map.class, Json.parse(json)); + assertTrue(json.contains("audio/opus")); + assertTrue(json.contains("javaVersion")); + assertTrue(json.contains("architecture")); + for (String secret : new String[] { "192.0.2.123", "secret.example", "private-", "track", "codecId" }) { + assertFalse(json.contains(secret), json); + } + map.get("codec").getAttributes().put("mimeType", "audio/private-codec"); + assertFalse(read(1, map).diagnosticReport().contains("private-codec")); + } + + @Test + @SuppressWarnings("unchecked") + void exportIncludesExplanationAndPolicy() { + cpu(0, "cpu"); + cpu(1, "cpu"); + cpu(2, "cpu"); + Map report = (Map) Json.parse(cpu(3, "cpu").diagnosticReport()); + Map policy = (Map) report.get("policy"); + assertEquals(3.0, policy.get("confirmationSeconds")); + assertEquals(3.0, policy.get("minimumMeasurements")); + assertEquals(0.05, policy.get("lossEnter")); + assertEquals(0.02, policy.get("lossExit")); + List> causes = (List>) report.get("conditions"); + assertEquals("VIDEO_CPU_LIMITED", causes.get(0).get("cause")); + assertEquals(QualityCondition.VIDEO_CPU_LIMITED.reason(), causes.get(0).get("reason")); + assertEquals(true, causes.get(0).get("problem")); + } + + @Test + void exportHandlesNonFiniteMeasurements() { + Map map = audio(10); + map.get("audio").getAttributes().putAll(attributes("jitter", Double.NaN, "audioLevel", Double.POSITIVE_INFINITY)); + add(map, RTCStatsType.OUTBOUND_RTP, "video", "kind", "video", "framesPerSecond", Double.NaN); + PeerStats sample = read(0, map); + assertNull(sample.inboundAudio().get(0).jitter()); + assertNull(sample.inboundAudio().get(0).audioLevel()); + assertInstanceOf(Map.class, Json.parse(sample.diagnosticReport())); + } + + @Test + @SuppressWarnings("unchecked") + void eventsOnlyOnChangesAndCanStop() throws Throwable { + AtomicReference> callback = new AtomicReference<>(); + boolean[] stopped = { false }; + // Exercise the default API without a native connection or timer. + Peer peer = (Peer) Proxy.newProxyInstance(Peer.class.getClassLoader(), new Class[] { Peer.class }, + (proxy, method, args) -> { + if (method.getName().equals("onStats")) { + callback.set((Consumer) args[1]); + return (Subscription) () -> stopped[0] = true; + } + if (method.isDefault()) { + return java.lang.invoke.MethodHandles.privateLookupIn(Peer.class, java.lang.invoke.MethodHandles.lookup()) + .unreflectSpecial(method, Peer.class).bindTo(proxy).invokeWithArguments(args); + } + throw new UnsupportedOperationException(method.getName()); + }); + List events = new ArrayList<>(); + Subscription subscription = peer.onDiagnostics(Duration.ofSeconds(1), events::add); + callback.get().accept(cpu(0, "cpu")); + callback.get().accept(cpu(1, "cpu")); + callback.get().accept(cpu(2, "cpu")); + callback.get().accept(cpu(3, "cpu")); + callback.get().accept(cpu(4, "cpu")); + assertEquals(3, events.size()); // observing, healthy, degraded + assertEquals(PeerDiagnostics.State.DEGRADED, events.get(2).state()); + subscription.close(); + assertTrue(stopped[0]); + } + + private PeerStats cpu(int second, String reason) { + Map map = new HashMap<>(); + add(map, RTCStatsType.OUTBOUND_RTP, "video", "kind", "video", "qualityLimitationReason", reason); + return read(second, map); + } + + private PeerStats loss(int second, long received, long lost) { + Map map = audio(received); + map.get("audio").getAttributes().put("packetsLost", lost); + return read(second, map); + } + + private PeerStats pcm(int second, long packets, long callbacks, long generation) { + return reader.read(report(second, audio(packets)), + Collections.singletonMap("track", new long[] { generation, callbacks })); + } + + private PeerStats read(int second, Map map) { + return reader.read(report(second, map)); + } + + private static RTCStatsReport report(int second, Map map) { + return new RTCStatsReport(map, 1_700_000_000_000_000L + second * 1_000_000L) { }; + } + + private static Map audio(long received) { + Map map = new HashMap<>(); + add(map, RTCStatsType.INBOUND_RTP, "audio", "kind", "audio", "trackIdentifier", "track", + "packetsReceived", received, "packetsLost", 0L); + return map; + } + + private static boolean has(PeerStats stats, QualityCondition cause) { + return stats.diagnostics().conditions().contains(cause); + } + + private static void add(Map map, RTCStatsType type, String id, Object... values) { + map.put(id, new RTCStats(0, type, id, attributes(values)) { }); + } + + private static Map attributes(Object... values) { + Map map = new HashMap<>(); + for (int i = 0; i < values.length; i += 2) map.put((String) values[i], values[i + 1]); + return map; + } +} diff --git a/webrtc/src/test/java/dev/onvoid/webrtc/client/DispatcherTests.java b/webrtc/src/test/java/dev/onvoid/webrtc/client/DispatcherTests.java index 75e9d7780..e3f3eb882 100644 --- a/webrtc/src/test/java/dev/onvoid/webrtc/client/DispatcherTests.java +++ b/webrtc/src/test/java/dev/onvoid/webrtc/client/DispatcherTests.java @@ -53,6 +53,23 @@ class DispatcherTests { private final Workload work = new Workload(() -> now); + @Test + void pcmCountsBeforeHandler() { + List heard = new ArrayList<>(); + AudioDispatcher audio = audio(Duration.ofMillis(10), chunk -> { + if (chunk.frames() > 0) heard.add(number(chunk)); + }); + push(audio, 1); + push(audio, 2); + assertEquals(2, audio.pcmCallbacks()); + assertTrue(heard.isEmpty()); + audio.onData(new byte[0], 16, 48000, 1, 0); + audio.onData(new byte[0], 8, 48000, 1, 1); + assertEquals(2, audio.pcmCallbacks()); + pool.runAll(); + assertTrue(errors.isEmpty()); + } + @Test void audioBufferDropsOldest() { List heard = new ArrayList<>(); diff --git a/webrtc/src/test/java/dev/onvoid/webrtc/client/MediaTests.java b/webrtc/src/test/java/dev/onvoid/webrtc/client/MediaTests.java index a91a84d8c..37dd7bb24 100644 --- a/webrtc/src/test/java/dev/onvoid/webrtc/client/MediaTests.java +++ b/webrtc/src/test/java/dev/onvoid/webrtc/client/MediaTests.java @@ -17,7 +17,9 @@ package dev.onvoid.webrtc.client; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; @@ -76,7 +78,16 @@ void pushedAudioArrives() throws Exception { CustomAudio voice = rtc.media().customAudio("voice"); CompletableFuture heard = new CompletableFuture<>(); - peers[1].onTrack(track -> track.asAudio().onAudio(heard::complete)); + peers[1].onTrack(track -> track.asAudio().onAudio(chunk -> { + // The playout clock can deliver silence before any RTP arrives. + // A non-silent chunk confirms that the pushed tone was decoded. + for (byte sample : chunk.samples()) { + if (sample != 0) { + heard.complete(chunk); + break; + } + } + })); peers[0].addTrack(voice); // One second in one go; the track sends it in 10 ms chunks. @@ -87,6 +98,13 @@ void pushedAudioArrives() throws Exception { assertTrue(chunk.frames() > 0); assertEquals(chunk.frames() * chunk.channels() * 2, chunk.samples().length); + PeerStats received = await(peers[1].stats()); + assertEquals(1, received.inboundAudio().size(), () -> received.report().toString()); + AudioStats audioStats = received.inboundAudio().get(0); + assertNotNull(audioStats.pcmCallbacks()); + assertTrue(audioStats.pcmCallbacks() > 0); + assertNotNull(audioStats.codec()); + assertFalse(received.diagnostics().conditions().contains(QualityCondition.AUDIO_PACKETS_WITHOUT_PCM)); } @Test