Skip to content

feat: send device-captured and pushed audio from one factory - #571

Merged
devopvoid merged 1 commit into
feat/device-audio-sourcefrom
feat/mixed-audio-sources
Nov 8, 2026
Merged

devopvoid merged 1 commit into
feat/device-audio-sourcefrom
feat/mixed-audio-sources

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Stacked on #570.

Each audio sender now gets the audio of its own track, so a factory no longer commits to one kind of audio input: device-captured, pushed and forwarded audio can be sent side by side, also on one peer connection.

  • The proxy's capture opens the recording device only while a device source has a sink and WebRTC asks for a recording. WebRTC's own InitRecording/StartRecording/StopRecording no longer reach the device, so a factory sending only pushed audio never opens it. setAudioRecording(false) still stops it.
  • The device is started and stopped on the worker thread; sink changes post the update there instead of blocking on it.
  • Removes the factory's audio input mode guard and its JNI functions.
  • Client API: customAudio() and audioTrack(CustomAudioSource) work in AudioMode.DEVICES; CUSTOM now means the microphone is never opened.
  • Docs updated where they described the restriction.

Each audio sender now gets the audio of its own track, so the factory no
longer commits to one kind of audio input. The proxy's capture opens the
recording device only while a device source has a sink and WebRTC asks
for a recording; WebRTC's own recording calls no longer reach the
device. The client API allows customAudio() in AudioMode.DEVICES.
@devopvoid
devopvoid added this pull request to stack #574 November 8, 2026 15:24
@devopvoid
devopvoid merged commit c60a287 into main Nov 8, 2026
10 of 20 checks passed
@devopvoid
devopvoid deleted the feat/mixed-audio-sources branch November 8, 2026 15:26
devopvoid added a commit that referenced this pull request Nov 13, 2026
* docs: write the WHIP and WHEP page in shorter sentences

Same content: long sentences split, sequences as numbered lists, rules as bullet lists.

* docs: write the media files page in shorter sentences

Same content: platforms and measurements as tables, rules as lists, and the page names its client API counterpart.

* docs: write the end-to-end encryption page in shorter sentences

Same content: long sentences split, rules and failure cases as lists.

* docs: write the video codecs page in shorter sentences

Long sentences split. Also corrects which factories offer H.265: the hardware factories on every platform where the GPU handles it, not only on Macs.

* docs: write the media recording page in shorter sentences

Same content, with one point made exact: a video track without a key frame within three seconds starts the new segment at its next key frame.

* docs: write the statistics page in shorter sentences

Same content: long table cells and rules moved into lists below the tables.

* docs: write the sending media page in shorter sentences

Also corrects the audio of a media file: it is pushed audio, which works in every audio mode, not only CUSTOM and HEADLESS.

* docs: write the peers and signaling page in shorter sentences

Same content: recovery rules and JSON rules as lists.

* docs: write the encoded transforms page in shorter sentences

Same content: rules as lists, long paragraphs split.

* docs: write the core API page in shorter sentences

Same content: the rules of disposal as a list.

* docs: write the threads page in shorter sentences

Same content: the exceptions of the thread table as a list below it, and screens() and windows() named as asking the system on every call.

* docs: write the receiving media page in shorter sentences

Same content: the memory rules of the video views and the speech detection settings as lists.

* docs: write the client transforms page in shorter sentences

Same content, long paragraphs split.

* docs: write the client API overview in shorter sentences

Same content, long sentences split.

* docs: drop the outdated warning on injected audio

Since #571 every audio sender gets the audio of its own track, so captured audio no longer reaches an injecting sender.

* docs: write the system audio source page in shorter sentences

Same content: the client API first, the core API after it, and the rules of starting, stopping and feedback as lists.

* docs: write the media constraints page in shorter sentences

Same content: content hints and layers as lists.

* docs: write the custom video source page in shorter sentences

Same content, rules about buffers as a list.

* docs: write the native image page in shorter sentences

Same content, long paragraphs split.

* docs: write the peer connection configuration page in shorter sentences

Same content, long table cells and paragraphs split.

* docs: write the camera video page in shorter sentences

Same content, long paragraphs split.

* docs: write the custom audio source page in shorter sentences

Same content, long sentences split.

* docs: write the headless audio page in shorter sentences

Same content, the effects of the old capture path as a list.

* docs: write the file transfer page in shorter sentences

Same content, long paragraphs split.

* docs: write the release notes of the build page in shorter sentences

Same content, long paragraphs split.

* docs: split a long paragraph of the examples page

Same content.

* docs: write the speech detection rules of audio processing as a list

Same content.

* docs: write the client data channels page in shorter sentences

Same content, the receive limit and message previews as lists.

* docs: split a long paragraph of the core data channels page

Same content.

* docs: split the introduction of the media devices page

Same content.

* docs: split a long sentence of the field trials page

Same content.

* docs: split the macOS paragraph of the desktop video page

Same content.

* docs: write the adapter notes of voice activity detection as a list

Same content.

* docs: split a long sentence of the video buffer converter page

Same content.

* docs: split a paragraph of the sending media page

Same content.

* docs: correct device query failures on the client API design page

A failed device query, also for an unavailable audio backend, throws MediaException; it does not come back empty. Also names screens() and windows() as asking the system on every call, and lists the blocking calls below the thread table.

* docs: drop the conclusion sections

They restated the page. The advice two of them held stays as Best Practices, and the CPU note of audio processing joins its Best Practices.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant