Audio capture
This guide describes how to capture browser audio, configure its format, receive audio samples, and manage capture sessions.
Since JxBrowser 9.6.0, you can capture all audio produced by a Browser as
uncompressed, 32-bit floating-point samples. You can use these samples for
audio processing, recording, or speech recognition.
Starting capture
To start capturing audio, use the Audio.startCapture() method:
var session = browser.audio().startCapture();
val session = browser.audio.startCapture()
The method returns an active AudioCaptureSession. By default, it captures
stereo audio at 48,000 Hz in buffers of 1,024 frames, and playback remains
audible. Each frame contains one sample per channel.
A browser can have only one active capture session. Calling startCapture()
while another session is active throws IllegalStateException.
Register the session’s event observers before the page starts playing to receive audio from its beginning. Buffers captured before an observer is registered are not delivered to that observer.
Configuring capture
To change the audio format, pass AudioCaptureOptions to startCapture().
The following example captures mono audio at 16,000 Hz in buffers of 160
frames (10 milliseconds) and mutes playback while capturing:
var options = AudioCaptureOptions.newBuilder()
.channelLayout(AudioChannelLayout.MONO)
.sampleRate(16_000)
.framesPerBuffer(160)
.muteSource()
.build();
var session = browser.audio().startCapture(options);
import com.teamdev.jxbrowser.dsl.media.AudioCaptureOptions
...
val options = AudioCaptureOptions(
channelLayout = AudioChannelLayout.MONO,
sampleRate = 16_000,
framesPerBuffer = 160,
muteSource = true
)
val session = browser.audio.startCapture(options)
Stop any active session before starting one with different options. You can
retrieve the options used by a session with session.options().
The sample rate must be from 3,000 to 768,000 Hz. The number of frames per
buffer must be positive, and the buffer duration must be from 10 to 500
milliseconds. Invalid values cause IllegalArgumentException when setting
the options or building them. startCapture(options) also throws
IllegalArgumentException if the options are not supported.
The channel layout defines the number and order of the channels:
| Layout | Channel order |
|---|---|
MONO | Front center |
STEREO | Front left, front right |
QUAD | Front left, front right, back left, back right |
SURROUND_5_1 | Front left, front right, front center, low-frequency effects, side left, side right |
SURROUND_7_1 | Front left, front right, front center, low-frequency effects, back left, back right, side left, side right |
Receiving audio data
Subscribe to AudioDataCaptured events on the capture session to receive
audio buffers:
session.on(AudioDataCaptured.class, event -> {
var channelCount = event.channelCount();
var frameCount = event.frameCount();
var samples = event.channelData(0);
var timestamp = event.timestamp();
// Process the samples of the first channel.
});
session.subscribe<AudioDataCaptured> { event ->
val channelCount = event.channelCount()
val frameCount = event.frameCount()
val samples = event.channelData(0)
val timestamp = event.timestamp()
// Process the samples of the first channel.
}
The samples are stored separately for each channel. channelData(index)
returns a new float[] array (FloatArray in Kotlin) containing
frameCount() samples. Channel indices start at zero and follow the order
of the configured channel layout.
The timestamp is a java.time.Duration measured from the first captured
frame of the session to the first frame of the buffer. Normally, consecutive
timestamps differ by the buffer duration. A larger difference indicates
that audio between the buffers was lost.
The session continues delivering buffers containing silence when the browser is not playing audio.
Muting playback
The muteSource option mutes playback only while the capture session is
active. It does not affect the captured samples or change the value
returned by Audio.isMuted().
Muting the browser with Audio.mute() also leaves the captured samples
unaffected. However, starting Chromium with the --mute-audio switch
causes the captured audio to be silent.
Stopping capture
When you no longer need audio data, stop the session:
session.stop();
session.stop()
Use session.isActive() to check whether the session is capturing audio.
Calling stop() on an inactive session has no effect. Closing the browser
stops its capture session automatically.
Once a session stops or fails, it discards buffers whose delivery has not
started. Registering an observer on that session throws
ObjectClosedException. To resume capturing, start a new session and
register its observers. Events from the previous session are not delivered
to the new session’s observers.
While capture is active, Chromium treats the browser as visible, so the
page is not throttled and its document.visibilityState is visible.
Capture also prevents the display from going to sleep. Stop the session
when you no longer need it.
Capture failures
To detect a capture failure, subscribe to AudioCaptureFailed on the active
session:
session.on(AudioCaptureFailed.class, event -> {
System.out.println(event.errorMessage());
});
session.subscribe<AudioCaptureFailed> { event ->
println(event.errorMessage())
}
The event provides a message describing the failure. A failed session is no longer active and cannot be reused.