Media
This guide gives an overview of the supported video and audio formats, describes how to control audio, get information about available web cameras and microphones, etc.
Codecs
Google Chrome and Chromium differ in several ways, including the sets of audio and video codecs they support.
The table below displays which codecs are supported by the codebase of corresponding browsers.
Chromium | Google Chrome | |
---|---|---|
AAC | yes | |
AV1 | yes | yes |
FLAC | yes | yes |
H.264 | yes | |
HEVC | yes | |
MP3 | yes | yes |
Opus | yes | yes |
Theora | yes | yes |
Vorbis | yes | yes |
VP8 | yes | yes |
VP9 | yes | yes |
WAV | yes | yes |
As you may see, Google Chrome supports certain codecs that Chromium does not. The reason is that these codecs are proprietary and cannot be used in an open-source or a commercial project without obtaining licenses from corresponding patent holders.
Different codecs have different patent holders. For example, to use H.264, companies must acquire the license from Via LA company. You can read more about their license terms on the website.
Proprietary codecs
Patent holders do not license codecs to the software that represents only a part of the final product deployed to the end users, e.g. libraries like JxBrowser.
In order to support proprietary codecs in your products, you need to acquire appropriate licenses and enable the following proprietary features:
Engine engine = Engine.newInstance(
EngineOptions.newBuilder(renderingMode)
.enableProprietaryFeature(ProprietaryFeature.AAC)
.enableProprietaryFeature(ProprietaryFeature.H_264)
.enableProprietaryFeature(ProprietaryFeature.HEVC)
.build());
val engine = Engine(renderingMode) {
proprietaryFeatures = setOf(
ProprietaryFeature.AAC,
ProprietaryFeature.H_264,
ProprietaryFeature.HEVC
)
}
With the license and the enabled proprietary features you will be able to load web pages with the AAC, HEVC, and H.264 formats, and play audio and video files, just like in Google Chrome. By default, the proprietary codecs are disabled.
Important: The H.264, HEVC, and AAC codecs are the proprietary components. By enabling these codecs you state that you are aware that H.264, HEVC, and AAC are the proprietary components, and you should have a license in order to use them. For more information, you could contact patent holders: Via LA Licensing. TeamDev shall not be responsible for your use of the H.264, HEVC, and AAC codecs.
Video
JxBrowser fully supports HTML5 <video>
element and can play video in the supported formats.
If the library cannot play a video, or a video format is unsupported, JxBrowser suggests to download the video file. Please see Downloads for guidance on managing downloads.
Audio
Controlling audio
Using Audio
you can find out whether audio is playing on the loaded web page:
boolean audioPlaying = audio.isPlaying();
val audioPlaying = audio.isPlaying
You can mute or unmute audio on the loaded web page if required:
audio.mute();
audio.unmute();
audio.mute()
audio.unmute()
To check whether audio is muted use the following code:
boolean audioMuted = audio.isMuted();
val audioMuted = audio.isMuted
Audio events
To find out whether audio has started/stopped playing on the loaded web page you can subscribe to the following events:
browser.audio().on(AudioStartedPlaying.class, event -> {});
browser.audio().on(AudioStoppedPlaying.class, event -> {});
val audio = browser.audio
audio.subscribe<AudioStartedPlaying> { event -> }
audio.subscribe<AudioStoppedPlaying> { event -> }
DRM
Widevine
The web services like Netflix or Amazon Prime use Widevine to distribute their DRM-encoded content. Widevine is a Google proprietary component that is disabled by default. In order to enable it and play the DRM-encoded content, use the following approach:
Engine engine = Engine.newInstance(
EngineOptions.newBuilder(renderingMode)
.enableProprietaryFeature(ProprietaryFeature.WIDEVINE)
.build());
val engine = Engine(renderingMode) {
proprietaryFeatures = setOf(ProprietaryFeature.WIDEVINE)
}
The Chromium version used by the library supports Widevine on the Windows and macOS platforms only. It is not supported on Linux. As soon as Chromium enables support of Widevine on Linux, we enable it in JxBrowser as well.
Important: Widevine is a Google proprietary component, governed by its own terms of use. For more information, see https://www.widevine.com/.
Camera & microphone
JxBrowser supports web camera and microphone.
You can get information about all available media stream devices using the following code:
MediaDevices mediaDevices = engine.mediaDevices();
// Get all available video devices, e.g. web cameras.
List<MediaDevice> videoDevices = mediaDevices.list(MediaDeviceType.VIDEO_DEVICE);
// Get all available audio devices, e.g. microphones.
List<MediaDevice> audioDevices = mediaDevices.list(MediaDeviceType.AUDIO_DEVICE);
val mediaDevices = engine.mediaDevices
// Get all available video devices, e.g. web cameras.
val videoDevices = mediaDevices.list(MediaDeviceType.VIDEO_DEVICE)
// Get all available audio devices, e.g. microphones.
val audioDevices = mediaDevices.list(MediaDeviceType.AUDIO_DEVICE)
You can detect when a media capturing starts or stops using these events:
browser.on(MediaStreamCaptureStarted.class, e -> {
System.out.println("Started capturing " + e.mediaStreamType());
});
browser.on(MediaStreamCaptureStopped.class, e -> {
System.out.println("Stopped capturing " + e.mediaStreamType());
});
browser.subscribe<MediaStreamCaptureStarted> { event ->
println("Started capturing ${event.mediaStreamType()}")
}
browser.subscribe<MediaStreamCaptureStopped> { event ->
println("Stopped capturing ${event.mediaStreamType()}")
}
Selecting media device
When a web page wants to use a web camera or a microphone, you can use SelectMediaDeviceCallback
to tell the web page
which device to use.
The following example demonstrates how to select the first device from the list of available devices:
mediaDevices.set(SelectMediaDeviceCallback.class, params ->
Response.select(params.mediaDevices().get(0)));
mediaDevices.register(SelectMediaDeviceCallback { params ->
val firstDevice = params.mediaDevices().first()
Response.select(firstDevice)
})
The callback will not be invoked if there are no media input devices of the requested type.
To disable access to microphones and web cameras, use RequestPermissionCallback
as shown below:
engine.permissions().set(RequestPermissionCallback.class, (params, tell) -> {
PermissionType type = params.permissionType();
if (type == PermissionType.VIDEO_CAPTURE || type == PermissionType.AUDIO_CAPTURE) {
tell.deny();
} else {
tell.grant();
}
});
engine.permissions.register(RequestPermissionCallback { params, tell ->
val type = params.permissionType()
if (type == PermissionType.VIDEO_CAPTURE || type == PermissionType.AUDIO_CAPTURE) {
tell.deny()
} else {
tell.grant()
}
})
Casting
Chromium has built-in functionality that allows casting media content to devices supporting different wireless technologies such as Chromecast, Miracast, DLNA, AirPlay, or similar. It can be smart TVs, projectors, and other devices.
Preliminary step
By default, we disable Chromium from scanning your network for media devices. To enable it and let Chromium find the potential receivers, use the engine option:
EngineOptions options = EngineOptions.newBuilder(renderingMode)
.enableMediaRouting()
.build();
Engine engine = Engine.newInstance(options);
val engine = Engine(renderingMode) {
mediaRoutingEnabled = true
}
Media receivers
To start casting media content to a receiver, you need to get one. For this purpose, JxBrowser provides a separate
profile service MediaReceivers
which can be obtained this way:
MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
val mediaReceivers = profile.mediaCasting().mediaReceivers()
To understand when a new receiver has been discovered, JxBrowser provides the MediaReceiverDiscovered
event:
MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
mediaReceivers.on(MediaReceiverDiscovered.class, event -> {
MediaReceiver receiver = event.mediaReceiver();
});
val mediaReceivers = profile.mediaCasting().mediaReceivers()
mediaReceivers.subscribe<MediaReceiverDiscovered> { event ->
val receiver = event.mediaReceiver()
}
For your convenience, JxBrowser keeps track of the discovered receivers. If you want to get the list of currently
discovered media receivers, use the MediaReceivers.list()
method:
MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
List<MediaReceiver> receivers = mediaReceivers.list();
val mediaReceivers = profile.mediaCasting().mediaReceivers()
val receivers = mediaReceivers.list()
If you look for a specific receiver, you can obtain it via the MediaReceivers.await(Predicate<MediaReceiver>)
convenience method. It waits until the first receiver matching the predicate is discovered and returns it.
MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
MediaReceiver receiver = mediaReceivers.await(it -> it.name().equals("Samsung Smart TV"));
val mediaReceivers = profile.mediaCasting().mediaReceivers()
val receiver = mediaReceivers.await { it.name() == "Samsung Smart TV" }
To detect that the media receiver has been disconnected, i.e. unplugged or disconnected from the network, use
the MediaReceiverDisconnected
event:
receiver.on(MediaReceiverDisconnected.class, event -> {
MediaReceiver mediaReceiver = event.mediaReceiver();
});
receiver.subscribe<MediaReceiverDisconnected> { event ->
val mediaReceiver = event.mediaReceiver()
}
Casting content
JxBrowser API allows casting content of browsers, screens and presentation using the JavaScript Presentation API.
Media receivers can support different media sources. A media source represents a type of content that can be cast to a media receiver. Before you start casting, please make sure that the selected media receiver supports the corresponding media source.
Casting a browser
To start casting browser content, use the Browser.cast(MediaReceiver)
method:
MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
if (receiver.supports(MediaSource.browser())) {
CompletableFuture<CastSession> future = browser.cast(receiver);
}
val receiver = mediaReceivers.await { it.name().contains("Samsung") }
if (receiver.supports(MediaSource.browser())) {
val future: CompletableFuture<CastSession> = browser.cast(receiver)
}
Each session of casting media content to a media receiver is represented by the instance of the CastSession
class.
Default presentation request
If the web page contains the
default PresentationRequest
,
the browser starts casting the content specified in this request instead of the browser’s content.
To check if the browser contains the default PresentationRequest
, use:
MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung Smart TV"));
browser.defaultPresentationRequest().ifPresent(request -> {
if (receiver.supports(request)) {
CompletableFuture<CastSession> future = browser.cast(receiver);
}
});
val receiver = mediaReceivers.await { it.name().contains("Samsung") }
browser.defaultPresentationRequest().ifPresent { request ->
if (receiver.supports(request)) {
val future = browser.cast(receiver)
}
}
Casting a screen
To start casting the screen content, use Browser.castScreen(MediaReceiver)
. This method will show a standard Chromium
dialog for picking the screen to cast.
MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
if (receiver.supports(MediaSource.screen())) {
CompletableFuture<CastSession> future = browser.castScreen(receiver);
}
val receiver = mediaReceivers.await { it.name().contains("Samsung") }
if (receiver.supports(MediaSource.screen())) {
val future: CompletableFuture<CastSession> =
browser.castScreen(receiver)
}
If you want to select the screen programmatically, use the Browser.castScreen(MediaReceiver, ScreenCastOptions)
method. Find the screen you need using the Screens
service:
MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
Screen screen = profile.mediaCasting().screens().defaultScreen();
ScreenCastOptions options = ScreenCastOptions.newBuilder(screen)
.withAudio()
.build();
if (receiver.supports(MediaSource.screen())) {
CompletableFuture<CastSession> future = browser.castScreen(receiver, options);
}
val receiver = mediaReceivers.await { it.name().contains("Samsung") }
val screen = profile.mediaCasting().screens().defaultScreen()
val options = ScreenCastOptions(screen, withAudio = true)
if (receiver.supports(MediaSource.screen())) {
val future: CompletableFuture<CastSession> =
browser.castScreen(receiver, options)
}
For now, Chromium supports audio casting only on Windows. So enabling it on macOS/Linux via
ScreenCastOptions.Builder.withAudio()
is a no-op. On Windows, when selecting the screen in the picker dialog,
Chromium provides a separate checkbox for selecting audio-casting.
Presentation API
JxBrowser allows working with JavaScript Presentation API.
When the PresentationRequest.start()
method is called on the JavaScript side, JxBrowser invokes StartPresentationCallback
where you can decide to start or
cancel the presentation.
To start the presentation to a receiver, use the StartPresentationCallback.Action.start(MediaReceiver)
method:
browser.set(StartPresentationCallback.class, (params, tell) -> {
MediaReceiver receiver = params.mediaReceivers().await(it -> {
return it.name().contains("Samsung");
});
if (receiver.supports(params.presentationRequest())) {
tell.start(receiver);
} else {
tell.cancel();
}
});
browser.register(StartPresentationCallback { params, tell ->
val receiver = params.mediaReceivers().await { it.name().contains("Samsung") }
if (receiver.supports(params.presentationRequest())) {
tell.start(receiver)
} else {
tell.cancel()
}
})
Discovering cast sessions
To get notified when a cast session has been discovered, JxBrowser provides the CastSessionDiscovered
event:
profile.mediaCasting().castSessions().on(CastSessionDiscovered.class, event -> {
CastSession castSession = event.castSession();
});
val castSessions = profile.mediaCasting().castSessions()
castSessions.subscribe<CastSessionDiscovered> { event ->
val castSession = event.castSession()
}
Chromium can discover sessions started by other applications or instances of Chromium. To indicate that
the cast session has been started by this profile, JxBrowser provides the CastSession.isLocal()
method. So if a cast
session is started by another profile or even another Chromium process the method will return false
.
Stopping cast sessions
To stop a cast session, use the CastSession.stop()
method. If you want to get notified when a cast session has been
stopped, use the CastSessionStopped
event:
CastSession session = profile.mediaCasting().castSessions().list().get(0);
session.on(CastSessionStopped.class, event -> {
// Do something.
});
...
session.stop();
val session = profile.mediaCasting().castSessions().list().first()
session.subscribe<CastSessionStopped> { event ->
// Do something.
}
...
session.stop()
Session can be stopped by other applications or instances of Chromium, i.e. Google Chrome. In this case, the event will be invoked as well.
Failures
Sometimes, Chromium may fail to start a new cast session, i.e. if the media receiver is not found or had
suddenly disconnected. To detect that, use the CastSessionStartFailed
event:
MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
profile.mediaCasting().castSessions().on(CastSessionStartFailed.class, event -> {
System.out.println(event.errorMessage());
});
CompletableFuture<CastSession> future = browser.cast(receiver);
val receiver = mediaReceivers.await { it.name().contains("Samsung") }
val castSessions = profile.mediaCasting().castSessions()
castSessions.subscribe<CastSessionStartFailed> { event ->
println(event.errorMessage())
}
val future: CompletableFuture<CastSession> = browser.cast(receiver)
This is intentionally an event due to asynchronous nature of media casting.
Since Browser.cast...
methods return CompletableFuture
you can detect that the start of the cast session
has failed. In this case, JxBrowser completes the future with the CastStartFailedException
:
CompletableFuture<CastSession> future = browser.cast(receiver);
future.exceptionally(throwable -> {
System.out.println(throwable.getMessage());
return null;
});
val future: CompletableFuture<CastSession> = browser.cast(receiver)
future.exceptionally { throwable ->
println(throwable.message)
null
}