Documentation

Contents

Introduction

Adding to project

Guides

Troubleshooting

Media

This guide gives an overview of the supported video and audio formats, describes how to control audio, get information about available web cameras and microphones, etc.

Codecs

Google Chrome and Chromium differ in several ways, including the sets of audio and video codecs they support.

The table below displays which codecs are supported by the codebase of corresponding browsers.

	Chromium	Google Chrome
AAC		yes
AV1	yes	yes
FLAC	yes	yes
H.264		yes
HEVC		yes
MP3	yes	yes
Opus	yes	yes
Theora	yes	yes
Vorbis	yes	yes
VP8	yes	yes
VP9	yes	yes
WAV	yes	yes

As you may see, Google Chrome supports certain codecs that Chromium does not. The reason is that these codecs are proprietary and cannot be used in an open-source or a commercial project without obtaining licenses from corresponding patent holders.

Different codecs have different patent holders. For example, to use H.264, companies must acquire the license from Via LA company. You can read more about their license terms on the website.

Proprietary codecs

Patent holders do not license codecs to the software that represents only a part of the final product deployed to the end users, e.g. libraries like JxBrowser.

In order to support proprietary codecs in your products, you need to acquire appropriate licenses and enable the following proprietary features:

Java
Kotlin

Engine engine = Engine.newInstance(
        EngineOptions.newBuilder(renderingMode)
                .enableProprietaryFeature(ProprietaryFeature.AAC)
                .enableProprietaryFeature(ProprietaryFeature.H_264)
                .enableProprietaryFeature(ProprietaryFeature.HEVC)
                .build());

val engine = Engine.newInstance(
    EngineOptions.newBuilder(renderingMode)
        .enableProprietaryFeature(ProprietaryFeature.AAC)
        .enableProprietaryFeature(ProprietaryFeature.H_264)
        .enableProprietaryFeature(ProprietaryFeature.HEVC)
        .build())

With the license and the enabled proprietary features you will be able to load web pages with the AAC, HEVC, and H.264 formats, and play audio and video files, just like in Google Chrome. By default, the proprietary codecs are disabled.

Important: The H.264, HEVC, and AAC codecs are the proprietary components. By enabling these codecs you state that you are aware that H.264, HEVC, and AAC are the proprietary components, and you should have a license in order to use them. For more information, you could contact patent holders: Via LA Licensing. TeamDev shall not be responsible for your use of the H.264, HEVC, and AAC codecs.

Video

JxBrowser fully supports HTML5 <video> element and can play video in the supported formats.

If the library cannot play a video, or a video format is unsupported, JxBrowser suggests to download the video file. Please see Downloads for guidance on managing downloads.

HTML5 Video

Audio

Controlling audio

Using Audio you can find out whether audio is playing on the loaded web page:

Java
Kotlin

boolean audioPlaying = audio.isPlaying();

val audioPlaying = audio.isPlaying()

You can mute or unmute audio on the loaded web page if required:

Java
Kotlin

audio.mute();
audio.unmute();

audio.mute()
audio.unmute()

To check whether audio is muted use the following code:

Java
Kotlin

boolean audioMuted = audio.isMuted();

val audioMuted = audio.isMuted()

Audio events

To find out whether audio has started/stopped playing on the loaded web page you can subscribe to the following events:

Java
Kotlin

browser.audio().on(AudioStartedPlaying.class, event -> {});
browser.audio().on(AudioStoppedPlaying.class, event -> {});

browser.audio().on(AudioStartedPlaying::class.java) { event -> }
browser.audio().on(AudioStoppedPlaying::class.java) { event -> }

DRM

Widevine

The web services like Netflix or Amazon Prime use Widevine to distribute their DRM-encoded content. Widevine is a Google proprietary component that is disabled by default. In order to enable it and play the DRM-encoded content, use the following approach:

Java
Kotlin

Engine engine = Engine.newInstance(
        EngineOptions.newBuilder(renderingMode)
                .enableProprietaryFeature(ProprietaryFeature.WIDEVINE)
                .build());

val engine = Engine.newInstance(
    EngineOptions.newBuilder(renderingMode)
        .enableProprietaryFeature(ProprietaryFeature.WIDEVINE)
        .build())

The Chromium version used by the library supports Widevine on the Windows and macOS platforms only. It is not supported on Linux. As soon as Chromium enables support of Widevine on Linux, we enable it in JxBrowser as well.

Important: Widevine is a Google proprietary component, governed by its own terms of use. For more information, see https://www.widevine.com/.

Camera & microphone

JxBrowser supports web camera and microphone.

You can get information about all available media stream devices using the following code:

Java
Kotlin

MediaDevices mediaDevices = engine.mediaDevices();

// Get all available video devices, e.g. web cameras.
List<MediaDevice> videoDevices = mediaDevices.list(MediaDeviceType.VIDEO_DEVICE);

// Get all available audio devices, e.g. microphones.
List<MediaDevice> audioDevices = mediaDevices.list(MediaDeviceType.AUDIO_DEVICE);

val mediaDevices = engine.mediaDevices()

// Get all available video devices, e.g. web cameras.
val videoDevices = mediaDevices.list(MediaDeviceType.VIDEO_DEVICE)

// Get all available audio devices, e.g. microphones.
val audioDevices = mediaDevices.list(MediaDeviceType.AUDIO_DEVICE)

You can detect when a media capturing starts or stops using these events:

Java
Kotlin

browser.on(MediaStreamCaptureStarted.class, e -> {
    System.out.println("Started capturing " + e.mediaStreamType());
});

browser.on(MediaStreamCaptureStopped.class, e -> {
    System.out.println("Stopped capturing " + e.mediaStreamType());
});

browser.on(MediaStreamCaptureStarted::class.java) { e ->
    println("Started capturing " + e.mediaStreamType())
}

browser.on(MediaStreamCaptureStopped::class.java) { e ->
    println("Stopped capturing " + e.mediaStreamType())
}

Selecting media device

When a web page wants to use a web camera or a microphone, you can use SelectMediaDeviceCallback to tell the web page which device to use.

The following example demonstrates how to select the first device from the list of available devices:

Java
Kotlin

mediaDevices.set(SelectMediaDeviceCallback.class, params ->
        Response.select(params.mediaDevices().get(0)));

mediaDevices.set(SelectMediaDeviceCallback::class.java,
    SelectMediaDeviceCallback { params ->
        Response.select(params.mediaDevices().first())
    }
)

The callback will not be invoked if there are no media input devices of the requested type.

To disable access to microphones and web cameras, use RequestPermissionCallback as shown below:

Java
Kotlin

engine.permissions().set(RequestPermissionCallback.class, (params, tell) -> {
    PermissionType type = params.permissionType();
    if (type == PermissionType.VIDEO_CAPTURE || type == PermissionType.AUDIO_CAPTURE) {
        tell.deny();
    } else {
        tell.grant();
    }
});

engine.permissions().set(RequestPermissionCallback::class.java,
    RequestPermissionCallback { params, tell ->
        val type = params.permissionType()
        if (type == PermissionType.VIDEO_CAPTURE || type == PermissionType.AUDIO_CAPTURE) {
            tell.deny()
        } else {
            tell.grant()
        }
    }
)

Casting

Chromium has built-in functionality that allows casting media content to devices supporting different wireless technologies such as Chromecast, Miracast, DLNA, AirPlay, or similar. It can be smart TVs, projectors, and other devices.

The Cast Diagram

Preliminary step

By default, we disable Chromium from scanning your network for media devices. To enable it and let Chromium find the potential receivers, use the engine option:

Java
Kotlin

EngineOptions options = EngineOptions.newBuilder(renderingMode)
        .enableMediaRouting()
        .build();
Engine engine = Engine.newInstance(options);

val options = EngineOptions.newBuilder(renderingMode)
    .enableMediaRouting()
    .build()
val engine = Engine.newInstance(options)

Media receivers

To start casting media content to a receiver, you need to get one. For this purpose, JxBrowser provides a separate profile service MediaReceivers which can be obtained this way:

Java
Kotlin

MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();

val mediaReceivers = profile.mediaCasting().mediaReceivers()

To understand when a new receiver has been discovered, JxBrowser provides the MediaReceiverDiscovered event:

Java
Kotlin

MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
mediaReceivers.on(MediaReceiverDiscovered.class, event -> {
    MediaReceiver receiver = event.mediaReceiver();
});

val mediaReceivers = profile.mediaCasting().mediaReceivers()
mediaReceivers.on(MediaReceiverDiscovered::class.java) { event ->
    val receiver = event.mediaReceiver()
}

For your convenience, JxBrowser keeps track of the discovered receivers. If you want to get the list of currently discovered media receivers, use the MediaReceivers.list() method:

Java
Kotlin

MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
List<MediaReceiver> receivers = mediaReceivers.list();

val mediaReceivers = profile.mediaCasting().mediaReceivers()
val receivers = mediaReceivers.list()

If you look for a specific receiver, you can obtain it via the MediaReceivers.await(Predicate<MediaReceiver>) convenience method. It waits until the first receiver matching the predicate is discovered and returns it.

Java
Kotlin

MediaReceivers mediaReceivers = profile.mediaCasting().mediaReceivers();
MediaReceiver receiver = mediaReceivers.await(it -> it.name().equals("Samsung Smart TV"));

val mediaReceivers = profile.mediaCasting().mediaReceivers()
val receiver = mediaReceivers.await { it.name() == "Samsung Smart TV" }

To detect that the media receiver has been disconnected, i.e. unplugged or disconnected from the network, use the MediaReceiverDisconnected event:

Java
Kotlin

receiver.on(MediaReceiverDisconnected.class, event -> {
    MediaReceiver mediaReceiver = event.mediaReceiver();
});

receiver.on(MediaReceiverDisconnected::class.java) { event ->
    val mediaReceiver = event.mediaReceiver()
}

Casting content

JxBrowser API allows casting content of browsers, screens and presentation using the JavaScript Presentation API.

Media receivers can support different media sources. A media source represents a type of content that can be cast to a media receiver. Before you start casting, please make sure that the selected media receiver supports the corresponding media source.

Casting a browser

To start casting browser content, use the Browser.cast(MediaReceiver) method:

Java
Kotlin

MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
if (receiver.supports(MediaSource.browser())) {
    CompletableFuture<CastSession> future = browser.cast(receiver);
}

val receiver = mediaReceivers.await { it.name().contains("Samsung") }
if (receiver.supports(MediaSource.browser())) {
    val future: CompletableFuture<CastSession> = browser.cast(receiver)
}

Each session of casting media content to a media receiver is represented by the instance of the CastSession class.

Default presentation request

If the web page contains the default PresentationRequest, the browser starts casting the content specified in this request instead of the browser’s content.

To check if the browser contains the default PresentationRequest, use:

Java
Kotlin

MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung Smart TV"));
browser.defaultPresentationRequest().ifPresent(request -> {
    if (receiver.supports(request)) {
        CompletableFuture<CastSession> future = browser.cast(receiver);
    }
});

val receiver = mediaReceivers.await { it.name().contains("Samsung Smart TV") }
browser.defaultPresentationRequest().ifPresent { request ->
    if (receiver.supports(request)) {
        val future = browser.cast(receiver)
    }
}

Casting a screen

To start casting the screen content, use Browser.castScreen(MediaReceiver). This method will show a standard Chromium dialog for picking the screen to cast.

Java
Kotlin

MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
if (receiver.supports(MediaSource.screen())) {
    CompletableFuture<CastSession> future = browser.castScreen(receiver);
}

val receiver = mediaReceivers.await { it.name().contains("Samsung") }
if (receiver.supports(MediaSource.screen())) {
    val future: CompletableFuture<CastSession> = browser.castScreen(receiver)
}

If you want to select the screen programmatically, use the Browser.castScreen(MediaReceiver, ScreenCastOptions) method. Find the screen you need using the Screens service:

Java
Kotlin

MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
Screen screen = profile.mediaCasting().screens().defaultScreen();
ScreenCastOptions options = ScreenCastOptions.newBuilder(screen)
        .withAudio()
        .build();
if (receiver.supports(MediaSource.screen())) {
    CompletableFuture<CastSession> future = browser.castScreen(receiver, options);
}

val receiver = mediaReceivers.await { it.name().contains("Samsung") }
val screen = profile.mediaCasting().screens().defaultScreen()
val options = ScreenCastOptions.newBuilder(screen)
    .withAudio()
    .build()
if (receiver.supports(MediaSource.screen())) {
    val future: CompletableFuture<CastSession> = browser.castScreen(receiver, options)
}

For now, Chromium supports audio casting only on Windows. So enabling it on macOS/Linux via ScreenCastOptions.Builder.withAudio() is a no-op. On Windows, when selecting the screen in the picker dialog, Chromium provides a separate checkbox for selecting audio-casting.

Presentation API

JxBrowser allows working with JavaScript Presentation API.

When the PresentationRequest.start() method is called on the JavaScript side, JxBrowser invokes StartPresentationCallback where you can decide to start or cancel the presentation.

To start the presentation to a receiver, use the StartPresentationCallback.Action.start(MediaReceiver) method:

Java
Kotlin

browser.set(StartPresentationCallback.class, (params, tell) -> {
    MediaReceiver receiver = params.mediaReceivers().await(it -> {
        return it.name().contains("Samsung");
    });
    if (receiver.supports(params.presentationRequest())) {
        tell.start(receiver);
    } else {
        tell.cancel();
    }
});

browser.set(StartPresentationCallback::class.java,
    StartPresentationCallback { params, tell ->
        val receiver = params.mediaReceivers().await { it.name().contains("Samsung") }
        if (receiver.supports(params.presentationRequest())) {
            tell.start(receiver)
        } else {
            tell.cancel()
        }
    }
)

Discovering cast sessions

To get notified when a cast session has been discovered, JxBrowser provides the CastSessionDiscovered event:

Java
Kotlin

profile.mediaCasting().castSessions().on(CastSessionDiscovered.class, event -> {
    CastSession castSession = event.castSession();
});

profile.mediaCasting().castSessions().on(CastSessionDiscovered::class.java) { event ->
    val castSession = event.castSession()
}

Chromium can discover sessions started by other applications or instances of Chromium. To indicate that the cast session has been started by this profile, JxBrowser provides the CastSession.isLocal() method. So if a cast session is started by another profile or even another Chromium process the method will return false.

Stopping cast sessions

To stop a cast session, use the CastSession.stop() method. If you want to get notified when a cast session has been stopped, use the CastSessionStopped event:

Java
Kotlin

CastSession session = profile.mediaCasting().castSessions().list().get(0);
session.on(CastSessionStopped.class, event -> {
    // Do something.
});
...
session.stop();

val session = profile.mediaCasting().castSessions().list().first()
session.on(CastSessionStopped::class.java) { event ->
    // Do something.
}
...
session.stop()

Session can be stopped by other applications or instances of Chromium, i.e. Google Chrome. In this case, the event will be invoked as well.

Failures

Sometimes, Chromium may fail to start a new cast session, i.e. if the media receiver is not found or had suddenly disconnected. To detect that, use the CastSessionStartFailed event:

Java
Kotlin

MediaReceiver receiver = mediaReceivers.await(it -> it.name().contains("Samsung"));
profile.mediaCasting().castSessions().on(CastSessionStartFailed.class, event -> {
    System.out.println(event.errorMessage());
});
CompletableFuture<CastSession> future = browser.cast(receiver);

val receiver = mediaReceivers.await { it.name().contains("Samsung") }
profile.mediaCasting().castSessions().on(CastSessionStartFailed::class.java) { event ->
    println(event.errorMessage())
}
val future: CompletableFuture<CastSession> = browser.cast(receiver)

This is intentionally an event due to asynchronous nature of media casting.

Since Browser.cast... methods return CompletableFuture you can detect that the start of the cast session has failed. In this case, JxBrowser completes the future with the CastStartFailedException:

Java
Kotlin

CompletableFuture<CastSession> future = browser.cast(receiver);
future.exceptionally(throwable -> {
    System.out.println(throwable.getMessage());
    return null;
});

val future: CompletableFuture<CastSession> = browser.cast(receiver)
future.exceptionally { throwable ->
    println(throwable.message)
    null
}