Sample - Video player

Table of Contents

• Building the sample
• Prerequisites
• Key points
• Sample walk-through

This sample demonstrates how to use Coherent GT's audio and video capabilities create a modern-looking video player in a few HTML lines.

Note

Supplied code is not representative for a high-performance solution. It's written for simplicity and its purpose is to demonstrate features of Coherent GT. Most notably, the provided audio systems are not optimal.

Building the sample

The sample solution is based on a minimal game framework that provides basic functionality.

The sample demonstrates how to use both Microsoft's XAudio2 API and the open-source OpenAL with GT which require some additional preparations.

• To use XAudio2, you need to install the Windows SDK if you are using Windows 8 or Windows 10. If you are using a previous Windows version, you'll need to:
  • Download the DirectX SDK
  • Add '$(DXSDK_DIR)\Include' as an additional include path in the project settings.
  • Add '$(DXSDK_DIR)\Lib\$(Platform)' as an additional library path in the project settings.
• To use OpenAL
  • On Mac / Linux, make sure the library's shared object is in the loader's path. (one way to do that is to set LD_LIBRARY_PATH)
  • On Windows, copy Sample_VideoPlayer/ThirdParty/openal-soft-1.15.1/Win64/soft_oal.dll either next to the sample executable or in your system directory (C:/Windows/system32) and rename it to OpenAL32.dll.

By default, the sample uses XAudio2 on Windows and OpenAL on Mac / Linux. If you are running Windows but only installed OpenAL, open Audio/Config.h and uncomment the line

//#define FORCE_OPENAL

After completing these simple steps, compile and run the application.

The output will be in the Coherent/bin directory.

Prerequisites

This sample builds upon the HelloGT sample and assumes that you understand it.

Key points

1. VideoPlayerViewListener is a custom implementation of Coherent::UIGT::ViewListener which bridges Coherent GT's audio streams with a client audio library. It also handles events send by JS - such as maximizing the window and changing the volume.
2. The interfaces IAudioSystem and ISound provide abstractions over the concrete libraries used. There's a sample implementation for both XAudio2 and OpenAL.
3. The class AudioSystemWrapper hides all of the details of the audio implementation.

Sample walk-through

The sample shows a simple video player. It plays a single file (Coherent Labs' reel for GDC 2015). It provides modern-looking, minimalistic UI inside a borderless window.

As noted in the docs, Coherent GT only supports playing webm videos. Therefore, our first step is to transcode our video to webm using ffmpeg.

Having done that, move to the HTML. Add the video element and some controls:

<video>
    <source src="video/coherent_reel.webm" type="video/webm">
</video>
<div id="ui">
    <div class="video-controls">
        <a class="video-control-button" id="playpause-button">Play/Pause</a>
        <label id="time-left"></label>
        <a class="video-control-button" id="repeat-button">Repeat</a>
        <label id="volume-down-label">VolumeDown</label>
        <div class="volume-control">
            <span></span>
        </div>
        <label id="volume-up-label">VolumeUp</label>
        <a class="video-control-button" id="fullscreen-button">Fullscreen</a>
        </br>
        <div class="video-progress">
             <span></span>
        </div>
    </div>
    <div class="app-controls">
        <a id="close-button" class="video-control-button">Quit</a>
    </div>
</div>

The snippet above creates:

1. The video element with its source pointing to our file.
2. Video controls for:
   • Play / pause button
   • A label showing the time left
   • A stateful repeat button. When pressed, the video will repeat once it ends.
   • Div acting as a slider for volume
   • Button to maximize the player
   • Button to close the player

Let's style some of these elements:

video {
    width: 100vw;
    height: 100vh;
    object-fit: cover; /* Make sure the video scales to cover the entire window*/
}
.video-controls {
    position: absolute;
    bottom: 0; /* Place the video controls at the bottom of the screen */
    width: 100%;
    height: 6vh;
    min-height: 45px;
    margin: 0;
    background: linear-gradient(to bottom, transparent, black); /* Add a black-to-transparent gradient to improve visibility */
}
.video-progress {
    display: inline-block;
    width: 100%;
    height: 4vh;
    min-height: 30px;
    background: rgb(7, 13, 28);
    cursor: pointer; /* Let the user know that the progress bar is clickable */
}
.video-progress span {
    /* We'll use this span to fill the parent div and show the progress. */
    background: rgb(107, 70, 141);
    display: inline-block;
    position: absolute;
    height: 100%;
}

/* For the rest of the styles see videoplayer.css */

To improve the visuals, in the final version we have replaced the text with icons using FontAwesome.

We now need to hook up the buttons with some JavaScript:

var video = document.querySelector('video');
var playButton = document.querySelector('#playpause-button');

var togglePlayPause = function () {
    if (video.paused) {
        video.play();
    }
    else {
        video.pause();
    }
};
// Toggle play / pause whenever:
// 1. The user clicks the button
// 2. The user clicks on the video
// 3. The user hits the spacebar
playButton.addEventListener('click', togglePlayPause);
video.addEventListener('click', togglePlayPause);
document.body.addEventListener('keydown', function (eventArgs) {
    if (eventArgs.keyCode === Keyboard.space) {
        togglePlayPause();
    }
});

// Fill the progress bar with the current time
var progressFiller = document.querySelector('.video-progress span');
var timeLeftLabel = document.querySelector('#time-left');
var updateProgress = function () {
    progressFiller.style.width = (video.currentTime / video.duration) * 100 + '%';
    // Before the video has fully loaded, duration might be NaN
    if (!isNaN(video.duration)) {
        timeLeftLabel.textContent = video.currentTime + ' / ' + video.duration;
    }
    requestAnimationFrame(updateProgress);
};
updateProgress();

/* See videoplayer.js for the rest of the code */

We need some interaction with the CPP code to handle events such as exiting the app. To test the interaction in the browser, we'll mock the calls to engine.trigger:

var closeButton = document.querySelector('#close-button');
closeButton.addEventListener('click', function () {
    engine.trigger('Quit');
});
...
if (!engine.isAttached) {
    engine.mock('Quit', function () {
        document.body.innerHTML = '<h1>You closed the player!</h1>';
    });
}
/* See videoplayer.js for the rest of the code */

Let's now take a look at the C++ side.

Firstly, we initialize the Application class using a Style object unlike the rest of the samples. This allows us to make the window borderless and / or transparent:

Coherent::Application::PlatformWindow::Style style;
style.ClientRect.Top = style.ClientRect.Left = 100;
style.ClientRect.Width = width;
style.ClientRect.Height = height;
style.Title = "Sample_VideoPlayer";
style.Fullscreen = false;
style.Centered = false;
style.ShowWindow = true;
style.Borderless = true;
//style.Opacity = 0.7f; // Uncomment to make the window transparent
app.Initialize(style);

The key part of the sample is the class VideoPlayerViewListener. As an implementation of Coherent::UIGT::ViewListener it overrides the audio handling methods:

virtual void OnAudioStreamCreated(int id, int bitDepth, int channels,
    float samplingRate) override
{
    bool ret = gAudioWrapper->CreateStream(id, bitDepth, channels, samplingRate);
    assert(ret);
}
virtual void OnAudioStreamPlay(int id) override
{
    bool ret = gAudioWrapper->PlayStream(id);
    assert(ret);
}
virtual void OnAudioDataReceived(int id, int samples, float** pcm,
    int channels) override
{
    gAudioWrapper->ReceivePCMData(id, samples, pcm, channels);
}
/* See Sample_VideoPlayer.cpp for the rest of the code */

The audio calls are handled by an instance of AudioSystemWrapper. This class hides the details by the actual implementation. Internally, it manages instances of IAudioSystem and ISound which are implemented using both XAudio2 and OpenAL. We won't go into details of the implementation as the code is too long but on the hand it is straightforward.

VideoPlayerViewListener also provides the interaction with the JavaScript:

void SetVolume(float volume)
    {
        gAudioWrapper->SetMasterVolume(volume);
    }
    void Quit()
    {
        m_App.Quit(0);
    }
    ..
    virtual void OnReadyForBindings() override
    {
        m_View->RegisterForEvent("SetVolume", Coherent::UIGT::MakeHandler(this, &VideoPlayerViewListener::SetVolume));
        m_View->RegisterForEvent("Quit", Coherent::UIGT::MakeHandler(this, &VideoPlayerViewListener::Quit));
        m_View->RegisterForEvent("ToggleMaximizeWindow", Coherent::UIGT::MakeHandler(this, &VideoPlayerViewListener::ToggleMaximizeWindow));
    }