How to encode videos

Although we support both VP8 and VP9, VP8 is much faster both for decoding and encoding, we recommend it. You can experiment with your own videos by encoding them through "ffmpeg". To encode a video, please download the ffmpeg-encoder from here The command line that gives best results is:

ffmpeg.exe -i "MYVIDEO.mp4" -vcodec libvpx -b:v 1000k -s 1280x720 -frame-parallel 1 -tile-rows 0 -tile-columns 6 -slices 4 -acodec libvorbis -b:a 128k movie_vp8_1000.webm

You can set the resolution and bitrate. Please make sure you use the other parameters as described because they affect the decoding and encoding performance.

Multithreaded decoding

Multithreaded video decoding is on by default and it cannot be changed currently. The maximum number of threads the decoder can use is 3.

Warning: Since a decoder is initialized for every video that is playing, it is highly recommended not to play multiple videos at the same time (especially on consoles), as that many threads might have a big impact on the game. Some frameworks (e.g. SPA-based ones) can keep video elements alive even when the page has changed and they are not displayed anymore, thus the corresponding threads will not be uninitialized properly.

The decoder threads are killed when the video element is removed from the DOM.

Transparent video support

Starting with Coherent GT 1.9 WebM videos with an encoded alpha channel can also be displayed. The basic authoring process of such a video goes roughly as follows:

Create video for chroma keying: Either record or create with some video editing tool a video which has one color assigned as "transparent" (key color).
Split the video into an image sequence: The video must be split into images that support transparency (e.g. PNGs). You can do this with most popular video editing software, or by using ffmpeg via the command line.

Following is a ffmpeg example which removes pink color (0xFF00FF) through a filter and outputs an image sequence:

ffmpeg -i vid.mp4 -vf colorkey=0xFF00FF:0.3:0.8 sequence/image-%05d.png

The FFMPEG manual states this regarding the colorkey filter:

colorkey

RGB colorspace color keying.

The filter accepts the following options:

color

The color which will be replaced with transparency.

similarity

Similarity percentage with the key color.

0.01 matches only the exact key color, while 1.0 matches everything.

blend

Blend percentage.

0.0 makes pixels either fully transparent, or not transparent at all.

Higher values result in semi-transparent pixels, with a higher transparency the more similar the pixels color is to the key color.

In the example a similarity of 0.3 and blend of 0.8 are used.

Convert the image sequence back into a video: One of the easiest ways to do this is using FFMPEG:

ffmpeg -i sequence/image-%05d.png -c:v libvpx -pix_fmt yuva420p -metadata:s:v:0 alpha_mode="1" output.webm

If you want to add audio, you can simply pass another input to ffmpeg:

ffmpeg -i sequence/image-%05d.png -i audio.wav -c:v libvpx -pix_fmt yuva420p -metadata:s:v:0 alpha_mode="1" output.webm

Other parameters can be passed as well, of course. Add values for the video bitrate, resolution, etc. to control the video quality.

ffmpeg -i sequence/image-%05d.png -c:v libvpx -pix_fmt yuva420p -metadata:s:v:0 alpha_mode="1" -b:v 2000k -s 1920x1080 -frame-parallel 1 -tile-rows 0 -tile-columns 6 -slices 4 output.webm

Note: Make sure you're using a new version of FFMPEG, otherwise the colorkey filter might not be available.

Showing video preview

By default, Coherent GT does not render video frames unless playback is explicitly required. This means that there will be no image of the video shown before you actually play it. If you want to show a preview, you can either play a very small portion of it, or add a poster image.

Adding a poster allows you to display a custom image. It can be done using the poster attribute of the <video> element:

<video id="myVideo" poster="back.jpg">
    <source src="movie.webm" type="video/webm">
</video>

The second option for a preview is to explicitly play the video and pause it almost immediately. This can be done by listening for the playing event of the video, which is fired after the video started playing, and then request a pause in the callback. Here's an example of doing that:

<script>
function onVideoStartedPlaying() {
    // This callback function should only be used for showing the first frame of
    // the video, since it removes itself from the event listeners list after
    // it's done. If you require another handler for the 'playing' event you
    // can register another callback with your custom code.
    // Stop the video right after it has started playing and rendered a frame
    this.pause();
    // Remove the callback so it's executed only once
    this.removeEventListener('playing', onVideoStartedPlaying);
}
function showFirstFrame(vid) {
    vid.addEventListener('playing', onVideoStartedPlaying);
    // Start playing. Playback will be stopped shortly when the 'playing' event
    // is received, in the 'onVideoStartedPlaying' callback registered above.
    vid.play();
}
var myVideo = document.getElementById("myVideo");
showFirstFrame(myVideo);
</script>

FileHandler

The custom FileHandler in OnResourceRead must call SetStatus and SetExpectedLength method on response object, before calling ReceiveData. For example:

char* buffer;
long long fileSize;
// read file here
response->SetStatus(200);
response->SetExpectedLength(fileSize);
response->ReceiveData(buffer, fileSize);
response->SignalSuccess();

If you want to load the video partially you need to implement Partial Requests. For them you need to parse the request headers for Range entry and load only part of the file. After that set response status code to 206.

YouTube

Supported features
- 360p playback
- Seeking
Unsupported features
- 360° video
- Quality different than 360p
Known issues
- Rapidly seeking multiple times sometimes causes the video decoder to fail and the video would freeze

Note: Video decoding is very slow in 1080p and above. This is due to the fact that GT uses software decoding.

Table of Contents