2.9.16
Coherent GT
A modern user interface library for games
Live views (picture-in-picture)

Coherent GT supports a powerful feature called Live Views. With it developers can embed dynamic content rendered by the 3D engine in their UI. The feature is ideal for 3D player portraits, equipment screens, rear view mirrors in racing games, but also advanced effects like adding particle effects or complex 3D content inside the interface.

The Live View is a dynamic preloaded texture which and can be repainted whenever you need to.

A working sample of Live Views can be seen in the Sample_LiveViews sample distributed in the Coherent GT package.

Setup

Live views are implemented as dynamic preloaded textures. They work exactly the same way as preloaded textures with the only difference being in that they can be changed while the game is running.

  1. Prepare your textures:
    void PreloadTexture()
    {
        for (const auto& textureURL : texturesToPreload)
        {
            liveViewTextures[textureURL] = LoadTexture(textureURL);
        }
    }
    
  2. Override Coherent::UIGT::ResourceHandler::OnResourceRead in your resource handler. Here's a pseudocode implementation:
    void MyResourceHandler::OnResourceRead(const Coherent::UIGT::ResourceRequestUIGT* request,
        Coherent::UIGT::ResourceResponseUIGT* response)
    {
        std::string url = request->GetURL();
        auto texture = liveViewTextures[url];
    
        // This texture isn't loaded yet, let Coherent GT read the raw file.
        if (!texture)
        {
            ReadRawFile(request, response);
            return;
        }
    
        // Tell Coherent GT that the image is live view
        using ImageData = Coherent::UIGT::ResourceResponseUIGT::UserImageData;
        ImageData data;
        data.Width = texture->Width;
        data.Height = texture->Height;
        data.ContentRectX = 0;
        data.ContentRectY = 0;
        data.ContentRectWidth = data.Width;
        data.ContentRectHeight = data.Height;
        data.Format = texture->Format;
        data.Texture = texture->Resource;
        data.ImageHandle = texture->UniqueHandle;
        response->ReceiveUserImage(data);
        response->SignalSuccess();
        response->Release();
    }
    
    Set the image's handle to a unique positive value in order to be able to repaint it later. This tells Coherent GT that the user image is a live view. For example how content rect work check Sample - Preloaded resources.
    Note
    ReadRawFile is called for resources that the application assumes not preloaded.
  3. Modify the rendering backends of your choice to support creating a texture from whatever you passed in your resource handler. Alternatively, your resource handler may return the data in the format used by that backend. See the provided sample and the code for the backends you intend to use.
    void MyBackend::WrapUserTexture(void* userObject,
        const Texture2D& description,
        Texture2DObject object)
    {
        // userObject contains whatever you passed to ResourceResponseUIGT::UserImageData::Texture
        auto textureResource = (MyTextureResource*) userObject;
        // Prepare the texture
    }
    
    When the texture is no longer needed, MyBackend::DestroyTexture will be called as it would for any other texture.
Note
Coherent GT expects all textures to have premultiplied alpha, keep that in mind when loading your images (unless you replace the default shaders / backends).

Repainting

The repainting of a live view's texture is via the Coherent::UIGT::UISystem::UserImagesChanged method. Here is a pseudocode implementation:

void UpdateLiveView()
{
    // Get the texture that needs a repaint.
    auto texture = liveViewTextures[GetTextureUrl()];
    GetUISystem()->UserImagesChanged(&texture->ImageHandle, 1);
}
Note
The UserImagesChanged method takes an array of image handles, so multiple textures can be marked as changed at once.