2.9.16
Coherent GT
A modern user interface library for games
Coherent GT incorporates the proprietary Renoir graphics library and performs all rendering on the GPU. Renoir backends are provided for all major graphics APIs and multi-threaded rendering engines. Users can also customize the low-level rendering code and even write their own backend that uses the application engine.
Coherent GT uses high level rendering commands internally that are passed to the Renoir graphics library. The Renoir Core performs the actual rendering by generating vertices, indices, binding textures etc. The Core library is API-agnostic - it has been ported to wildly different devices and APIs - from mobile phones with GLES 2 to powerful consoles like PlayStation 4 and Xbox One. The platform-specific piece that connects the Renoir Core to the API (DirectX, OPenGL etc.) is called a Renoir backend and is an implementation of the renoir::RendererBackend interface. Users can implement their own backend for tighter integration with engines or to support new APIs.
Coherent GT comes with a collection of ready-to-use backends for the major APIs:
More backends are coming soon.
When drawing through OpenGL, DirectX 9, DirectX 11 and DirectX 12, the provided backends use the client application's context/device and thus seamlessly plugs itself in the rendering pipeline.
All backends are available in the "RenoirBackends" folder both as source and pre-built. Applications should link with the backend they intend to use.
Dx11Backend::Fillcaps in Dx11Backend.cpp). Set SupportsMSAATextures and RequiresMSAAResolve to false.Coherent GT introduces a separation between the main logic of the UI system and it's rendering. The main classes UISystem and View take care of all the JavaScript logic, the layout of the pages, the interaction with the input from the client application. Those classes feed the rendering pipeline that is represented through the UISystemRenderer and ViewRenderer classes. You can paint your Views only through the renderer classes. The reason for this clear separation is easier support for threaded rendering engines. The rendering classes can be used in a thread separate from the one where the main logic is happening. For more information see the Threaded rendering section.
After having created you UI System, you should initialize the rendering backend. This is done with code like this:
auto backend = new renoir::Dx11Backend(); auto m_UISystemRenderer = m_UISystem->CreateRenderer(backend);
After we have created the System renderer, we need to create a renderer for each View we wish to use.
auto viewRenderer = m_UISystemRenderer->CreateViewRenderer(m_View,
myTexture, width, height, sampleCount);
Each ViewRenderer is created from the UISystem renderer. The first parameter is the View that this renderer will draw. The second is an object of type NativeRenderTarget that is a structure with opaque pointers that depend on the actual backend used. These objects will be directly sent to the backend where they can be cast to API-specific objects (on DirectX 11 for instance they will usually be ID3D11RenderTargetView*; on DirectX 9 they are IDirect3DSurface9*, on OpenGL they are GLuint which are names of textures). The user can also use other types that make sense for her backend. Width and height tell the size of the render target, while sample count is the number of MSAA samples in the RT. We strongly recommend to not use MSAA render targets. Use a UI RT with 1 sample. Renoir does anti-aliasing with alternative techniques and you'll have great looking UI without the memory cost of MSAA.
Coherent GT does NOT take ownership of the Texture objects. After you have finished using them you must release them yourself.
You must destroy all renderer objects before their main counterparts. In particular you must always destroy the ViewRenderers before their respective Views. Resource destruction must happen in the rendering thread if such is used.
If you want to remove a particular view, you must destroys the view renderer and its corresponding view (in this order), thus freeing the resources used by the specified view. Destruction of view renderers and views goes hand in hand!
For more information on how to create and destroy Views and ViewRenderers please visit the Sample Application documentation section.
Every frame Coherent GT has to do basically 3 things:
In code the three steps become:
unsigned Application::Update(float dt)
{
UpdateGameState();
// Advance the time in the UI
m_UISystem->Advance();
// Re-layout the current views
unsigned frameId = m_View->Layout();
}
void Application::DrawScene(unsigned frameId)
{
// Draw the rendering commands with the supplied id in a texture
// This will end up calling the rendering backend
m_ViewRenderer->Paint(frameId);
DrawGame();
}
Note that because Coherent GT supports threaded rendering engines, the ViewRenderer methods (in this case Paint(frameId)) can be called on a rendering thread. The Paint(frameId) method will end up calling the methods of the rendering backend.
After rendering, the Coherent GT backends do not restore the rendering state of the API. The user has to restore any state she needs. A complete list of changed states can be seen in the code of the backends themselves.
Coherent GT requires a RGBA texture to draw into with a depth-stencil texture with 8 stencil bits. Although supported, we encourage users to not use an MSAA texture. Renoir does its own AA that gives better results than MSAA and doesn't incur the multi-sampling overhead.
Coherent GT extends the classic image formats supported by HTML rendering engines and browsers with formats that are used in games. The current version of the library works with PNG, JPG, DDS, TGA, PSD (simple), GNF, SVG. We strongly encourage users to employ compressed image formats (DDS, GNF) in shipping configurations.
Coherent GT support a "click-through" functionality that answers the question "Is the cursor on the UI?" based on the alpha value of the pixel. This feature is implemented in the Renoir backend through GPU queries.
Coherent GT support engines and games that use a separate rendering thread. To ease the support for such architectures the UI System object and the Views are logically separated in two parts. The main UI code must always happen in one thread - usually the main thread and there all the JavaScript is executed, input is passed, callbacks are invoked and the layout is performed.
The UI system and each View have renderer objects that can be used from a thread different than the main thread - usually the rendering thread. Their methods use the rendering API of the application and hence must happen where it does its drawing. The renderers are the only objects in the Coherent GT API that can be called from a thread different than the main UI thread.
Whenever a call is made to the rendering API for the first time, the ID of the thread is saved. Then, for every consequent rendering call, the saved ID and the current thread's ID are compared. If they do not match, this will cause an assertion failure, signalling that these calls are made from the wrong thread.
true if your engine allows changing the rendering thread.Coherent GT minimizes the rendering work it performs by re-drawing only the parts of the textures that have changed. Users can keep track of what has been re-drawn each frame (if anything). The Coherent::UIGT::ViewRenderer::Paint method returns the count of rects in the texture that have been re-drawn. The Coherent::UIGT::ViewRenderer::GetLastPaintedRects retrieves the list of those rectangles. Users can also visually inspect what gets redrawn each frame by calling the Coherent::UIGT::View::ShowPaintRects method or clicking "Show paint rects" in the Debugger.
Coherent GT will emit performance warnings in the log file. You can disable those warnings in the SystemSettings or per-View in the ViewInfo structure. You can also change the threshold values that will cause a warning to emit. Warnings help identifying quickly during development that some change has caused a performance drop. You should disable them in shipped products.
When you receive a performance warning, you can use the Debugger to profile eventual inefficiencies in the interface and the Auditor to understand what parts of your CSS and HTML are causing the inefficiencies. Please also consult the Performance guide that gives plenty of information about the characteristics of different workloads and tips on how to optimize the UI. The guide is available in the Coherent GT package alongside this file.
HTML <canvas> elements are GPU accelerated. Because of that reading pixels back (via the getImageData() canvas method) is not supported. Reading back from a GPU resource is very expensive and disabled.
When drawing to a <canvas>, the draw calls are recorded in a command buffer. The recorded commands are executed when the canvas itself is actually drawn in the view. This means that drawing to a <canvas> that is not visible, will store all the commands until it becomes visible. The number of stored commands may grow significantly if the canvas is updated every frame. Therefore it is recommened to avoid drawing to a in-visible canvas. Canvases are invisible when they are not attached to the body of the document or are outside of the screen. To avoid creating very large command buffers, you can reset the canvas buffer by setting its width or height to the same value.
var canvas = document.getElementById('offscreenCanvas');canvas.width = canvas.width;
When Coherent GT detects that a layer is being overdrawn many times, without being painted, Coherent GT issues a warning message:
Detected drawing to a canvas for more than 2500 times.
Due to the way offscreen canvases and image caching work, drawing images to a <canvas> may fail as the image resources are not kept alive until commands are executed to prevent excessive cache growing. In case an image resource is not available in the cache when the recorded commands are executed, Coherent GT issues a warning message:
Missing texture for rendering: (id)
To resolve any of the two issues, you can enable offscreen canvas rendering through the Coherent::UIGT::ViewInfo::OffscreenCanvasRendering member when creating the view or by calling Coherent::UIGT::View::SetOffscreenCanvasRendering. By enabling the option Coherent GT will execute the rendering commands of all offscreen canvases each frame.
In order to provide the best possible performance, the Renoir library introduces some limitations on the rendered content.
Users are free to implement their own rendering backend in order to more tightly integrate GT in their application. This will require implementing the renoir::RendererBackend interface. We advise looking at the provided implementation for major APIs as a sample on how to create the backend. Typically creating a backend for an API that is close to DirectX 11 in design would take around 3 work days to a developer familiar with the application API. As a first step we advise user to use or customize the backends provided with the library, study their flow and only if necessary go on and implement their own backend.
Coherent GT employs several caches to accelerate the rendering of the pages. Currently there are 5 caches whose size and content can be controlled by the user:
The following 6 caches are rendering-oriented and their current state can be queried and modified with the Coherent::UIGT::View::GetCacheStats, Coherent::UIGT::View::QueueSetCacheStats; Coherent::UIGT::View::QueueClearCaches methods.
Coherent GT provides the functionality to paint directly into a specified texture using the method Coherent::UIGT::ViewRenderer::PaintToTexture instead of Coherent::UIGT::ViewRenderer::Paint. The functionality allows you to paint directly into the backbuffer of the game and avoid creation of an extra texture for the UI, which in terms avoids one copy of a texture on the GPU and the cost associated with copying that texture into the backbuffer. The improvement of the performance scales proportionally to the game resolution and inversely proportional to the size of the elements in the UI. For example, a 4k UI showing only a health bar will benefit more from the option than a 1080p UI with many menus. The method also allows you to set an offset and a content rectangle, in which the view will be drawn.