Live game views for Unreal Engine 4

by Nick July. 09, 14 01 Comment

We’re happy to introduce you to our latest feature for Unreal Engine 4 – Live game views (a.k.a. Picture-in-Picture)! We’ve had this feature in our core product for a while now and the time for an UE4 implementaion came with version 2.2.

 

Let’s go back a little and explain what a “live view” actually is, for anyone who doesn’t know. Simply put, it’s a high performance way to transfer image data from your game engine into a HTML canvas in your UI. You can read more about this feature in this blog post.

 

Getting right to the point – how can you use live views in Unreal Engine 4? We tried to make the integration extremely simple so here’s what you need to do:

  • Add a Coherent UI component
  • Add a Coherent UI Live View component (it depends on the Coherent UI component)
  • Set up the UTexture that is going to be transferred to the HTML UI
  • Add code in your HTML page that will render the image data into a canvas
  • Done!

 

This is all very easy so let’s go through a practical example – we’ll set up a secondary camera that renders the scene to a texture and we’ll send that texture to Coherent UI. We’ll start with the Blueprint Third Person template and the Coherent UI plugin added to it.  Next, we need a secondary camera that renders to a texture target. Grab the “Screen Capture 2D” and place it somewhere in your scene so it displays something interesting 🙂 .

Scene Capture 2D

Scene Capture 2D

 

After that, create a new “Render Target” and assign it as a “Target Texture” of the Screen Capture 2D actor.

Creating a new Render Target

Creating a new Render Target

 

Assigning a texture to the screen capture actor

Assigning a texture to the screen capture actor

 

Double click on the texture and disable the HDR check – it’s currently not supported for live views. You can optionally change the resolution.

Texture properties

Texture properties

 

We’ll now create a Coherent UI HUD and display a page that renders the texture from the “Screen Capture 2D” . To do that, we need to make a blueprint for the HUD. First, select the CoherentUIGameHUD as a parent.

New blueprint with CoherentUIGameHUD parent

New blueprint with CoherentUIGameHUD parent

 

Open the blueprint for editing and add a Coherent UI Live View component, then set its Texture property to the “ScreenCapture_RT” texture. Leave the “Update every frame” checkbox ticked. This will send the texture to the UI every frame – otherwise you can do it manually. Note that the same steps can be applied on a 3D surface with Coherent UI  (live views are not limited to HUDs).

Adding a Coherent UI Live View component

Adding a Coherent UI Live View component

 

Setting Coherent UI Live View properties

Setting Coherent UI Live View properties

 

That’s everything that’s needed for the configuration of the live view! Now, let’s set up our game to use the custom blueprint HUD we created. To do that, open the “World Settings” panel, create a new game mode and set it up like the picture below:

Using the custom HUD blueprint we made

Using the custom HUD blueprint we made

 

All that’s left to do in UE4 is to make a Coherent UI HUD – here’s the blueprint I used:

HUD blueprint

HUD blueprint

 

Here’s the final piece of the puzzle, the live_view.html:

 

Basically, it’s just a <canvas> that’s as big as the texture we set up in UE4. The important function is the onEngineImageDataUpdated callback, which is called every frame in our case (remember the checkbox in the Coherent UI Live View component earlier?). The name argument is the name of the live view link that was set in the component and the image argument is an ImageData object that contains the pixels from the texture. In the sample implementation I changed the alpha value of the received ImageData to about 50% and added some text on top, just for kicks. I also added a CSS perspective for the whole canvas so it looks groovy 🙂 .   And here’s the final result:

Final result

Final result

 

I hope you like our new feature which enables you to add minimaps, character portraits and much more in your UI. You can even do some post-processing on the canvas object if you feel like it.   We’ll also be making a video tutorial out of this to see it in action so stay tuned!

Try Coherent UI and add picture-in-picture in your UE4 UI.

Social Shares

1 Comments

Leave Reply

Leave a Comment