Gameface for UE4

Input overview

The input in Gameface is managed through a Slate widget

  • SCohtmlInputForward. This widget is an empty widget that covers the whole screen and forwards input to a Cohtml View. For simplicity we have created the ACohtmlInputActor actor, which encapsulates the logic behind the widget and is accessible through both C++ and blueprints. All you have to do is spawn the actor and call its initialize method.

Keyboard, Mouse and Gamepad focus in UE4

Gameface will process keyboard events only when the Input Forward widget has keyboard focus.

Mouse events don't require focus and they are propagated to the widgets under the cursor until one of the widgets handles the event. The Input Forward widget is generally on the top of the viewport so it can process mouse events first. The processing in the widget works as follows:

  • If the widget does not have keyboard focus and the widget isn't set to always accept mouse events anyway, the event is unhandled. By default, the widget does not always allow mouse events so you need to have keyboard focus to receive them.
  • Check the clicked/touched DOM node from the UI for every HUD view. If node is HTMLTag::BODY and event phase is InputEventPhase::AT_TARGET or this node or one of it's parent nodes has CSS class that marks the node as transparent for the input then the event is left unhandled, in every other case event is handled. CSS classes that should mark a node as transparent for the input can be set from Gameface settings in UE4 editor.
  • If none of the HUD views should handle the mouse event, a raycast through the mouse position is made. Depending on the widget settings it can trace either multiple objects, or just the first object the ray hits. If the hit object has a Gameface component the event is handled.
  • If neither a HUD view, nor a 3D object view should receive the mouse event, it's left unhandled and it's propagated to the next widgets.
Setting up Input Transparent CSS classes

Gamepad input works by default when the viewport widget has focus. Keyboard focus implies gamepad focus, which means that if the Gameface widget has keyboard focus, the game will stop processing gamepad input. If that's not intended, the input actor has a method for keeping gamepad focus to the viewport after setting keyboard focus.

Each view component has a property (Receive Input) that controls whether a specific view takes part in the mouse event forwarding.

Input (Blueprints)

To add input in a blueprint only game, you need to spawn an actor that will set up the SCoherentInputForward widget. This is an empty widget that covers the whole screen and forwards input to a Cohtml View.

The widget should be added in the front of the viewport so it can process mouse and keyboard messages first. Due to Unreal Engine 4's architecture, the widget can process keyboard messages only if it's focused. Check the ExampleMap's SetupInput method in the level blueprint for an example.

The Input actor setup part shows how to spawn and initialize the input actor, which in turn adds the SCohtmlInputForward widget to the viewport.

At any time there should be only one input actor spawned, otherwise the input might not work correctly.

The Game to UI Focus part is for managing input focus - when you press Tab, the character input will stop and the UI will take it. This would seem to be enough, but currently UE4 stops firing key events when the focus is not on the viewport so we need other means to get focus back to the game. This is shown in the UI to game focus part. The input actor provides events for key/mouse up/down - in the sample we're using the key down event. When it's fired there's a check for the key enum to see if it's the Tab key, and if it is, the focus is toggled.

The Toggle Input Focus function provides means for toggling input without tracking it. If you want to, there are also functions for querying and setting the focus explicitly.

Input (C++)

The input in Gameface is managed through a Slate widget (SCohtmlInputForward). This widget is an empty widget that covers the whole screen and forwards input to a Cohtml View. The widget should be added in the front of the viewport so it can process mouse and keyboard messages first. Due to Unreal Engine 4's architecture, the widget can process keyboard messages only if it's focused. To focus the widget, you can use the FSlateApplication::Get().SetKeyboardFocus(Widget) method. Focus management must be implemented by the client.

You can also show the system mouse cursor using FSlateApplication::Get().ResetToDefaultInputSettings();. If you want to set focus back to the game, FSlateApplication::Get().SetFocusToGameViewport(); will do that.

If the SCohtmlInputForward widget has focus, then keyboard events are sent to the focused Cohtml View. To focus a view, you should click anywhere on it.

Mouse events are sent only to the Cohtml View that is under the cursor.

To setup the slate input forward widget, you only need to spawn an instance of ACohtmlInputActor and call ACohtmlInputActor::Initialize.

Of course, you can also use the cohtml::View API to create your own input events and send them to Cohtml. The View provides the MouseEvent, KeyEvent and TouchEvent methods that send input to the it. Each input event has to be populated with properties that can be trivially translated from the UE4 events.

Input use cases

Here we outline some common usages of the input system. The examples are given using blueprints and the Input actor as these maps directly to C++ code. We'll assume that you already have a spawned and initialized Input actor:

Initialize Input Actor

Toggling input focus between the UI and the Game

Toggling input focus can be done 2 ways:

  • Clicking with the mouse on a Cohtml view gives focus to the view since the Unreal's Slate system will focus the widget that handles the click. If you click back somewhere in the game viewport, the game will take focus.
  • The "click to focus" method works, but in a more realistic use case you'd want to be able to change the focus programmatically. You can do that using the input actor's ToggleInputFocus method. The actor keeps track of the current focus internally so it can toggle between game and UI at any time. If you want to set it explicitly, you can use SetInputFocus, or query the state with IsFocused.

After the Input Actor has focused the input forwarding widget, you need to set which view should receive the keyboard input. This can be done with the SetViewFocus method. An alternative is to simply let the user click on the input field that she wants to type in.

Propagating keyboard and gamepad events when UI is focused

When the UI is focused, keyboard events are consumed by the Input forward widget by default. You can change that using the SetInputPropagationBehaviour method. The available options are to forward keyboard events, gamepad events, both or none to other widgets.

This can be useful if you need to have your action handlers (e.g. "Jump", "Fire", etc.) executed even if the focused widget isn't the game viewport.

Note that gamepad forwarding is only taken into account when using the Input actor's methods for setting (or toggling) focus.

Forwarding mouse input events to Gameface regardless of the focused widget

If you want your mouse input to be forwarded to Gameface first for hover events, but still receive keyboard events in your game, then you can use the AlwaysAcceptMouseInput method. When set to true, mouse events will be received by Gameface, regardless of whether the input forward widget is focused or not.

Including or removing specific Gameface components in the input forwarding

To control whether a Cohtml view receives mouse events you can change the View component's Receive Input property. Only when this property is set to true will the component be considered for receiving input events.

Set receive input property from components tab

Input on multiple Gameface components in the 3D world

When you have multiple views on 3D objects and they are overlapping when viewed with the current camera, the front one will receive the mouse input events, and the back one will get nothing. If, for example, you have disabled input on the front one, the back one will still get nothing, because 3D objects under the cursor are gathered with a raycast from UE4, which is independent from the input forward widget's logic.

For performance reasons, the raycast returns only the first object hit by default. In a scenario like the one described above, where you have two overlapping objects and you want to forward input to the one in the back, the raycast type must be changed to return multiple objects. This is needed even if the object in the front is marked to not receive input.

Virtual Keyboard Input

On a variety of platforms, text input can be achieved only by using a virtual keyboard. Gameface supports this feature out of the box. When focusing an html input field or textarea, Gameface will try to spawn virtual keyboard on screen, using FSlateApplication::ShowVirtualKeybord and supplying FCohtmlVirtualKeyboardEntry as an argument.

note: FSlateApplication::ShowVirtualKeybord has platform-specific implementation and there is no default virtual keyboard available on Windows.

Virtual Keyboard layout

Specialized virtual keyboard layouts may be used for input fields of type password, number, url and email, depending on the platform-specific implementation. Default text virtual keyboard layout will be used for all other input field types. Multiline text virtual keyboard layout will be used for textarea fields.

Overriding Gameface Virtual Keyboard behavior

The default behavior described above can be changed by binding an override function to the ICohtmlPlugin::OnShowVirtualKeyboard delegate.

Inside that function you have to create an instance of FCohtmlVirtualKeyboardEntry or a derived class. You can use FVirtualKeyboardOptions as an optional parameter. Lastly, you need to show the virtual keyboard either through FSlateApplication::ShowVirtualKeyboard or your own implementation. You can also completely deactivate this feature for a given platform by leaving the implementation of the bound function empty.

void ShowVirtualKeyboardOverride(TSharedPtr<FCohtmlVirtualKeyboardEntry>& CohtmlVirtualKeyboardEntry, int UserIndex, cohtml::View* OwningView)
    // fill in the CohtmlVirtualKeyboardEntry shared ptr and show the virtual keyboard
    CohtmlVirtualKeyboardEntry = MakeShareable(
        new FCohtmlVirtualKeyboardEntry(OwningView, FVirtualKeyboardOptions())
        true, UserIndex, CohtmlVirtualKeyboardEntry);
    // leave empty to disable for a given platform

Example usage of the ICohtmlPlugin::OnShowVirtualKeyboard delegate can be found inside the sample game in CoherentSample.cpp.