Qt Quick 3D - XR 3D Interaction example

This example shows how to create components that move and resize 3D objects in a scene. It uses the same ray-picking strategy as the xr_input example.

Controller input

In this example, we create a reusable component named AimController. This performs ray picking to select a Model, and optionally allows the user to grab the selected Model and move it around.

AimController defines signals that allows us to implement custom interactions:

signal objectPressed(obj: Model, pos: vector3d, direction: vector3d)
signal objectHovered(obj: Model)
signal moved(pos: vector3d, direction: vector3d)
signal released()

Gadgets

We define an abstract component, XrGadget, that has two functions: handleControllerPress and handleControllerMove. In C++, these functions would have been virtual. Since this example is implemented completely in QML, we instead emit signals that can be handled in sub-components.

For example, TranslateGadget moves the controlled object along the gadget's axis based on the onMoved signal:

onMoved: (pos, dir) => {
    let moveDirection = delta.normalized()
    let mapped_axis = controlledObject.mapDirectionToScene(axisDirection).normalized()
    let dot = mapped_axis.dotProduct(moveDirection)
    let offset = mapped_axis.times(delta.length() * dot)
    controlledObject.position = originalPos.plus(offset)
}

Tying it all together

We define a component GadgetBox that keeps track of which object is selected and shows a translucent box around the selected object, in addition to showing gadgets around the object. When a selected object is pressed, the GadgetBox will cycle between the three different types of gadgets (translate, rotate, and resize).

In main.qml we react to the signals from the AimController and call the functions in GadgetBox:

AimController {
    id: rightAim
    controller: XrController.ControllerRight

    onObjectPressed: (obj, pos, dir) => {
        gadgetBox.handlePress(obj, pos, dir)
    }
    onObjectHovered: (obj) => {
        gadgetBox.handleHover(obj)
        hapticFeedback.handleHover(obj)
    }
    onMoved: (pos, dir) => {
        gadgetBox.handleMove(pos, dir)
    }
    onReleased: {
        gadgetBox.handleRelease()
    }

    grabMoveEnabled: !gadgetBox.gadgetActive
}

We also provide haptic feedback when the hovered object or gadget changes:

XrHapticFeedback {
    id: hapticFeedback
    controller: XrHapticFeedback.RightController
    hapticEffect: XrSimpleHapticEffect {
        amplitude: 0.5
        duration: 30
        frequency: 3000
    }
    property Model prevObj: null
    function handleHover(obj: Model) {
        if (obj && obj !== prevObj)
            start()
        prevObj = obj
    }
}

Example project @ code.qt.io