Teleoperation and Remote Control

Remote control of a robot from a Qt application involves two message types:

• joy (Joy) — raw axis and button state from a physical gamepad or joystick, published by the joy_node driver.
• twist (Twist) — a linear and angular velocity command sent to the robot, typically on the /cmd_vel topic.

A Qt application can sit anywhere in this chain: it can subscribe to Joy and re-publish Twist after applying your own mapping, or it can bypass Joy entirely and publish Twist directly from an on-screen virtual joystick.

Velocity commands: the Twist message

twist (Twist) carries two three-dimensional vectors:

For a typical differential-drive robot, only linear.x (drive forward/back) and angular.z (steer) are used. All other components should be zero.

TwistStamped

Some robot drivers — in particular the ROS 2 Nav2 stack and many newer hardware interfaces — require twistStamped (TwistStamped) instead of plain Twist. The only difference is a std_msgs/Header field that carries a timestamp and frame ID. Check your robot's documentation or the value of the robot_description parameter to see which topic type it expects.

Subscribing to a physical joystick

The joy package (installed separately from the Qt ROS2 Bridge) provides joy_node, which reads a Linux input device and publishes on /joy:

import QtRos2.Core
import QtRos2.SensorMsgs

JoySubscriber {
    id: joyInput
    topic: "/joy"
    node: myNode
    onMessageReceived: (msg) => handleJoy(msg)
}

The joy (Joy) message contains:

• axes — float32[] normalised to −1.0…+1.0. The mapping from array index to physical stick axis is controller-dependent; most gamepads follow the convention left-stick-X=0, left-stick-Y=1, triggers=2/5, right-stick-X=3, right-stick-Y=4.
• buttons — int32[] with 0 (released) or 1 (pressed).

Mapping Joy to Twist

Translate axis values into a Twist and publish on /cmd_vel:

import QtRos2.Core
import QtRos2.GeometryMsgs
import QtRos2.SensorMsgs

Node { id: myNode; nodeName: "qt_teleop" }

JoySubscriber {
    topic: "/joy"
    node: myNode
    onMessageReceived: (joy) => {
        // Axis 1: left stick Y — forward/back.  Axis 0: left stick X — turn.
        // Invert Y because stick-up is typically negative on most gamepads.
        const deadman = joy.buttons[4] === 1  // left shoulder button
        const vx = deadman ? -joy.axes[1] * maxLinear  : 0
        const wz = deadman ?  joy.axes[0] * maxAngular : 0
        drivePublisher.publishVelocity(vx, wz)
    }
}

TwistPublisher {
    id: drivePublisher
    topic: "/cmd_vel"
    node: myNode

    property real maxLinear:  0.5  // m/s
    property real maxAngular: 1.0  // rad/s

    function publishVelocity(vx, wz) {
        var msg = Qt.createQmlObject("import QtRos2.GeometryMsgs; Twist {}", this)
        msg.linear.x  = vx
        msg.angular.z = wz
        publish(msg)
    }
}

On-screen virtual joystick

When no physical gamepad is available, a MultiPointTouchArea (or MouseArea on desktop) can act as a virtual joystick. The principle is the same: map a normalised 2D offset into Twist fields and publish while the user is touching the control.

import QtQuick
import QtRos2.Core
import QtRos2.GeometryMsgs

Node { id: myNode; nodeName: "qt_teleop" }

TwistPublisher {
    id: cmdVel
    topic: "/cmd_vel"
    node: myNode
}

// Joystick pad: a fixed circle with a movable thumb indicator
Rectangle {
    id: pad
    width: 160; height: 160
    radius: 80
    color: "#44ffffff"

    property real thumbX: 0  // −1.0 … +1.0
    property real thumbY: 0  // −1.0 … +1.0

    // Thumb indicator
    Rectangle {
        width: 40; height: 40; radius: 20
        color: "white"
        x: (pad.width  - width)  / 2 + pad.thumbX * (pad.width  / 2 - 20)
        y: (pad.height - height) / 2 - pad.thumbY * (pad.height / 2 - 20)
    }

    MultiPointTouchArea {
        anchors.fill: parent
        maximumTouchPoints: 1

        touchPoints: [ TouchPoint { id: tp } ]

        onUpdated: {
            const cx = pad.width  / 2
            const cy = pad.height / 2
            const dx = (tp.x - cx) / cx
            const dy = (cy - tp.y) / cy  // invert: up is positive
            // Clamp to unit circle
            const len = Math.min(1.0, Math.sqrt(dx*dx + dy*dy))
            const angle = Math.atan2(dy, dx)
            pad.thumbX = len * Math.cos(angle)
            pad.thumbY = len * Math.sin(angle)
            publishFromThumb()
        }

        onReleased: {
            pad.thumbX = 0
            pad.thumbY = 0
            publishFromThumb()  // send zero — see Safety note below
        }
    }

    function publishFromThumb() {
        var msg = Qt.createQmlObject("import QtRos2.GeometryMsgs; Twist {}", pad)
        msg.linear.x  = thumbY * 0.5   // forward/back, max 0.5 m/s
        msg.angular.z = -thumbX * 1.0  // turn, max 1.0 rad/s
        cmdVel.publish(msg)
    }
}

Split-axis layout

Many teleoperation UIs place two separate one-axis sliders: one for throttle (forward/back, linear.x) and one for steering (turn, angular.z), matching the feel of a radio-control car transmitter. The same TwistPublisher approach applies; only the axis mapping changes.

Safety: the deadman switch

A robot receiving a sustained non-zero Twist will keep moving if the controlling application crashes, loses network connectivity, or its window loses focus. Always implement a deadman mechanism:

• Physical gamepad: only publish non-zero velocity while a designated button (the "deadman button") is held. On button release, publish one explicit zero Twist before stopping publication.
• On-screen joystick: publish a zero Twist in the onReleased handler, as shown above. Also publish zero in Component.onDestruction and whenever the enclosing Window loses activeFocus.
• Velocity limits: clamp Twist values to safe maximums before publishing. Never pass raw axis values multiplied by an arbitrarily large scale factor.
• Watchdog on the robot: ideally the robot firmware or base driver should apply its own timeout — stopping automatically if no Twist arrives within a configured interval (e.g. 500 ms). This guards against application crashes and network outages regardless of what the UI does.

Navigation goals (click-to-navigate)

Sending a single goal pose to an autonomous navigation stack — rather than continuously streaming velocity — uses a different message type:

• Publish a poseStamped (PoseStamped) on /goal_pose. Nav2 picks this up, plans a path, and drives the robot autonomously.
• The pose's header.frame_id must match the map frame (typically "map").
• The orientation quaternion encodes the desired heading at the goal; leave it as the identity quaternion if the final heading does not matter.

This approach is complementary to direct velocity teleoperation: a Qt UI can let the user switch between "manual drive" mode (Twist) and "click destination on the map" mode (PoseStamped).

Haptic feedback

Physical gamepads that support rumble or LED colour can receive feedback via joyFeedbackArray (JoyFeedbackArray). Publish an array of joyFeedback (JoyFeedback) entries on /joy/set_feedback:

• type: 0 = LED, 1 = rumble, 2 = buzzer
• id: selects which LED or motor when the device has several
• intensity: 0.0 (off) … 1.0 (full)

This is useful for alerting the operator when the robot reaches a goal, encounters an obstacle, or the battery is low.