Qt Edge AI overview

Qt Edge AI is for applications whose logic or user interface must react to what a camera sees: an AI model on the device analyses the video stream, and the application consumes the results. Integrating an on-device AI runtime normally means writing pipeline code against one vendor's stack; Qt Edge AI instead provides C++ classes and QML types that describe the inference pipeline independently of the AI runtime that runs it.

A Qt Edge AI application declares an inference flow: a video source step that reads frames from a Qt Multimedia capture session, a vision inference step that runs a model on those frames, and a result parser step that turns the model output into data the application can consume. Steps exchange data through typed ports. Consecutive steps connect automatically when their port types match, and connections can also be declared explicitly, per port. The actual pipeline work is done by a backend, which gets its flow and step providers from backend plugins on the target, so the same application code runs on any platform that has a matching backend plugin.

Qt Edge AI is a Technical Preview and its API may change. The module ships one backend, built on GStreamer 1.20 or later, with backend plugins that provide inference step providers for the Qualcomm Intelligent Multimedia SDK, NVIDIA DeepStream, and Edge Impulse models; the provided step types cover video input, vision inference, and result parsing. Without a matching backend plugin on the target, flows cannot run.

Qt Edge AI does not capture or render video itself: Qt Multimedia owns the camera through its capture session, and Qt Edge AI reads frames from that session, without copying them when the setup allows it; see Zero-copy Video Frames. Use Qt Multimedia alone to capture, play back, or display video; add Qt Edge AI when AI results computed from the video should drive the application.

Getting started

To use the QML types, add the following import statement to your .qml file:

import QtEdgeAi

To link against the C++ library, add the following to your project's CMakeLists.txt file:

find_package(Qt6 REQUIRED COMPONENTS EdgeAi Multimedia)
target_link_libraries(mytarget PRIVATE Qt6::EdgeAi Qt6::Multimedia)

A minimal working example

The following application shows live camera video and runs an Edge Impulse model on the same frames, logging each inference result to the console:

import QtQuick
import QtMultimedia
import QtEdgeAi

Window {
    id: window
    width: 640
    height: 480
    visible: true

    Component.onCompleted: {
        session.camera.start()
        pipeline.start(window)
    }

    CaptureSession {
        id: session
        camera: Camera {}
        videoOutput: videoOutput
    }

    VideoOutput {
        id: videoOutput
        anchors.fill: parent
    }

    InferenceFlow {
        id: pipeline

        InferenceVideoSource {
            captureSession: session
        }

        VisionInference {
            provider: "edgeimpulse"
            configFile: "configs/edgeimpulse/inference.json"
        }

        InferenceResultParser {
            onDataAvailable: data => console.log(data)
        }
    }
}

The vision step reads its configuration, including the path to the model, from a configuration file; see Qt Edge AI Configuration Files. The Simple Face Detection example walks through this application.

Explore Qt Edge AI

• Qt Edge AI QML Types - declare inference flows in QML
• Qt Edge AI C++ Classes - build flows in C++, or implement a new backend plugin
• Qt Edge AI Configuration Files - configure how a step provider runs an inference step
• Qt Edge AI Examples

Platform support and dependencies

The C++ module depends on Qt Core and Qt Multimedia; the QML types additionally use Qt Qml and Qt Quick. Inference runs through backend plugins.

The gstflowbackend plugin provides the flow provider that builds the GStreamer pipeline, and the step providers of the video source step and the result parser step. It requires GStreamer 1.20 or later. The following plugins provide inference step providers for the GStreamer backend, so they have the same requirement in addition to their own:

The QT_INFERENCE_FLOW_BACKEND environment variable selects the backend; the default is gstreamer.

Module status and licensing

Qt Edge AI 1.0 is a Technical Preview based on Qt 6.12. The API is under development and may change in future releases.

The Qt Edge AI module is available under commercial licenses from The Qt Company. See Qt Licensing for further details.