Defining an inference flow

An inference flow is the pipeline that turns live camera video into data that an application can use. You describe the flow as a list of steps; Qt Edge AI wires the steps together, and a backend runs them. See Qt Edge AI steps and ports for what a step is and how steps exchange data.

A typical flow has three steps, in this order:

1. A video source step, InferenceVideoSource or QInferenceVideoSource, that reads frames from a Qt Multimedia capture session.
2. A vision inference step, VisionInference or QVisionInference, that runs a model on the frames.
3. A result parser step, InferenceResultParser or QInferenceResultParser, that turns the model output into JSON.

Defining a Flow in QML

Declare the steps as children of an InferenceFlow, in the order the data travels through them:

InferenceFlow {
    id: pipeline

    InferenceVideoSource { captureSession: session }

    VisionInference {
        provider: "imsdk"
        configFile: "configs/imsdk/inference.json"
    }

    InferenceResultParser {
        onDataAvailable: data => console.log(data)
    }
}

When the component completes, the flow builds the underlying C++ QInferenceFlow from its steps. Call start() to run the pipeline.

Defining a Flow in C++

Pass the steps to QInferenceFlow::fromSteps(), in the order the data travels through them, and call start():

QInferenceFlow *flow = QInferenceFlow::fromSteps({ videoSource, detection, parser });
flow->start();

The flow takes ownership of the steps. The returned flow has no parent, so you own it and can reparent it. For a complete example, including how the steps are created, see Qt Edge AI C++ Classes.

How Steps Are Connected

The flow connects the steps in two passes:

1. It applies the connections you declared explicitly on each step.
2. It chains each remaining step to the next one in the list, connecting every output port of the step to an input port of the next step that carries the same data type.

A step that has declared connections is not chained, and a step without output ports, such as a final parser, is skipped when chaining. If a step cannot be chained because no output and input port types match, the flow does not become ready, and a warning is logged.

Declaring Connections Explicitly

Chaining by type is enough for a linear flow. To connect specific ports, declare a DataPortConnection as a child of the source step. The type's properties describe which ports to connect, and what an empty port name selects:

VisionInference {
    id: detection
    provider: "imsdk"
    configFile: "configs/imsdk/inference.json"

    DataPortConnection {
        sourcePort: "text-out"
        targetStep: parser
        targetPort: "text-in"
    }
}

InferenceResultParser { id: parser }

A connection that names a port the step does not have, or a target step that is not set, is skipped with a warning. The port names of the built-in steps are listed in Qt Edge AI steps and ports.

Starting a Flow

A flow is ready when every step has been added to the backend and its ports are connected. A step can become ready later than the others, for example while its backend loads a model, so readiness is asynchronous. The flow emits ready() when it can run, and started() once the pipeline is running.

If the flow is not ready yet when you call start(), the start is deferred and happens as soon as the flow becomes ready. A flow without a backend plugin on the target cannot run, and start() does nothing.

In QML, the InferenceFlow builds the flow from its steps when the component has completed. Calling start() before that has no effect and is not deferred, so call it from Component.onCompleted or later, as the Simple Face Detection example does.

If a step fails to initialize, for example because its configuration file is missing, it never becomes ready and the flow does not start. The step logs a warning that names the problem; see Qt Edge AI Configuration Files.