RHI Window Example#

This example shows how to create a minimal QWindow-based application using QRhi.

Qt 6.6 starts offering its accelerated 3D API and shader abstraction layer for application use as well. Applications can now use the same 3D graphics classes Qt itself uses to implement the Qt Quick scenegraph or the Qt Quick 3D engine. In earlier Qt versions QRhi and the related classes were all private APIs. From 6.6 on these classes are in a similar category as QPA family of classes: neither fully public nor private, but something in-between, with a more limited compatibility promise compared to public APIs. On the other hand, QRhi and the related classes now come with full documentation similarly to public APIs.

There are multiple ways to use QRhi, the example here shows the most low-level approach: targeting a QWindow, while not using Qt Quick, Qt Quick 3D, or Widgets in any form, and setting up all the rendering and windowing infrastructure in the application.

In contrast, when writing a QML application with Qt Quick or Qt Quick 3D, and wanting to add QRhi-based rendering to it, such an application is going to rely on the window and rendering infrastructure Qt Quick has already initialized, and it is likely going to query an existing QRhi instance from the QQuickWindow. There dealing with QRhi::create(), platform/API specifics or correctly handling QExposeEvent and resize events for the window are all managed by Qt Quick. Whereas in this example, all that is managed and taken care of by the application itself.

Note

For QWidget-based applications in particular, it should be noted that QWidget::createWindowContainer() allows embedding a QWindow (backed by a native window) into the widget-based user interface. Therefore, the HelloWindow class from this example is reusable in QWidget-based applications, assuming the necessary initialization from main() is in place as well.

Shaders#

Due to being a Qt GUI/Python module example, this example cannot have a dependency on the Qt Shader Tools module. This means that CMake helper functions such as qt_add_shaders are not available for use. Therefore, the example has the pre-processed .qsb files included in the shaders/prebuilt folder, and they are simply included within the executable via a resource file}. This approach is not generally recommended for applications.

RHI Window Example

Download this example

# Copyright (C) 2023 The Qt Company Ltd.
# SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

from argparse import ArgumentParser, RawDescriptionHelpFormatter
import sys

from PySide6.QtCore import QCoreApplication
from PySide6.QtGui import QGuiApplication, QRhi, QSurfaceFormat

from rhiwindow import HelloWindow
import rc_rhiwindow

if __name__ == "__main__":
    app = QGuiApplication(sys.argv)

    # Use platform-specific defaults when no command-line arguments given.
    graphicsApi = QRhi.OpenGLES2
    if sys.platform == "win32":
        graphicsApi = QRhi.D3D11
    elif sys.platform == "darwin":
        graphicsApi = QRhi.Metal

    parser = ArgumentParser(formatter_class=RawDescriptionHelpFormatter,
                            description="QRhi render example")
    parser.add_argument("--null", "-n", action="store_true", help="Null")
    parser.add_argument("--opengl", "-g", action="store_true", help="OpenGL")
    parser.add_argument("--d3d11", "-d", action="store_true",
                        help="Direct3D 11")
    parser.add_argument("--d3d12", "-D", action="store_true",
                        help="Direct3D 12")
    parser.add_argument("--metal", "-m", action="store_true",
                        help="Metal")
    args = parser.parse_args()
    if args.null:
        graphicsApi = QRhi.Null
    elif args.opengl:
        graphicsApi = QRhi.OpenGLES2
    elif args.d3d11:
        graphicsApi = QRhi.D3D11
    elif args.d3d12:
        graphicsApi = QRhi.D3D12
    elif args.metal:
        graphicsApi = QRhi.Metal

    # graphicsApi = QRhi.Vulkan?  detect? needs QVulkanInstance

    # For OpenGL, to ensure there is a depth/stencil buffer for the window.
    # With other APIs this is under the application's control
    # (QRhiRenderBuffer etc.) and so no special setup is needed for those.
    fmt = QSurfaceFormat()
    fmt.setDepthBufferSize(24)
    fmt.setStencilBufferSize(8)
    # Special case macOS to allow using OpenGL there.
    # (the default Metal is the recommended approach, though)
    # gl_VertexID is a GLSL 130 feature, and so the default OpenGL 2.1 context
    # we get on macOS is not sufficient.
    if sys.platform == "darwin":
        fmt.setVersion(4, 1)
        fmt.setProfile(QSurfaceFormat.CoreProfile)
    QSurfaceFormat.setDefaultFormat(fmt)

    window = HelloWindow(graphicsApi)

    window.resize(1280, 720)
    title = QCoreApplication.applicationName() + " - " + window.graphicsApiName()
    window.setTitle(title)
    window.show()

    ret = app.exec()

    # RhiWindow::event() will not get invoked when the
    # PlatformSurfaceAboutToBeDestroyed event is sent during the QWindow
    # destruction. That happens only when exiting via app::quit() instead of
    # the more common QWindow::close(). Take care of it: if the QPlatformWindow
    # is still around (there was no close() yet), get rid of the swapchain
    # while it's not too late.
    if window.isVisible():
        window.releaseSwapChain()

    sys.exit(ret)
# Copyright (C) 2023 The Qt Company Ltd.
# SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

import numpy
import sys

from PySide6.QtCore import (QEvent, QFile, QIODevice, QPointF, QRectF, QSize,
                            qFatal, qWarning, Qt)
from PySide6.QtGui import (QColor, QFont, QGradient, QImage, QMatrix4x4,
                           QPainter, QPlatformSurfaceEvent, QSurface, QWindow)
from PySide6.QtGui import (QRhi, QRhiBuffer,
                           QRhiDepthStencilClearValue,
                           QRhiGraphicsPipeline, QRhiNullInitParams,
                           QRhiGles2InitParams, QRhiRenderBuffer,
                           QRhiSampler, QRhiShaderResourceBinding,
                           QRhiShaderStage, QRhiTexture,
                           QRhiVertexInputAttribute, QRhiVertexInputBinding,
                           QRhiVertexInputLayout, QRhiViewport,
                           QShader)
from PySide6.support import VoidPtr

if sys.platform == "win32":
    from PySide6.QtGui import QRhiD3D11InitParams, QRhiD3D12InitParams
elif sys.platform == "darwin":
    from PySide6.QtGui import QRhiMetalInitParams


# Y up (note clipSpaceCorrMatrix in m_viewProjection), CCW
VERTEX_DATA = numpy.array([
    0.0, 0.5, 1.0, 0.0, 0.0,
    -0.5, -0.5, 0.0, 1.0, 0.0,
    0.5, -0.5, 0.0, 0.0, 1.0], dtype=numpy.float32)


UBUF_SIZE = 68


def getShader(name):
    f = QFile(name)
    if f.open(QIODevice.ReadOnly):
        result = QShader.fromSerialized(f.readAll())
        f.close()
        return result
    return QShader()


class RhiWindow(QWindow):

    def __init__(self, graphicsApi):
        super().__init__()
        self.m_graphicsApi = QRhi.Null
        self.m_initialized = False
        self.m_notExposed = False
        self.m_newlyExposed = False

        self.m_fallbackSurface = None
        self.m_rhi = None
        self.m_sc = None
        self.m_ds = None
        self.m_rp = None
        self.m_hasSwapChain = False
        self.m_viewProjection = QMatrix4x4()

        self.m_graphicsApi = graphicsApi

        if graphicsApi == QRhi.OpenGLES2:
            self.setSurfaceType(QSurface.SurfaceType.OpenGLSurface)
        elif graphicsApi == QRhi.Vulkan:
            self.setSurfaceType(QSurface.SurfaceType.VulkanSurface)
        elif graphicsApi == QRhi.D3D11 or graphicsApi == QRhi.D3D12:
            self.setSurfaceType(QSurface.SurfaceType.Direct3DSurface)
        elif graphicsApi == QRhi.Metal:
            self.setSurfaceType(QSurface.SurfaceType.MetalSurface)
        elif graphicsApi == QRhi.Null:
            pass  # RasterSurface

    def __del__(self):
        # destruction order matters to a certain degree: the fallbackSurface
        # must outlive the rhi, the rhi must outlive all other resources.
        # The resources need no special order when destroying.
        del self.m_rp
        self.m_rp = None
        del self.m_ds
        self.m_ds = None
        del self.m_sc
        self.m_sc = None
        del self.m_rhi
        self.m_rhi = None
        if self.m_fallbackSurface:
            del self.m_fallbackSurface
            self.m_fallbackSurface = None

    def graphicsApiName(self):
        if self.m_graphicsApi == QRhi.Null:
            return "Null (no output)"
        if self.m_graphicsApi == QRhi.OpenGLES2:
            return "OpenGL"
        if self.m_graphicsApi == QRhi.Vulkan:
            return "Vulkan"
        if self.m_graphicsApi == QRhi.D3D11:
            return "Direct3D 11"
        if self.m_graphicsApi == QRhi.D3D12:
            return "Direct3D 12"
        if self.m_graphicsApi == QRhi.Metal:
            return "Metal"
        return ""

    def customInit(self):
        pass

    def customRender(self):
        pass

    def exposeEvent(self, e):
        # initialize and start rendering when the window becomes usable
        # for graphics purposes
        is_exposed = self.isExposed()
        if is_exposed and not self.m_initialized:
            self.init()
            self.resizeSwapChain()
            self.m_initialized = True

        surfaceSize = self.m_sc.surfacePixelSize() if self.m_hasSwapChain else QSize()

        # stop pushing frames when not exposed (or size is 0)
        if ((not is_exposed or (self.m_hasSwapChain and surfaceSize.isEmpty()))
                and self.m_initialized and not self.m_notExposed):
            self.m_notExposed = True

        # Continue when exposed again and the surface has a valid size. Note
        # that surfaceSize can be (0, 0) even though size() reports a valid
        # one, hence trusting surfacePixelSize() and not QWindow.
        if is_exposed and self.m_initialized and self.m_notExposed and not surfaceSize.isEmpty():
            self.m_notExposed = False
            self.m_newlyExposed = True

        # always render a frame on exposeEvent() (when exposed) in order to
        # update immediately on window resize.
        if is_exposed and not surfaceSize.isEmpty():
            self.render()

    def event(self, e):
        if e.type() == QEvent.UpdateRequest:
            self.render()
        elif e.type() == QEvent.PlatformSurface:
            # this is the proper time to tear down the swapchain (while
            # the native window and surface are still around)
            if e.surfaceEventType() == QPlatformSurfaceEvent.SurfaceAboutToBeDestroyed:
                self.releaseSwapChain()

        return super().event(e)

    def init(self):
        if self.m_graphicsApi == QRhi.Null:
            params = QRhiNullInitParams()
            self.m_rhi = QRhi.create(QRhi.Null, params)

        if self.m_graphicsApi == QRhi.OpenGLES2:
            self.m_fallbackSurface = QRhiGles2InitParams.newFallbackSurface()
            params = QRhiGles2InitParams()
            params.fallbackSurface = self.m_fallbackSurface
            params.window = self
            self.m_rhi = QRhi.create(QRhi.OpenGLES2, params)
        elif self.m_graphicsApi == QRhi.D3D11:
            params = QRhiD3D11InitParams()
            # Enable the debug layer, if available. This is optional
            # and should be avoided in production builds.
            params.enableDebugLayer = True
            self.m_rhi = QRhi.create(QRhi.D3D11, params)
        elif self.m_graphicsApi == QRhi.D3D12:
            params = QRhiD3D12InitParams()
            # Enable the debug layer, if available. This is optional
            # and should be avoided in production builds.
            params.enableDebugLayer = True
            self.m_rhi = QRhi.create(QRhi.D3D12, params)
        elif self.m_graphicsApi == QRhi.Metal:
            params = QRhiMetalInitParams()
            self.m_rhi.reset(QRhi.create(QRhi.Metal, params))

        if not self.m_rhi:
            qFatal("Failed to create RHI backend")

        self.m_sc = self.m_rhi.newSwapChain()
        # no need to set the size here, due to UsedWithSwapChainOnly
        self.m_ds = self.m_rhi.newRenderBuffer(QRhiRenderBuffer.DepthStencil,
                                               QSize(), 1,
                                               QRhiRenderBuffer.UsedWithSwapChainOnly)
        self.m_sc.setWindow(self)
        self.m_sc.setDepthStencil(self.m_ds)
        self.m_rp = self.m_sc.newCompatibleRenderPassDescriptor()
        self.m_sc.setRenderPassDescriptor(self.m_rp)
        self.customInit()

    def resizeSwapChain(self):
        self.m_hasSwapChain = self.m_sc.createOrResize()  # also handles self.m_ds
        outputSize = self.m_sc.currentPixelSize()
        self.m_viewProjection = self.m_rhi.clipSpaceCorrMatrix()
        r = float(outputSize.width()) / float(outputSize.height())
        self.m_viewProjection.perspective(45.0, r, 0.01, 1000.0)
        self.m_viewProjection.translate(0, 0, -4)

    def releaseSwapChain(self):
        if self.m_hasSwapChain:
            self.m_hasSwapChain = False
            self.m_sc.destroy()

    def render(self):
        if not self.m_hasSwapChain or self.m_notExposed:
            return

        # If the window got resized or newly exposed, resize the swapchain.
        # (the newly-exposed case is not actually required by some platforms,
        # but is here for robustness and portability)
        #
        # This (exposeEvent + the logic here) is the only safe way to perform
        # resize handling. Note the usage of the RHI's surfacePixelSize(), and
        # never QWindow::size(). (the two may or may not be the same under the
        # hood, # depending on the backend and platform)
        if self.m_sc.currentPixelSize() != self.m_sc.surfacePixelSize() or self.m_newlyExposed:
            self.resizeSwapChain()
            if not self.m_hasSwapChain:
                return
            self.m_newlyExposed = False

        result = self.m_rhi.beginFrame(self.m_sc)
        if result == QRhi.FrameOpSwapChainOutOfDate:
            self.resizeSwapChain()
            if not self.m_hasSwapChain:
                return
            result = self.m_rhi.beginFrame(self.m_sc)

        if result != QRhi.FrameOpSuccess:
            qWarning(f"beginFrame failed with {result}, will retry")
            self.requestUpdate()
            return

        self.customRender()

        self.m_rhi.endFrame(self.m_sc)

        # Always request the next frame via requestUpdate(). On some platforms
        # this is backed by a platform-specific solution, e.g. CVDisplayLink
        # on macOS, which is potentially more efficient than a timer,
        # queued metacalls, etc.
        self.requestUpdate()


class HelloWindow(RhiWindow):

    def __init__(self, graphicsApi):
        super().__init__(graphicsApi)
        self.m_vbuf = None
        self.m_ubuf = None
        self.m_texture = None
        self.m_sampler = None
        self.m_colorTriSrb = None
        self.m_colorPipeline = None
        self.m_fullscreenQuadSrb = None
        self.m_fullscreenQuadPipeline = None
        self.m_initialUpdates = None

        self.m_rotation = 0
        self.m_opacity = 1
        self.m_opacityDir = -1

    def ensureFullscreenTexture(self, pixelSize, u):
        if self.m_texture and self.m_texture.pixelSize() == pixelSize:
            return

        if not self.m_texture:
            self.m_texture = self.m_rhi.newTexture(QRhiTexture.RGBA8, pixelSize)
        else:
            self.m_texture.setPixelSize(pixelSize)
        self.m_texture.create()
        image = QImage(pixelSize, QImage.Format_RGBA8888_Premultiplied)
        with QPainter(image) as painter:
            painter.fillRect(QRectF(QPointF(0, 0), pixelSize),
                             QColor.fromRgbF(0.4, 0.7, 0.0, 1.0))
            painter.setPen(Qt.transparent)
            painter.setBrush(QGradient(QGradient.DeepBlue))
            painter.drawRoundedRect(QRectF(QPointF(20, 20), pixelSize - QSize(40, 40)),
                                    16, 16)
            painter.setPen(Qt.black)
            font = QFont()
            font.setPixelSize(0.05 * min(pixelSize.width(), pixelSize.height()))
            painter.setFont(font)
            name = self.graphicsApiName()
            t = (f"Rendering with QRhi to a resizable QWindow.\nThe 3D API is {name}."
                 "\nUse the command-line options to choose a different API.")
            painter.drawText(QRectF(QPointF(60, 60), pixelSize - QSize(120, 120)), 0, t)

        if self.m_rhi.isYUpInNDC():
            image = image.mirrored()

        u.uploadTexture(self.m_texture, image)

    def customInit(self):
        self.m_initialUpdates = self.m_rhi.nextResourceUpdateBatch()

        vertex_size = 4 * VERTEX_DATA.size
        self.m_vbuf = self.m_rhi.newBuffer(QRhiBuffer.Immutable, QRhiBuffer.VertexBuffer,
                                           vertex_size)
        self.m_vbuf.create()
        self.m_initialUpdates.uploadStaticBuffer(self.m_vbuf,
                                                 VoidPtr(VERTEX_DATA.tobytes(), vertex_size))

        self.m_ubuf = self.m_rhi.newBuffer(QRhiBuffer.Dynamic,
                                           QRhiBuffer.UniformBuffer, UBUF_SIZE)
        self.m_ubuf.create()

        self.ensureFullscreenTexture(self.m_sc.surfacePixelSize(), self.m_initialUpdates)

        self.m_sampler = self.m_rhi.newSampler(QRhiSampler.Linear, QRhiSampler.Linear,
                                               QRhiSampler.None_,
                                               QRhiSampler.ClampToEdge, QRhiSampler.ClampToEdge)
        self.m_sampler.create()

        self.m_colorTriSrb = self.m_rhi.newShaderResourceBindings()
        visibility = (QRhiShaderResourceBinding.VertexStage
                      | QRhiShaderResourceBinding.FragmentStage)
        bindings = [
            QRhiShaderResourceBinding.uniformBuffer(0, visibility, self.m_ubuf)
        ]
        self.m_colorTriSrb.setBindings(bindings)
        self.m_colorTriSrb.create()

        self.m_colorPipeline = self.m_rhi.newGraphicsPipeline()
        # Enable depth testing; not quite needed for a simple triangle, but we
        # have a depth-stencil buffer so why not.
        self.m_colorPipeline.setDepthTest(True)
        self.m_colorPipeline.setDepthWrite(True)
        # Blend factors default to One, OneOneMinusSrcAlpha, which is convenient.
        premulAlphaBlend = QRhiGraphicsPipeline.TargetBlend()
        premulAlphaBlend.enable = True
        self.m_colorPipeline.setTargetBlends([premulAlphaBlend])
        stages = [
            QRhiShaderStage(QRhiShaderStage.Vertex, getShader(":/color.vert.qsb")),
            QRhiShaderStage(QRhiShaderStage.Fragment, getShader(":/color.frag.qsb"))
        ]
        self.m_colorPipeline.setShaderStages(stages)
        inputLayout = QRhiVertexInputLayout()
        input_bindings = [QRhiVertexInputBinding(5 * 4)]  # sizeof(float)
        inputLayout.setBindings(input_bindings)
        attributes = [
            QRhiVertexInputAttribute(0, 0, QRhiVertexInputAttribute.Float2, 0),
            QRhiVertexInputAttribute(0, 1, QRhiVertexInputAttribute.Float3, 2 * 4)]  # sizeof(float)
        inputLayout.setAttributes(attributes)
        self.m_colorPipeline.setVertexInputLayout(inputLayout)
        self.m_colorPipeline.setShaderResourceBindings(self.m_colorTriSrb)
        self.m_colorPipeline.setRenderPassDescriptor(self.m_rp)
        self.m_colorPipeline.create()

        self.m_fullscreenQuadSrb = self.m_rhi.newShaderResourceBindings()
        bindings = [
            QRhiShaderResourceBinding.sampledTexture(0, QRhiShaderResourceBinding.FragmentStage,
                                                     self.m_texture, self.m_sampler)
        ]
        self.m_fullscreenQuadSrb.setBindings(bindings)
        self.m_fullscreenQuadSrb.create()

        self.m_fullscreenQuadPipeline = self.m_rhi.newGraphicsPipeline()
        stages = [
            QRhiShaderStage(QRhiShaderStage.Vertex, getShader(":/quad.vert.qsb")),
            QRhiShaderStage(QRhiShaderStage.Fragment, getShader(":/quad.frag.qsb"))
        ]
        self.m_fullscreenQuadPipeline.setShaderStages(stages)
        layout = QRhiVertexInputLayout()
        self.m_fullscreenQuadPipeline.setVertexInputLayout(layout)
        self.m_fullscreenQuadPipeline.setShaderResourceBindings(self.m_fullscreenQuadSrb)
        self.m_fullscreenQuadPipeline.setRenderPassDescriptor(self.m_rp)
        self.m_fullscreenQuadPipeline.create()

    def customRender(self):
        resourceUpdates = self.m_rhi.nextResourceUpdateBatch()

        if self.m_initialUpdates:
            resourceUpdates.merge(self.m_initialUpdates)
            self.m_initialUpdates = None

        self.m_rotation += 1.0
        modelViewProjection = self.m_viewProjection
        modelViewProjection.rotate(self.m_rotation, 0, 1, 0)
        projection = numpy.array(modelViewProjection.data(),
                                 dtype=numpy.float32)
        resourceUpdates.updateDynamicBuffer(self.m_ubuf, 0, 64,
                                            projection.tobytes())

        self.m_opacity += self.m_opacityDir * 0.005
        if self.m_opacity < 0.0 or self.m_opacity > 1.0:
            self.m_opacityDir *= -1
            self.m_opacity = max(0.0, min(1.0, self.m_opacity))

        opacity = numpy.array([self.m_opacity], dtype=numpy.float32)
        resourceUpdates.updateDynamicBuffer(self.m_ubuf, 64, 4,
                                            opacity.tobytes())

        cb = self.m_sc.currentFrameCommandBuffer()
        outputSizeInPixels = self.m_sc.currentPixelSize()

        # (re)create the texture with a size matching the output surface size,
        # when necessary.
        self.ensureFullscreenTexture(outputSizeInPixels, resourceUpdates)

        cv = QRhiDepthStencilClearValue(1.0, 0)
        cb.beginPass(self.m_sc.currentFrameRenderTarget(), Qt.black,
                     cv, resourceUpdates)

        cb.setGraphicsPipeline(self.m_fullscreenQuadPipeline)
        viewport = QRhiViewport(0, 0, outputSizeInPixels.width(),
                                outputSizeInPixels.height())
        cb.setViewport(viewport)
        cb.setShaderResources()
        cb.draw(3)

        cb.setGraphicsPipeline(self.m_colorPipeline)
        cb.setShaderResources()
        vbufBinding = (self.m_vbuf, 0)
        cb.setVertexInput(0, [vbufBinding])
        cb.draw(3)
        cb.endPass()
<!DOCTYPE RCC><RCC version="1.0">
<qresource>
  <file alias="color.vert.qsb">shaders/prebuilt/color.vert.qsb</file>
  <file alias="color.frag.qsb">shaders/prebuilt/color.frag.qsb</file>
  <file alias="quad.vert.qsb">shaders/prebuilt/quad.vert.qsb</file>
  <file alias="quad.frag.qsb">shaders/prebuilt/quad.frag.qsb</file>
</qresource>
</RCC>
#version 440

layout(location = 0) in vec3 v_color;

layout(location = 0) out vec4 fragColor;

layout(std140, binding = 0) uniform buf {
    mat4 mvp;
    float opacity;
};

void main()
{
    fragColor = vec4(v_color * opacity, opacity);
}
#version 440

layout(location = 0) in vec4 position;
layout(location = 1) in vec3 color;

layout(location = 0) out vec3 v_color;

layout(std140, binding = 0) uniform buf {
    mat4 mvp;
    float opacity;
};

void main()
{
    v_color = color;
    gl_Position = mvp * position;
}
#version 440

layout(location = 0) in vec2 v_uv;
layout(location = 0) out vec4 fragColor;
layout(binding = 0) uniform sampler2D tex;

void main()
{
    vec4 c = texture(tex, v_uv);
    fragColor = vec4(c.rgb * c.a, c.a);
}
#version 440

layout (location = 0) out vec2 v_uv;

void main()
{
    // https://www.saschawillems.de/blog/2016/08/13/vulkan-tutorial-on-rendering-a-fullscreen-quad-without-buffers/
    v_uv = vec2((gl_VertexIndex << 1) & 2, gl_VertexIndex & 2);
    gl_Position = vec4(v_uv * 2.0 - 1.0, 0.0, 1.0);
}