Getting graphics on the screen Implementing vector graphics support

C

Implementing hardware layer support

Some hardware platforms do support layers, which means you can draw content onto multiple independent visual layers and let the hardware blend them together. The hardware might also let you define layer content directly from image data in flash. This should help reduce volatile memory usage by using images for static background content and framebuffers for dynamic content.

This chapter explains how hardware layer support could be implemented if the target platform supports it. If not, this entire chapter can be skipped, and the layer related code paths can be removed from the platform code.

The layer engine interface

First, override the Qul::PlatformInterface::LayerEngine interface, which is used by Qt Quick Ultralite core to allocate, update, and deallocate platform layers.

class ExampleLayerEngine : public PlatformInterface::LayerEngine
{
public:
    ItemLayer *allocateItemLayer(const PlatformInterface::Screen *,
                                 const ItemLayerProperties &props,
                                 SpriteLayer *spriteLayer = NULL) QUL_DECL_OVERRIDE;
    ImageLayer *allocateImageLayer(const PlatformInterface::Screen *,
                                   const ImageLayerProperties &props,
                                   SpriteLayer *spriteLayer = NULL) QUL_DECL_OVERRIDE;
    SpriteLayer *allocateSpriteLayer(const PlatformInterface::Screen *,
                                     const SpriteLayerProperties &props) QUL_DECL_OVERRIDE;

    void deallocateItemLayer(ItemLayer *layer) QUL_DECL_OVERRIDE;
    void deallocateImageLayer(ImageLayer *layer) QUL_DECL_OVERRIDE;
    void deallocateSpriteLayer(SpriteLayer *layer) QUL_DECL_OVERRIDE;

    void updateItemLayer(ItemLayer *layer, const ItemLayerProperties &props) QUL_DECL_OVERRIDE;
    void updateImageLayer(ImageLayer *layer, const ImageLayerProperties &props) QUL_DECL_OVERRIDE;
    void updateSpriteLayer(SpriteLayer *layer, const SpriteLayerProperties &props) QUL_DECL_OVERRIDE;

    static PlatformInterface::DrawingDevice *beginFrame(const PlatformInterface::LayerEngine::ItemLayer *,
                                                        const PlatformInterface::Rect &,
                                                        int refreshInterval);
    static void endFrame(const PlatformInterface::LayerEngine::ItemLayer *);
    static FrameStatistics presentFrame(const PlatformInterface::Screen *screen, const PlatformInterface::Rect &rect);

    static Qul::Platform::FramebufferFormat framebufferFormat(const PlatformInterface::LayerEngine::ItemLayer *layer);
};

The example class also includes custom implementations of PlatformContext::beginFrame, PlatformContext::endFrame, and PlatformContext::presentFrame. If enableLayerEngine is set in the example platform_context.cpp, these functions will forward to the ExampleLayerEngine versions instead of the default non-layer implementation.

In order to use the ExampleLayerEngine, enable layer support in the example platform. The enableLayerEngine value must be set to true (by default it is false as shown below):

const bool enableLayerEngine = false;

If enableLayerEngine is set to true, the skipped parts in the PlatformContext::beginFrame, PlatformContext::endFrame, and PlatformContext::presentFrame methods could be removed.

In addition, the PlatformContext::layerEngine method must be implemented to return a pointer to an instance of Qul::PlatformInterface::LayerEngine. In this case, it is a statically allocated ExampleLayerEngine instance:

PlatformInterface::LayerEngine *ExamplePlatform::layerEngine()
{
    if (enableLayerEngine) {
        static ExampleLayerEngine layerEngine;
        return &layerEngine;
    } else {
        return NULL;
    }
}

Implementation of the layer engine interface

In order to implement the layer engine interface, create sub-classes of the Qul::PlatformInterface::LayerEngine::ItemLayer, Qul::PlatformInterface::LayerEngine::ImageLayer, and Qul::PlatformInterface::LayerEngine::SpriteLayer opaque types. These sub-classes can be used to store platform-specific layer data.

Image and item layers that are inside a sprite layer require a different implementation than image and item layers that are root layers. In the example platform, the ExampleImageLayer and ExampleImageSprite classes are used to represent a root image layer and a sprite image layer respectively. Similarly, the ExampleItemLayer and ExampleItemSprite classes represent the item layer, while the ExampleSpriteLayer class represents the sprite layer.

The following snippet shows how these are allocated in the example platform. If multiple screen support is desired, the Qul::PlatformInterface::Screen pointer must be taken into account to display the layer on the correct screen.

PlatformInterface::LayerEngine::ItemLayer *ExampleLayerEngine::allocateItemLayer(const PlatformInterface::Screen *,
                                                                                 const ItemLayerProperties &props,
                                                                                 SpriteLayer *spriteLayer)
{
    ItemLayer *layer;
    if (spriteLayer) {
        ExampleSpriteLayer *exampleSpriteLayer = static_cast<ExampleSpriteLayer *>(spriteLayer);
        layer = new ExampleItemSprite(props, exampleSpriteLayer);
    } else {
        layer = new ExampleItemLayer(props);
    }

    return layer;
}

PlatformInterface::LayerEngine::ImageLayer *ExampleLayerEngine::allocateImageLayer(const PlatformInterface::Screen *,
                                                                                   const ImageLayerProperties &props,
                                                                                   SpriteLayer *spriteLayer)
{
    ImageLayer *layer;
    if (spriteLayer) {
        ExampleSpriteLayer *exampleSpriteLayer = static_cast<ExampleSpriteLayer *>(spriteLayer);
        layer = new ExampleImageSprite(props, exampleSpriteLayer);
    } else {
        layer = new ExampleImageLayer(props);
    }

    return layer;
}

PlatformInterface::LayerEngine::SpriteLayer *ExampleLayerEngine::allocateSpriteLayer(const PlatformInterface::Screen *,
                                                                                     const SpriteLayerProperties &props)
{
    return new ExampleSpriteLayer(props);
}

In order to safely downcast the Qul::PlatformInterface::LayerEngine::ImageLayer opaque type, the ExampleImageLayer and ExampleImageSprite inherit from ExampleImageLayerBase, which in-turn inherits from Qul::PlatformInterface::LayerEngine::ImageLayer:

struct ExampleImageLayerBase : public Qul::PlatformInterface::LayerEngine::ImageLayer
{
    virtual ~ExampleImageLayerBase() {}
    virtual void updateProperties(const Qul::PlatformInterface::LayerEngine::ImageLayerProperties &props) = 0;
};

This contains a virtual destructor so that the correct sub-class destructor is called, and also a virtual function to update the properties of the image layer when they change.

Similarly, there's ExampleItemLayerBase that is inheritted by ExampleItemLayer and ExampleSpriteLayer:

struct ExampleItemLayerBase : public Qul::PlatformInterface::LayerEngine::ItemLayer
{
    ExampleItemLayerBase(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &props);

    virtual ~ExampleItemLayerBase() {}
    virtual void updateProperties(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &props) = 0;
    virtual unsigned char *getNextDrawBuffer() = 0;
    virtual void swap() = 0;

    Qul::PlatformInterface::DrawingDevice drawingDevice;

    unsigned int lastSwapFrame;
    unsigned int targetFrame;
};

In addition to the virtual destructor and updateProperties method, there are also virtual methods to get the next drawbuffer and swap the buffers of the item layer once a frame has been rendered. There are also variables to support dynamic refresh interval and track the frame that the item layer updates are meant for.

With the virtual destructors, the deallocation methods can be implemented by downcasting to the appropriate platform class and calling delete:

void ExampleLayerEngine::deallocateItemLayer(PlatformInterface::LayerEngine::ItemLayer *layer)
{
    delete static_cast<ExampleItemLayerBase *>(layer);
}

void ExampleLayerEngine::deallocateImageLayer(PlatformInterface::LayerEngine::ImageLayer *layer)
{
    delete static_cast<ExampleImageLayerBase *>(layer);
}

void ExampleLayerEngine::deallocateSpriteLayer(PlatformInterface::LayerEngine::SpriteLayer *layer)
{
    delete static_cast<ExampleSpriteLayer *>(layer);
}

The methods used to update layer properties are also implemented in a similar way, forwarding to the appropriate sub-class:

void ExampleLayerEngine::updateItemLayer(PlatformInterface::LayerEngine::ItemLayer *layer,
                                         const ItemLayerProperties &props)
{
    static_cast<ExampleItemLayerBase *>(layer)->updateProperties(props);
}

void ExampleLayerEngine::updateImageLayer(PlatformInterface::LayerEngine::ImageLayer *layer,
                                          const ImageLayerProperties &props)
{
    static_cast<ExampleImageLayerBase *>(layer)->updateProperties(props);
}

void ExampleLayerEngine::updateSpriteLayer(PlatformInterface::LayerEngine::SpriteLayer *layer,
                                           const SpriteLayerProperties &props)
{
    static_cast<ExampleSpriteLayer *>(layer)->updateProperties(props);
}

Hardware layer base class

The ExampleHardwareLayer class is the base class for the ExampleItemLayer, ExampleImageLayer, and ExampleSpriteLayer classes. It demonstrates how a root (non-sprite) hardware layer can be initialized and updated.

Using a dummy hardware-acceleration API, this is how the constructor might look:

struct ExampleHardwareLayer
{
    Qul::PlatformInterface::LayerEngine::LayerPropertiesBase props;
    Qul::PlatformInterface::Size size;
    // hw_layer_t hwLayer;

    enum Type { SpriteLayer, FramebufferLayer };

    ExampleHardwareLayer(const Qul::PlatformInterface::LayerEngine::LayerPropertiesBase &p,
                         const Qul::PlatformInterface::Size &s,
                         HwPixelFormat pixelFormat,
                         Type type)
        : props(p)
        , size(s)
    {
        // hw_layer_create_data_t layerCreateData;
        // layerCreateData.opacity = p.opacity;
        // layerCreateData.x = p.position.x();
        // layerCreateData.y = p.position.y();
        // layerCreateData.z = p.z;
        // layerCreateData.width = size.width();
        // layerCreateData.height = size.height();
        // layerCreateData.pixelFormat = pixelFormat;
        // layerCreateData.type = type == SpriteLayer ? HW_LAYER_TYPE_SPRITE : HW_LAYER_TYPE_FRAMEBUFFER;

        // HW_LayerCreate(&hwLayer, &layerData);
    }

It sets up all the properties of the hardware layer and calls a hardware layer API function to create the hardware layer.

Next, updateProperties() updates the hardware layer based on the basic properties that are common for all the different layer types, such as size, position, opacity, and enabled status:

void updateProperties(const Qul::PlatformInterface::LayerEngine::LayerPropertiesBase &p,
                      const Qul::PlatformInterface::Size &s)
{
    if (size != s) {
        // HW_LayerResize(&hwLayer, p.size.width(), p.size.height());
    }

    if (props.position != p.position || props.z != p.z) {
        // HW_LayerMove(&hwLayer, p.position.x(), p.position.y(), p.z);
    }

    if (props.opacity != p.opacity) {
        // HW_LayerSetOpacity(&hwLayer, p.opacity);
    }

    if (p.enabled != props.enabled) {
        if (p.enabled) {
            // HW_LayerEnable(&hwLayer);
        } else {
            // HW_LayerDisable(&hwLayer);
        }
    }

    size = s;
    props = p;
}

The destructor deletes the hardware layer resource:

~ExampleHardwareLayer()
{
    // HW_LayerDelete(&hwLayer);
}

Item layer classes

First, the ExampleItemLayer class allocates and destroys framebuffers for the given hardware layer:

struct ExampleItemLayer : public ExampleItemLayerBase, public ExampleHardwareLayer
{
    ExampleItemLayer(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &props)
        : ExampleItemLayerBase(props)
        , ExampleHardwareLayer(props,
                               props.size,
                               toHwPixelFormat(props.colorDepth),
                               ExampleHardwareLayer::FramebufferLayer)
    {
        // Allocate double buffers for hardware framebuffer layer
        // HW_CreateFramebuffers(&hwLayer, props.size.width(), props.size.height(), toHwPixelFormat(props.colorFormat), 2);
    }

    ~ExampleItemLayer()
    {
        // HW_DestroyFramebuffers(&hwLayer, props.size.width(), props.size.height(), toHwPixelFormat(props.colorFormat), 2);
    }

The updateProperties method forwards to the ExampleHardwareLayer base implementation:

virtual void updateProperties(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &props) QUL_DECL_OVERRIDE
{
    ExampleHardwareLayer::updateProperties(props, props.size);
}

The getNextDrawBuffer waits until the back buffer is ready before returning its address, and swap calls a hardware layer method to swap the front and back buffers:

virtual unsigned char *getNextDrawBuffer() QUL_DECL_OVERRIDE
{
    unsigned char *bits = NULL;
    do {
        // bits = HW_GetNextFramebuffer(&hwLayer);
    } while (!bits);
    return bits;
}

virtual void swap() QUL_DECL_OVERRIDE
{
    // HW_FinishRendering();
    // HW_SwapFramebuffers(&hwLayer);
}

Assuming that the address of the sprite data can be any valid pointer, in the example platform the ExampleItemSprite class the two framebuffers are allocated and freed manually:

struct ExampleItemSprite : public ExampleItemLayerBase
{
    ExampleItemSprite(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &p, ExampleSpriteLayer *spriteLayer)
        : ExampleItemLayerBase(props)
        , props(p)
        , spriteLayer(spriteLayer)
        , didCreate(false)
        , frontBufferIndex(0)
    {
        // For item sprites we manually allocate the framebuffers
        for (int i = 0; i < 2; ++i)
            framebuffers[i] = (unsigned char *) qul_malloc(p.size.width() * p.size.height()
                                                           * bytesPerPixel(p.colorDepth));
    }

    ~ExampleItemSprite()
    {
        for (int i = 0; i < 2; ++i)
            qul_free(framebuffers[i]);

        if (didCreate) {
            // HW_SpriteDelete(&hwSprite);
        }
    }

The sprite related properties that need updating are position and whether the sprite is enabled or not:

virtual void updateProperties(const Qul::PlatformInterface::LayerEngine::ItemLayerProperties &p) QUL_DECL_OVERRIDE
{
    if (!didCreate)
        return;

    if (props.position != p.position || props.z != p.z) {
        // HW_SpriteMove(&hwSprite, p.position.x(), p.position.y(), p.z);
    }

    if (props.enabled != p.enabled) {
        if (p.enabled) {
            // HW_SpriteEnable(&hwSprite);
        } else {
            // HW_SpriteDisable(&hwSprite);
        }
    }

    props = p;
}

The getNextDrawBuffer returns a pointer to the back buffer, and the swap override updates the sprite data, or creates the sprite if it was not yet created:

virtual unsigned char *getNextDrawBuffer() QUL_DECL_OVERRIDE { return framebuffers[!frontBufferIndex]; }

virtual void swap() QUL_DECL_OVERRIDE
{
    // HW_FinishRendering();

    frontBufferIndex = !frontBufferIndex;

    if (didCreate) {
        // HW_SpriteSetData(&hwSprite, framebuffers[frontBufferIndex]);
    } else {
        create();
        didCreate = true;
    }
}

Creating a hardware sprite might look like this:

void create()
{
    // hw_sprite_create_data_t spriteCreateData;
    // spriteCreateData.x = props.position.x();
    // spriteCreateData.y = props.position.y();
    // spriteCreateData.z = props.z;
    // spriteCreateData.width = props.size.width();
    // spriteCreateData.height = props.size.height();
    // spriteCreateData.pixelFormat = toHwPixelFormat(props.colorDepth);
    // spriteCreateData.data = framebuffers[frontBufferIndex];
    // spriteCreateData.layer = &spriteLayer->hwLayer;
    // HW_SpriteCreate(&hwSprite, &spriteLayer->hwLayer);
}

Image layer classes

The ExampleImageLayer class relies on the base class ExampleHardwareLayer for creation and property updates, and in addition directly sets the texture data as the framebuffer:

struct ExampleImageLayer : public ExampleImageLayerBase, public ExampleHardwareLayer
{
    ExampleImageLayer(const Qul::PlatformInterface::LayerEngine::ImageLayerProperties &p)
        : ExampleHardwareLayer(p,
                               p.texture.size(),
                               toHwPixelFormat(p.texture.format()),
                               ExampleHardwareLayer::FramebufferLayer)
    {
        // HW_LayerSetFramebuffer(&hwLayer, p.texture.data(), p.texture.bytesPerLine());
    }

    void updateProperties(const Qul::PlatformInterface::LayerEngine::ImageLayerProperties &p) QUL_DECL_OVERRIDE
    {
        ExampleHardwareLayer::updateProperties(p, p.texture.size());
        // HW_LayerSetFramebuffer(&hwLayer, p.texture.data(), p.texture.bytesPerLine());
    }

Qt Quick Ultralite core will ensure the texture data remains valid as long as the image layer is visible.

For ExampleImageSprite, the hardware sprite can be created immediately in the constructor, and can be destroyed in the destructor:

struct ExampleImageSprite : public ExampleImageLayerBase
{
    ExampleImageSprite(const Qul::PlatformInterface::LayerEngine::ImageLayerProperties &p,
                       ExampleSpriteLayer *spriteLayer)
        : props(p)
    {
        // hw_sprite_create_data_t spriteCreateData;
        // spriteCreateData.x = p.position.x();
        // spriteCreateData.y = p.position.y();
        // spriteCreateData.z = p.z;
        // spriteCreateData.width = p.size.width();
        // spriteCreateData.height = p.size.height();
        // spriteCreateData.pixelFormat = toHwPixelFormat(p.texture.format());
        // spriteCreateData.data = p.texture.data();
        // spriteCreateData.layer = &spriteLayer->hwLayer;

        // HW_SpriteCreate(&hwSprite, &spriteLayer->hwLayer);
    }

    ~ExampleImageSprite()
    {
        // HW_SpriteDelete(&hwSprite);
    }

The updateProperties override is similar to the one for the item sprite, except that the texture data might also have changed:

void updateProperties(const Qul::PlatformInterface::LayerEngine::ImageLayerProperties &p) QUL_DECL_OVERRIDE
{
    if (props.texture.data() != p.texture.data()) {
        // HW_SpriteSetData(&hwSprite, p.texture.data());
    }

    if (props.position != p.position || props.z != p.z) {
        // HW_SpriteMove(&hwSprite, p.position.x(), p.position.y(), p.z);
    }

    if (props.enabled != p.enabled) {
        if (p.enabled) {
            // HW_SpriteEnable(&hwSprite);
        } else {
            // HW_SpriteDisable(&hwSprite);
        }
    }

    props = p;
}

Sprite layer class

The sprite layer is a simple container for item and image sprites. The ExampleSpriteLayer class doesn't need to handle anything specific as the position, opacity, and size properties are handled by the ExampleHardwareLayer base class.

struct ExampleSpriteLayer : public Qul::PlatformInterface::LayerEngine::SpriteLayer, public ExampleHardwareLayer
{
    ExampleSpriteLayer(const Qul::PlatformInterface::LayerEngine::SpriteLayerProperties &props)
        : ExampleHardwareLayer(props, props.size, toHwPixelFormat(props.colorDepth), ExampleHardwareLayer::SpriteLayer)
    {}

    void updateProperties(const Qul::PlatformInterface::LayerEngine::SpriteLayerProperties &props)
    {
        ExampleHardwareLayer::updateProperties(props, props.size);
    }

    static void *operator new(std::size_t size) { return qul_malloc(size); }
    static void operator delete(void *ptr) { qul_free(ptr); }
};

Platform `beginFrame`, `endFrame`, and `presentFrame` APIs

When layer support is enabled, the PlatformContext::beginFrame method forwards to ExampleLayerEngine::beginFrame, which is implemented as follows:

PlatformInterface::DrawingDevice *ExampleLayerEngine::beginFrame(const PlatformInterface::LayerEngine::ItemLayer *layer,
                                                                 const PlatformInterface::Rect &,
                                                                 int refreshInterval)
{
    ExampleItemLayerBase *itemLayer = const_cast<ExampleItemLayerBase *>(
        static_cast<const ExampleItemLayerBase *>(layer));

    if (itemLayer->lastSwapFrame == currentFrame) {
        // waitForVblank();
    }

    if (refreshInterval == -1)
        itemLayer->targetFrame = currentFrame + 1;
    else
        itemLayer->targetFrame = itemLayer->lastSwapFrame + refreshInterval;

    unsigned char *bits = itemLayer->getNextDrawBuffer();
    // Tell the HW drawing API which framebuffer to render to
    // HW_FramebufferSet(bits, drawingDevice.width(), drawingDevice.height(), toHwPixelFormat(drawingDevice.format()));

    // The drawing device also needs the framebuffer pointer for the CPU rendering fallbacks to work
    itemLayer->drawingDevice.setBits(bits);

    return &itemLayer->drawingDevice;
}

The beginFrame method ensures that the back buffer is ready to be rendered. It checks whether a vertical refresh has happened since the last buffer swap, and sets the target frame for the next swap based on the refresh interval. It also gets the pointer to the back buffer that is ready, and tells the hardware API and Qul::PlatformInterface::DrawingDevice to render to that address.

Next, the PlatformContext::endFrame method forwards to ExampleLayerEngine::endFrame, which is implemented as follows:

bool waitingForTargetFrame(ExampleItemLayerBase *layer)
{
    const unsigned int delta = layer->targetFrame - currentFrame;

    // guard against wrap-around (swap intervals above 0x100 are unlikely)
    return delta != 0 && delta < 0x100;
}

void ExampleLayerEngine::endFrame(const PlatformInterface::LayerEngine::ItemLayer *layer)
{
    ExampleItemLayerBase *itemLayer = const_cast<ExampleItemLayerBase *>(
        static_cast<const ExampleItemLayerBase *>(layer));
    while (waitingForTargetFrame(itemLayer))
        ;
    itemLayer->swap();
    itemLayer->lastSwapFrame = currentFrame;
}

It waits until the layer update for the target frame completes, before calling the swap method to swap the front and back buffers of the item layer.

Finally, the PlatformContext::presentFrame method forwards to ExampleLayerEngine::presentFrame, which is shown below:

FrameStatistics ExampleLayerEngine::presentFrame(const PlatformInterface::Screen *screen,
                                                 const PlatformInterface::Rect &rect)
{
    static unsigned int lastFrame = 0xffffffff;
    while (currentFrame == lastFrame) {
        // The device may yield or go into sleep mode
    }
    lastFrame = currentFrame;

    PlatformInterface::Rgba32 color = screen->backgroundColor();
    // HW_SetScreenBackgroundColor(color.red(), color.blue(), color.green());

    // No frame skip compensation implemented for layers
    return FrameStatistics();
}

It ensures that updates are throttled to the update rate of the screen, even when no item layer is being rendered. In addition, it sets the background color of the screen using a hardware API.

To compute the frame statistics when multiple layers are animating, a default-constructed Qul::Platform::FrameStatistics object is returned to disable frame skip compensation in Qt Quick Ultralite core.

Getting graphics on the screen Implementing vector graphics support

Available under certain Qt licenses.
Find out more.

C

Implementing hardware layer support

Implementing hardware layer support

The layer engine interface

Implementation of the layer engine interface

Hardware layer base class

Item layer classes

Image layer classes

Sprite layer class

Platform beginFrame, endFrame, and presentFrame APIs

Platform `beginFrame`, `endFrame`, and `presentFrame` APIs