Sample Bindings Example

This example showcases how to generate Python bindings for a non-Qt C++ library.

The example defines a CMake project that builds two libraries:

  • libuniverse - a sample library with two C++ classes.

  • Universe - the generated Python extension module that contains bindings to the library above.

The project file is structured in such a way that a user can copy-paste in into their own project, and be able to build it with a minimal amount of modifications.

Description

The libuniverse library declares two classes: Icecream and Truck.

Icecream objects have a flavor, and an accessor for returning the flavor.

Truck instances store a vector of Icecream objects, and have various methods for adding new flavors, printing available flavors, delivering icecream, etc.

From a C++ perspective, Icecream instances are treated as object types (pointer semantics) because the class declares virtual methods.

In contrast Truck does not define virtual methods and is treated as a value type (copy semantics).

Because Truck is a value type and it stores a vector of Icecream pointers, the rule of five has to be taken into account (implement the copy constructor, assignment operator, move constructor, move assignment operator and destructor).

And due to Icecream objects being copyable, the type has to define an implementation of the clone() method, to avoid type slicing issues.

Both of these types and their methods will be exposed to Python by generating CPython code. The code is generated by shiboken and placed in separate .cpp files named after each C++ type. The code is then compiled and linked into a shared library. The shared library is a CPython extension module, which is loaded by the Python interpreter.

Because the C++ language has different semantics to Python, shiboken needs help in figuring out how to generate the bindings code. This is done by specifying a special XML file called a typesystem file.

In the typesystem file you specify things like:

  • Which C++ classes should have bindings (Icecream, Truck) and with what kind of semantics (value / object)

  • Ownership rules (who deletes the C++ objects, C++ or Python)

  • Code injection (for various special cases that shiboken doesn’t know about)

  • Package name (name of package as imported from Python)

In this example we declare Icecream as an object type and Truck as a value type. The clone() and addIcecreamFlavor(Icecream*) need additional info about who owns the parameter objects when passing them across language boundaries (in this case C++ will delete the objects).

The Truck has getters and setters for the string arrivalMessage. In the type system file, we declare this to be a property in Python:

<property type="std::string" name="arrivalMessage" get="getArrivalMessage" set="setArrivalMessage"/>

It can then be used in a more pythonic way:

special_truck.arrivalMessage = "A new SPECIAL icecream truck has arrived!\n"

After shiboken generates the C++ code and CMake makes an extension module from the code, the types can be accessed in Python simply by importing them using the original C++ names.

from Universe import Icecream, Truck

Constructing C++ wrapped objects is the same as in Python

icecream = Icecream("vanilla")
truck = Truck()

And actual C++ constructors are mapped to the Python __init__ method.

class VanillaChocolateIcecream(Icecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

C++ methods can be accessed as regular Python methods using the C++ names

truck.addIcecreamFlavor(icecream)

Inheritance works as with regular Python classes, and virtual C++ methods can be overridden simply by definining a method with the same name as in the C++ class.

class VanillaChocolateIcecream(Icecream):
    # ...
    def getFlavor(self):
        return "vanilla sprinked with chocolate"

The main.py script demonstrates usages of these types.

The CMake project file contains many comments explaining all the build rules for those interested in the build process.

Building the project

This example can only be built using CMake. The following requirements need to be met:

  • A PySide package is installed into the current active Python environment (system or virtualenv)

  • A new enough version of CMake (3.16+).

  • ninja

For Windows you will also need:

  • a Visual Studio environment to be active in your terminal

  • Correct visual studio architecture chosen (32 vs 64 bit)

  • Make sure that your Python intepreter and bindings project build configuration is the same (all Release, which is more likely, or all Debug).

The build uses the Shiboken6, Shiboken6Tools, and PySide6 CMake packages to configure the project with the current PySide/Shiboken installation.

Using CMake

You can build and run this example by executing the following commands (slightly adapted to your file system layout) in a terminal:

Run CMake on macOS/Linux:

cd ~/pyside-setup/examples/samplebinding
mkdir build
cd build
cmake .. -B. -G Ninja -DCMAKE_BUILD_TYPE=Release

Run CMake on Windows:

cd C:\pyside-setup\examples\samplebinding
mkdir build
cd build
cmake .. -B. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=cl.exe

To build:

ninja
ninja install
cd ..

Use the Python module

The final example can then be run by:

python main.py

In the main.py script, two types are derived from Icecream for different “flavors” after importing the classes from the Universe module. Then, a truck is created to deliver some regular flavored Icecreams and two special ones.

If the delivery fails, a new truck is created with the old flavors copied over, and a new magical flavor that will surely satisfy all customers.

Try running it to see if the ice creams are delivered.

Windows troubleshooting

It is possible that CMake can pick up the wrong compiler for a different architecture, but it can be addressed explicitly by setting the CC environment variable:

set CC=cl

passing the compiler on the command line:

cmake -S.. -B. -DCMAKE_C_COMPILER=cl.exe -DCMAKE_CXX_COMPILER=cl.exe

or by using the -G option:

cmake -S.. -B. -G "Visual Studio 14 Win64"

If the -G "Visual Studio 14 Win64" option is used, a sln file will be generated, and can be used with MSBuild instead of ninja. The easiest way to both build and install in this case, is to use the cmake executable:

cmake --build . --target install --config Release

Note that using the "Ninja" generator is preferred to the MSBuild one, because the MSBuild one generates configs for both Debug and Release, and this might lead to building errors if you accidentally build the wrong config at least once.

Virtualenv Support

If the python application is started from a terminal with an activated python virtual environment, that environment’s packages will be used for the python module import process. In this case, make sure that the bindings were built while the virtualenv was active, so that the build system picks up the correct python shared library and PySide6 / shiboken package.

Linux Shared Libraries Notes

For this example’s purpose, we link against the absolute path of the dependent shared library libshiboken because the installation of the library is done via a wheel, and there is no clean solution to include symbolic links in a wheel package (so that passing -lshiboken to the linker would work).

Windows Notes

The build config of the bindings (Debug or Release) should match the PySide build config, otherwise the application will not properly work.

In practice this means the only supported configurations are:

  1. release config build of the bindings + PySide setup.py without --debug flag + python.exe for the PySide build process + python39.dll for the linked in shared library.

  2. debug config build of the application + PySide setup.py with --debug flag + python_d.exe for the PySide build process + python39_d.dll for the linked in shared library.

This is necessary because all the shared libraries in question have to link to the same C++ runtime library (msvcrt.dll or msvcrtd.dll). To make the example as self-contained as possible, the shared libraries in use (pyside6.dll, shiboken6.dll) are hard-linked into the build folder of the application.

Download this example

The Shiboken generator needs a header file that includes the types we are interested in:

// Copyright (C) 2018 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#ifndef BINDINGS_H
#define BINDINGS_H

#include "icecream.h"
#include "truck.h"

#endif // BINDINGS_H

Shiboken requires an XML-based typesystem file that defines the relationship between C++ and Python types.

It declares the two aforementioned classes. One of them as an “object-type” and the other as a “value-type”. The main difference is that object-types are passed around in generated code as pointers, whereas value-types are copied (value semantics).

By specifying the names of these classes in the typesystem file, Shiboken automatically tries to generate bindings for all methods of those classes. You need not mention all the methods manually in the XML file, unless you want to modify them.

Object ownership rules

Shiboken doesn’t know if Python or C++ are responsible for freeing the C++ objects that were allocated in the Python code, and assuming this might lead to errors. There can be cases where Python should release the C++ memory when the reference count of the Python object becomes zero, but it should never delete the underlying C++ object just from assuming that it will not be deleted by underlying C++ library, or if it’s maybe parented to another object (like QWidgets).

In our case, the clone() method is only called inside the C++ library, and we assume that the C++ code takes care of releasing the cloned object.

As for addIcecreamFlavor(), we know that a Truck owns the Icecream object, and will remove it once the Truck is destroyed. That’s why the ownership is set to “c++” in the typesystem file, so that the C++ objects are not deleted when the corresponding Python names go out of scope.

<?xml version="1.0"?>
<!--
// Copyright (C) 2021 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause
-->
<typesystem package="Universe">

    <object-type name="Icecream">
        <!-- By default the ownership of an object created in Python is tied
             to the Python name pointing to it. In order for the underlying
             C++ object not to get deleted when the Python name goes out of
             scope, we have to transfer ownership to C++.
             -->
        <modify-function signature="clone()">
            <modify-argument index="0">
                <define-ownership owner="c++"/>
            </modify-argument>
        </modify-function>
    </object-type>

    <value-type name="Truck">
        <!-- Same ownership caveat applies here. -->
        <property type="std::string" name="arrivalMessage" get="getArrivalMessage" set="setArrivalMessage"/>
        <modify-function signature="addIcecreamFlavor(Icecream*)">
            <modify-argument index="1">
                <define-ownership owner="c++"/>
            </modify-argument>
        </modify-function>
    </value-type>

</typesystem>

// Copyright (C) 2022 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#include "icecream.h"

#include <iostream>

Icecream::Icecream(const std::string &flavor) : m_flavor(flavor) {}

Icecream::~Icecream() = default;

std::string Icecream::getFlavor() const
{
    return m_flavor;
}

Icecream *Icecream::clone()
{
    return new Icecream(*this);
}

std::ostream &operator<<(std::ostream &str, const Icecream &i)
{
    str << i.getFlavor();
    return str;
}

// Copyright (C) 2022 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#ifndef ICECREAM_H
#define ICECREAM_H

#include "macros.h"

#include <iosfwd>
#include <string>

class BINDINGS_API Icecream
{
public:
    explicit Icecream(const std::string &flavor);
    virtual Icecream *clone();
    virtual ~Icecream();
    virtual std::string getFlavor() const;

private:
    std::string m_flavor;
};

std::ostream &operator<<(std::ostream &str, const Icecream &i);

#endif // ICECREAM_H

// Copyright (C) 2018 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#ifndef MACROS_H
#define MACROS_H

#if defined _WIN32 || defined __CYGWIN__
    // Export symbols when creating .dll and .lib, and import them when using .lib.
    #if BINDINGS_BUILD
        #define BINDINGS_API __declspec(dllexport)
    #else
        #define BINDINGS_API __declspec(dllimport)
    #endif
    // Disable warnings about exporting STL types being a bad idea. Don't use this in production
    // code.
    #pragma warning( disable : 4251 )
#else
    #define BINDINGS_API
#endif

#endif // MACROS_H

# Copyright (C) 2022 The Qt Company Ltd.
# SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause
from __future__ import annotations

"""An example showcasing how to use bindings for a custom non-Qt C++ library"""

from Universe import Icecream, Truck


class VanillaChocolateIcecream(Icecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

    def clone(self):
        return VanillaChocolateIcecream(self.getFlavor())

    def getFlavor(self):
        return "vanilla sprinked with chocolate"


class VanillaChocolateCherryIcecream(VanillaChocolateIcecream):
    def __init__(self, flavor=""):
        super().__init__(flavor)

    def clone(self):
        return VanillaChocolateCherryIcecream(self.getFlavor())

    def getFlavor(self):
        base_flavor = super(VanillaChocolateCherryIcecream, self).getFlavor()
        return f"{base_flavor} and a cherry"


if __name__ == '__main__':
    leave_on_destruction = True
    truck = Truck(leave_on_destruction)

    flavors = ["vanilla", "chocolate", "strawberry"]
    for f in flavors:
        icecream = Icecream(f)
        truck.addIcecreamFlavor(icecream)

    truck.addIcecreamFlavor(VanillaChocolateIcecream())
    truck.addIcecreamFlavor(VanillaChocolateCherryIcecream())

    truck.arrive()
    truck.printAvailableFlavors()
    result = truck.deliver()

    if result:
        print("All the kids got some icecream!")
    else:
        print("Aww, someone didn't get the flavor they wanted...")

    if not result:
        special_truck = Truck(truck)
        del truck

        print("")
        special_truck.arrivalMessage = "A new SPECIAL icecream truck has arrived!\n"
        special_truck.arrive()
        special_truck.addIcecreamFlavor(Icecream("SPECIAL *magical* icecream"))
        special_truck.printAvailableFlavors()
        special_truck.deliver()
        print("Now everyone got the flavor they wanted!")
        special_truck.leave()

// Copyright (C) 2022 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#include <iostream>
#include <random>

#include "truck.h"

Truck::Truck(bool leaveOnDestruction) : m_leaveOnDestruction(leaveOnDestruction) {}

Truck::Truck(const Truck &other)
{
    assign(other);
}

Truck &Truck::operator=(const Truck &other)
{
    if (this != &other) {
        m_flavors.clear();
        assign(other);
    }
    return *this;
}

Truck::Truck(Truck &&other) = default;

Truck& Truck::operator=(Truck &&other) = default;

Truck::~Truck()
{
    if (m_leaveOnDestruction)
        leave();
}

void Truck::addIcecreamFlavor(Icecream *icecream)
{
    m_flavors.push_back(IcecreamPtr(icecream));
}

void Truck::printAvailableFlavors() const
{
    std::cout << "It sells the following flavors: \n";
    for (const auto &flavor : m_flavors)
        std::cout << "  * "  << *flavor << '\n';
    std::cout << '\n';
}

void Truck::arrive() const
{
    std::cout << m_arrivalMessage;
}

void Truck::leave() const
{
    std::cout << "The truck left the neighborhood.\n";
}

void Truck::setLeaveOnDestruction(bool value)
{
    m_leaveOnDestruction = value;
}

void Truck::setArrivalMessage(const std::string &message)
{
    m_arrivalMessage = message;
}

std::string Truck::getArrivalMessage() const
{
    return m_arrivalMessage;
}

void Truck::assign(const Truck &other)
{
    m_flavors.reserve(other.m_flavors.size());
    for (const auto &f : other.m_flavors)
        m_flavors.push_back(IcecreamPtr(f->clone()));
}

bool Truck::deliver() const
{
    std::random_device rd;
    std::mt19937 mt(rd());
    std::uniform_int_distribution<int> dist(1, 2);

    std::cout << "The truck started delivering icecream to all the kids in the neighborhood.\n";
    bool result = false;

    if (dist(mt) == 2)
        result = true;

    return result;
}

// Copyright (C) 2022 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR BSD-3-Clause

#ifndef TRUCK_H
#define TRUCK_H

#include "icecream.h"
#include "macros.h"

#include <memory>
#include <vector>

class BINDINGS_API Truck
{
public:
    explicit Truck(bool leaveOnDestruction = false);
    Truck(const Truck &other);
    Truck& operator=(const Truck &other);
    Truck(Truck &&other);
    Truck& operator=(Truck &&other);

    ~Truck();

    void addIcecreamFlavor(Icecream *icecream);
    void printAvailableFlavors() const;

    bool deliver() const;
    void arrive() const;
    void leave() const;

    void setLeaveOnDestruction(bool value);

    void setArrivalMessage(const std::string &message);
    std::string getArrivalMessage() const;

private:
    using IcecreamPtr = std::shared_ptr<Icecream>;

    void assign(const Truck &other);

    bool m_leaveOnDestruction = false;
    std::string m_arrivalMessage = "A new icecream truck has arrived!\n";
    std::vector<IcecreamPtr> m_flavors;
};

#endif // TRUCK_H

# Copyright (C) 2023 The Qt Company Ltd.
# SPDX-License-Identifier: BSD-3-Clause

cmake_minimum_required(VERSION 3.18)
cmake_policy(VERSION 3.18)

# Enable policy to not use RPATH settings for install_name on macOS.
if(POLICY CMP0068)
  cmake_policy(SET CMP0068 NEW)
endif()

# Consider changing the project name to something relevant for you.
project(SampleBinding)

# ================================ General configuration ======================================

# Set CPP standard to C++17 minimum.
set(CMAKE_CXX_STANDARD 17)

# The sample library for which we will create bindings. You can change the name to something
# relevant for your project.
set(sample_library "libuniverse")

# The name of the generated bindings module (as imported in Python). You can change the name
# to something relevant for your project.
set(bindings_library "Universe")

# The header file with all the types and functions for which bindings will be generated.
# Usually it simply includes other headers of the library you are creating bindings for.
set(wrapped_header ${CMAKE_SOURCE_DIR}/bindings.h)

# The typesystem xml file which defines the relationships between the C++ types / functions
# and the corresponding Python equivalents.
set(typesystem_file ${CMAKE_SOURCE_DIR}/bindings.xml)

# Specify which C++ files will be generated by shiboken. This includes the module wrapper
# and a '.cpp' file per C++ type. These are needed for generating the module shared
# library.
set(generated_sources
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/universe_module_wrapper.cpp
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/icecream_wrapper.cpp
    ${CMAKE_CURRENT_BINARY_DIR}/${bindings_library}/truck_wrapper.cpp)


# ================================== Dependency detection ======================================

find_package(Python COMPONENTS Interpreter Development REQUIRED)
# On RHEL and some other distros, Python wheels and site-packages may be installed under 'lib64'
# instead of 'lib'. The FindPython CMake module may set Python_SITELIB to 'lib', which is incorrect
# for these cases. To ensure compatibility, we override Python_SITELIB by querying Python directly.
# This guarantees the correct site-packages path is used regardless of platform or Python build.
execute_process(
    COMMAND ${Python_EXECUTABLE} -c
    "import site; print(next(p for p in site.getsitepackages() if 'site-packages' in p))"
    OUTPUT_VARIABLE Python_SITELIB
    OUTPUT_STRIP_TRAILING_WHITESPACE
)
message(STATUS "Python site-packages directory: ${Python_SITELIB}")
list(APPEND CMAKE_PREFIX_PATH
    "${Python_SITELIB}/shiboken6_generator/lib/cmake"
)
find_package(Shiboken6Tools REQUIRED)

# ==================================== RPATH configuration ====================================


# =============================================================================================
# !!! (The section below is deployment related, so in a real world application you will want to
# take care of this properly with some custom script or tool).
# =============================================================================================
# Enable rpaths so that the built shared libraries find their dependencies.
set(CMAKE_SKIP_BUILD_RPATH FALSE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)
set(CMAKE_INSTALL_RPATH ${CMAKE_CURRENT_SOURCE_DIR})
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
# =============================================================================================
# !!! End of dubious section.
# =============================================================================================


# =============================== CMake target - sample_library ===============================


# Define the sample shared library for which we will create bindings.
set(${sample_library}_sources icecream.cpp truck.cpp)
add_library(${sample_library} SHARED ${${sample_library}_sources})
set_property(TARGET ${sample_library} PROPERTY PREFIX "")

# Needed mostly on Windows to export symbols, and create a .lib file, otherwise the binding
# library can't link to the sample library.
target_compile_definitions(${sample_library} PRIVATE BINDINGS_BUILD)

# =============================== CMake target - bindings_library =============================

# Create Python bindings using Shiboken6Tools macro
shiboken_generator_create_binding(
    EXTENSION_TARGET ${bindings_library}
    GENERATED_SOURCES ${generated_sources}
    HEADERS ${wrapped_header}
    TYPESYSTEM_FILE ${typesystem_file}
    LIBRARY_TARGET ${sample_library}
    FORCE_LIMITED_API
)
# ================================= Dubious deployment section ================================

set(windows_shiboken_shared_libraries)

if(WIN32)
    # =========================================================================================
    # !!! (The section below is deployment related, so in a real world application you will
    # want to take care of this properly (this is simply to eliminate errors that users usually
    # encounter.
    # =========================================================================================
    # Circumvent some "#pragma comment(lib)"s in "include/pyconfig.h" which might force to link
    # against a wrong python shared library.

    set(python_versions_list 3 36 37 38 39)
    set(python_additional_link_flags "")
    foreach(ver ${python_versions_list})
        set(python_additional_link_flags
            "${python_additional_link_flags} /NODEFAULTLIB:\"python${ver}_d.lib\"")
        set(python_additional_link_flags
            "${python_additional_link_flags} /NODEFAULTLIB:\"python${ver}.lib\"")
    endforeach()

    set_target_properties(${bindings_library}
                           PROPERTIES LINK_FLAGS "${python_additional_link_flags}")

    # Get the correct DLL path for the current build type
    if(CMAKE_BUILD_TYPE STREQUAL "Debug")
        get_target_property(dll_path Shiboken6::libshiboken IMPORTED_LOCATION_DEBUG)
    else()
        get_target_property(dll_path Shiboken6::libshiboken IMPORTED_LOCATION_RELEASE)
    endif()
    file(TO_CMAKE_PATH "${dll_path}" dll_path)
    set(windows_shiboken_shared_libraries "${dll_path}")
    # =========================================================================================
    # !!! End of dubious section.
    # =========================================================================================
endif()

# =============================================================================================
# !!! (The section below is deployment related, so in a real world application you will want to
# take care of this properly with some custom script or tool).
# =============================================================================================
# Install the library and the bindings module into the source folder near the main.py file, so
# that the Python interpeter successfully imports the used module.
install(TARGETS ${bindings_library} ${sample_library}
        LIBRARY DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}
        RUNTIME DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}
        )
install(FILES ${windows_shiboken_shared_libraries} DESTINATION ${CMAKE_CURRENT_SOURCE_DIR})
# =============================================================================================
# !!! End of dubious section.
# =============================================================================================