Integrating C++ With Python Using PyBind11

Integrating C++ with Python using PyBind11

In my work as a developer writing high-performance C++ libraries, I I sometimes need to integrate them with Python using PyBind11. This lightweight tool allows me to maintain the speed and efficiency of C++ while providing Python’s flexibility for scripting and automation. In this post, I will share tips and techniques for exposing C++ functions, classes, and handling memory directly between Python and C++.

Binding a C++ function

One of the simplest ways to expose C++ functionality to Python is through functions. With PyBind11, I can quickly bind a C++ function for use in Python. Here is an example of how I bind a C++ function using PyBind11:

extern "C" int add(int i = 1, int j = 1);

PYBIND11_MODULE(pybind11_iface, m)
{
    m.def("add", &add, "A function which adds two numbers", pybind11::arg("i") = 1, pybind11::arg("j") = 1);
}

In this example, the C++ function add takes two arguments, adds them together, and returns the result. By using m.def in PyBind11, I expose this function to Python, making it callable with or without arguments, thanks to the default values.

Exposing a C++ class

Exposing C++ classes is just as simple as binding functions. For example, I frequently work with classes that encapsulate important logic. PyBind11 allows me to make these classes accessible from Python with ease. Here’s how I bind a C++ class to Python:

class CppClass
{
public:
    CppClass();
    ~CppClass();
    uint32_t class_function();
};

PYBIND11_MODULE(pybind11_iface, m)
{
    pybind11::class_<CppClass>(m, "CppClass")
        .def(pybind11::init<>())  // Constructor binding
        .def("class_function", &CppClass::class_function);  // Method binding
}

In this example, I expose the CppClass constructor and one of its member functions, class_function. Once bound, I can instantiate CppClass and call its method from Python while keeping the performance characteristics of C++.

Handling memory with numpy and vectors

One of the challenges I often face is efficiently passing data between C++ and Python, particularly when working with std::vector and numpy.array. I use PyBind11 to handle memory efficiently by working with these data structures directly. Here’s an example of how I bind a C++ vector and manipulate it from Python:

void UpdateVector(std::vector<double>& vec)
{
    for (size_t i = 0; i < vec.size(); ++i)
    {
        vec[i] = (static_cast<double>(i) + 1) * 1.1;
    }
}

PYBIND11_MODULE(pybind11_iface, m)
{
    pybind11::bind_vector<std::vector<double>>(m, "VectorDouble");
}

With this binding, I can create and modify std::vector<double> objects in Python without unnecessary copying between Python and C++. This method is extremely useful when working with large datasets where performance is critical.

Working with numpy arrays in C++

In situations where I need to manipulate numpy.array objects directly in C++, PyBind11 provides a simple mechanism to interact with these arrays without losing efficiency. I often use the py::array_t type to work with numpy.array data:

void UpdateVectorNumpy(py::array_t<double> array)
{
    py::buffer_info buf = array.request();
    double* ptr = static_cast<double*>(buf.ptr);
    size_t size = static_cast<size_t>(buf.size);

    for (size_t i = 0; i < size; ++i)
    {
        ptr[i] = (i + 1) * 1.1;
    }
}

PYBIND11_MODULE(pybind11_iface, m)
{
    m.def("update_vector_numpy", &UpdateVectorNumpy);
}

In this example, I modify a numpy.array directly in C++, updating its values in place. By avoiding unnecessary data copies, I can ensure that the performance remains optimal, even when working with large arrays.

Conclusion

PyBind11 has become an essential tool in my workflow, allowing me to bridge the gap between Python and C++ without sacrificing performance. Whether I’m exposing simple functions or working with complex data structures like numpy.array and std::vector, PyBind11 makes the integration seamless. The examples I shared here demonstrate just a few of the ways I use PyBind11 in my own projects.

For more insights into this topic, you can find the details here.