Metadata-Version: 2.2
Name: swigbind11
Version: 1.1.0
Summary: Coupling of pybind11 object and swig object.
Author-Email: "Ramandeep Jain, Severin Strobl" <ramandeep.jain@dlr.de>
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.7
Description-Content-Type: text/markdown

# swigbind11 - Interfacing SWIG and pybind11 <!-- omit in toc -->

- [Overview](#overview)
- [Motivation](#motivation)
  - [Basics of *pybind11*](#basics-of-pybind11)
  - [Combining libraries using *SWIG* and *pybind11*](#combining-libraries-using-swig-and-pybind11)
- [Using *swigbind11*](#using-swigbind11)
- [Example: Mesh Plugin](#example-mesh-plugin)

## Overview

Many existing C++ codes use [*SWIG*](http://www.swig.org/) to automatically
generate bindings for Python, but over the last years
[*pybind11*](https://pybind11.readthedocs.io/) has gained a lot of popularity.
Most newer projects rely on *pybind11* and even large exiting code bases, such
as, *tensorflow*[^1], made the transition to *pybind11*. While SWIG is extremely
powerful with its support for a large number of languages, it has its
shortcomings w.r.t. supporting modern C++ standards. There seems to be little
development in this direction and with the increasing complexity of C++ it is
unlikely SWIG will ever catch up. The scope of *pybind11* is much more focused:
the goal is to enable the generation of Python bindings employing modern C++
itself. As such, *pybind11* does not have to process and transform C++ code to
generate the glue code for the Python bindings, but rather *pybind11* provides a
C++ library to enable the developer to generate this glue code easily, as part
of regular C++ library.

While creating Python modules using *pybind11* is surprisingly easy, things get
tricky when interacting with libraries using *SWIG* to generate their Python
bindings. As long as the modules generated via *SWIG* and *pybind11* interact
only on the Python level, the method the bindings were generated with does not
matter. But as soon as a C++ library wrapped to Python via *pybind11* tries to
interact with a library wrapped to Python via *SWIG* by exchanging C++ objects
via the Python layer, incompatibilities arise due to the different methods of
wrapping the underlying C++ types.

## Motivation

### Basics of *pybind11*

Assume we use the following code to generate the Python module `pybind_example`
via *pybind11*:

<!-- **pybind_example.cpp** -->
```cpp
#include "pybind11/pybind11.h"

namespace py = pybind11;

namespace pybind_example {

class A {
public:
  auto get_answer() -> int { return 42; }
};

auto foo(A *a) -> int {
  return a->get_answer();
};

PYBIND11_MODULE(pybind_example, m) {
  py::class_<A>(m, "A").def(py::init<>()).def("get_answer", &A::get_answer);

  m.def("foo", &foo);
}

}  // namespace pybind_example
```
There we have the class `A` with its (default) constructor and the member
function `get_answer` exposed via the Python module `pybind_example`, along with
the function `foo` taking a pointer to an instance of `A` and calling the
`get_answer` member function. Using this trivial module in Python is then
straight forward:
<!-- **pybind_example.py** -->
```python
import pybind_example as pe
a = pe.A()
pe.foo(a)
```
All the boilerplate code required is automatically generated by *pybind11* and
wrapping additional C++ classes and functions is easily done. Also interfacing
with certain builtin Python types (e.g., `str` or `list`) is directly supported
by *pybind11*, as well as support and, where applicable, conversion to C++
standard data structures, such as, `std::string` or `std::vector` (including
handling of `numpy` arrays!).

Arbitrary C++ libraries can be included for the implementation of the required
functionality and, if necessary, the corresponding types can be exposed via the
Python API by wrapping them using *pybind11*.

### Combining libraries using *SWIG* and *pybind11*

Now suppose we have an existing C++ library *A* wrapped to Python via *SWIG*,
resulting in the Python module *a_swig*. In our C++ library *B*, which we wrap
to Python using *pybind11* (giving us the module *b_pybind*), we now want to use
*A* and, to make matters worse, not only internally, but as part of the public
API. This is a common use case for a framework library with a series of plugin
components. Consider the following code snippets:

#### Bar.hpp (part of C++ library *A*) <!-- omit in toc -->
```cpp
namespace a {

class Bar {
  // [...]
};

}
```

#### Foo.hpp (part of C++ library *B*) <!-- omit in toc -->
```cpp
// include header provided by external library A
#include "Bar.hpp"

namespace b {

class Foo {
public:
  explicit Foo(a::Bar* bar);
  // [...]
};

}
```

#### b_pybind.cpp (Python wrappers for library *B*) <!-- omit in toc -->
```cpp
#include "pybind11/pybind11.h"

#include "Foo.hpp"

namespace py = pybind11;

namespace b {

PYBIND11_MODULE(b_pybind, m) {
  py::class_<Foo>(m, "Foo").def(py::init<a::Bar*>());
}

}
```
While the Python module compiles just fine, when trying to create a new instance
of `Foo` on the Python side via

```python
import a_swig as a
import b_pybind as b

bar = a.Bar()
foo = b.Foo(bar)
```
an exception similar to the one below is triggered due to incompatible types:

```
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. b_pybind.Foo(arg0: Bar)

Invoked with: <a_swig.Bar; proxy of <Swig Object of type 'Bar *' at 0x7fd79e3f12a0> >
```
So while we have Python bindings for both libraries, it is not possible to pass
instances of the underlying C++ types between the two libraries as the two
wrapping types (*SWIG* and *pybind11*) generate incompatible types on the Python
side. One could be tempted to additionally wrap `a::Bar` using *pybind11*, but
this would not resolve the problem: Instances created in Python via the *SWIG*
bindings from the module `a_swig` would have a different type compared to
instances created for the same underlying class `a::Bar` but via the *pybind11*
bindings. But, as we shall see in the next section, there is a solution!

## Installation
### Include with PyPI
To install *swigbind11* as a standard Python package within a virtual environment, run the following commands:
```bash
python3 -m venv venv
source venv/bin/activate
pip install .
```

This installation provides a CLI utility and configures the necessary metadata for CMake to locate the package.

To use *swigbind11* in your CMake project, you can help CMake locate its configuration files by dynamically retrieving this path using ``execute_process``.
Add the following snippet to your CMakeLists.txt:

````cmake
# Get the installation directory from the swigbind11 module
execute_process(
    COMMAND ${Python3_EXECUTABLE} -m swigbind11 --cmakedir
    OUTPUT_VARIABLE swigbind11_DIR
    OUTPUT_STRIP_TRAILING_WHITESPACE
)

# Locate the package using the path retrieved above
find_package(swigbind11 REQUIRED)
````

**NOTE**:
Ensure that ``${Python3_EXECUTABLE}`` points to the interpreter inside the virtual environment where the package was installed.

### Include as a submodule
To include this library as a dependency using Git Submodules, follow these steps:

```bash
git submodule add https://github.com/dlr-sp/swigbind11.git third_party/swigbind11
git submodule update --init --recursive
```

We assume that you are placing swigbind11 in *third_party* folder.

## Building with CMake
This assumes that you have activated a python environment and would like to install it in site-packages.
```bash
# 1. Configure the project
cmake -S . -B build -DCMAKE_INSTALL_PREFIX=$VIRTUAL_ENV/lib/python3.10/site-packages

# 2. Build and Install
cmake --build build
cmake --install build
```

## Using *swigbind11*

Note: It is the responsibility of the user to add the *pybind11* headers to the header-search path of his build-system (see [Link](https://pybind11.readthedocs.io/en/stable/installing.html)).
*swigbind11* is not shipped with a *pybind11* clone for this purpose!

Interfacing of libraries wrapped via *SWIG* and *pybind11* is quite easy using
*swigbind11*. It allows the automatic conversion between *SWIG*-wrapped objects
and the corresponding C++ types by means of using a custom *pybind11* *type
caster*. This *type caster* transforms a Python object holding, e.g., a `a::Bar`
into a `std::shared_ptr<a::Bar>` and vice versa. A `std::shared_ptr<>` is required
to properly track the lifetime of the underlying object on the Python and C++ side
simultaneously.

<!-- #### b_swigpybind11.cpp (Python wrappers for library *B* using *swigbind11*) -->
```cpp
#include "pybind11/pybind11.h"
#include "swigbind11/swigbind11.hpp"

#include "Foo.hpp"

namespace py = pybind11;

SWIGBIND11_TYPE_CASTER(a::Bar, "a_swig.Bar");

namespace b {

PYBIND11_MODULE(b_pybind, m) {
  py::class_<Foo>(m, "Foo").def(py::init<std::shared_ptr<a::Bar>>(), py::arg("bar"));
}

}
```

The only other required change is to switch the interface of `b::Foo` to accept
a `std::shared_ptr<a::Bar>` instead of a raw pointer. Using a smart pointer here
at the interface of C*+ and Python instead of a raw pointer is often a good idea
anyway, as this avoids hard to debug issues with interference between the
automatic Python garbage collection and the use of the underlying object on both
sides of the interface.

In addition to the helper macro to define the custom type casters, *swigbind11*
also provides some low-level functions to perform type conversions:

Conversion from Python object to underlying C++ object:
```cpp
template<typename T>
std::shared_ptr<T> swig_py_cast(pybind11::handle obj, std::string_view python_type_name)
```

Wrapping an existing C++ object using the externally defined SWIG bindings in a Python object:
```cpp
template<typename T>
pybind11::handle py_swig_cast(std::shared_ptr<T> obj, std::string_view python_type_name)
```
```cpp
template<typename T>
pybind11::handle py_swig_cast(std::unique_ptr<T> obj, std::string_view python_type_name)
```

In all cases the user has to provide the correct mapping between the C++ type
and the corresponding Python type generated by *SWIG*. If these two types do not
match up, there is no way for *swigbind11* to detect this, and one will (as a
best case scenario) encounter segmentation faults and similar errors.

## Example: Mesh Plugin

A demonstration of the usage of *swigbind11* for a real-world example (plugin
for a *mesh_library*) can be found in the [example directory](example/).

[^1]: https://github.com/tensorflow/community/blob/master/rfcs/20190208-pybind11.md
