본문으로 건너뛰기

Python package using Pybind11

설치​

sudo apt install -y libffi-dev python3 python3-pip python3-dev \
&& python3 -m pip install -U pip setuptools wheel twine keyrings.alt pybind11 build

Package directory structure​

workspace
├── LICENSE
├── MANIFEST.in
├── README.md
├── CHANGELOG
├── setup.cfg
├── setup.py
├── c_src
│   └── ...
└── py_src
    └── package
        ├── test
        |   ├── __init__.py
        |   └── ...
        ├── subPackage
        │   ├── __init__.py
        │   └── module1.py
        ├── __init__.py
        └── __main__.py

C/C++​

function​

#include <pybind11/pybind11.h>

namespace py = pybind11;

int add(int i, int j) { return i + j; }

PYBIND11_MODULE(_<package>, m) {
    // m.def( "add", &add );
    m.def("add",
          &add,
          "A function which adds two numbers",
          py::arg("i") = 1,
          py::arg("j") = 2);
}

variable​

#include <pybind11/pybind11.h>

namespace py = pybind11;

PYBIND11_MODULE(_<package>, m) {
    // Built-in types and general objects (more on that later)
    // are automatically converted.
    m.attr("var1") = 10;

    // Can be explicitly converted using the function `py::cast`.
    py::object var2 = py::cast("It is string");
    m.attr("var2")  = var2;
}

class​

#include <pybind11/pybind11.h>
#include <string>

namespace py = pybind11;

class TestClass {
public:
    TestClass(int a);
    ~TestClass();
    int         add(int a, int b);
    int         sub(int a, int b);
    float       sub(float a, float b);
    std::string get_name(void);
    void        set_name(std::string name);

private:
    std::string name;
};

PYBIND11_MODULE(_<package>, m) {
    py::class_<TestClass>(m, "TestClass")
        .def(py::init<int>())
        .def("add", &TestClass::add)
        .def("sub", (int (TestClass::*)(int, int)) & TestClass::sub)
        .def("sub", (float (TestClass::*)(float, float)) & TestClass::sub)
        // .def_readwrite( "name", &TestClass::name )    // public
        .def_property(
            "name", &TestClass::get_name, &TestClass::set_name);    // private
}

Python​

__init__.py​

from <package>._<package> import *

...

Build​

pyproject.toml​

[build-system]
requires = [
    "setuptools>=42",
    "wheel",
    "pybind11>=2.6.0",
]
build-backend = "setuptools.build_meta"
경고

build-system.requires는 build할 때만 사용되는 패키지입니다. 설치 이후에 사용될 패키지는 setup.cfg에서 관리하세요.

setup.cfg​

Static metadata를 관리하는 파일입니다. 설치 상황에 따라 바뀌지 않는 내용을 적으세요.

[metadata]
name = yolov4
url = https://github.com/hhk7734/tensorflow-yolov4
project_urls =
    Documentation = https://wiki.loliot.net/docs/lang/python/libraries/yolov4/python-yolov4-about
    Source = https://github.com/hhk7734/tensorflow-yolov4

author = Hyeonki Hong
author_email = [email protected]
description = YOLOv4: Optimal Speed and Accuracy of Object Detection
long-description = file: README.md, CHANGELOG
long_description_content_type = text/markdown
keywords = tensorflow, yolo, AI, TPU
license = MIT
classifiers =
    Programming Language :: Python :: 3
    License :: OSI Approved :: MIT License
    Operating System :: POSIX :: Linux
    Intended Audience :: Developers
    Intended Audience :: Science/Research
    Topic :: Scientific/Engineering
    Topic :: Scientific/Engineering :: Artificial Intelligence
    Topic :: Software Development
    Topic :: System :: Hardware


[options]
package_dir =
    = py_src
packages = find:
zip_safe = False
install_requires =
    numpy>=1.18.0

[options.packages.find]
where = py_src
정보

install_requires에 같이 설치되어야할 패키지를 적으면 됩니다.

TODO: extras_require

setup.py​

Dynamic metadata를 관리하는 파일입니다. 설치 상황에 따라 바뀔 수 있는 내용을 프로그래밍할 수 있습니다.

from glob import glob
from os import path

from setuptools import setup
from pybind11.setup_helpers import ParallelCompile, Pybind11Extension, build_ext

BASE_DIR = path.dirname(path.abspath(__file__))
CHANGELOG_PATH = path.join(BASE_DIR, "CHANGELOG")

with open(CHANGELOG_PATH, "r") as f:
    __version__ = f.readline()
    __version__ = __version__.split()
    __version__ = __version__[1][1:-1]

ParallelCompile("NPY_NUM_BUILD_JOBS").install()

ext_modules = [
    Pybind11Extension(
        # Ref: distutils.extension.Extension
        name="yolov4.common._common",
        sources=sorted(glob("c_src/**/*.cpp", recursive=True)),
        include_dirs=["c_src/"],  # -I
        define_macros=[(("VERSION_INFO", __version__))],  # -D<string>=<string>
        undef_macros=[],  # [string] -D<string>
        library_dirs=[],  # [string] -L<string>
        libraries=[],  # [string] -l<string>
        runtime_library_dirs=[],  # [string] -rpath=<string>
        extra_objects=[],  # [string]
        extra_compile_args=[],  # [string]
        extra_link_args=[],  # [string]
    ),
]


setup(
    version=__version__,
    ext_modules=ext_modules,
    cmdclass={"build_ext": build_ext},
)

__version__ 값을 .cpp파일 안에서 호출할 수 있습니다.

#define MACRO_STRINGIFY(x) #x

PYBIND11_MODULE(_<package>, m) {
#ifdef VERSION_INFO
    m.attr("__version__") = MACRO_STRINGIFY(VERSION_INFO);
#else
    m.attr("__version__") = "dev";
#endif
}

MANIFEST.in​

include LICENSE
include README.md
include CHANGELOG
include c_src/*

Test install/uninstall with pip​

python3 -m pip install <directory|git|package>
python3 -m pip uninstall <package>

pip 등록​

dist​

python3 -m build

Test 등록/설치​

python3 -m twine upload --repository testpypi dist/*
python3 -m pip install \
    --index-url https://test.pypi.org/simple/ \
    --verbose \
    --user \
    --no-deps \
    <package>

정식 등록/설치​

python3 -m twine upload dist/*
python3 -m pip install <package>

Reference​