Rust bindings for the Python interpreter
Find a file
Gregory Szorc c9c606f7c6 build: enable suppression of cargo:rustc-link-* lines
PyOxidizer requires advanced control over the settings used to link
libpython. We recently implemented support for configuration files
defining explicit lines to emit from build scripts to give callers
control over what lines to emit from build scripts so use cases
like PyOxidizer's are feasible without hacks in PyO3's code base.

However, the default logic in `emit_link_config()` may not be
appropriate in scenarios where link settings are provided via this
"extra lines" mechanism. The default logic may prohibit use of or
interfere with desired settings provided externally.

This commit defines a new field on the interpreter config that
suppresses the emission of the default link control logic from the
`pyo3` build script. It effectively gives advanced consumers like
PyOxidizer full control over link logic while minimally polluting
PyO3's build logic.

I thought about implementing this control as a crate feature. But
given the expected target audience size of ~1, I thought a crate
feature was too visible for a power user feature and decided to
implement it via the configuration file.
2021-08-30 10:01:36 -07:00
.github ci: pin parking_lot to 0.11.1 on MSRV 2021-08-28 08:17:28 +01:00
benches - PyList, PyTuple and PySequence's get_item now accepts only usize indices instead of isize. 2021-08-17 15:01:22 +01:00
examples examples: make word-count example comparison fairer 2021-08-13 14:19:46 +01:00
guide changelog: remove function parentheses 2021-08-26 18:59:07 +01:00
pyo3-build-config build: enable suppression of cargo:rustc-link-* lines 2021-08-30 10:01:36 -07:00
pyo3-macros changelog: updates for 0.14.4 2021-08-29 08:07:44 +01:00
pyo3-macros-backend Merge pull request #1846 from davidhewitt/changelog-release-0.14.4 2021-08-30 08:19:53 +01:00
src Merge pull request #1841 from PyO3/exception_hygiene 2021-08-29 07:14:36 +01:00
tests Merge pull request #1843 from PyO3/pymethods_hygiene 2021-08-29 11:26:10 +01:00
.gitignore Some api improvements 2019-02-23 18:01:22 +01:00
Architecture.md docs: use pyo3.rs/latest instead of pyo3.rs/main 2021-07-24 08:47:02 +01:00
build.rs build: enable suppression of cargo:rustc-link-* lines 2021-08-30 10:01:36 -07:00
Cargo.toml changelog: updates for 0.14.4 2021-08-29 08:07:44 +01:00
CHANGELOG.md changelog: updates for 0.14.4 2021-08-29 08:07:44 +01:00
Code-of-Conduct.md Remove myself from the CoC 2020-04-13 13:38:41 +02:00
codecov.yml coverage: ignore tests 2021-06-29 22:11:08 +01:00
Contributing.md Rewrite module.md for clarity and add tip on code organization (#1693) 2021-07-22 08:10:32 +01:00
LICENSE Rename LICENSE-APACHE to LICENSE 2017-10-04 08:56:57 -07:00
Makefile docs: sync README and lib.rs examples 2021-08-02 23:03:25 +01:00
pyproject.toml Format examples with black (#590) 2019-09-06 01:16:09 +02:00
README.md changelog: updates for 0.14.4 2021-08-29 08:07:44 +01:00
tox.ini Add Python 3.8 to CI 2020-02-01 15:08:41 +01:00

PyO3

actions status benchmark codecov crates.io minimum rustc 1.41 dev chat contributing notes

Rust bindings for Python, including tools for creating native Python extension modules. Running and interacting with Python code from a Rust binary is also supported.

Usage

PyO3 supports the following software versions:

  • Python 3.6 and up (CPython and PyPy)
  • Rust 1.41 and up

You can use PyO3 to write a native Python module in Rust, or to embed Python in a Rust binary. The following sections explain each of these in turn.

Using Rust from Python

PyO3 can be used to generate a native Python module. The easiest way to try this out for the first time is to use maturin. maturin is a tool for building and publishing Rust-based Python packages with minimal configuration. The following steps set up some files for an example Python module, install maturin, and then show how build and import the Python module.

First, create a new folder (let's call it string_sum) containing the following two files:

Cargo.toml

[package]
name = "string-sum"
version = "0.1.0"
edition = "2018"

[lib]
name = "string_sum"
# "cdylib" is necessary to produce a shared library for Python to import from.
#
# Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able
# to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.:
# crate-type = ["cdylib", "rlib"]
crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.14.4"
features = ["extension-module"]

src/lib.rs

use pyo3::prelude::*;

/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

/// A Python module implemented in Rust. The name of this function must match
/// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to
/// import the module.
#[pymodule]
fn string_sum(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;

    Ok(())
}

With those two files in place, now maturin needs to be installed. This can be done using Python's package manager pip. First, load up a new Python virtualenv, and install maturin into it:

$ cd string_sum
$ python -m venv .env
$ source .env/bin/activate
$ pip install maturin

Now build and execute the module:

$ maturin develop
# lots of progress output as maturin runs the compilation...
$ python
>>> import string_sum
>>> string_sum.sum_as_string(5, 20)
'25'

As well as with maturin, it is possible to build using setuptools-rust or manually. Both offer more flexibility than maturin but require further configuration.

Using Python from Rust

To embed Python into a Rust binary, you need to ensure that your Python installation contains a shared library. The following steps demonstrate how to ensure this (for Ubuntu), and then give some example code which runs an embedded Python interpreter.

To install the Python shared library on Ubuntu:

sudo apt install python3-dev

Start a new project with cargo new and add pyo3 to the Cargo.toml like this:

[dependencies.pyo3]
version = "0.14.4"
features = ["auto-initialize"]

Example program displaying the value of sys.version and the current user name:

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> PyResult<()> {
    Python::with_gil(|py| {
        let sys = py.import("sys")?;
        let version: String = sys.get("version")?.extract()?;

        let locals = [("os", py.import("os")?)].into_py_dict(py);
        let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
        let user: String = py.eval(code, None, Some(&locals))?.extract()?;

        println!("Hello {}, I'm Python {}", user, version);
        Ok(())
    })
}

The guide has a section with lots of examples about this topic.

Tools and libraries

  • maturin Zero configuration build tool for Rust-made Python extensions.
  • setuptools-rust Setuptools plugin for Rust support.
  • pyo3-built Simple macro to expose metadata obtained with the built crate as a PyDict
  • rust-numpy Rust binding of NumPy C-API
  • dict-derive Derive FromPyObject to automatically transform Python dicts into Rust structs
  • pyo3-log Bridge from Rust to Python logging
  • pythonize Serde serializer for converting Rust objects to JSON-compatible Python objects
  • pyo3-asyncio Utilities for working with Python's Asyncio library and async functions

Examples

  • hyperjson A hyper-fast Python module for reading/writing JSON data using Rust's serde-json
  • html-py-ever Using html5ever through kuchiki to speed up html parsing and css-selecting.
  • point-process High level API for pointprocesses as a Python library
  • autopy A simple, cross-platform GUI automation library for Python and Rust.
    • Contains an example of building wheels on TravisCI and appveyor using cibuildwheel
  • orjson Fast Python JSON library
  • inline-python Inline Python code directly in your Rust code
  • Rogue-Gym Customizable rogue-like game for AI experiments
    • Contains an example of building wheels on Azure Pipelines
  • fastuuid Python bindings to Rust's UUID library
  • wasmer-python Python library to run WebAssembly binaries
  • mocpy Astronomical Python library offering data structures for describing any arbitrary coverage regions on the unit sphere
  • tokenizers Python bindings to the Hugging Face tokenizers (NLP) written in Rust
  • pyre Fast Python HTTP server written in Rust
  • jsonschema-rs Fast JSON Schema validation library
  • css-inline CSS inlining for Python implemented in Rust
  • cryptography Python cryptography library with some functionality in Rust
  • polaroid Hyper Fast and safe image manipulation library for Python written in Rust
  • ormsgpack Fast Python msgpack library

Contributing

Everyone is welcomed to contribute to PyO3! There are many ways to support the project, such as:

  • help PyO3 users with issues on Github and Gitter
  • improve documentation
  • write features and bugfixes
  • publish blogs and examples of how to use PyO3

Our contributing notes and architecture guide have more resources if you wish to volunteer time for PyO3 and are searching where to start.

If you don't have time to contribute yourself but still wish to support the project's future success, some of our maintainers have Github sponsorship pages:

License

PyO3 is licensed under the Apache-2.0 license. Python is licensed under the Python License.