pyo3/README.md

159 lines
7.1 KiB
Markdown
Raw Normal View History

2018-05-17 21:59:11 +00:00
# PyO3
2015-04-19 03:29:19 +00:00
2018-07-30 20:52:22 +00:00
[![Build Status](https://travis-ci.org/PyO3/pyo3.svg?branch=master)](https://travis-ci.org/PyO3/pyo3)
[![Actions Status](https://github.com/PyO3/pyo3/workflows/Test/badge.svg)](https://github.com/PyO3/pyo3/actions)
[![codecov](https://codecov.io/gh/PyO3/pyo3/branch/master/graph/badge.svg)](https://codecov.io/gh/PyO3/pyo3)
2018-07-30 20:52:22 +00:00
[![crates.io](http://meritbadge.herokuapp.com/pyo3)](https://crates.io/crates/pyo3)
[![Join the dev chat](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/PyO3/Lobby)
2018-05-17 21:59:11 +00:00
[Rust](http://www.rust-lang.org/) bindings for [Python](https://www.python.org/). This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules.
2015-04-19 03:29:19 +00:00
2018-06-01 13:03:04 +00:00
* User Guide: [stable](https://pyo3.rs) | [master](https://pyo3.rs/master)
2015-04-19 03:29:19 +00:00
2019-10-26 13:47:19 +00:00
* API Documentation: [stable](https://docs.rs/pyo3/) | [master](https://pyo3.rs/master/doc)
2018-10-04 06:46:15 +00:00
2019-04-27 20:14:34 +00:00
A comparison with rust-cpython can be found [in the guide](https://pyo3.rs/master/rust_cpython.html).
2017-05-21 12:37:56 +00:00
2018-05-17 21:59:11 +00:00
## Usage
2017-05-21 12:37:56 +00:00
PyO3 supports Python 3.5 and up. The minimum required Rust version is 1.42.0-nightly 2020-01-21.
2015-09-20 17:43:24 +00:00
2019-11-23 06:29:39 +00:00
If you have never used nightly Rust, the official guide has
[a great section](https://doc.rust-lang.org/book/appendix-07-nightly-rust.html#rustup-and-the-role-of-rust-nightly)
about installing it.
Support rust extensions for PyPy via cpyext (#393) * wip * removed stuff * removed another change * implemented minimum amouth of ifdefs to make pypy3 hello world to compile * implemented minimum amount of ifdefs to make pypy3 hello world to compile * hacking on build.rs * compiler is happy! * few todos remain * extracted build logic to seperate module * added pypy test * finally fixed pypy structs * removed some todos * test should now be machine independent * fixed all pypy3 symbols * added pypy feature * removed `is_pypy` * added pypy2 declerations also * fix for cpython2 * improved libpypy detection * added all pypy2 macros * fixed errneous type * more fixes * fix python2 string macros * modsupport symbol * fix * fixed and added many symbols * fixes * remove dup * remove mac-specific config * fix all name mangling macros * unite imports * missing symbol * fix pybool * implemented another missing symbol * it works * fix merge conflict * uncomment non default features * cargo.toml * Cargo fmt * small merge fixes * use newer build version * whoops * fix build script * more build hacks * some random hiccups * small fixes * it builds! * it builds and runs * revert everything in FFI2 * revert changes to ffi2 * check python3 for pypy * tiny fix * revert ffi2 for real * revert weird formatting changes * bring back missing feature * tiny error * fix py3.7 issue * add pypy3.5 6.0 to travis * remove dbg! * another tiny fix * removed some useless annotations, and fixed inlines annotations * removed `pretty_assertions` * removed pypy feature from cargo.toml * fix for Py_CompileStringFlags * tox runs word_count! * __dict__ changes are not supported for PyPy * fix 3.7 and copy comment * fix test script :flushed: * transfer ownership of strings to cpython when possible * remove cstr! macro * added missing nuls * as_bytes() -> b’’ string * symbol removed by mistake * properly shim pypy date time API, some tests are passing! * extension_module tests now not crashing! (some still skipped) * maybe travis has new pypy version? * small error on windows (build script) * fix conditional compilation * try to make tests run on travis.. * invert condition * added pytest-faulthandler to facilitate debugging * correctly name dir * use full paths * say —yes to conda * fix * syntax error * change PATH * fixed a terrible bug with PyTypeObjects in PyPy * fix PyTypeObject defs * re-enabled tests! * all tests are passing! * make the fix ad-hoc for now * removed build module * revert changes that cause an additional GC bug * prevented buggy test from failing pypy * removed unused comment * don’t run coverage on pypy * removed some erroneous symbols from function calls which are actually macros * restore py37 pyunicode missing def * use only `link_name` in PyPy specific declarations * only setup PyPy when testing against PyPy * annotation that was eaten during merge * remove change to comment by mistake + unnecessary changes to cargo.toml * xfail dates test only on pypy * changed comment to be a little more helpful * cleaned up some warnings * Update src/ffi3/ceval.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin PR notes * rustfmt * some documentation * if configured via env var only, default to cpython * remove extra unsafe * refer users to guide for pypy * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin applied patch * check that pypy at least build * search explicitly for libpypy * added note about some known unsupported features * use ld_version * export PYTHON_SYS_EXECUTABLE to `cargo build` test * inverted if * always link pypy dynamically * remove unused imports * Apply @kngwyu’s suggestion * fix tox configuration * try conda virtualenv * try to simply not install python at all inside pypy environment * setup pypy before using “python" * use system_site_packages * revert change to .travis * moved cpyext datetime documentation to module level, and revised it. * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * rustfmt * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * kept only notes that are relevant to users. * invert if * use bash and not sh
2019-04-23 11:18:42 +00:00
PyPy is also supported (via cpyext) for Python 3.5 only, targeted PyPy version is 7.0.0.
Please refer to the [pypy section in the guide](https://pyo3.rs/master/pypy.html).
Support rust extensions for PyPy via cpyext (#393) * wip * removed stuff * removed another change * implemented minimum amouth of ifdefs to make pypy3 hello world to compile * implemented minimum amount of ifdefs to make pypy3 hello world to compile * hacking on build.rs * compiler is happy! * few todos remain * extracted build logic to seperate module * added pypy test * finally fixed pypy structs * removed some todos * test should now be machine independent * fixed all pypy3 symbols * added pypy feature * removed `is_pypy` * added pypy2 declerations also * fix for cpython2 * improved libpypy detection * added all pypy2 macros * fixed errneous type * more fixes * fix python2 string macros * modsupport symbol * fix * fixed and added many symbols * fixes * remove dup * remove mac-specific config * fix all name mangling macros * unite imports * missing symbol * fix pybool * implemented another missing symbol * it works * fix merge conflict * uncomment non default features * cargo.toml * Cargo fmt * small merge fixes * use newer build version * whoops * fix build script * more build hacks * some random hiccups * small fixes * it builds! * it builds and runs * revert everything in FFI2 * revert changes to ffi2 * check python3 for pypy * tiny fix * revert ffi2 for real * revert weird formatting changes * bring back missing feature * tiny error * fix py3.7 issue * add pypy3.5 6.0 to travis * remove dbg! * another tiny fix * removed some useless annotations, and fixed inlines annotations * removed `pretty_assertions` * removed pypy feature from cargo.toml * fix for Py_CompileStringFlags * tox runs word_count! * __dict__ changes are not supported for PyPy * fix 3.7 and copy comment * fix test script :flushed: * transfer ownership of strings to cpython when possible * remove cstr! macro * added missing nuls * as_bytes() -> b’’ string * symbol removed by mistake * properly shim pypy date time API, some tests are passing! * extension_module tests now not crashing! (some still skipped) * maybe travis has new pypy version? * small error on windows (build script) * fix conditional compilation * try to make tests run on travis.. * invert condition * added pytest-faulthandler to facilitate debugging * correctly name dir * use full paths * say —yes to conda * fix * syntax error * change PATH * fixed a terrible bug with PyTypeObjects in PyPy * fix PyTypeObject defs * re-enabled tests! * all tests are passing! * make the fix ad-hoc for now * removed build module * revert changes that cause an additional GC bug * prevented buggy test from failing pypy * removed unused comment * don’t run coverage on pypy * removed some erroneous symbols from function calls which are actually macros * restore py37 pyunicode missing def * use only `link_name` in PyPy specific declarations * only setup PyPy when testing against PyPy * annotation that was eaten during merge * remove change to comment by mistake + unnecessary changes to cargo.toml * xfail dates test only on pypy * changed comment to be a little more helpful * cleaned up some warnings * Update src/ffi3/ceval.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin PR notes * rustfmt * some documentation * if configured via env var only, default to cpython * remove extra unsafe * refer users to guide for pypy * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin applied patch * check that pypy at least build * search explicitly for libpypy * added note about some known unsupported features * use ld_version * export PYTHON_SYS_EXECUTABLE to `cargo build` test * inverted if * always link pypy dynamically * remove unused imports * Apply @kngwyu’s suggestion * fix tox configuration * try conda virtualenv * try to simply not install python at all inside pypy environment * setup pypy before using “python" * use system_site_packages * revert change to .travis * moved cpyext datetime documentation to module level, and revised it. * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * rustfmt * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * kept only notes that are relevant to users. * invert if * use bash and not sh
2019-04-23 11:18:42 +00:00
You can either write a native Python module in Rust, or use Python from a Rust binary.
2015-04-19 03:29:19 +00:00
However, on some OSs, you need some additional packages. E.g. if you are on *Ubuntu 18.04*, please run
2018-11-26 04:31:49 +00:00
```bash
sudo apt install python3-dev python-dev
```
## Using Rust from Python
2015-04-19 03:29:19 +00:00
PyO3 can be used to generate a native Python module.
2019-02-01 13:01:18 +00:00
**`Cargo.toml`**
2017-05-21 12:37:56 +00:00
```toml
[package]
2018-08-30 21:19:23 +00:00
name = "string-sum"
version = "0.1.0"
2019-02-01 13:01:18 +00:00
edition = "2018"
[lib]
2018-08-30 21:19:23 +00:00
name = "string_sum"
2016-12-17 20:41:14 +00:00
crate-type = ["cdylib"]
2017-05-19 04:35:08 +00:00
[dependencies.pyo3]
2020-05-14 11:50:33 +00:00
version = "0.10.1"
2016-12-17 20:41:14 +00:00
features = ["extension-module"]
```
**`src/lib.rs`**
2017-05-21 12:37:56 +00:00
```rust
2018-07-30 20:52:22 +00:00
use pyo3::prelude::*;
2019-02-01 13:01:18 +00:00
use pyo3::wrap_pyfunction;
/// Formats the sum of two numbers as string.
2020-06-07 12:06:15 +00:00
#[pyfunction]
2018-07-30 20:52:22 +00:00
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
2018-05-01 13:31:11 +00:00
/// A Python module implemented in Rust.
2020-06-07 12:06:15 +00:00
#[pymodule]
2018-08-30 21:19:23 +00:00
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
2019-02-01 13:01:18 +00:00
m.add_wrapped(wrap_pyfunction!(sum_as_string))?;
Ok(())
2017-06-12 06:57:21 +00:00
}
```
2017-03-12 11:54:21 +00:00
On Windows and Linux, you can build normally with `cargo build --release`. On macOS, you need to set additional linker arguments. One option is to compile with `cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup`, the other is to create a `.cargo/config` with the following content:
```toml
[target.x86_64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
```
While developing, you can symlink (or copy) and rename the shared library from the target folder: On MacOS, rename `libstring_sum.dylib` to `string_sum.so`, on Windows `libstring_sum.dll` to `string_sum.pyd`, and on Linux `libstring_sum.so` to `string_sum.so`. Then open a Python shell in the same folder and you'll be able to `import string_sum`.
To build, test and publish your crate as a Python module, you can use [maturin](https://github.com/PyO3/maturin) or [setuptools-rust](https://github.com/PyO3/setuptools-rust). You can find an example for setuptools-rust in [examples/word-count](https://github.com/PyO3/pyo3/tree/master/examples/word-count), while maturin should work on your crate without any configuration.
2017-05-13 05:51:14 +00:00
## Using Python from Rust
2018-07-30 20:52:22 +00:00
If you want your Rust application to create a Python interpreter internally and
use it to run Python code, add `pyo3` to your `Cargo.toml` like this:
2018-07-30 20:52:22 +00:00
```toml
[dependencies]
2020-05-14 11:50:33 +00:00
pyo3 = "0.10.1"
2018-07-30 20:52:22 +00:00
```
Example program displaying the value of `sys.version` and the current user name:
2018-07-30 20:52:22 +00:00
```rust
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
2018-07-30 20:52:22 +00:00
2019-09-15 10:55:01 +00:00
fn main() -> Result<(), ()> {
2018-07-30 20:52:22 +00:00
let gil = Python::acquire_gil();
let py = gil.python();
2019-09-15 10:55:01 +00:00
main_(py).map_err(|e| {
// We can't display Python exceptions via std::fmt::Display,
// so print the error here manually.
2019-09-15 10:55:01 +00:00
e.print_and_set_sys_last_vars(py);
})
}
fn main_(py: Python) -> PyResult<()> {
2018-07-30 20:52:22 +00:00
let sys = py.import("sys")?;
let version: String = sys.get("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py);
2019-02-01 13:01:18 +00:00
let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
let user: String = py.eval(code, None, Some(&locals))?.extract()?;
2018-07-30 20:52:22 +00:00
println!("Hello {}, I'm Python {}", user, version);
Ok(())
}
```
Our guide has [a section](https://pyo3.rs/master/python_from_rust.html) with lots of examples
about this topic.
2019-09-15 10:55:01 +00:00
## Tools and libraries
* [maturin](https://github.com/PyO3/maturin) _Zero configuration build tool for Rust-made Python extensions_.
* [setuptools-rust](https://github.com/PyO3/setuptools-rust) _Setuptools plugin for Rust support_.
* [pyo3-built](https://github.com/PyO3/pyo3-built) _Simple macro to expose metadata obtained with the [`built`](https://crates.io/crates/built) crate as a [`PyDict`](https://pyo3.github.io/pyo3/pyo3/struct.PyDict.html)_
* [rust-numpy](https://github.com/PyO3/rust-numpy) _Rust binding of NumPy C-API_
* [dict-derive](https://github.com/gperinazzo/dict-derive) _Derive FromPyObject to automatically transform Python dicts into Rust structs_
## Examples
2018-07-30 20:52:22 +00:00
* [hyperjson](https://github.com/mre/hyperjson) _A hyper-fast Python module for reading/writing JSON data using Rust's serde-json_
* [html-py-ever](https://github.com/PyO3/setuptools-rust/tree/master/html-py-ever) _Using [html5ever](https://github.com/servo/html5ever) through [kuchiki](https://github.com/kuchiki-rs/kuchiki) to speed up html parsing and css-selecting._
2018-08-11 15:35:03 +00:00
* [point-process](https://github.com/ManifoldFR/point-process-rust/tree/master/pylib) _High level API for pointprocesses as a Python library_
2018-11-23 19:28:36 +00:00
* [autopy](https://github.com/autopilot-rs/autopy) _A simple, cross-platform GUI automation library for Python and Rust._
2019-06-05 06:51:25 +00:00
* Contains an example of building wheels on TravisCI and appveyor using [cibuildwheel](https://github.com/joerick/cibuildwheel)
2018-11-23 19:28:36 +00:00
* [orjson](https://github.com/ijl/orjson) _Fast Python JSON library_
2019-05-12 12:41:01 +00:00
* [inline-python](https://github.com/dronesforwork/inline-python) _Inline Python code directly in your Rust code_
2019-06-08 14:35:24 +00:00
* [Rogue-Gym](https://github.com/kngwyu/rogue-gym) _Customizable rogue-like game for AI experiments_
2019-06-05 06:51:25 +00:00
* Contains an example of building wheels on Azure Pipelines
2019-06-05 17:54:26 +00:00
* [fastuuid](https://github.com/thedrow/fastuuid/) _Python bindings to Rust's UUID library_
2019-06-08 14:35:24 +00:00
* [python-ext-wasm](https://github.com/wasmerio/python-ext-wasm) _Python library to run WebAssembly binaries_
* [mocpy](https://github.com/cds-astro/mocpy) _Astronomical Python library offering data structures for describing any arbitrary coverage regions on the unit sphere_
2020-02-13 22:13:28 +00:00
* [tokenizers](https://github.com/huggingface/tokenizers/tree/master/bindings/python) _Python bindings to the Hugging Face tokenizers (NLP) written in Rust_
2018-07-30 20:52:22 +00:00
2018-05-17 21:59:11 +00:00
## License
2017-05-13 05:51:14 +00:00
2018-05-17 21:59:11 +00:00
PyO3 is licensed under the [Apache-2.0 license](http://opensource.org/licenses/APACHE-2.0).
2018-06-14 08:10:39 +00:00
Python is licensed under the [Python License](https://docs.python.org/2/license.html).