pyo3/README.md

161 lines
7.6 KiB
Markdown
Raw Normal View History

2018-05-17 21:59:11 +00:00
# PyO3
2015-04-19 03:29:19 +00:00
2018-07-30 20:52:22 +00:00
[![Build Status](https://travis-ci.org/PyO3/pyo3.svg?branch=master)](https://travis-ci.org/PyO3/pyo3)
[![Actions Status](https://github.com/PyO3/pyo3/workflows/Test/badge.svg)](https://github.com/PyO3/pyo3/actions)
[![codecov](https://codecov.io/gh/PyO3/pyo3/branch/master/graph/badge.svg)](https://codecov.io/gh/PyO3/pyo3)
2018-07-30 20:52:22 +00:00
[![crates.io](http://meritbadge.herokuapp.com/pyo3)](https://crates.io/crates/pyo3)
[![minimum rustc 1.39](https://img.shields.io/badge/rustc-1.39+-blue.svg)](https://rust-lang.github.io/rfcs/2495-min-rust-version.html)
2018-07-30 20:52:22 +00:00
[![Join the dev chat](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/PyO3/Lobby)
2018-05-17 21:59:11 +00:00
[Rust](http://www.rust-lang.org/) bindings for [Python](https://www.python.org/). This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules.
2015-04-19 03:29:19 +00:00
2018-06-01 13:03:04 +00:00
* User Guide: [stable](https://pyo3.rs) | [master](https://pyo3.rs/master)
2015-04-19 03:29:19 +00:00
2019-10-26 13:47:19 +00:00
* API Documentation: [stable](https://docs.rs/pyo3/) | [master](https://pyo3.rs/master/doc)
2018-10-04 06:46:15 +00:00
2019-04-27 20:14:34 +00:00
A comparison with rust-cpython can be found [in the guide](https://pyo3.rs/master/rust_cpython.html).
2017-05-21 12:37:56 +00:00
2018-05-17 21:59:11 +00:00
## Usage
2017-05-21 12:37:56 +00:00
2020-06-18 09:49:43 +00:00
PyO3 supports Python 3.5 and up. The minimum required Rust version is 1.39.0.
2019-11-23 06:29:39 +00:00
Support rust extensions for PyPy via cpyext (#393) * wip * removed stuff * removed another change * implemented minimum amouth of ifdefs to make pypy3 hello world to compile * implemented minimum amount of ifdefs to make pypy3 hello world to compile * hacking on build.rs * compiler is happy! * few todos remain * extracted build logic to seperate module * added pypy test * finally fixed pypy structs * removed some todos * test should now be machine independent * fixed all pypy3 symbols * added pypy feature * removed `is_pypy` * added pypy2 declerations also * fix for cpython2 * improved libpypy detection * added all pypy2 macros * fixed errneous type * more fixes * fix python2 string macros * modsupport symbol * fix * fixed and added many symbols * fixes * remove dup * remove mac-specific config * fix all name mangling macros * unite imports * missing symbol * fix pybool * implemented another missing symbol * it works * fix merge conflict * uncomment non default features * cargo.toml * Cargo fmt * small merge fixes * use newer build version * whoops * fix build script * more build hacks * some random hiccups * small fixes * it builds! * it builds and runs * revert everything in FFI2 * revert changes to ffi2 * check python3 for pypy * tiny fix * revert ffi2 for real * revert weird formatting changes * bring back missing feature * tiny error * fix py3.7 issue * add pypy3.5 6.0 to travis * remove dbg! * another tiny fix * removed some useless annotations, and fixed inlines annotations * removed `pretty_assertions` * removed pypy feature from cargo.toml * fix for Py_CompileStringFlags * tox runs word_count! * __dict__ changes are not supported for PyPy * fix 3.7 and copy comment * fix test script :flushed: * transfer ownership of strings to cpython when possible * remove cstr! macro * added missing nuls * as_bytes() -> b’’ string * symbol removed by mistake * properly shim pypy date time API, some tests are passing! * extension_module tests now not crashing! (some still skipped) * maybe travis has new pypy version? * small error on windows (build script) * fix conditional compilation * try to make tests run on travis.. * invert condition * added pytest-faulthandler to facilitate debugging * correctly name dir * use full paths * say —yes to conda * fix * syntax error * change PATH * fixed a terrible bug with PyTypeObjects in PyPy * fix PyTypeObject defs * re-enabled tests! * all tests are passing! * make the fix ad-hoc for now * removed build module * revert changes that cause an additional GC bug * prevented buggy test from failing pypy * removed unused comment * don’t run coverage on pypy * removed some erroneous symbols from function calls which are actually macros * restore py37 pyunicode missing def * use only `link_name` in PyPy specific declarations * only setup PyPy when testing against PyPy * annotation that was eaten during merge * remove change to comment by mistake + unnecessary changes to cargo.toml * xfail dates test only on pypy * changed comment to be a little more helpful * cleaned up some warnings * Update src/ffi3/ceval.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin PR notes * rustfmt * some documentation * if configured via env var only, default to cpython * remove extra unsafe * refer users to guide for pypy * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin applied patch * check that pypy at least build * search explicitly for libpypy * added note about some known unsupported features * use ld_version * export PYTHON_SYS_EXECUTABLE to `cargo build` test * inverted if * always link pypy dynamically * remove unused imports * Apply @kngwyu’s suggestion * fix tox configuration * try conda virtualenv * try to simply not install python at all inside pypy environment * setup pypy before using “python" * use system_site_packages * revert change to .travis * moved cpyext datetime documentation to module level, and revised it. * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * rustfmt * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * kept only notes that are relevant to users. * invert if * use bash and not sh
2019-04-23 11:18:42 +00:00
PyPy is also supported (via cpyext) for Python 3.5 only, targeted PyPy version is 7.0.0.
Please refer to the [pypy section in the guide](https://pyo3.rs/master/pypy.html).
Support rust extensions for PyPy via cpyext (#393) * wip * removed stuff * removed another change * implemented minimum amouth of ifdefs to make pypy3 hello world to compile * implemented minimum amount of ifdefs to make pypy3 hello world to compile * hacking on build.rs * compiler is happy! * few todos remain * extracted build logic to seperate module * added pypy test * finally fixed pypy structs * removed some todos * test should now be machine independent * fixed all pypy3 symbols * added pypy feature * removed `is_pypy` * added pypy2 declerations also * fix for cpython2 * improved libpypy detection * added all pypy2 macros * fixed errneous type * more fixes * fix python2 string macros * modsupport symbol * fix * fixed and added many symbols * fixes * remove dup * remove mac-specific config * fix all name mangling macros * unite imports * missing symbol * fix pybool * implemented another missing symbol * it works * fix merge conflict * uncomment non default features * cargo.toml * Cargo fmt * small merge fixes * use newer build version * whoops * fix build script * more build hacks * some random hiccups * small fixes * it builds! * it builds and runs * revert everything in FFI2 * revert changes to ffi2 * check python3 for pypy * tiny fix * revert ffi2 for real * revert weird formatting changes * bring back missing feature * tiny error * fix py3.7 issue * add pypy3.5 6.0 to travis * remove dbg! * another tiny fix * removed some useless annotations, and fixed inlines annotations * removed `pretty_assertions` * removed pypy feature from cargo.toml * fix for Py_CompileStringFlags * tox runs word_count! * __dict__ changes are not supported for PyPy * fix 3.7 and copy comment * fix test script :flushed: * transfer ownership of strings to cpython when possible * remove cstr! macro * added missing nuls * as_bytes() -> b’’ string * symbol removed by mistake * properly shim pypy date time API, some tests are passing! * extension_module tests now not crashing! (some still skipped) * maybe travis has new pypy version? * small error on windows (build script) * fix conditional compilation * try to make tests run on travis.. * invert condition * added pytest-faulthandler to facilitate debugging * correctly name dir * use full paths * say —yes to conda * fix * syntax error * change PATH * fixed a terrible bug with PyTypeObjects in PyPy * fix PyTypeObject defs * re-enabled tests! * all tests are passing! * make the fix ad-hoc for now * removed build module * revert changes that cause an additional GC bug * prevented buggy test from failing pypy * removed unused comment * don’t run coverage on pypy * removed some erroneous symbols from function calls which are actually macros * restore py37 pyunicode missing def * use only `link_name` in PyPy specific declarations * only setup PyPy when testing against PyPy * annotation that was eaten during merge * remove change to comment by mistake + unnecessary changes to cargo.toml * xfail dates test only on pypy * changed comment to be a little more helpful * cleaned up some warnings * Update src/ffi3/ceval.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin PR notes * rustfmt * some documentation * if configured via env var only, default to cpython * remove extra unsafe * refer users to guide for pypy * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * Update guide/src/pypy.md Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * @konstin applied patch * check that pypy at least build * search explicitly for libpypy * added note about some known unsupported features * use ld_version * export PYTHON_SYS_EXECUTABLE to `cargo build` test * inverted if * always link pypy dynamically * remove unused imports * Apply @kngwyu’s suggestion * fix tox configuration * try conda virtualenv * try to simply not install python at all inside pypy environment * setup pypy before using “python" * use system_site_packages * revert change to .travis * moved cpyext datetime documentation to module level, and revised it. * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * rustfmt * Update src/ffi/datetime.rs Co-Authored-By: omerbenamram <omerbenamram@gmail.com> * kept only notes that are relevant to users. * invert if * use bash and not sh
2019-04-23 11:18:42 +00:00
You can either write a native Python module in Rust, or use Python from a Rust binary.
2015-04-19 03:29:19 +00:00
However, on some OSs, you need some additional packages. E.g. if you are on *Ubuntu 18.04*, please run
2018-11-26 04:31:49 +00:00
```bash
sudo apt install python3-dev python-dev
```
## Using Rust from Python
2015-04-19 03:29:19 +00:00
PyO3 can be used to generate a native Python module.
2019-02-01 13:01:18 +00:00
**`Cargo.toml`**
2017-05-21 12:37:56 +00:00
```toml
[package]
2018-08-30 21:19:23 +00:00
name = "string-sum"
version = "0.1.0"
2019-02-01 13:01:18 +00:00
edition = "2018"
[lib]
2018-08-30 21:19:23 +00:00
name = "string_sum"
2016-12-17 20:41:14 +00:00
crate-type = ["cdylib"]
2017-05-19 04:35:08 +00:00
[dependencies.pyo3]
2020-06-30 05:53:55 +00:00
version = "0.11.1"
2016-12-17 20:41:14 +00:00
features = ["extension-module"]
```
**`src/lib.rs`**
2017-05-21 12:37:56 +00:00
```rust
2018-07-30 20:52:22 +00:00
use pyo3::prelude::*;
2019-02-01 13:01:18 +00:00
use pyo3::wrap_pyfunction;
/// Formats the sum of two numbers as string.
2020-06-07 12:06:15 +00:00
#[pyfunction]
2018-07-30 20:52:22 +00:00
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
2018-05-01 13:31:11 +00:00
/// A Python module implemented in Rust.
2020-06-07 12:06:15 +00:00
#[pymodule]
2018-08-30 21:19:23 +00:00
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
2019-02-01 13:01:18 +00:00
m.add_wrapped(wrap_pyfunction!(sum_as_string))?;
Ok(())
2017-06-12 06:57:21 +00:00
}
```
2017-03-12 11:54:21 +00:00
On Windows and Linux, you can build normally with `cargo build --release`. On macOS, you need to set additional linker arguments. One option is to compile with `cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup`, the other is to create a `.cargo/config` with the following content:
```toml
[target.x86_64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
```
While developing, you can symlink (or copy) and rename the shared library from the target folder: On MacOS, rename `libstring_sum.dylib` to `string_sum.so`, on Windows `libstring_sum.dll` to `string_sum.pyd`, and on Linux `libstring_sum.so` to `string_sum.so`. Then open a Python shell in the same folder and you'll be able to `import string_sum`.
Adding the `cdylib` arguments in the `Cargo.toml` files changes the way your crate is compiled.
Other Rust projects using your crate will have to link against the `.so` or `.pyd` file rather than include your library directly as normal.
In order to make available your crate in the usual way for Rust user, you you might want to consider using both `crate-type = ["cdylib", "rlib"]` so that Rust users can use the `rlib` (the default lib crate type).
Another possibility is to create a new crate to perform the binding.
To build, test and publish your crate as a Python module, you can use [maturin](https://github.com/PyO3/maturin) or [setuptools-rust](https://github.com/PyO3/setuptools-rust). You can find an example for setuptools-rust in [examples/word-count](https://github.com/PyO3/pyo3/tree/master/examples/word-count), while maturin should work on your crate without any configuration.
2017-05-13 05:51:14 +00:00
## Using Python from Rust
2018-07-30 20:52:22 +00:00
If you want your Rust application to create a Python interpreter internally and
use it to run Python code, add `pyo3` to your `Cargo.toml` like this:
2018-07-30 20:52:22 +00:00
```toml
[dependencies]
2020-06-30 05:53:55 +00:00
pyo3 = "0.11.1"
2018-07-30 20:52:22 +00:00
```
Example program displaying the value of `sys.version` and the current user name:
2018-07-30 20:52:22 +00:00
```rust
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
2018-07-30 20:52:22 +00:00
2019-09-15 10:55:01 +00:00
fn main() -> Result<(), ()> {
2018-07-30 20:52:22 +00:00
let gil = Python::acquire_gil();
let py = gil.python();
2019-09-15 10:55:01 +00:00
main_(py).map_err(|e| {
// We can't display Python exceptions via std::fmt::Display,
// so print the error here manually.
2019-09-15 10:55:01 +00:00
e.print_and_set_sys_last_vars(py);
})
}
fn main_(py: Python) -> PyResult<()> {
2018-07-30 20:52:22 +00:00
let sys = py.import("sys")?;
let version: String = sys.get("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py);
2019-02-01 13:01:18 +00:00
let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
let user: String = py.eval(code, None, Some(&locals))?.extract()?;
2018-07-30 20:52:22 +00:00
println!("Hello {}, I'm Python {}", user, version);
Ok(())
}
```
Our guide has [a section](https://pyo3.rs/master/python_from_rust.html) with lots of examples
about this topic.
2019-09-15 10:55:01 +00:00
## Tools and libraries
* [maturin](https://github.com/PyO3/maturin) _Zero configuration build tool for Rust-made Python extensions_.
* [setuptools-rust](https://github.com/PyO3/setuptools-rust) _Setuptools plugin for Rust support_.
* [pyo3-built](https://github.com/PyO3/pyo3-built) _Simple macro to expose metadata obtained with the [`built`](https://crates.io/crates/built) crate as a [`PyDict`](https://pyo3.github.io/pyo3/pyo3/struct.PyDict.html)_
* [rust-numpy](https://github.com/PyO3/rust-numpy) _Rust binding of NumPy C-API_
* [dict-derive](https://github.com/gperinazzo/dict-derive) _Derive FromPyObject to automatically transform Python dicts into Rust structs_
## Examples
2018-07-30 20:52:22 +00:00
* [hyperjson](https://github.com/mre/hyperjson) _A hyper-fast Python module for reading/writing JSON data using Rust's serde-json_
* [html-py-ever](https://github.com/PyO3/setuptools-rust/tree/master/html-py-ever) _Using [html5ever](https://github.com/servo/html5ever) through [kuchiki](https://github.com/kuchiki-rs/kuchiki) to speed up html parsing and css-selecting._
2018-08-11 15:35:03 +00:00
* [point-process](https://github.com/ManifoldFR/point-process-rust/tree/master/pylib) _High level API for pointprocesses as a Python library_
2018-11-23 19:28:36 +00:00
* [autopy](https://github.com/autopilot-rs/autopy) _A simple, cross-platform GUI automation library for Python and Rust._
2019-06-05 06:51:25 +00:00
* Contains an example of building wheels on TravisCI and appveyor using [cibuildwheel](https://github.com/joerick/cibuildwheel)
2018-11-23 19:28:36 +00:00
* [orjson](https://github.com/ijl/orjson) _Fast Python JSON library_
2019-05-12 12:41:01 +00:00
* [inline-python](https://github.com/dronesforwork/inline-python) _Inline Python code directly in your Rust code_
2019-06-08 14:35:24 +00:00
* [Rogue-Gym](https://github.com/kngwyu/rogue-gym) _Customizable rogue-like game for AI experiments_
2019-06-05 06:51:25 +00:00
* Contains an example of building wheels on Azure Pipelines
2019-06-05 17:54:26 +00:00
* [fastuuid](https://github.com/thedrow/fastuuid/) _Python bindings to Rust's UUID library_
2019-06-08 14:35:24 +00:00
* [python-ext-wasm](https://github.com/wasmerio/python-ext-wasm) _Python library to run WebAssembly binaries_
* [mocpy](https://github.com/cds-astro/mocpy) _Astronomical Python library offering data structures for describing any arbitrary coverage regions on the unit sphere_
2020-02-13 22:13:28 +00:00
* [tokenizers](https://github.com/huggingface/tokenizers/tree/master/bindings/python) _Python bindings to the Hugging Face tokenizers (NLP) written in Rust_
2018-07-30 20:52:22 +00:00
2018-05-17 21:59:11 +00:00
## License
2017-05-13 05:51:14 +00:00
2018-05-17 21:59:11 +00:00
PyO3 is licensed under the [Apache-2.0 license](http://opensource.org/licenses/APACHE-2.0).
2018-06-14 08:10:39 +00:00
Python is licensed under the [Python License](https://docs.python.org/2/license.html).