docs: major rewrite for Bound API (#3906)
* wip bound docs * Update guide/src/python_from_rust/calling-existing-code.md Co-authored-by: Lily Foote <code@lilyf.org> * continue to move and tidy up * Apply suggestions from code review Co-authored-by: Lily Foote <code@lilyf.org> * update URL * complete python-from-rust.md * progress on types.md; probably more to go * update doctest paths * review: Icxolu * finish updating `types.md` to Bound API * update remainder of the guide to Bound API * Update guide/src/performance.md Co-authored-by: Icxolu <10486322+Icxolu@users.noreply.github.com> * Update guide/src/types.md Co-authored-by: Icxolu <10486322+Icxolu@users.noreply.github.com> * Update src/lib.rs * review: Icxolu * Update guide/src/python-from-rust.md Co-authored-by: Adam Reichold <adamreichold@users.noreply.github.com> * Update guide/src/async-await.md Co-authored-by: Adam Reichold <adamreichold@users.noreply.github.com> * review: adamreichold --------- Co-authored-by: Lily Foote <code@lilyf.org> Co-authored-by: Icxolu <10486322+Icxolu@users.noreply.github.com> Co-authored-by: Adam Reichold <adamreichold@users.noreply.github.com>
This commit is contained in:
David Hewitt
GitHub
parent
75af678f43
commit
9145fcfe19
|
|
@ -59,6 +59,7 @@ Those flags are set in [`build.rs`](#6-buildrs-and-pyo3-build-config).
|
|||
[`src/types`] contains bindings to [built-in types](https://docs.python.org/3/library/stdtypes.html)
|
||||
of Python, such as `dict` and `list`.
|
||||
For historical reasons, Python's `object` is called `PyAny` in PyO3 and located in [`src/types/any.rs`].
|
||||
|
||||
Currently, `PyAny` is a straightforward wrapper of `ffi::PyObject`, defined as:
|
||||
|
||||
```rust
|
||||
|
|
@ -66,38 +67,16 @@ Currently, `PyAny` is a straightforward wrapper of `ffi::PyObject`, defined as:
|
|||
pub struct PyAny(UnsafeCell<ffi::PyObject>);
|
||||
```
|
||||
|
||||
All built-in types are defined as a C struct.
|
||||
For example, `dict` is defined as:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
/* Base object */
|
||||
PyObject ob_base;
|
||||
/* Number of items in the dictionary */
|
||||
Py_ssize_t ma_used;
|
||||
/* Dictionary version */
|
||||
uint64_t ma_version_tag;
|
||||
PyDictKeysObject *ma_keys;
|
||||
PyObject **ma_values;
|
||||
} PyDictObject;
|
||||
```
|
||||
|
||||
However, we cannot access such a specific data structure with `#[cfg(Py_LIMITED_API)]` set.
|
||||
Thus, all builtin objects are implemented as opaque types by wrapping `PyAny`, e.g.,:
|
||||
Concrete Python objects are implemented by wrapping `PyAny`, e.g.,:
|
||||
|
||||
```rust
|
||||
#[repr(transparent)]
|
||||
pub struct PyDict(PyAny);
|
||||
```
|
||||
|
||||
Note that `PyAny` is not a pointer, and it is usually used as a pointer to the object in the
|
||||
Python heap, as `&PyAny`.
|
||||
This design choice can be changed
|
||||
(see the discussion in [#1056](https://github.com/PyO3/pyo3/issues/1056)).
|
||||
These types are not intended to be accessed directly, and instead are used through the `Py<T>` and `Bound<T>` smart pointers.
|
||||
|
||||
Since we need lots of boilerplate for implementing common traits for these types
|
||||
(e.g., `AsPyPointer`, `AsRef<PyAny>`, and `Debug`), we have some macros in
|
||||
[`src/types/mod.rs`].
|
||||
We have some macros in [`src/types/mod.rs`] which make it easier to implement APIs for concrete Python types.
|
||||
|
||||
## 3. `PyClass` and related functionalities
|
||||
|
||||
|
|
|
|||
|
|
@ -5,22 +5,25 @@
|
|||
---
|
||||
|
||||
- [Getting started](getting-started.md)
|
||||
- [Python modules](module.md)
|
||||
- [Python functions](function.md)
|
||||
- [Function signatures](function/signature.md)
|
||||
- [Error handling](function/error-handling.md)
|
||||
- [Python classes](class.md)
|
||||
- [Class customizations](class/protocols.md)
|
||||
- [Basic object customization](class/object.md)
|
||||
- [Emulating numeric types](class/numeric.md)
|
||||
- [Emulating callable objects](class/call.md)
|
||||
- [Using Rust from Python](rust-from-python.md)
|
||||
- [Python modules](module.md)
|
||||
- [Python functions](function.md)
|
||||
- [Function signatures](function/signature.md)
|
||||
- [Error handling](function/error-handling.md)
|
||||
- [Python classes](class.md)
|
||||
- [Class customizations](class/protocols.md)
|
||||
- [Basic object customization](class/object.md)
|
||||
- [Emulating numeric types](class/numeric.md)
|
||||
- [Emulating callable objects](class/call.md)
|
||||
- [Calling Python from Rust](python-from-rust.md)
|
||||
- [Python object types](types.md)
|
||||
- [Python exceptions](exception.md)
|
||||
- [Calling Python functions](python-from-rust/function-calls.md)
|
||||
- [Executing existing Python code](python-from-rust/calling-existing-code.md)
|
||||
- [Type conversions](conversions.md)
|
||||
- [Mapping of Rust types to Python types](conversions/tables.md)
|
||||
- [Conversion traits](conversions/traits.md)
|
||||
- [Python exceptions](exception.md)
|
||||
- [Calling Python from Rust](python-from-rust.md)
|
||||
- [Using `async` and `await`](async-await.md)
|
||||
- [GIL, mutability and object types](types.md)
|
||||
- [Parallelism](parallelism.md)
|
||||
- [Debugging](debugging.md)
|
||||
- [Features reference](features.md)
|
||||
|
|
|
|||
|
|
@ -5,10 +5,3 @@
|
|||
PyO3 exposes much of Python's C API through the `ffi` module.
|
||||
|
||||
The C API is naturally unsafe and requires you to manage reference counts, errors and specific invariants yourself. Please refer to the [C API Reference Manual](https://docs.python.org/3/c-api/) and [The Rustonomicon](https://doc.rust-lang.org/nightly/nomicon/ffi.html) before using any function from that API.
|
||||
|
||||
## Memory management
|
||||
|
||||
PyO3's `&PyAny` "owned references" and `Py<PyAny>` smart pointers are used to
|
||||
access memory stored in Python's heap. This memory sometimes lives for longer
|
||||
than expected because of differences in Rust and Python's memory models. See
|
||||
the chapter on [memory management](./memory.md) for more information.
|
||||
|
|
|
|||
|
|
@ -30,13 +30,13 @@ async fn sleep(seconds: f64, result: Option<PyObject>) -> Option<PyObject> {
|
|||
|
||||
Resulting future of an `async fn` decorated by `#[pyfunction]` must be `Send + 'static` to be embedded in a Python object.
|
||||
|
||||
As a consequence, `async fn` parameters and return types must also be `Send + 'static`, so it is not possible to have a signature like `async fn does_not_compile(arg: &PyAny, py: Python<'_>) -> &PyAny`.
|
||||
As a consequence, `async fn` parameters and return types must also be `Send + 'static`, so it is not possible to have a signature like `async fn does_not_compile<'py>(arg: Bound<'py, PyAny>) -> Bound<'py, PyAny>`.
|
||||
|
||||
However, there is an exception for method receiver, so async methods can accept `&self`/`&mut self`. Note that this means that the class instance is borrowed for as long as the returned future is not completed, even across yield points and while waiting for I/O operations to complete. Hence, other methods cannot obtain exclusive borrows while the future is still being polled. This is the same as how async methods in Rust generally work but it is more problematic for Rust code interfacing with Python code due to pervasive shared mutability. This strongly suggests to prefer shared borrows `&self` to exclusive ones `&mut self` to avoid racy borrow check failures at runtime.
|
||||
However, there is an exception for method receivers, so async methods can accept `&self`/`&mut self`. Note that this means that the class instance is borrowed for as long as the returned future is not completed, even across yield points and while waiting for I/O operations to complete. Hence, other methods cannot obtain exclusive borrows while the future is still being polled. This is the same as how async methods in Rust generally work but it is more problematic for Rust code interfacing with Python code due to pervasive shared mutability. This strongly suggests to prefer shared borrows `&self` over exclusive ones `&mut self` to avoid racy borrow check failures at runtime.
|
||||
|
||||
## Implicit GIL holding
|
||||
|
||||
Even if it is not possible to pass a `py: Python<'_>` parameter to `async fn`, the GIL is still held during the execution of the future – it's also the case for regular `fn` without `Python<'_>`/`&PyAny` parameter, yet the GIL is held.
|
||||
Even if it is not possible to pass a `py: Python<'py>` parameter to `async fn`, the GIL is still held during the execution of the future – it's also the case for regular `fn` without `Python<'py>`/`Bound<'py, PyAny>` parameter, yet the GIL is held.
|
||||
|
||||
It is still possible to get a `Python` marker using [`Python::with_gil`]({{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.with_gil); because `with_gil` is reentrant and optimized, the cost will be negligible.
|
||||
|
||||
|
|
@ -47,7 +47,11 @@ There is currently no simple way to release the GIL when awaiting a future, *but
|
|||
Here is the advised workaround for now:
|
||||
|
||||
```rust,ignore
|
||||
use std::{future::Future, pin::{Pin, pin}, task::{Context, Poll}};
|
||||
use std::{
|
||||
future::Future,
|
||||
pin::{Pin, pin},
|
||||
task::{Context, Poll},
|
||||
};
|
||||
use pyo3::prelude::*;
|
||||
|
||||
struct AllowThreads<F>(F);
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ enum Shape {
|
|||
Circle { radius: f64 },
|
||||
Rectangle { width: f64, height: f64 },
|
||||
RegularPolygon { side_count: u32, radius: f64 },
|
||||
Nothing { },
|
||||
Nothing {},
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -89,7 +89,7 @@ Currently, the best alternative is to write a macro which expands to a new `#[py
|
|||
use pyo3::prelude::*;
|
||||
|
||||
struct GenericClass<T> {
|
||||
data: T
|
||||
data: T,
|
||||
}
|
||||
|
||||
macro_rules! create_interface {
|
||||
|
|
@ -102,7 +102,9 @@ macro_rules! create_interface {
|
|||
impl $name {
|
||||
#[new]
|
||||
pub fn new(data: $type) -> Self {
|
||||
Self { inner: GenericClass { data: data } }
|
||||
Self {
|
||||
inner: GenericClass { data: data },
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
|
@ -187,23 +189,21 @@ fn my_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
|
|||
}
|
||||
```
|
||||
|
||||
## Bound and interior mutability
|
||||
## Bound<T> and interior mutability
|
||||
|
||||
You sometimes need to convert your `#[pyclass]` into a Python object and access it
|
||||
from Rust code (e.g., for testing it).
|
||||
[`Bound`] is the primary interface for that.
|
||||
Often is is useful to turn a `#[pyclass]` type `T` into a Python object and access it from Rust code. The [`Py<T>`] and [`Bound<'py, T>`] smart pointers are the ways to represent a Python object in PyO3's API. More detail can be found about them [in the Python objects](./types.md#pyo3s-smart-pointers) section of the guide.
|
||||
|
||||
To mutate data behind a `Bound<'_, T>` safely, PyO3 employs the
|
||||
[Interior Mutability Pattern](https://doc.rust-lang.org/book/ch15-05-interior-mutability.html)
|
||||
like [`RefCell`].
|
||||
Most Python objects do not offer exclusive (`&mut`) access (see the [section on Python's memory model](./python-from-rust.md#pythons-memory-model)). However, Rust structs wrapped as Python objects (called `pyclass` types) often *do* need `&mut` access. Due to the GIL, PyO3 *can* guarantee exclusive access to them.
|
||||
|
||||
Users who are familiar with `RefCell` can use `Bound` just like `RefCell`.
|
||||
The Rust borrow checker cannot reason about `&mut` references once an object's ownership has been passed to the Python interpreter. This means that borrow checking is done at runtime using with a scheme very similar to `std::cell::RefCell<T>`. This is known as [interior mutability](https://doc.rust-lang.org/book/ch15-05-interior-mutability.html).
|
||||
|
||||
For users who are not very familiar with `RefCell`, here is a reminder of Rust's rules of borrowing:
|
||||
Users who are familiar with `RefCell<T>` can use `Py<T>` and `Bound<'py, T>` just like `RefCell<T>`.
|
||||
|
||||
For users who are not very familiar with `RefCell<T>`, here is a reminder of Rust's rules of borrowing:
|
||||
- At any given time, you can have either (but not both of) one mutable reference or any number of immutable references.
|
||||
- References must always be valid.
|
||||
- References can never outlast the data they refer to.
|
||||
|
||||
`Bound`, like `RefCell`, ensures these borrowing rules by tracking references at runtime.
|
||||
`Py<T>` and `Bound<'py, T>`, like `RefCell<T>`, ensure these borrowing rules by tracking references at runtime.
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
|
|
@ -233,10 +233,8 @@ Python::with_gil(|py| {
|
|||
});
|
||||
```
|
||||
|
||||
A `Bound<'py, T>` is restricted to the GIL lifetime `'py`.
|
||||
To make the object longer lived (for example, to store it in a struct on the
|
||||
Rust side), you can use `Py<T>`, which stores an object longer than the GIL
|
||||
lifetime, and therefore needs a `Python<'_>` token to access.
|
||||
A `Bound<'py, T>` is restricted to the GIL lifetime `'py`. To make the object longer lived (for example, to store it in a struct on the
|
||||
Rust side), use `Py<T>`. `Py<T>` needs a `Python<'_>` token to allow access:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
|
|
@ -252,7 +250,7 @@ fn return_myclass() -> Py<MyClass> {
|
|||
let obj = return_myclass();
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let bound = obj.bind(py); // Py<MyClass>::bind returns &Bound<'_, MyClass>
|
||||
let bound = obj.bind(py); // Py<MyClass>::bind returns &Bound<'py, MyClass>
|
||||
let obj_ref = bound.borrow(); // Get PyRef<T>
|
||||
assert_eq!(obj_ref.num, 1);
|
||||
});
|
||||
|
|
@ -431,7 +429,7 @@ impl DictWithCounter {
|
|||
Self::default()
|
||||
}
|
||||
|
||||
fn set(slf: &Bound<'_, Self>, key: String, value: &PyAny) -> PyResult<()> {
|
||||
fn set(slf: &Bound<'_, Self>, key: String, value: Bound<'_, PyAny>) -> PyResult<()> {
|
||||
slf.borrow_mut().counter.entry(key.clone()).or_insert(0);
|
||||
let dict = slf.downcast::<PyDict>()?;
|
||||
dict.set_item(key, value)
|
||||
|
|
@ -1319,7 +1317,7 @@ impl pyo3::impl_::pyclass::PyClassImpl for MyClass {
|
|||
[`PyTypeInfo`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeInfo.html
|
||||
|
||||
[`Py`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html
|
||||
[`Bound`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Bound.html
|
||||
[`Bound<'_, T>`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Bound.html
|
||||
[`PyClass`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/trait.PyClass.html
|
||||
[`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html
|
||||
[`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRefMut.html
|
||||
|
|
|
|||
|
|
@ -12,49 +12,49 @@ The table below contains the Python type and the corresponding function argument
|
|||
|
||||
| Python | Rust | Rust (Python-native) |
|
||||
| ------------- |:-------------------------------:|:--------------------:|
|
||||
| `object` | - | `&PyAny` |
|
||||
| `str` | `String`, `Cow<str>`, `&str`, `char`, `OsString`, `PathBuf`, `Path` | `&PyString`, `&PyUnicode` |
|
||||
| `bytes` | `Vec<u8>`, `&[u8]`, `Cow<[u8]>` | `&PyBytes` |
|
||||
| `bool` | `bool` | `&PyBool` |
|
||||
| `int` | `i8`, `u8`, `i16`, `u16`, `i32`, `u32`, `i64`, `u64`, `i128`, `u128`, `isize`, `usize`, `num_bigint::BigInt`[^1], `num_bigint::BigUint`[^1] | `&PyLong` |
|
||||
| `float` | `f32`, `f64` | `&PyFloat` |
|
||||
| `complex` | `num_complex::Complex`[^2] | `&PyComplex` |
|
||||
| `list[T]` | `Vec<T>` | `&PyList` |
|
||||
| `dict[K, V]` | `HashMap<K, V>`, `BTreeMap<K, V>`, `hashbrown::HashMap<K, V>`[^3], `indexmap::IndexMap<K, V>`[^4] | `&PyDict` |
|
||||
| `tuple[T, U]` | `(T, U)`, `Vec<T>` | `&PyTuple` |
|
||||
| `set[T]` | `HashSet<T>`, `BTreeSet<T>`, `hashbrown::HashSet<T>`[^3] | `&PySet` |
|
||||
| `frozenset[T]` | `HashSet<T>`, `BTreeSet<T>`, `hashbrown::HashSet<T>`[^3] | `&PyFrozenSet` |
|
||||
| `bytearray` | `Vec<u8>`, `Cow<[u8]>` | `&PyByteArray` |
|
||||
| `slice` | - | `&PySlice` |
|
||||
| `type` | - | `&PyType` |
|
||||
| `module` | - | `&PyModule` |
|
||||
| `object` | - | `PyAny` |
|
||||
| `str` | `String`, `Cow<str>`, `&str`, `char`, `OsString`, `PathBuf`, `Path` | `PyString`, `PyUnicode` |
|
||||
| `bytes` | `Vec<u8>`, `&[u8]`, `Cow<[u8]>` | `PyBytes` |
|
||||
| `bool` | `bool` | `PyBool` |
|
||||
| `int` | `i8`, `u8`, `i16`, `u16`, `i32`, `u32`, `i64`, `u64`, `i128`, `u128`, `isize`, `usize`, `num_bigint::BigInt`[^1], `num_bigint::BigUint`[^1] | `PyLong` |
|
||||
| `float` | `f32`, `f64` | `PyFloat` |
|
||||
| `complex` | `num_complex::Complex`[^2] | `PyComplex` |
|
||||
| `list[T]` | `Vec<T>` | `PyList` |
|
||||
| `dict[K, V]` | `HashMap<K, V>`, `BTreeMap<K, V>`, `hashbrown::HashMap<K, V>`[^3], `indexmap::IndexMap<K, V>`[^4] | `PyDict` |
|
||||
| `tuple[T, U]` | `(T, U)`, `Vec<T>` | `PyTuple` |
|
||||
| `set[T]` | `HashSet<T>`, `BTreeSet<T>`, `hashbrown::HashSet<T>`[^3] | `PySet` |
|
||||
| `frozenset[T]` | `HashSet<T>`, `BTreeSet<T>`, `hashbrown::HashSet<T>`[^3] | `PyFrozenSet` |
|
||||
| `bytearray` | `Vec<u8>`, `Cow<[u8]>` | `PyByteArray` |
|
||||
| `slice` | - | `PySlice` |
|
||||
| `type` | - | `PyType` |
|
||||
| `module` | - | `PyModule` |
|
||||
| `collections.abc.Buffer` | - | `PyBuffer<T>` |
|
||||
| `datetime.datetime` | `SystemTime`, `chrono::DateTime<Tz>`[^5], `chrono::NaiveDateTime`[^5] | `&PyDateTime` |
|
||||
| `datetime.date` | `chrono::NaiveDate`[^5] | `&PyDate` |
|
||||
| `datetime.time` | `chrono::NaiveTime`[^5] | `&PyTime` |
|
||||
| `datetime.tzinfo` | `chrono::FixedOffset`[^5], `chrono::Utc`[^5], `chrono_tz::TimeZone`[^6] | `&PyTzInfo` |
|
||||
| `datetime.timedelta` | `Duration`, `chrono::Duration`[^5] | `&PyDelta` |
|
||||
| `datetime.datetime` | `SystemTime`, `chrono::DateTime<Tz>`[^5], `chrono::NaiveDateTime`[^5] | `PyDateTime` |
|
||||
| `datetime.date` | `chrono::NaiveDate`[^5] | `PyDate` |
|
||||
| `datetime.time` | `chrono::NaiveTime`[^5] | `PyTime` |
|
||||
| `datetime.tzinfo` | `chrono::FixedOffset`[^5], `chrono::Utc`[^5], `chrono_tz::TimeZone`[^6] | `PyTzInfo` |
|
||||
| `datetime.timedelta` | `Duration`, `chrono::Duration`[^5] | `PyDelta` |
|
||||
| `decimal.Decimal` | `rust_decimal::Decimal`[^7] | - |
|
||||
| `ipaddress.IPv4Address` | `std::net::IpAddr`, `std::net::IpV4Addr` | - |
|
||||
| `ipaddress.IPv6Address` | `std::net::IpAddr`, `std::net::IpV6Addr` | - |
|
||||
| `os.PathLike ` | `PathBuf`, `Path` | `&PyString`, `&PyUnicode` |
|
||||
| `pathlib.Path` | `PathBuf`, `Path` | `&PyString`, `&PyUnicode` |
|
||||
| `os.PathLike ` | `PathBuf`, `Path` | `PyString`, `PyUnicode` |
|
||||
| `pathlib.Path` | `PathBuf`, `Path` | `PyString`, `PyUnicode` |
|
||||
| `typing.Optional[T]` | `Option<T>` | - |
|
||||
| `typing.Sequence[T]` | `Vec<T>` | `&PySequence` |
|
||||
| `typing.Sequence[T]` | `Vec<T>` | `PySequence` |
|
||||
| `typing.Mapping[K, V]` | `HashMap<K, V>`, `BTreeMap<K, V>`, `hashbrown::HashMap<K, V>`[^3], `indexmap::IndexMap<K, V>`[^4] | `&PyMapping` |
|
||||
| `typing.Iterator[Any]` | - | `&PyIterator` |
|
||||
| `typing.Iterator[Any]` | - | `PyIterator` |
|
||||
| `typing.Union[...]` | See [`#[derive(FromPyObject)]`](traits.md#deriving-frompyobject-for-enums) | - |
|
||||
|
||||
There are also a few special types related to the GIL and Rust-defined `#[pyclass]`es which may come in useful:
|
||||
It is also worth remembering the following special types:
|
||||
|
||||
| What | Description |
|
||||
| ------------- | ------------------------------------- |
|
||||
| `Python` | A GIL token, used to pass to PyO3 constructors to prove ownership of the GIL |
|
||||
| `Py<T>` | A Python object isolated from the GIL lifetime. This can be sent to other threads. |
|
||||
| `PyObject` | An alias for `Py<PyAny>` |
|
||||
| `&PyCell<T>` | A `#[pyclass]` value owned by Python. |
|
||||
| `PyRef<T>` | A `#[pyclass]` borrowed immutably. |
|
||||
| `PyRefMut<T>` | A `#[pyclass]` borrowed mutably. |
|
||||
| What | Description |
|
||||
| ---------------- | ------------------------------------- |
|
||||
| `Python<'py>` | A GIL token, used to pass to PyO3 constructors to prove ownership of the GIL. |
|
||||
| `Bound<'py, T>` | A Python object connected to the GIL lifetime. This provides access to most of PyO3's APIs. |
|
||||
| `Py<T>` | A Python object isolated from the GIL lifetime. This can be sent to other threads. |
|
||||
| `PyObject` | An alias for `Py<PyAny>` |
|
||||
| `PyRef<T>` | A `#[pyclass]` borrowed immutably. |
|
||||
| `PyRefMut<T>` | A `#[pyclass]` borrowed mutably. |
|
||||
|
||||
For more detail on accepting `#[pyclass]` values as function arguments, see [the section of this guide on Python Classes](../class.md).
|
||||
|
||||
|
|
@ -72,9 +72,9 @@ For most PyO3 usage the conversion cost is worth paying to get these benefits. A
|
|||
|
||||
### Returning Rust values to Python
|
||||
|
||||
When returning values from functions callable from Python, Python-native types (`&PyAny`, `&PyDict` etc.) can be used with zero cost.
|
||||
When returning values from functions callable from Python, [PyO3's smart pointers](../types.md#pyo3s-smart-pointers) (`Py<T>`, `Bound<'py, T>`, and `Borrowed<'a, 'py, T>`) can be used with zero cost.
|
||||
|
||||
Because these types are references, in some situations the Rust compiler may ask for lifetime annotations. If this is the case, you should use `Py<PyAny>`, `Py<PyDict>` etc. instead - which are also zero-cost. For all of these Python-native types `T`, `Py<T>` can be created from `T` with an `.into()` conversion.
|
||||
Because `Bound<'py, T>` and `Borrowed<'a, 'py, T>` have lifetime parameters, the Rust compiler may ask for lifetime annotations to be added to your function. See the [section of the guide dedicated to this](../types.md#function-argument-lifetimes).
|
||||
|
||||
If your function is fallible, it should return `PyResult<T>` or `Result<T, E>` where `E` implements `From<E> for PyErr`. This will raise a `Python` exception if the `Err` variant is returned.
|
||||
|
||||
|
|
@ -95,7 +95,8 @@ Finally, the following Rust types are also able to convert to Python as return v
|
|||
| `BTreeMap<K, V>` | `Dict[K, V]` |
|
||||
| `HashSet<T>` | `Set[T]` |
|
||||
| `BTreeSet<T>` | `Set[T]` |
|
||||
| `&PyCell<T: PyClass>` | `T` |
|
||||
| `Py<T>` | `T` |
|
||||
| `Bound<T>` | `T` |
|
||||
| `PyRef<T: PyClass>` | `T` |
|
||||
| `PyRefMut<T: PyClass>` | `T` |
|
||||
|
||||
|
|
|
|||
|
|
@ -484,7 +484,7 @@ If the input is neither a string nor an integer, the error message will be:
|
|||
- `pyo3(from_py_with = "...")`
|
||||
- apply a custom function to convert the field from Python the desired Rust type.
|
||||
- the argument must be the name of the function as a string.
|
||||
- the function signature must be `fn(&PyAny) -> PyResult<T>` where `T` is the Rust type of the argument.
|
||||
- the function signature must be `fn(&Bound<PyAny>) -> PyResult<T>` where `T` is the Rust type of the argument.
|
||||
|
||||
### `IntoPy<T>`
|
||||
|
||||
|
|
|
|||
|
|
@ -120,7 +120,7 @@ Export an async function that makes use of `async-std`:
|
|||
use pyo3::{prelude::*, wrap_pyfunction};
|
||||
|
||||
#[pyfunction]
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> {
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&Bound<'_, PyAny>>> {
|
||||
pyo3_asyncio::async_std::future_into_py(py, async {
|
||||
async_std::task::sleep(std::time::Duration::from_secs(1)).await;
|
||||
Ok(Python::with_gil(|py| py.None()))
|
||||
|
|
@ -143,7 +143,7 @@ If you want to use `tokio` instead, here's what your module should look like:
|
|||
use pyo3::{prelude::*, wrap_pyfunction};
|
||||
|
||||
#[pyfunction]
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> {
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&Bound<'_, PyAny>>> {
|
||||
pyo3_asyncio::tokio::future_into_py(py, async {
|
||||
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
|
||||
Ok(Python::with_gil(|py| py.None()))
|
||||
|
|
@ -233,7 +233,7 @@ a coroutine argument:
|
|||
|
||||
```rust
|
||||
#[pyfunction]
|
||||
fn await_coro(coro: &PyAny) -> PyResult<()> {
|
||||
fn await_coro(coro: &Bound<'_, PyAny>>) -> PyResult<()> {
|
||||
// convert the coroutine into a Rust future using the
|
||||
// async_std runtime
|
||||
let f = pyo3_asyncio::async_std::into_future(coro)?;
|
||||
|
|
@ -261,7 +261,7 @@ If for you wanted to pass a callable function to the `#[pyfunction]` instead, (i
|
|||
|
||||
```rust
|
||||
#[pyfunction]
|
||||
fn await_coro(callable: &PyAny) -> PyResult<()> {
|
||||
fn await_coro(callable: &Bound<'_, PyAny>>) -> PyResult<()> {
|
||||
// get the coroutine by calling the callable
|
||||
let coro = callable.call0()?;
|
||||
|
||||
|
|
@ -317,7 +317,7 @@ async fn rust_sleep() {
|
|||
}
|
||||
|
||||
#[pyfunction]
|
||||
fn call_rust_sleep(py: Python<'_>) -> PyResult<&PyAny> {
|
||||
fn call_rust_sleep(py: Python<'_>) -> PyResult<&Bound<'_, PyAny>>> {
|
||||
pyo3_asyncio::async_std::future_into_py(py, async move {
|
||||
rust_sleep().await;
|
||||
Ok(Python::with_gil(|py| py.None()))
|
||||
|
|
@ -467,7 +467,7 @@ tokio = "1.4"
|
|||
use pyo3::{prelude::*, wrap_pyfunction};
|
||||
|
||||
#[pyfunction]
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> {
|
||||
fn rust_sleep(py: Python<'_>) -> PyResult<&Bound<'_, PyAny>>> {
|
||||
pyo3_asyncio::tokio::future_into_py(py, async {
|
||||
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
|
||||
Ok(Python::with_gil(|py| py.None()))
|
||||
|
|
|
|||
|
|
@ -111,7 +111,7 @@ mod io {
|
|||
pyo3::import_exception!(io, UnsupportedOperation);
|
||||
}
|
||||
|
||||
fn tell(file: &PyAny) -> PyResult<u64> {
|
||||
fn tell(file: &Bound<'_, PyAny>) -> PyResult<u64> {
|
||||
match file.call_method0("tell") {
|
||||
Err(_) => Err(io::UnsupportedOperation::new_err("not supported: tell")),
|
||||
Ok(x) => x.extract::<u64>(),
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
# Calling Python functions
|
||||
|
|
@ -87,7 +87,9 @@ The `#[pyo3]` attribute can be used to modify properties of the generated Python
|
|||
|
||||
#[pyfunction]
|
||||
#[pyo3(pass_module)]
|
||||
fn pyfunction_with_module<'py>(module: &Bound<'py, PyModule>) -> PyResult<Bound<'py, PyString>> {
|
||||
fn pyfunction_with_module<'py>(
|
||||
module: &Bound<'py, PyModule>,
|
||||
) -> PyResult<Bound<'py, PyString>> {
|
||||
module.name()
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,22 @@
|
|||
|
||||
Welcome to the PyO3 user guide! This book is a companion to [PyO3's API docs](https://docs.rs/pyo3). It contains examples and documentation to explain all of PyO3's use cases in detail.
|
||||
|
||||
The rough order of material in this user guide is as follows:
|
||||
1. Getting started
|
||||
2. Wrapping Rust code for use from Python
|
||||
3. How to use Python code from Rust
|
||||
4. Remaining topics which go into advanced concepts in detail
|
||||
|
||||
Please choose from the chapters on the left to jump to individual topics, or continue below to start with PyO3's README.
|
||||
|
||||
<div class="warning">
|
||||
⚠️ Warning: API update in progress 🛠️
|
||||
|
||||
PyO3 0.21 has introduced a significant new API, termed the "Bound" API after the new smart pointer `Bound<T>`.
|
||||
|
||||
While most of this guide has been updated to the new API, it is possible some stray references to the older "GIL Refs" API such as `&PyAny` remain.
|
||||
</div>
|
||||
|
||||
<hr style="opacity:0.2">
|
||||
|
||||
{{#include ../../README.md}}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,15 @@
|
|||
# Memory management
|
||||
|
||||
<div class="warning">
|
||||
⚠️ Warning: API update in progress 🛠️
|
||||
|
||||
PyO3 0.21 has introduced a significant new API, termed the "Bound" API after the new smart pointer `Bound<T>`.
|
||||
|
||||
This section on memory management is heavily weighted towards the now-deprecated "GIL Refs" API, which suffered from the drawbacks detailed here as well as CPU overheads.
|
||||
|
||||
See [the smart pointer types](./types.md#pyo3s-smart-pointers) for description on the new, simplified, memory model of the Bound API, which is built as a thin wrapper on Python reference counting.
|
||||
</div>
|
||||
|
||||
Rust and Python have very different notions of memory management. Rust has
|
||||
a strict memory model with concepts of ownership, borrowing, and lifetimes,
|
||||
where memory is freed at predictable points in program execution. Python has
|
||||
|
|
@ -10,12 +20,12 @@ Memory in Python is freed eventually by the garbage collector, but not usually
|
|||
in a predictable way.
|
||||
|
||||
PyO3 bridges the Rust and Python memory models with two different strategies for
|
||||
accessing memory allocated on Python's heap from inside Rust. These are
|
||||
GIL-bound, or "owned" references, and GIL-independent `Py<Any>` smart pointers.
|
||||
accessing memory allocated on Python's heap from inside Rust. These are
|
||||
GIL Refs such as `&'py PyAny`, and GIL-independent `Py<Any>` smart pointers.
|
||||
|
||||
## GIL-bound memory
|
||||
|
||||
PyO3's GIL-bound, "owned references" (`&PyAny` etc.) make PyO3 more ergonomic to
|
||||
PyO3's GIL Refs such as `&'py PyAny` make PyO3 more ergonomic to
|
||||
use by ensuring that their lifetime can never be longer than the duration the
|
||||
Python GIL is held. This means that most of PyO3's API can assume the GIL is
|
||||
held. (If PyO3 could not assume this, every PyO3 API would need to take a
|
||||
|
|
@ -27,7 +37,10 @@ very simple and easy-to-understand programs like this:
|
|||
# use pyo3::types::PyString;
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
let hello = py.eval_bound("\"Hello World!\"", None, None)?.downcast_into::<PyString>()?;
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello = py
|
||||
.eval("\"Hello World!\"", None, None)?
|
||||
.downcast::<PyString>()?;
|
||||
println!("Python says: {}", hello);
|
||||
Ok(())
|
||||
})?;
|
||||
|
|
@ -48,7 +61,10 @@ of the time we don't have to think about this, but consider the following:
|
|||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
for _ in 0..10 {
|
||||
let hello = py.eval_bound("\"Hello World!\"", None, None)?.downcast_into::<PyString>()?;
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello = py
|
||||
.eval("\"Hello World!\"", None, None)?
|
||||
.downcast::<PyString>()?;
|
||||
println!("Python says: {}", hello);
|
||||
}
|
||||
// There are 10 copies of `hello` on Python's heap here.
|
||||
|
|
@ -76,7 +92,10 @@ is to acquire and release the GIL with each iteration of the loop.
|
|||
# fn main() -> PyResult<()> {
|
||||
for _ in 0..10 {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
let hello = py.eval_bound("\"Hello World!\"", None, None)?.downcast_into::<PyString>()?;
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello = py
|
||||
.eval("\"Hello World!\"", None, None)?
|
||||
.downcast::<PyString>()?;
|
||||
println!("Python says: {}", hello);
|
||||
Ok(())
|
||||
})?; // only one copy of `hello` at a time
|
||||
|
|
@ -97,7 +116,10 @@ Python::with_gil(|py| -> PyResult<()> {
|
|||
for _ in 0..10 {
|
||||
let pool = unsafe { py.new_pool() };
|
||||
let py = pool.python();
|
||||
let hello = py.eval_bound("\"Hello World!\"", None, None)?.downcast_into::<PyString>()?;
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello = py
|
||||
.eval("\"Hello World!\"", None, None)?
|
||||
.downcast::<PyString>()?;
|
||||
println!("Python says: {}", hello);
|
||||
}
|
||||
Ok(())
|
||||
|
|
@ -144,8 +166,12 @@ reference count reaches zero? It depends whether or not we are holding the GIL.
|
|||
# use pyo3::types::PyString;
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
let hello: Py<PyString> = py.eval_bound("\"Hello World!\"", None, None)?.extract()?;
|
||||
println!("Python says: {}", hello.bind(py));
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello: Py<PyString> = py.eval("\"Hello World!\"", None, None)?.extract()?;
|
||||
#[allow(deprecated)] // as_ref is part of the GIL Refs API
|
||||
{
|
||||
println!("Python says: {}", hello.as_ref(py));
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
# Ok(())
|
||||
|
|
@ -166,7 +192,8 @@ we are *not* holding the GIL?
|
|||
# use pyo3::types::PyString;
|
||||
# fn main() -> PyResult<()> {
|
||||
let hello: Py<PyString> = Python::with_gil(|py| {
|
||||
py.eval_bound("\"Hello World!\"", None, None)?.extract()
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
py.eval("\"Hello World!\"", None, None)?.extract()
|
||||
})?;
|
||||
// Do some stuff...
|
||||
// Now sometime later in the program we want to access `hello`.
|
||||
|
|
@ -198,12 +225,16 @@ We can avoid the delay in releasing memory if we are careful to drop the
|
|||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyString;
|
||||
# fn main() -> PyResult<()> {
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello: Py<PyString> =
|
||||
Python::with_gil(|py| py.eval_bound("\"Hello World!\"", None, None)?.extract())?;
|
||||
Python::with_gil(|py| py.eval("\"Hello World!\"", None, None)?.extract())?;
|
||||
// Do some stuff...
|
||||
// Now sometime later in the program:
|
||||
Python::with_gil(|py| {
|
||||
println!("Python says: {}", hello.bind(py));
|
||||
#[allow(deprecated)] // as_ref is part of the GIL Refs API
|
||||
{
|
||||
println!("Python says: {}", hello.as_ref(py));
|
||||
}
|
||||
drop(hello); // Memory released here.
|
||||
});
|
||||
# Ok(())
|
||||
|
|
@ -220,12 +251,16 @@ until the GIL is dropped.
|
|||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyString;
|
||||
# fn main() -> PyResult<()> {
|
||||
#[allow(deprecated)] // py.eval() is part of the GIL Refs API
|
||||
let hello: Py<PyString> =
|
||||
Python::with_gil(|py| py.eval_bound("\"Hello World!\"", None, None)?.extract())?;
|
||||
Python::with_gil(|py| py.eval("\"Hello World!\"", None, None)?.extract())?;
|
||||
// Do some stuff...
|
||||
// Now sometime later in the program:
|
||||
Python::with_gil(|py| {
|
||||
println!("Python says: {}", hello.into_bound(py));
|
||||
#[allow(deprecated)] // into_ref is part of the GIL Refs API
|
||||
{
|
||||
println!("Python says: {}", hello.into_ref(py));
|
||||
}
|
||||
// Memory not released yet.
|
||||
// Do more stuff...
|
||||
// Memory released here at end of `with_gil()` closure.
|
||||
|
|
|
|||
|
|
@ -190,7 +190,9 @@ struct PyClassAsyncIter {
|
|||
impl PyClassAsyncIter {
|
||||
fn __anext__(&mut self) -> PyClassAwaitable {
|
||||
self.number += 1;
|
||||
PyClassAwaitable { number: self.number }
|
||||
PyClassAwaitable {
|
||||
number: self.number,
|
||||
}
|
||||
}
|
||||
|
||||
fn __aiter__(slf: Py<Self>) -> Py<Self> {
|
||||
|
|
@ -312,7 +314,10 @@ Python::with_gil(|py| {
|
|||
// `b` is not in the dictionary
|
||||
assert!(dict.get_item("b").is_none());
|
||||
// `dict` is not hashable, so this fails with a `TypeError`
|
||||
assert!(dict.get_item_with_error(dict).unwrap_err().is_instance_of::<PyTypeError>(py));
|
||||
assert!(dict
|
||||
.get_item_with_error(dict)
|
||||
.unwrap_err()
|
||||
.is_instance_of::<PyTypeError>(py));
|
||||
});
|
||||
# }
|
||||
```
|
||||
|
|
@ -333,7 +338,10 @@ Python::with_gil(|py| -> PyResult<()> {
|
|||
// `b` is not in the dictionary
|
||||
assert!(dict.get_item("b")?.is_none());
|
||||
// `dict` is not hashable, so this fails with a `TypeError`
|
||||
assert!(dict.get_item(dict).unwrap_err().is_instance_of::<PyTypeError>(py));
|
||||
assert!(dict
|
||||
.get_item(dict)
|
||||
.unwrap_err()
|
||||
.is_instance_of::<PyTypeError>(py));
|
||||
|
||||
Ok(())
|
||||
});
|
||||
|
|
@ -495,7 +503,7 @@ drop(first);
|
|||
drop(second);
|
||||
```
|
||||
|
||||
The replacement is [`Python::with_gil`]() which is more cumbersome but enforces the proper nesting by design, e.g.
|
||||
The replacement is [`Python::with_gil`](https://docs.rs/pyo3/0.18.3/pyo3/marker/struct.Python.html#method.with_gil) which is more cumbersome but enforces the proper nesting by design, e.g.
|
||||
|
||||
```rust
|
||||
# #![allow(dead_code)]
|
||||
|
|
|
|||
|
|
@ -4,26 +4,26 @@ To achieve the best possible performance, it is useful to be aware of several tr
|
|||
|
||||
## `extract` versus `downcast`
|
||||
|
||||
Pythonic API implemented using PyO3 are often polymorphic, i.e. they will accept `&PyAny` and try to turn this into multiple more concrete types to which the requested operation is applied. This often leads to chains of calls to `extract`, e.g.
|
||||
Pythonic API implemented using PyO3 are often polymorphic, i.e. they will accept `&Bound<'_, PyAny>` and try to turn this into multiple more concrete types to which the requested operation is applied. This often leads to chains of calls to `extract`, e.g.
|
||||
|
||||
```rust
|
||||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::{exceptions::PyTypeError, types::PyList};
|
||||
|
||||
fn frobnicate_list(list: &PyList) -> PyResult<&PyAny> {
|
||||
fn frobnicate_list<'py>(list: &Bound<'_, PyList>) -> PyResult<Bound<'py, PyAny>> {
|
||||
todo!()
|
||||
}
|
||||
|
||||
fn frobnicate_vec(vec: Vec<&PyAny>) -> PyResult<&PyAny> {
|
||||
fn frobnicate_vec<'py>(vec: Vec<Bound<'py, PyAny>>) -> PyResult<Bound<'py, PyAny>> {
|
||||
todo!()
|
||||
}
|
||||
|
||||
#[pyfunction]
|
||||
fn frobnicate(value: &PyAny) -> PyResult<&PyAny> {
|
||||
if let Ok(list) = value.extract::<&PyList>() {
|
||||
frobnicate_list(list)
|
||||
} else if let Ok(vec) = value.extract::<Vec<&PyAny>>() {
|
||||
fn frobnicate<'py>(value: &Bound<'py, PyAny>) -> PyResult<Bound<'py, PyAny>> {
|
||||
if let Ok(list) = value.extract::<Bound<'_, PyList>>() {
|
||||
frobnicate_list(&list)
|
||||
} else if let Ok(vec) = value.extract::<Vec<Bound<'_, PyAny>>>() {
|
||||
frobnicate_vec(vec)
|
||||
} else {
|
||||
Err(PyTypeError::new_err("Cannot frobnicate that type."))
|
||||
|
|
@ -37,15 +37,15 @@ This suboptimal as the `FromPyObject<T>` trait requires `extract` to have a `Res
|
|||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::{exceptions::PyTypeError, types::PyList};
|
||||
# fn frobnicate_list(list: &PyList) -> PyResult<&PyAny> { todo!() }
|
||||
# fn frobnicate_vec(vec: Vec<&PyAny>) -> PyResult<&PyAny> { todo!() }
|
||||
# fn frobnicate_list<'py>(list: &Bound<'_, PyList>) -> PyResult<Bound<'py, PyAny>> { todo!() }
|
||||
# fn frobnicate_vec<'py>(vec: Vec<Bound<'py, PyAny>>) -> PyResult<Bound<'py, PyAny>> { todo!() }
|
||||
#
|
||||
#[pyfunction]
|
||||
fn frobnicate(value: &PyAny) -> PyResult<&PyAny> {
|
||||
fn frobnicate<'py>(value: &Bound<'py, PyAny>) -> PyResult<Bound<'py, PyAny>> {
|
||||
// Use `downcast` instead of `extract` as turning `PyDowncastError` into `PyErr` is quite costly.
|
||||
if let Ok(list) = value.downcast::<PyList>() {
|
||||
frobnicate_list(list)
|
||||
} else if let Ok(vec) = value.extract::<Vec<&PyAny>>() {
|
||||
} else if let Ok(vec) = value.extract::<Vec<Bound<'_, PyAny>>>() {
|
||||
frobnicate_vec(vec)
|
||||
} else {
|
||||
Err(PyTypeError::new_err("Cannot frobnicate that type."))
|
||||
|
|
@ -53,9 +53,9 @@ fn frobnicate(value: &PyAny) -> PyResult<&PyAny> {
|
|||
}
|
||||
```
|
||||
|
||||
## Access to GIL-bound reference implies access to GIL token
|
||||
## Access to Bound implies access to GIL token
|
||||
|
||||
Calling `Python::with_gil` is effectively a no-op when the GIL is already held, but checking that this is the case still has a cost. If an existing GIL token can not be accessed, for example when implementing a pre-existing trait, but a GIL-bound reference is available, this cost can be avoided by exploiting that access to GIL-bound reference gives zero-cost access to a GIL token via `PyAny::py`.
|
||||
Calling `Python::with_gil` is effectively a no-op when the GIL is already held, but checking that this is the case still has a cost. If an existing GIL token can not be accessed, for example when implementing a pre-existing trait, but a GIL-bound reference is available, this cost can be avoided by exploiting that access to GIL-bound reference gives zero-cost access to a GIL token via `Bound::py`.
|
||||
|
||||
For example, instead of writing
|
||||
|
||||
|
|
@ -66,34 +66,32 @@ For example, instead of writing
|
|||
|
||||
struct Foo(Py<PyList>);
|
||||
|
||||
struct FooRef<'a>(&'a PyList);
|
||||
struct FooBound<'py>(Bound<'py, PyList>);
|
||||
|
||||
impl PartialEq<Foo> for FooRef<'_> {
|
||||
impl PartialEq<Foo> for FooBound<'_> {
|
||||
fn eq(&self, other: &Foo) -> bool {
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let len = other.0.as_ref(py).len();
|
||||
let len = other.0.bind(py).len();
|
||||
self.0.len() == len
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
use more efficient
|
||||
use the more efficient
|
||||
|
||||
```rust
|
||||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
# struct Foo(Py<PyList>);
|
||||
# struct FooRef<'a>(&'a PyList);
|
||||
# struct FooBound<'py>(Bound<'py, PyList>);
|
||||
#
|
||||
impl PartialEq<Foo> for FooRef<'_> {
|
||||
impl PartialEq<Foo> for FooBound<'_> {
|
||||
fn eq(&self, other: &Foo) -> bool {
|
||||
// Access to `&'a PyAny` implies access to `Python<'a>`.
|
||||
// Access to `&Bound<'py, PyAny>` implies access to `Python<'py>`.
|
||||
let py = self.0.py();
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let len = other.0.as_ref(py).len();
|
||||
let len = other.0.bind(py).len();
|
||||
self.0.len() == len
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,511 +1,46 @@
|
|||
# Calling Python in Rust code
|
||||
|
||||
This chapter of the guide documents some ways to interact with Python code from Rust:
|
||||
This chapter of the guide documents some ways to interact with Python code from Rust.
|
||||
|
||||
Below is an introduction to the `'py` lifetime and some general remarks about how PyO3's API reasons about Python code.
|
||||
|
||||
The subchapters also cover the following topics:
|
||||
- Python object types available in PyO3's API
|
||||
- How to work with Python exceptions
|
||||
- How to call Python functions
|
||||
- How to execute existing Python code
|
||||
|
||||
## Calling Python functions
|
||||
## The `'py` lifetime
|
||||
|
||||
Any Python-native object reference (such as `&PyAny`, `&PyList`, or `&PyCell<MyClass>`) can be used to call Python functions.
|
||||
To safely interact with the Python interpreter a Rust thread must have a corresponding Python thread state and hold the [Global Interpreter Lock (GIL)](#the-global-interpreter-lock). PyO3 has a `Python<'py>` token that is used to prove that these conditions
|
||||
are met. Its lifetime `'py` is a central part of PyO3's API.
|
||||
|
||||
PyO3 offers two APIs to make function calls:
|
||||
The `Python<'py>` token serves three purposes:
|
||||
|
||||
* [`call`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call) - call any callable Python object.
|
||||
* [`call_method`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method) - call a method on the Python object.
|
||||
* It provides global APIs for the Python interpreter, such as [`py.eval_bound()`][eval] and [`py.import_bound()`][import].
|
||||
* It can be passed to functions that require a proof of holding the GIL, such as [`Py::clone_ref`][clone_ref].
|
||||
* Its lifetime `'py` is used to bind many of PyO3's types to the Python interpreter, such as [`Bound<'py, T>`][Bound].
|
||||
|
||||
Both of these APIs take `args` and `kwargs` arguments (for positional and keyword arguments respectively). There are variants for less complex calls:
|
||||
PyO3's types that are bound to the `'py` lifetime, for example `Bound<'py, T>`, all contain a `Python<'py>` token. This means they have full access to the Python interpreter and offer a complete API for interacting with Python objects.
|
||||
|
||||
* [`call1`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call1) and [`call_method1`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method1) to call only with positional `args`.
|
||||
* [`call0`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call0) and [`call_method0`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method0) to call with no arguments.
|
||||
Consult [PyO3's API documentation][obtaining-py] to learn how to acquire one of these tokens.
|
||||
|
||||
For convenience the [`Py<T>`](types.md#pyt-and-pyobject) smart pointer also exposes these same six API methods, but needs a `Python` token as an additional first argument to prove the GIL is held.
|
||||
### The Global Interpreter Lock
|
||||
|
||||
The example below calls a Python function behind a `PyObject` (aka `Py<PyAny>`) reference:
|
||||
Concurrent programming in Python is aided by the Global Interpreter Lock (GIL), which ensures that only one Python thread can use the Python interpreter and its API at the same time. This allows it to be used to synchronize code. See the [`pyo3::sync`] module for synchronization tools PyO3 offers that are based on the GIL's guarantees.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyTuple;
|
||||
Non-Python operations (system calls and native Rust code) can unlock the GIL. See [the section on parallelism](parallelism.md) for how to do that using PyO3's API.
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let arg1 = "arg1";
|
||||
let arg2 = "arg2";
|
||||
let arg3 = "arg3";
|
||||
## Python's memory model
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let fun: Py<PyAny> = PyModule::from_code_bound(
|
||||
py,
|
||||
"def example(*args, **kwargs):
|
||||
if args != ():
|
||||
print('called with args', args)
|
||||
if kwargs != {}:
|
||||
print('called with kwargs', kwargs)
|
||||
if args == () and kwargs == {}:
|
||||
print('called with no arguments')",
|
||||
"",
|
||||
"",
|
||||
)?
|
||||
.getattr("example")?
|
||||
.into();
|
||||
Python's memory model differs from Rust's memory model in two key ways:
|
||||
- There is no concept of ownership; all Python objects are shared and usually implemented via reference counting
|
||||
- There is no concept of exclusive (`&mut`) references; any reference can mutate a Python object
|
||||
|
||||
// call object without any arguments
|
||||
fun.call0(py)?;
|
||||
PyO3's API reflects this by providing [smart pointer][smart-pointers] types, `Py<T>`, `Bound<'py, T>`, and (the very rarely used) `Borrowed<'a, 'py, T>`. These smart pointers all use Python reference counting. See the [subchapter on types](./types.md) for more detail on these types.
|
||||
|
||||
// call object with PyTuple
|
||||
let args = PyTuple::new_bound(py, &[arg1, arg2, arg3]);
|
||||
fun.call1(py, args)?;
|
||||
Because of the lack of exclusive `&mut` references, PyO3's APIs for Python objects, for example [`PyListMethods::append`], use shared references. This is safe because Python objects have internal mechanisms to prevent data races (as of time of writing, the Python GIL).
|
||||
|
||||
// pass arguments as rust tuple
|
||||
let args = (arg1, arg2, arg3);
|
||||
fun.call1(py, args)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### Creating keyword arguments
|
||||
|
||||
For the `call` and `call_method` APIs, `kwargs` can be `None` or `Some(&PyDict)`. You can use the [`IntoPyDict`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.IntoPyDict.html) trait to convert other dict-like containers, e.g. `HashMap` or `BTreeMap`, as well as tuples with up to 10 elements and `Vec`s where each element is a two-element tuple.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::IntoPyDict;
|
||||
use std::collections::HashMap;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let key1 = "key1";
|
||||
let val1 = 1;
|
||||
let key2 = "key2";
|
||||
let val2 = 2;
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let fun: Py<PyAny> = PyModule::from_code_bound(
|
||||
py,
|
||||
"def example(*args, **kwargs):
|
||||
if args != ():
|
||||
print('called with args', args)
|
||||
if kwargs != {}:
|
||||
print('called with kwargs', kwargs)
|
||||
if args == () and kwargs == {}:
|
||||
print('called with no arguments')",
|
||||
"",
|
||||
"",
|
||||
)?
|
||||
.getattr("example")?
|
||||
.into();
|
||||
|
||||
// call object with PyDict
|
||||
let kwargs = [(key1, val1)].into_py_dict_bound(py);
|
||||
fun.call_bound(py, (), Some(&kwargs))?;
|
||||
|
||||
// pass arguments as Vec
|
||||
let kwargs = vec![(key1, val1), (key2, val2)];
|
||||
fun.call_bound(py, (), Some(&kwargs.into_py_dict_bound(py)))?;
|
||||
|
||||
// pass arguments as HashMap
|
||||
let mut kwargs = HashMap::<&str, i32>::new();
|
||||
kwargs.insert(key1, 1);
|
||||
fun.call_bound(py, (), Some(&kwargs.into_py_dict_bound(py)))?;
|
||||
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
<div class="warning">
|
||||
|
||||
During PyO3's [migration from "GIL Refs" to the `Bound<T>` smart pointer](./migration.md#migrating-from-the-gil-refs-api-to-boundt), [`Py<T>::call`]({{#PYO3_DOCS_URL}}/pyo3/struct.Py.html#method.call) is temporarily named `call_bound` (and `call_method` is temporarily `call_method_bound`).
|
||||
|
||||
(This temporary naming is only the case for the `Py<T>` smart pointer. The methods on the `&PyAny` GIL Ref such as `call` have not been given replacements, and the methods on the `Bound<PyAny>` smart pointer such as [`Bound<PyAny>::call`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call) already use follow the newest API conventions.)
|
||||
|
||||
</div>
|
||||
|
||||
## Executing existing Python code
|
||||
|
||||
If you already have some existing Python code that you need to execute from Rust, the following FAQs can help you select the right PyO3 functionality for your situation:
|
||||
|
||||
### Want to access Python APIs? Then use `PyModule::import`.
|
||||
|
||||
[`Pymodule::import`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.import) can
|
||||
be used to get handle to a Python module from Rust. You can use this to import and use any Python
|
||||
module available in your environment.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
let builtins = PyModule::import_bound(py, "builtins")?;
|
||||
let total: i32 = builtins
|
||||
.getattr("sum")?
|
||||
.call1((vec![1, 2, 3],))?
|
||||
.extract()?;
|
||||
assert_eq!(total, 6);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### Want to run just an expression? Then use `eval`.
|
||||
|
||||
[`Python::eval`]({{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.eval) is
|
||||
a method to execute a [Python expression](https://docs.python.org/3.7/reference/expressions.html)
|
||||
and return the evaluated value as a `&PyAny` object.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
# fn main() -> Result<(), ()> {
|
||||
Python::with_gil(|py| {
|
||||
let result = py
|
||||
.eval_bound("[i * 10 for i in range(5)]", None, None)
|
||||
.map_err(|e| {
|
||||
e.print_and_set_sys_last_vars(py);
|
||||
})?;
|
||||
let res: Vec<i64> = result.extract().unwrap();
|
||||
assert_eq!(res, vec![0, 10, 20, 30, 40]);
|
||||
Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
### Want to run statements? Then use `run`.
|
||||
|
||||
[`Python::run`] is a method to execute one or more
|
||||
[Python statements](https://docs.python.org/3.7/reference/simple_stmts.html).
|
||||
This method returns nothing (like any Python statement), but you can get
|
||||
access to manipulated objects via the `locals` dict.
|
||||
|
||||
You can also use the [`py_run!`] macro, which is a shorthand for [`Python::run`].
|
||||
Since [`py_run!`] panics on exceptions, we recommend you use this macro only for
|
||||
quickly testing your Python extensions.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::py_run;
|
||||
|
||||
# fn main() {
|
||||
#[pyclass]
|
||||
struct UserData {
|
||||
id: u32,
|
||||
name: String,
|
||||
}
|
||||
|
||||
#[pymethods]
|
||||
impl UserData {
|
||||
fn as_tuple(&self) -> (u32, String) {
|
||||
(self.id, self.name.clone())
|
||||
}
|
||||
|
||||
fn __repr__(&self) -> PyResult<String> {
|
||||
Ok(format!("User {}(id: {})", self.name, self.id))
|
||||
}
|
||||
}
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let userdata = UserData {
|
||||
id: 34,
|
||||
name: "Yu".to_string(),
|
||||
};
|
||||
let userdata = Py::new(py, userdata).unwrap();
|
||||
let userdata_as_tuple = (34, "Yu");
|
||||
py_run!(py, userdata userdata_as_tuple, r#"
|
||||
assert repr(userdata) == "User Yu(id: 34)"
|
||||
assert userdata.as_tuple() == userdata_as_tuple
|
||||
"#);
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
## You have a Python file or code snippet? Then use `PyModule::from_code`.
|
||||
|
||||
[`PyModule::from_code`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.from_code)
|
||||
can be used to generate a Python module which can then be used just as if it was imported with
|
||||
`PyModule::import`.
|
||||
|
||||
**Warning**: This will compile and execute code. **Never** pass untrusted code
|
||||
to this function!
|
||||
|
||||
```rust
|
||||
use pyo3::{
|
||||
prelude::*,
|
||||
types::IntoPyDict,
|
||||
};
|
||||
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
let activators = PyModule::from_code_bound(
|
||||
py,
|
||||
r#"
|
||||
def relu(x):
|
||||
"""see https://en.wikipedia.org/wiki/Rectifier_(neural_networks)"""
|
||||
return max(0.0, x)
|
||||
|
||||
def leaky_relu(x, slope=0.01):
|
||||
return x if x >= 0 else x * slope
|
||||
"#,
|
||||
"activators.py",
|
||||
"activators",
|
||||
)?;
|
||||
|
||||
let relu_result: f64 = activators.getattr("relu")?.call1((-1.0,))?.extract()?;
|
||||
assert_eq!(relu_result, 0.0);
|
||||
|
||||
let kwargs = [("slope", 0.2)].into_py_dict_bound(py);
|
||||
let lrelu_result: f64 = activators
|
||||
.getattr("leaky_relu")?
|
||||
.call((-1.0,), Some(&kwargs))?
|
||||
.extract()?;
|
||||
assert_eq!(lrelu_result, -0.2);
|
||||
# Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
### Want to embed Python in Rust with additional modules?
|
||||
|
||||
Python maintains the `sys.modules` dict as a cache of all imported modules.
|
||||
An import in Python will first attempt to lookup the module from this dict,
|
||||
and if not present will use various strategies to attempt to locate and load
|
||||
the module.
|
||||
|
||||
The [`append_to_inittab`]({{#PYO3_DOCS_URL}}/pyo3/macro.append_to_inittab.html)
|
||||
macro can be used to add additional `#[pymodule]` modules to an embedded
|
||||
Python interpreter. The macro **must** be invoked _before_ initializing Python.
|
||||
|
||||
As an example, the below adds the module `foo` to the embedded interpreter:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
#[pyfunction]
|
||||
fn add_one(x: i64) -> i64 {
|
||||
x + 1
|
||||
}
|
||||
|
||||
#[pymodule]
|
||||
fn foo(foo_module: &Bound<'_, PyModule>) -> PyResult<()> {
|
||||
foo_module.add_function(wrap_pyfunction!(add_one, foo_module)?)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
pyo3::append_to_inittab!(foo);
|
||||
Python::with_gil(|py| Python::run_bound(py, "import foo; foo.add_one(6)", None, None))
|
||||
}
|
||||
```
|
||||
|
||||
If `append_to_inittab` cannot be used due to constraints in the program,
|
||||
an alternative is to create a module using [`PyModule::new`]
|
||||
and insert it manually into `sys.modules`:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyDict;
|
||||
|
||||
#[pyfunction]
|
||||
pub fn add_one(x: i64) -> i64 {
|
||||
x + 1
|
||||
}
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
// Create new module
|
||||
let foo_module = PyModule::new_bound(py, "foo")?;
|
||||
foo_module.add_function(wrap_pyfunction!(add_one, &foo_module)?)?;
|
||||
|
||||
// Import and get sys.modules
|
||||
let sys = PyModule::import_bound(py, "sys")?;
|
||||
let py_modules: Bound<'_, PyDict> = sys.getattr("modules")?.downcast_into()?;
|
||||
|
||||
// Insert foo into sys.modules
|
||||
py_modules.set_item("foo", foo_module)?;
|
||||
|
||||
// Now we can import + run our python code
|
||||
Python::run_bound(py, "import foo; foo.add_one(6)", None, None)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### Include multiple Python files
|
||||
|
||||
You can include a file at compile time by using
|
||||
[`std::include_str`](https://doc.rust-lang.org/std/macro.include_str.html) macro.
|
||||
|
||||
Or you can load a file at runtime by using
|
||||
[`std::fs::read_to_string`](https://doc.rust-lang.org/std/fs/fn.read_to_string.html) function.
|
||||
|
||||
Many Python files can be included and loaded as modules. If one file depends on
|
||||
another you must preserve correct order while declaring `PyModule`.
|
||||
|
||||
Example directory structure:
|
||||
```text
|
||||
.
|
||||
├── Cargo.lock
|
||||
├── Cargo.toml
|
||||
├── python_app
|
||||
│ ├── app.py
|
||||
│ └── utils
|
||||
│ └── foo.py
|
||||
└── src
|
||||
└── main.rs
|
||||
```
|
||||
|
||||
`python_app/app.py`:
|
||||
```python
|
||||
from utils.foo import bar
|
||||
|
||||
|
||||
def run():
|
||||
return bar()
|
||||
```
|
||||
|
||||
`python_app/utils/foo.py`:
|
||||
```python
|
||||
def bar():
|
||||
return "baz"
|
||||
```
|
||||
|
||||
The example below shows:
|
||||
* how to include content of `app.py` and `utils/foo.py` into your rust binary
|
||||
* how to call function `run()` (declared in `app.py`) that needs function
|
||||
imported from `utils/foo.py`
|
||||
|
||||
`src/main.rs`:
|
||||
```rust,ignore
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let py_foo = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/python_app/utils/foo.py"
|
||||
));
|
||||
let py_app = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/python_app/app.py"));
|
||||
let from_python = Python::with_gil(|py| -> PyResult<Py<PyAny>> {
|
||||
PyModule::from_code_bound(py, py_foo, "utils.foo", "utils.foo")?;
|
||||
let app: Py<PyAny> = PyModule::from_code_bound(py, py_app, "", "")?
|
||||
.getattr("run")?
|
||||
.into();
|
||||
app.call0(py)
|
||||
});
|
||||
|
||||
println!("py: {}", from_python?);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The example below shows:
|
||||
* how to load content of `app.py` at runtime so that it sees its dependencies
|
||||
automatically
|
||||
* how to call function `run()` (declared in `app.py`) that needs function
|
||||
imported from `utils/foo.py`
|
||||
|
||||
It is recommended to use absolute paths because then your binary can be run
|
||||
from anywhere as long as your `app.py` is in the expected directory (in this example
|
||||
that directory is `/usr/share/python_app`).
|
||||
|
||||
`src/main.rs`:
|
||||
```rust,no_run
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let path = Path::new("/usr/share/python_app");
|
||||
let py_app = fs::read_to_string(path.join("app.py"))?;
|
||||
let from_python = Python::with_gil(|py| -> PyResult<Py<PyAny>> {
|
||||
let syspath = py.import_bound("sys")?.getattr("path")?.downcast_into::<PyList>()?;
|
||||
syspath.insert(0, &path)?;
|
||||
let app: Py<PyAny> = PyModule::from_code_bound(py, &py_app, "", "")?
|
||||
.getattr("run")?
|
||||
.into();
|
||||
app.call0(py)
|
||||
});
|
||||
|
||||
println!("py: {}", from_python?);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
[`Python::run`]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.run
|
||||
[`py_run!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.py_run.html
|
||||
|
||||
## Need to use a context manager from Rust?
|
||||
|
||||
Use context managers by directly invoking `__enter__` and `__exit__`.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() {
|
||||
Python::with_gil(|py| {
|
||||
let custom_manager = PyModule::from_code_bound(
|
||||
py,
|
||||
r#"
|
||||
class House(object):
|
||||
def __init__(self, address):
|
||||
self.address = address
|
||||
def __enter__(self):
|
||||
print(f"Welcome to {self.address}!")
|
||||
def __exit__(self, type, value, traceback):
|
||||
if type:
|
||||
print(f"Sorry you had {type} trouble at {self.address}")
|
||||
else:
|
||||
print(f"Thank you for visiting {self.address}, come again soon!")
|
||||
|
||||
"#,
|
||||
"house.py",
|
||||
"house",
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let house_class = custom_manager.getattr("House").unwrap();
|
||||
let house = house_class.call1(("123 Main Street",)).unwrap();
|
||||
|
||||
house.call_method0("__enter__").unwrap();
|
||||
|
||||
let result = py.eval_bound("undefined_variable + 1", None, None);
|
||||
|
||||
// If the eval threw an exception we'll pass it through to the context manager.
|
||||
// Otherwise, __exit__ is called with empty arguments (Python "None").
|
||||
match result {
|
||||
Ok(_) => {
|
||||
let none = py.None();
|
||||
house
|
||||
.call_method1("__exit__", (&none, &none, &none))
|
||||
.unwrap();
|
||||
}
|
||||
Err(e) => {
|
||||
house
|
||||
.call_method1("__exit__", (e.get_type_bound(py), e.value_bound(py), e.traceback_bound(py)))
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Handling system signals/interrupts (Ctrl-C)
|
||||
|
||||
The best way to handle system signals when running Rust code is to periodically call `Python::check_signals` to handle any signals captured by Python's signal handler. See also [the FAQ entry](./faq.md#ctrl-c-doesnt-do-anything-while-my-rust-code-is-executing).
|
||||
|
||||
Alternatively, set Python's `signal` module to take the default action for a signal:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
let signal = py.import_bound("signal")?;
|
||||
// Set SIGINT to have the default action
|
||||
signal
|
||||
.getattr("signal")?
|
||||
.call1((signal.getattr("SIGINT")?, signal.getattr("SIG_DFL")?))?;
|
||||
Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
|
||||
[`PyModule::new`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.new
|
||||
[smart-pointers]: https://doc.rust-lang.org/book/ch15-00-smart-pointers.html
|
||||
[obtaining-py]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#obtaining-a-python-token
|
||||
[`pyo3::sync`]: {{#PYO3_DOCS_URL}}/pyo3/sync/index.html
|
||||
|
|
|
|||
|
|
@ -0,0 +1,397 @@
|
|||
# Executing existing Python code
|
||||
|
||||
If you already have some existing Python code that you need to execute from Rust, the following FAQs can help you select the right PyO3 functionality for your situation:
|
||||
|
||||
## Want to access Python APIs? Then use `PyModule::import`.
|
||||
|
||||
[`Pymodule::import`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.import) can
|
||||
be used to get handle to a Python module from Rust. You can use this to import and use any Python
|
||||
module available in your environment.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
let builtins = PyModule::import_bound(py, "builtins")?;
|
||||
let total: i32 = builtins
|
||||
.getattr("sum")?
|
||||
.call1((vec![1, 2, 3],))?
|
||||
.extract()?;
|
||||
assert_eq!(total, 6);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Want to run just an expression? Then use `eval`.
|
||||
|
||||
[`Python::eval`]({{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.eval) is
|
||||
a method to execute a [Python expression](https://docs.python.org/3.7/reference/expressions.html)
|
||||
and return the evaluated value as a `Bound<'py, PyAny>` object.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
# fn main() -> Result<(), ()> {
|
||||
Python::with_gil(|py| {
|
||||
let result = py
|
||||
.eval_bound("[i * 10 for i in range(5)]", None, None)
|
||||
.map_err(|e| {
|
||||
e.print_and_set_sys_last_vars(py);
|
||||
})?;
|
||||
let res: Vec<i64> = result.extract().unwrap();
|
||||
assert_eq!(res, vec![0, 10, 20, 30, 40]);
|
||||
Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
## Want to run statements? Then use `run`.
|
||||
|
||||
[`Python::run`] is a method to execute one or more
|
||||
[Python statements](https://docs.python.org/3.7/reference/simple_stmts.html).
|
||||
This method returns nothing (like any Python statement), but you can get
|
||||
access to manipulated objects via the `locals` dict.
|
||||
|
||||
You can also use the [`py_run!`] macro, which is a shorthand for [`Python::run`].
|
||||
Since [`py_run!`] panics on exceptions, we recommend you use this macro only for
|
||||
quickly testing your Python extensions.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::py_run;
|
||||
|
||||
# fn main() {
|
||||
#[pyclass]
|
||||
struct UserData {
|
||||
id: u32,
|
||||
name: String,
|
||||
}
|
||||
|
||||
#[pymethods]
|
||||
impl UserData {
|
||||
fn as_tuple(&self) -> (u32, String) {
|
||||
(self.id, self.name.clone())
|
||||
}
|
||||
|
||||
fn __repr__(&self) -> PyResult<String> {
|
||||
Ok(format!("User {}(id: {})", self.name, self.id))
|
||||
}
|
||||
}
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let userdata = UserData {
|
||||
id: 34,
|
||||
name: "Yu".to_string(),
|
||||
};
|
||||
let userdata = Py::new(py, userdata).unwrap();
|
||||
let userdata_as_tuple = (34, "Yu");
|
||||
py_run!(py, userdata userdata_as_tuple, r#"
|
||||
assert repr(userdata) == "User Yu(id: 34)"
|
||||
assert userdata.as_tuple() == userdata_as_tuple
|
||||
"#);
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
## You have a Python file or code snippet? Then use `PyModule::from_code`.
|
||||
|
||||
[`PyModule::from_code`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.from_code)
|
||||
can be used to generate a Python module which can then be used just as if it was imported with
|
||||
`PyModule::import`.
|
||||
|
||||
**Warning**: This will compile and execute code. **Never** pass untrusted code
|
||||
to this function!
|
||||
|
||||
```rust
|
||||
use pyo3::{prelude::*, types::IntoPyDict};
|
||||
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
let activators = PyModule::from_code_bound(
|
||||
py,
|
||||
r#"
|
||||
def relu(x):
|
||||
"""see https://en.wikipedia.org/wiki/Rectifier_(neural_networks)"""
|
||||
return max(0.0, x)
|
||||
|
||||
def leaky_relu(x, slope=0.01):
|
||||
return x if x >= 0 else x * slope
|
||||
"#,
|
||||
"activators.py",
|
||||
"activators",
|
||||
)?;
|
||||
|
||||
let relu_result: f64 = activators.getattr("relu")?.call1((-1.0,))?.extract()?;
|
||||
assert_eq!(relu_result, 0.0);
|
||||
|
||||
let kwargs = [("slope", 0.2)].into_py_dict_bound(py);
|
||||
let lrelu_result: f64 = activators
|
||||
.getattr("leaky_relu")?
|
||||
.call((-1.0,), Some(&kwargs))?
|
||||
.extract()?;
|
||||
assert_eq!(lrelu_result, -0.2);
|
||||
# Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
## Want to embed Python in Rust with additional modules?
|
||||
|
||||
Python maintains the `sys.modules` dict as a cache of all imported modules.
|
||||
An import in Python will first attempt to lookup the module from this dict,
|
||||
and if not present will use various strategies to attempt to locate and load
|
||||
the module.
|
||||
|
||||
The [`append_to_inittab`]({{#PYO3_DOCS_URL}}/pyo3/macro.append_to_inittab.html)
|
||||
macro can be used to add additional `#[pymodule]` modules to an embedded
|
||||
Python interpreter. The macro **must** be invoked _before_ initializing Python.
|
||||
|
||||
As an example, the below adds the module `foo` to the embedded interpreter:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
#[pyfunction]
|
||||
fn add_one(x: i64) -> i64 {
|
||||
x + 1
|
||||
}
|
||||
|
||||
#[pymodule]
|
||||
fn foo(foo_module: &Bound<'_, PyModule>) -> PyResult<()> {
|
||||
foo_module.add_function(wrap_pyfunction!(add_one, foo_module)?)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
pyo3::append_to_inittab!(foo);
|
||||
Python::with_gil(|py| Python::run_bound(py, "import foo; foo.add_one(6)", None, None))
|
||||
}
|
||||
```
|
||||
|
||||
If `append_to_inittab` cannot be used due to constraints in the program,
|
||||
an alternative is to create a module using [`PyModule::new`]
|
||||
and insert it manually into `sys.modules`:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyDict;
|
||||
|
||||
#[pyfunction]
|
||||
pub fn add_one(x: i64) -> i64 {
|
||||
x + 1
|
||||
}
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| {
|
||||
// Create new module
|
||||
let foo_module = PyModule::new_bound(py, "foo")?;
|
||||
foo_module.add_function(wrap_pyfunction!(add_one, &foo_module)?)?;
|
||||
|
||||
// Import and get sys.modules
|
||||
let sys = PyModule::import_bound(py, "sys")?;
|
||||
let py_modules: Bound<'_, PyDict> = sys.getattr("modules")?.downcast_into()?;
|
||||
|
||||
// Insert foo into sys.modules
|
||||
py_modules.set_item("foo", foo_module)?;
|
||||
|
||||
// Now we can import + run our python code
|
||||
Python::run_bound(py, "import foo; foo.add_one(6)", None, None)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Include multiple Python files
|
||||
|
||||
You can include a file at compile time by using
|
||||
[`std::include_str`](https://doc.rust-lang.org/std/macro.include_str.html) macro.
|
||||
|
||||
Or you can load a file at runtime by using
|
||||
[`std::fs::read_to_string`](https://doc.rust-lang.org/std/fs/fn.read_to_string.html) function.
|
||||
|
||||
Many Python files can be included and loaded as modules. If one file depends on
|
||||
another you must preserve correct order while declaring `PyModule`.
|
||||
|
||||
Example directory structure:
|
||||
```text
|
||||
.
|
||||
├── Cargo.lock
|
||||
├── Cargo.toml
|
||||
├── python_app
|
||||
│ ├── app.py
|
||||
│ └── utils
|
||||
│ └── foo.py
|
||||
└── src
|
||||
└── main.rs
|
||||
```
|
||||
|
||||
`python_app/app.py`:
|
||||
```python
|
||||
from utils.foo import bar
|
||||
|
||||
|
||||
def run():
|
||||
return bar()
|
||||
```
|
||||
|
||||
`python_app/utils/foo.py`:
|
||||
```python
|
||||
def bar():
|
||||
return "baz"
|
||||
```
|
||||
|
||||
The example below shows:
|
||||
* how to include content of `app.py` and `utils/foo.py` into your rust binary
|
||||
* how to call function `run()` (declared in `app.py`) that needs function
|
||||
imported from `utils/foo.py`
|
||||
|
||||
`src/main.rs`:
|
||||
```rust,ignore
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let py_foo = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/python_app/utils/foo.py"
|
||||
));
|
||||
let py_app = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/python_app/app.py"));
|
||||
let from_python = Python::with_gil(|py| -> PyResult<Py<PyAny>> {
|
||||
PyModule::from_code_bound(py, py_foo, "utils.foo", "utils.foo")?;
|
||||
let app: Py<PyAny> = PyModule::from_code_bound(py, py_app, "", "")?
|
||||
.getattr("run")?
|
||||
.into();
|
||||
app.call0(py)
|
||||
});
|
||||
|
||||
println!("py: {}", from_python?);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The example below shows:
|
||||
* how to load content of `app.py` at runtime so that it sees its dependencies
|
||||
automatically
|
||||
* how to call function `run()` (declared in `app.py`) that needs function
|
||||
imported from `utils/foo.py`
|
||||
|
||||
It is recommended to use absolute paths because then your binary can be run
|
||||
from anywhere as long as your `app.py` is in the expected directory (in this example
|
||||
that directory is `/usr/share/python_app`).
|
||||
|
||||
`src/main.rs`:
|
||||
```rust,no_run
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let path = Path::new("/usr/share/python_app");
|
||||
let py_app = fs::read_to_string(path.join("app.py"))?;
|
||||
let from_python = Python::with_gil(|py| -> PyResult<Py<PyAny>> {
|
||||
let syspath = py
|
||||
.import_bound("sys")?
|
||||
.getattr("path")?
|
||||
.downcast_into::<PyList>()?;
|
||||
syspath.insert(0, &path)?;
|
||||
let app: Py<PyAny> = PyModule::from_code_bound(py, &py_app, "", "")?
|
||||
.getattr("run")?
|
||||
.into();
|
||||
app.call0(py)
|
||||
});
|
||||
|
||||
println!("py: {}", from_python?);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
[`Python::run`]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.run
|
||||
[`py_run!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.py_run.html
|
||||
|
||||
## Need to use a context manager from Rust?
|
||||
|
||||
Use context managers by directly invoking `__enter__` and `__exit__`.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
fn main() {
|
||||
Python::with_gil(|py| {
|
||||
let custom_manager = PyModule::from_code_bound(
|
||||
py,
|
||||
r#"
|
||||
class House(object):
|
||||
def __init__(self, address):
|
||||
self.address = address
|
||||
def __enter__(self):
|
||||
print(f"Welcome to {self.address}!")
|
||||
def __exit__(self, type, value, traceback):
|
||||
if type:
|
||||
print(f"Sorry you had {type} trouble at {self.address}")
|
||||
else:
|
||||
print(f"Thank you for visiting {self.address}, come again soon!")
|
||||
|
||||
"#,
|
||||
"house.py",
|
||||
"house",
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let house_class = custom_manager.getattr("House").unwrap();
|
||||
let house = house_class.call1(("123 Main Street",)).unwrap();
|
||||
|
||||
house.call_method0("__enter__").unwrap();
|
||||
|
||||
let result = py.eval_bound("undefined_variable + 1", None, None);
|
||||
|
||||
// If the eval threw an exception we'll pass it through to the context manager.
|
||||
// Otherwise, __exit__ is called with empty arguments (Python "None").
|
||||
match result {
|
||||
Ok(_) => {
|
||||
let none = py.None();
|
||||
house
|
||||
.call_method1("__exit__", (&none, &none, &none))
|
||||
.unwrap();
|
||||
}
|
||||
Err(e) => {
|
||||
house
|
||||
.call_method1(
|
||||
"__exit__",
|
||||
(
|
||||
e.get_type_bound(py),
|
||||
e.value_bound(py),
|
||||
e.traceback_bound(py),
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Handling system signals/interrupts (Ctrl-C)
|
||||
|
||||
The best way to handle system signals when running Rust code is to periodically call `Python::check_signals` to handle any signals captured by Python's signal handler. See also [the FAQ entry](../faq.md#ctrl-c-doesnt-do-anything-while-my-rust-code-is-executing).
|
||||
|
||||
Alternatively, set Python's `signal` module to take the default action for a signal:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
# fn main() -> PyResult<()> {
|
||||
Python::with_gil(|py| -> PyResult<()> {
|
||||
let signal = py.import_bound("signal")?;
|
||||
// Set SIGINT to have the default action
|
||||
signal
|
||||
.getattr("signal")?
|
||||
.call1((signal.getattr("SIGINT")?, signal.getattr("SIG_DFL")?))?;
|
||||
Ok(())
|
||||
})
|
||||
# }
|
||||
```
|
||||
|
||||
|
||||
[`PyModule::new`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.new
|
||||
|
|
@ -0,0 +1,114 @@
|
|||
# Calling Python functions
|
||||
|
||||
The `Bound<'py, T>` smart pointer (such as `Bound<'py, PyAny>`, `Bound<'py, PyList>`, or `Bound<'py, MyClass>`) can be used to call Python functions.
|
||||
|
||||
PyO3 offers two APIs to make function calls:
|
||||
|
||||
* [`call`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call) - call any callable Python object.
|
||||
* [`call_method`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method) - call a method on the Python object.
|
||||
|
||||
Both of these APIs take `args` and `kwargs` arguments (for positional and keyword arguments respectively). There are variants for less complex calls:
|
||||
|
||||
* [`call1`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call1) and [`call_method1`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method1) to call only with positional `args`.
|
||||
* [`call0`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call0) and [`call_method0`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call_method0) to call with no arguments.
|
||||
|
||||
For convenience the [`Py<T>`](../types.md#pyt-and-pyobject) smart pointer also exposes these same six API methods, but needs a `Python` token as an additional first argument to prove the GIL is held.
|
||||
|
||||
The example below calls a Python function behind a `PyObject` (aka `Py<PyAny>`) reference:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyTuple;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let arg1 = "arg1";
|
||||
let arg2 = "arg2";
|
||||
let arg3 = "arg3";
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let fun: Py<PyAny> = PyModule::from_code_bound(
|
||||
py,
|
||||
"def example(*args, **kwargs):
|
||||
if args != ():
|
||||
print('called with args', args)
|
||||
if kwargs != {}:
|
||||
print('called with kwargs', kwargs)
|
||||
if args == () and kwargs == {}:
|
||||
print('called with no arguments')",
|
||||
"",
|
||||
"",
|
||||
)?
|
||||
.getattr("example")?
|
||||
.into();
|
||||
|
||||
// call object without any arguments
|
||||
fun.call0(py)?;
|
||||
|
||||
// pass object with Rust tuple of positional arguments
|
||||
let args = (arg1, arg2, arg3);
|
||||
fun.call1(py, args)?;
|
||||
|
||||
// call object with Python tuple of positional arguments
|
||||
let args = PyTuple::new_bound(py, &[arg1, arg2, arg3]);
|
||||
fun.call1(py, args)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Creating keyword arguments
|
||||
|
||||
For the `call` and `call_method` APIs, `kwargs` are `Option<&Bound<'py, PyDict>>`, so can either be `None` or `Some(&dict)`. You can use the [`IntoPyDict`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.IntoPyDict.html) trait to convert other dict-like containers, e.g. `HashMap` or `BTreeMap`, as well as tuples with up to 10 elements and `Vec`s where each element is a two-element tuple.
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::IntoPyDict;
|
||||
use std::collections::HashMap;
|
||||
|
||||
fn main() -> PyResult<()> {
|
||||
let key1 = "key1";
|
||||
let val1 = 1;
|
||||
let key2 = "key2";
|
||||
let val2 = 2;
|
||||
|
||||
Python::with_gil(|py| {
|
||||
let fun: Py<PyAny> = PyModule::from_code_bound(
|
||||
py,
|
||||
"def example(*args, **kwargs):
|
||||
if args != ():
|
||||
print('called with args', args)
|
||||
if kwargs != {}:
|
||||
print('called with kwargs', kwargs)
|
||||
if args == () and kwargs == {}:
|
||||
print('called with no arguments')",
|
||||
"",
|
||||
"",
|
||||
)?
|
||||
.getattr("example")?
|
||||
.into();
|
||||
|
||||
// call object with PyDict
|
||||
let kwargs = [(key1, val1)].into_py_dict_bound(py);
|
||||
fun.call_bound(py, (), Some(&kwargs))?;
|
||||
|
||||
// pass arguments as Vec
|
||||
let kwargs = vec![(key1, val1), (key2, val2)];
|
||||
fun.call_bound(py, (), Some(&kwargs.into_py_dict_bound(py)))?;
|
||||
|
||||
// pass arguments as HashMap
|
||||
let mut kwargs = HashMap::<&str, i32>::new();
|
||||
kwargs.insert(key1, 1);
|
||||
fun.call_bound(py, (), Some(&kwargs.into_py_dict_bound(py)))?;
|
||||
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
<div class="warning">
|
||||
|
||||
During PyO3's [migration from "GIL Refs" to the `Bound<T>` smart pointer](../migration.md#migrating-from-the-gil-refs-api-to-boundt), [`Py<T>::call`]({{#PYO3_DOCS_URL}}/pyo3/struct.Py.html#method.call) is temporarily named `call_bound` (and `call_method` is temporarily `call_method_bound`).
|
||||
|
||||
(This temporary naming is only the case for the `Py<T>` smart pointer. The methods on the `&PyAny` GIL Ref such as `call` have not been given replacements, and the methods on the `Bound<PyAny>` smart pointer such as [`Bound<PyAny>::call`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.call) already use follow the newest API conventions.)
|
||||
|
||||
</div>
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Using Rust from Python
|
||||
|
||||
This chapter of the guide is dedicated to explaining how to wrap Rust code into Python objects.
|
||||
|
||||
PyO3 uses Rust's "procedural macros" to provide a powerful yet simple API to denote what Rust code should map into Python objects.
|
||||
|
||||
The three types of Python objects which PyO3 can produce are:
|
||||
|
||||
- Python modules, via the `#[pymodule]` macro
|
||||
- Python functions, via the `#[pyfunction]` macro
|
||||
- Python classes, via the `#[pyclass]` macro (plus `#[pymethods]` to define methods for those clases)
|
||||
|
||||
The following subchapters go through each of these in turn.
|
||||
|
|
@ -66,6 +66,7 @@ The following wrapper will call the Python model from Rust, using a struct to ho
|
|||
```rust
|
||||
# #![allow(dead_code)]
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
|
||||
# pub trait Model {
|
||||
# fn set_variables(&mut self, inputs: &Vec<f64>);
|
||||
|
|
@ -81,12 +82,9 @@ impl Model for UserModel {
|
|||
fn set_variables(&mut self, var: &Vec<f64>) {
|
||||
println!("Rust calling Python to set the variables");
|
||||
Python::with_gil(|py| {
|
||||
let values: Vec<f64> = var.clone();
|
||||
let list: PyObject = values.into_py(py);
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let py_model = self.model.as_ref(py);
|
||||
py_model
|
||||
.call_method("set_variables", (list,), None)
|
||||
self.model
|
||||
.bind(py)
|
||||
.call_method("set_variables", (PyList::new_bound(py, var),), None)
|
||||
.unwrap();
|
||||
})
|
||||
}
|
||||
|
|
@ -94,9 +92,8 @@ impl Model for UserModel {
|
|||
fn get_results(&self) -> Vec<f64> {
|
||||
println!("Rust calling Python to get the results");
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
self.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("get_results", (), None)
|
||||
.unwrap()
|
||||
.extract()
|
||||
|
|
@ -107,9 +104,8 @@ impl Model for UserModel {
|
|||
fn compute(&mut self) {
|
||||
println!("Rust calling Python to perform the computation");
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
self.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("compute", (), None)
|
||||
.unwrap();
|
||||
})
|
||||
|
|
@ -168,6 +164,7 @@ This wrapper will also perform the type conversions between Python and Rust.
|
|||
```rust
|
||||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
#
|
||||
# pub trait Model {
|
||||
# fn set_variables(&mut self, inputs: &Vec<f64>);
|
||||
|
|
@ -184,12 +181,8 @@ This wrapper will also perform the type conversions between Python and Rust.
|
|||
# fn set_variables(&mut self, var: &Vec<f64>) {
|
||||
# println!("Rust calling Python to set the variables");
|
||||
# Python::with_gil(|py| {
|
||||
# let values: Vec<f64> = var.clone();
|
||||
# let list: PyObject = values.into_py(py);
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# let py_model = self.model.as_ref(py);
|
||||
# py_model
|
||||
# .call_method("set_variables", (list,), None)
|
||||
# self.model.bind(py)
|
||||
# .call_method("set_variables", (PyList::new_bound(py, var),), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
# }
|
||||
|
|
@ -197,9 +190,8 @@ This wrapper will also perform the type conversions between Python and Rust.
|
|||
# fn get_results(&self) -> Vec<f64> {
|
||||
# println!("Rust calling Python to get the results");
|
||||
# Python::with_gil(|py| {
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# self.model
|
||||
# .as_ref(py)
|
||||
# .bind(py)
|
||||
# .call_method("get_results", (), None)
|
||||
# .unwrap()
|
||||
# .extract()
|
||||
|
|
@ -210,9 +202,8 @@ This wrapper will also perform the type conversions between Python and Rust.
|
|||
# fn compute(&mut self) {
|
||||
# println!("Rust calling Python to perform the computation");
|
||||
# Python::with_gil(|py| {
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# self.model
|
||||
# .as_ref(py)
|
||||
# .bind(py)
|
||||
# .call_method("compute", (), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
|
|
@ -340,6 +331,7 @@ We used in our `get_results` method the following call that performs the type co
|
|||
```rust
|
||||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
#
|
||||
# pub trait Model {
|
||||
# fn set_variables(&mut self, inputs: &Vec<f64>);
|
||||
|
|
@ -355,10 +347,9 @@ We used in our `get_results` method the following call that performs the type co
|
|||
impl Model for UserModel {
|
||||
fn get_results(&self) -> Vec<f64> {
|
||||
println!("Rust calling Python to get the results");
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
Python::with_gil(|py| {
|
||||
self.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("get_results", (), None)
|
||||
.unwrap()
|
||||
.extract()
|
||||
|
|
@ -368,12 +359,8 @@ impl Model for UserModel {
|
|||
# fn set_variables(&mut self, var: &Vec<f64>) {
|
||||
# println!("Rust calling Python to set the variables");
|
||||
# Python::with_gil(|py| {
|
||||
# let values: Vec<f64> = var.clone();
|
||||
# let list: PyObject = values.into_py(py);
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# let py_model = self.model.as_ref(py);
|
||||
# py_model
|
||||
# .call_method("set_variables", (list,), None)
|
||||
# self.model.bind(py)
|
||||
# .call_method("set_variables", (PyList::new_bound(py, var),), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
# }
|
||||
|
|
@ -381,9 +368,8 @@ impl Model for UserModel {
|
|||
# fn compute(&mut self) {
|
||||
# println!("Rust calling Python to perform the computation");
|
||||
# Python::with_gil(|py| {
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# self.model
|
||||
# .as_ref(py)
|
||||
# .bind(py)
|
||||
# .call_method("compute", (), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
|
|
@ -396,6 +382,7 @@ Let's break it down in order to perform better error handling:
|
|||
```rust
|
||||
# #![allow(dead_code)]
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
#
|
||||
# pub trait Model {
|
||||
# fn set_variables(&mut self, inputs: &Vec<f64>);
|
||||
|
|
@ -412,10 +399,9 @@ impl Model for UserModel {
|
|||
fn get_results(&self) -> Vec<f64> {
|
||||
println!("Get results from Rust calling Python");
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let py_result: &PyAny = self
|
||||
let py_result: Bound<'_, PyAny> = self
|
||||
.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("get_results", (), None)
|
||||
.unwrap();
|
||||
|
||||
|
|
@ -432,12 +418,8 @@ impl Model for UserModel {
|
|||
# fn set_variables(&mut self, var: &Vec<f64>) {
|
||||
# println!("Rust calling Python to set the variables");
|
||||
# Python::with_gil(|py| {
|
||||
# let values: Vec<f64> = var.clone();
|
||||
# let list: PyObject = values.into_py(py);
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# let py_model = self.model.as_ref(py);
|
||||
# py_model
|
||||
# .call_method("set_variables", (list,), None)
|
||||
# let py_model = self.model.bind(py)
|
||||
# .call_method("set_variables", (PyList::new_bound(py, var),), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
# }
|
||||
|
|
@ -445,9 +427,8 @@ impl Model for UserModel {
|
|||
# fn compute(&mut self) {
|
||||
# println!("Rust calling Python to perform the computation");
|
||||
# Python::with_gil(|py| {
|
||||
# #[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
# self.model
|
||||
# .as_ref(py)
|
||||
# .bind(py)
|
||||
# .call_method("compute", (), None)
|
||||
# .unwrap();
|
||||
# })
|
||||
|
|
@ -478,6 +459,7 @@ It is also required to make the struct public.
|
|||
```rust
|
||||
# #![allow(dead_code)]
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
|
||||
pub trait Model {
|
||||
fn set_variables(&mut self, var: &Vec<f64>);
|
||||
|
|
@ -533,12 +515,9 @@ impl Model for UserModel {
|
|||
fn set_variables(&mut self, var: &Vec<f64>) {
|
||||
println!("Rust calling Python to set the variables");
|
||||
Python::with_gil(|py| {
|
||||
let values: Vec<f64> = var.clone();
|
||||
let list: PyObject = values.into_py(py);
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let py_model = self.model.as_ref(py);
|
||||
py_model
|
||||
.call_method("set_variables", (list,), None)
|
||||
self.model
|
||||
.bind(py)
|
||||
.call_method("set_variables", (PyList::new_bound(py, var),), None)
|
||||
.unwrap();
|
||||
})
|
||||
}
|
||||
|
|
@ -546,10 +525,9 @@ impl Model for UserModel {
|
|||
fn get_results(&self) -> Vec<f64> {
|
||||
println!("Get results from Rust calling Python");
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let py_result: &PyAny = self
|
||||
let py_result: Bound<'_, PyAny> = self
|
||||
.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("get_results", (), None)
|
||||
.unwrap();
|
||||
|
||||
|
|
@ -567,9 +545,8 @@ impl Model for UserModel {
|
|||
fn compute(&mut self) {
|
||||
println!("Rust calling Python to perform the computation");
|
||||
Python::with_gil(|py| {
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
self.model
|
||||
.as_ref(py)
|
||||
.bind(py)
|
||||
.call_method("compute", (), None)
|
||||
.unwrap();
|
||||
})
|
||||
|
|
|
|||
|
|
@ -1,66 +1,289 @@
|
|||
# GIL lifetimes, mutability and Python object types
|
||||
# Python object types
|
||||
|
||||
On first glance, PyO3 provides a huge number of different types that can be used
|
||||
to wrap or refer to Python objects. This page delves into the details and gives
|
||||
an overview of their intended meaning, with examples when each type is best
|
||||
used.
|
||||
PyO3 offers two main sets of types to interact with Python objects. This section of the guide expands into detail about these types and how to choose which to use.
|
||||
|
||||
The first set of types is are the [smart pointers][smart-pointers] which all Python objects are wrapped in. These are `Py<T>`, `Bound<'py, T>`, and `Borrowed<'a, 'py, T>`. The [first section below](#pyo3s-smart-pointers) expands on each of these in detail and why there are three of them.
|
||||
|
||||
## The Python GIL, mutability, and Rust types
|
||||
The second set of types are types which fill in the generic parameter `T` of the smart pointers. The most common is `PyAny`, which represents any Python object (similar to Python's `typing.Any`). There are also concrete types for many Python built-in types, such as `PyList`, `PyDict`, and `PyTuple`. User defined `#[pyclass]` types also fit this category. The [second section below](#concrete-python-types) expands on how to use these types.
|
||||
|
||||
Since Python has no concept of ownership, and works solely with boxed objects,
|
||||
any Python object can be referenced any number of times, and mutation is allowed
|
||||
from any reference.
|
||||
Before PyO3 0.21, PyO3's main API to interact with Python objects was a deprecated API known as the "GIL Refs" API, containing reference types such as `&PyAny`, `&PyList`, and `&PyCell<T>` for user-defined `#[pyclass]` types. The [third section below](#the-gil-refs-api) details this deprecated API.
|
||||
|
||||
The situation is helped a little by the Global Interpreter Lock (GIL), which
|
||||
ensures that only one thread can use the Python interpreter and its API at the
|
||||
same time, while non-Python operations (system calls and extension code) can
|
||||
unlock the GIL. (See [the section on parallelism](parallelism.md) for how to do
|
||||
that in PyO3.)
|
||||
## PyO3's smart pointers
|
||||
|
||||
In PyO3, holding the GIL is modeled by acquiring a token of the type
|
||||
`Python<'py>`, which serves three purposes:
|
||||
PyO3's API offers three generic smart pointers: `Py<T>`, `Bound<'py, T>` and `Borrowed<'a, 'py, T>`. For each of these the type parameter `T` will be filled by a [concrete Python type](#concrete-python-types). For example, a Python list object can be represented by `Py<PyList>`, `Bound<'py, PyList>`, and `Borrowed<'a, 'py, PyList>`.
|
||||
|
||||
* It provides some global API for the Python interpreter, such as
|
||||
[`eval`][eval].
|
||||
* It can be passed to functions that require a proof of holding the GIL,
|
||||
such as [`Py::clone_ref`][clone_ref].
|
||||
* Its lifetime can be used to create Rust references that implicitly guarantee
|
||||
holding the GIL, such as [`&'py PyAny`][PyAny].
|
||||
These smart pointers behave differently due to their lifetime parameters. `Py<T>` has no lifetime parameters, `Bound<'py, T>` has [the `'py` lifetime](./python-from-rust.md#the-py-lifetime) as a parameter, and `Borrowed<'a, 'py, T>` has the `'py` lifetime plus an additional lifetime `'a` to denote the lifetime it is borrowing data for. (You can read more about these lifetimes in the subsections below).
|
||||
|
||||
The latter two points are the reason why some APIs in PyO3 require the `py:
|
||||
Python` argument, while others don't.
|
||||
Python objects are reference counted, like [`std::sync::Arc`](https://doc.rust-lang.org/stable/std/sync/struct.Arc.html). A major reason for these smart pointers is to bring Python's reference counting to a Rust API.
|
||||
|
||||
The PyO3 API for Python objects is written such that instead of requiring a
|
||||
mutable Rust reference for mutating operations such as
|
||||
[`PyList::append`][PyList_append], a shared reference (which, in turn, can only
|
||||
be created through `Python<'_>` with a GIL lifetime) is sufficient.
|
||||
The recommendation of when to use each of these smart pointers is as follows:
|
||||
|
||||
However, Rust structs wrapped as Python objects (called `pyclass` types) usually
|
||||
*do* need `&mut` access. Due to the GIL, PyO3 *can* guarantee thread-safe access
|
||||
to them, but it cannot statically guarantee uniqueness of `&mut` references once
|
||||
an object's ownership has been passed to the Python interpreter, ensuring
|
||||
references is done at runtime using `PyCell`, a scheme very similar to
|
||||
`std::cell::RefCell`.
|
||||
- Use `Bound<'py, T>` for as much as possible, as it offers the most efficient and complete API.
|
||||
- Use `Py<T>` mostly just for storage inside Rust `struct`s which do not want to or can't add a lifetime parameter for `Bound<'py, T>`.
|
||||
- `Borrowed<'a, 'py, T>` is almost never used. It is occasionally present at the boundary between Rust and the Python interpreter, for example when borrowing data from Python tuples (which is safe because they are immutable).
|
||||
|
||||
### Accessing the Python GIL
|
||||
The sections below also explain these smart pointers in a little more detail.
|
||||
|
||||
To get hold of a `Python<'py>` token to prove the GIL is held, consult [PyO3's documentation][obtaining-py].
|
||||
### `Py<T>` (and `PyObject`)
|
||||
|
||||
## Object types
|
||||
[`Py<T>`][Py] is the foundational smart pointer in PyO3's API. The type parameter `T` denotes the type of the Python object. Very frequently this is `PyAny`, meaning any Python object. This is so common that `Py<PyAny>` has a type alias `PyObject`.
|
||||
|
||||
Because `Py<T>` is not bound to [the `'py` lifetime](./python-from-rust.md#the-py-lifetime), it is the type to use when storing a Python object inside a Rust `struct` or `enum` which do not want to have a lifetime parameter. In particular, [`#[pyclass]`][pyclass] types are not permitted to have a lifetime, so `Py<T>` is the correct type to store Python objects inside them.
|
||||
|
||||
The lack of binding to the `'py` lifetime also carries drawbacks:
|
||||
- Almost all methods on `Py<T>` require a `Python<'py>` token as the first argument
|
||||
- Other functionality, such as [`Drop`][Drop], needs to check at runtime for attachment to the Python GIL, at a small performance cost
|
||||
|
||||
Because of the drawbacks `Bound<'py, T>` is preferred for many of PyO3's APIs. In particular, `Bound<'py, T>` is the better for function arguments.
|
||||
|
||||
To convert a `Py<T>` into a `Bound<'py, T>`, the `Py::bind` and `Py::into_bound` methods are available. `Bound<'py, T>` can be converted back into `Py<T>` using [`Bound::unbind`].
|
||||
|
||||
### `Bound<'py, T>`
|
||||
|
||||
[`Bound<'py, T>`][Bound] is the counterpart to `Py<T>` which is also bound to the `'py` lifetime. It can be thought of as equivalent to the Rust tuple `(Python<'py>, Py<T>)`.
|
||||
|
||||
By having the binding to the `'py` lifetime, `Bound<'py, T>` can offer the complete PyO3 API at maximum efficiency. This means that in almost all cases where `Py<T>` is not necessary for lifetime reasons, `Bound<'py, T>` should be used.
|
||||
|
||||
`Bound<'py, T>` engages in Python reference counting. This means that `Bound<'py, T>` owns a Python object. Rust code which just wants to borrow a Python object should use a shared reference `&Bound<'py, T>`. Just like `std::sync::Arc`, using `.clone()` and `drop()` will cheaply increment and decrement the reference count of the object (just in this case, the reference counting is implemented by the Python interpreter itself).
|
||||
|
||||
To give an example of how `Bound<'py, T>` is PyO3's primary API type, consider the following Python code:
|
||||
|
||||
```python
|
||||
def example():
|
||||
x = list() # create a Python list
|
||||
x.append(1) # append the integer 1 to it
|
||||
y = x # create a second reference to the list
|
||||
del x # delete the original reference
|
||||
```
|
||||
|
||||
Using PyO3's API, and in particular `Bound<'py, PyList>`, this code translates into the following Rust code:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
|
||||
fn example<'py>(py: Python<'py>) -> PyResult<()> {
|
||||
let x: Bound<'py, PyList> = PyList::empty_bound(py);
|
||||
x.append(1)?;
|
||||
let y: Bound<'py, PyList> = x.clone(); // y is a new reference to the same list
|
||||
drop(x); // release the original reference x
|
||||
Ok(())
|
||||
}
|
||||
# Python::with_gil(example).unwrap();
|
||||
```
|
||||
|
||||
Or, without the type annotations:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
|
||||
# fn example(py: Python<'_>) -> PyResult<()> {
|
||||
let x = PyList::empty_bound(py);
|
||||
x.append(1)?;
|
||||
let y = x.clone();
|
||||
drop(x);
|
||||
Ok(())
|
||||
}
|
||||
# Python::with_gil(example).unwrap();
|
||||
```
|
||||
|
||||
#### Function argument lifetimes
|
||||
|
||||
Because the `'py` lifetime often appears in many function arguments as part of the `Bound<'py, T>` smart pointer, the Rust compiler will often require annotations of input and output lifetimes. This occurs when the function output has at least one lifetime, and there is more than one lifetime present on the inputs.
|
||||
|
||||
To demonstrate, consider this function which takes accepts Python objects and applies the [Python `+` operation][PyAnyMethods::add] to them:
|
||||
|
||||
```rust,compile_fail
|
||||
# use pyo3::prelude::*;
|
||||
fn add(left: &'_ Bound<'_, PyAny>, right: &'_ Bound<'_, PyAny>) -> PyResult<Bound<'_, PyAny>> {
|
||||
left.add(right)
|
||||
}
|
||||
```
|
||||
|
||||
Because the Python `+` operation might raise an exception, this function returns `PyResult<Bound<'_, PyAny>>`. It doesn't need ownership of the inputs, so it takes `&Bound<'_, PyAny>` shared references. To demonstrate the point, all lifetimes have used the wildcard `'_` to allow the Rust compiler to attempt to infer them. Because there are four input lifetimes (two lifetimes of the shared references, and two `'py` lifetimes unnamed inside the `Bound<'_, PyAny>` pointers), the compiler cannot reason about which must be connected to the output.
|
||||
|
||||
The correct way to solve this is to add the `'py` lifetime as a parameter for the function, and name all the `'py` lifetimes inside the `Bound<'py, PyAny>` smart pointers. For the shared references, it's also fine to reduce `&'_` to just `&`. The working end result is below:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
fn add<'py>(
|
||||
left: &Bound<'py, PyAny>,
|
||||
right: &Bound<'py, PyAny>,
|
||||
) -> PyResult<Bound<'py, PyAny>> {
|
||||
left.add(right)
|
||||
}
|
||||
# Python::with_gil(|py| {
|
||||
# let s = pyo3::types::PyString::new_bound(py, "s");
|
||||
# assert!(add(&s, &s).unwrap().eq("ss").unwrap());
|
||||
# })
|
||||
```
|
||||
|
||||
If naming the `'py` lifetime adds unwanted complexity to the function signature, it is also acceptable to return `PyObject` (aka `Py<PyAny>`), which has no lifetime. The cost is instead paid by a slight increase in implementation complexity, as seen by the introduction of a call to [`Bound::unbind`]:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
fn add(left: &Bound<'_, PyAny>, right: &Bound<'_, PyAny>) -> PyResult<PyObject> {
|
||||
let output: Bound<'_, PyAny> = left.add(right)?;
|
||||
Ok(output.unbind())
|
||||
}
|
||||
# Python::with_gil(|py| {
|
||||
# let s = pyo3::types::PyString::new_bound(py, "s");
|
||||
# assert!(add(&s, &s).unwrap().bind(py).eq("ss").unwrap());
|
||||
# })
|
||||
```
|
||||
|
||||
### `Borrowed<'a, 'py, T>`
|
||||
|
||||
[`Borrowed<'a, 'py, T>`][Borrowed] is an advanced type used just occasionally at the edge of interaction with the Python interpreter. It can be thought of as analogous to the shared reference `&'a Bound<'py, T>`. The difference is that `Borrowed<'a, 'py, T>` is just a smart pointer rather than a reference-to-a-smart-pointer, which is a helpful reduction in indirection in specific interactions with the Python interpreter.
|
||||
|
||||
`Borrowed<'a, 'py, T>` dereferences to `Bound<'py, T>`, so all methods on `Bound<'py, T>` are available on `Borrowed<'a, 'py, T>`.
|
||||
|
||||
An example where `Borrowed<'a, 'py, T>` is used is in [`PyTupleMethods::get_borrowed_item`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.PyTupleMethods.html#tymethod.get_item):
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyTuple;
|
||||
|
||||
# fn example<'py>(py: Python<'py>) -> PyResult<()> {
|
||||
// Create a new tuple with the elements (0, 1, 2)
|
||||
let t = PyTuple::new_bound(py, [0, 1, 2]);
|
||||
for i in 0..=2 {
|
||||
let entry: Borrowed<'_, 'py, PyAny> = t.get_borrowed_item(i)?;
|
||||
// `PyAnyMethods::extract` is available on `Borrowed`
|
||||
// via the dereference to `Bound`
|
||||
let value: usize = entry.extract()?;
|
||||
assert_eq!(i, value);
|
||||
}
|
||||
# Ok(())
|
||||
# }
|
||||
# Python::with_gil(example).unwrap();
|
||||
```
|
||||
|
||||
## Concrete Python types
|
||||
|
||||
In all of `Py<T>`, `Bound<'py, T>`, and `Borrowed<'a, 'py, T>`, the type parameter `T` denotes the type of the Python object referred to by the smart pointer.
|
||||
|
||||
This parameter `T` can be filled by:
|
||||
- [`PyAny`][PyAny], which represents any Python object,
|
||||
- Native Python types such as `PyList`, `PyTuple`, and `PyDict`, and
|
||||
- [`#[pyclass]`][pyclass] types defined from Rust
|
||||
|
||||
The following subsections covers some further detail about how to work with these types:
|
||||
- the APIs that are available for these concrete types,
|
||||
- how to cast `Bound<'py, T>` to a specific concrete type, and
|
||||
- how to get Rust data out of a `Bound<'py, T>`.
|
||||
|
||||
### Using APIs for concrete Python types
|
||||
|
||||
Each concrete Python type such as `PyAny`, `PyTuple` and `PyDict` exposes its API on the corresponding bound smart pointer `Bound<'py, PyAny>`, `Bound<'py, PyTuple>` and `Bound<'py, PyDict>`.
|
||||
|
||||
Each type's API is exposed as a trait: [`PyAnyMethods`], [`PyTupleMethods`], [`PyDictMethods`], and so on for all concrete types. Using traits rather than associated methods on the `Bound` smart pointer is done for a couple of reasons:
|
||||
- Clarity of documentation: each trait gets its own documentation page in the PyO3 API docs. If all methods were on the `Bound` smart pointer directly, the vast majority of PyO3's API would be on a single, extremely long, documentation page.
|
||||
- Consistency: downstream code implementing Rust APIs for existing Python types can also follow this pattern of using a trait. Downstream code would not be allowed to add new associated methods directly on the `Bound` type.
|
||||
- Future design: it is hoped that a future Rust with [arbitrary self types](https://github.com/rust-lang/rust/issues/44874) will remove the need for these traits in favour of placing the methods directly on `PyAny`, `PyTuple`, `PyDict`, and so on.
|
||||
|
||||
These traits are all included in the `pyo3::prelude` module, so with the glob import `use pyo3::prelude::*` the full PyO3 API is made available to downstream code.
|
||||
|
||||
The following function accesses the first item in the input Python list, using the `.get_item()` method from the `PyListMethods` trait:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::PyList;
|
||||
|
||||
fn get_first_item<'py>(list: &Bound<'py, PyList>) -> PyResult<Bound<'py, PyAny>> {
|
||||
list.get_item(0)
|
||||
}
|
||||
# Python::with_gil(|py| {
|
||||
# let l = PyList::new_bound(py, ["hello world"]);
|
||||
# assert!(get_first_item(&l).unwrap().eq("hello world").unwrap());
|
||||
# })
|
||||
```
|
||||
|
||||
### Casting between Python object types
|
||||
|
||||
To cast `Bound<'py, T>` smart pointers to some other type, use the [`.downcast()`][PyAnyMethods::downcast] family of functions. This converts `&Bound<'py, T>` to a different `&Bound<'py, U>`, without transferring ownership. There is also [`.downcast_into()`][PyAnyMethods::downcast_into] to convert `Bound<'py, T>` to `Bound<'py, U>` with transfer of ownership. These methods are available for all types `T` which implement the [`PyTypeCheck`] trait.
|
||||
|
||||
Casting to `Bound<'py, PyAny>` can be done with `.as_any()` or `.into_any()`.
|
||||
|
||||
For example, the following snippet shows how to cast `Bound<'py, PyAny>` to `Bound<'py, PyTuple>`:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyTuple;
|
||||
# fn example<'py>(py: Python<'py>) -> PyResult<()> {
|
||||
// create a new Python `tuple`, and use `.into_any()` to erase the type
|
||||
let obj: Bound<'py, PyAny> = PyTuple::empty_bound(py).into_any();
|
||||
|
||||
// use `.downcast()` to cast to `PyTuple` without transferring ownership
|
||||
let _: &Bound<'py, PyTuple> = obj.downcast()?;
|
||||
|
||||
// use `.downcast_into()` to cast to `PyTuple` with transfer of ownership
|
||||
let _: Bound<'py, PyTuple> = obj.downcast_into()?;
|
||||
# Ok(())
|
||||
# }
|
||||
# Python::with_gil(example).unwrap()
|
||||
```
|
||||
|
||||
Custom [`#[pyclass]`][pyclass] types implement [`PyTypeCheck`], so `.downcast()` also works for these types. The snippet below is the same as the snippet above casting instead to a custom type `MyClass`:
|
||||
|
||||
```rust
|
||||
use pyo3::prelude::*;
|
||||
|
||||
#[pyclass]
|
||||
struct MyClass {}
|
||||
|
||||
# fn example<'py>(py: Python<'py>) -> PyResult<()> {
|
||||
// create a new Python `tuple`, and use `.into_any()` to erase the type
|
||||
let obj: Bound<'py, PyAny> = Bound::new(py, MyClass {})?.into_any();
|
||||
|
||||
// use `.downcast()` to cast to `MyClass` without transferring ownership
|
||||
let _: &Bound<'py, MyClass> = obj.downcast()?;
|
||||
|
||||
// use `.downcast_into()` to cast to `MyClass` with transfer of ownership
|
||||
let _: Bound<'py, MyClass> = obj.downcast_into()?;
|
||||
# Ok(())
|
||||
# }
|
||||
# Python::with_gil(example).unwrap()
|
||||
```
|
||||
|
||||
### Extracting Rust data from Python objects
|
||||
|
||||
To extract Rust data from Python objects, use [`.extract()`][PyAnyMethods::extract] instead of `.downcast()`. This method is available for all types which implement the [`FromPyObject`] trait.
|
||||
|
||||
For example, the following snippet extracts a Rust tuple of integers from a Python tuple:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyTuple;
|
||||
# fn example<'py>(py: Python<'py>) -> PyResult<()> {
|
||||
// create a new Python `tuple`, and use `.into_any()` to erase the type
|
||||
let obj: Bound<'py, PyAny> = PyTuple::new_bound(py, [1, 2, 3]).into_any();
|
||||
|
||||
// extracting the Python `tuple` to a rust `(i32, i32, i32)` tuple
|
||||
let (x, y, z) = obj.extract::<(i32, i32, i32)>()?;
|
||||
assert_eq!((x, y, z), (1, 2, 3));
|
||||
# Ok(())
|
||||
# }
|
||||
# Python::with_gil(example).unwrap()
|
||||
```
|
||||
|
||||
To avoid copying data, [`#[pyclass]`][pyclass] types can directly reference Rust data stored within the Python objects without needing to `.extract()`. See the [corresponding documentation in the class section of the guide](./class.
|
||||
md#bound-and-interior-mutability) for more detail.
|
||||
|
||||
## The GIL Refs API
|
||||
|
||||
The GIL Refs API was PyO3's primary API prior to PyO3 0.21. The main difference was that instead of the `Bound<'py, PyAny>` smart pointer, the "GIL Reference" `&'py PyAny` was used. (This was similar for other Python types.)
|
||||
|
||||
As of PyO3 0.21, the GIL Refs API is deprecated. See the [migration guide](./migration.md#from-020-to-021) for details on how to upgrade.
|
||||
|
||||
The following sections note some historical detail about the GIL Refs API.
|
||||
|
||||
### [`PyAny`][PyAny]
|
||||
|
||||
**Represents:** a Python object of unspecified type, restricted to a GIL
|
||||
lifetime. Currently, `PyAny` can only ever occur as a reference, `&PyAny`.
|
||||
**Represented:** a Python object of unspecified type. In the GIL Refs API, this was only accessed as the GIL Ref `&'py PyAny`.
|
||||
|
||||
**Used:** Whenever you want to refer to some Python object and will have the
|
||||
GIL for the whole duration you need to access that object. For example,
|
||||
intermediate values and arguments to `pyfunction`s or `pymethod`s implemented
|
||||
in Rust where any type is allowed.
|
||||
|
||||
Many general methods for interacting with Python objects are on the `PyAny` struct,
|
||||
such as `getattr`, `setattr`, and `.call`.
|
||||
**Used:** `&'py PyAny` was used to refer to some Python object when the GIL lifetime was available for the whole duration access was needed. For example, intermediate values and arguments to `pyfunction`s or `pymethod`s implemented in Rust where any type is allowed.
|
||||
|
||||
**Conversions:**
|
||||
|
||||
|
|
@ -71,7 +294,7 @@ a list:
|
|||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
#[allow(deprecated)] // PyList::empty is part of the deprecated "GIL Refs" API.
|
||||
#[allow(deprecated)] // PyList::empty is part of the deprecated "GIL Refs" API.
|
||||
let obj: &PyAny = PyList::empty(py);
|
||||
|
||||
// To &PyList with PyAny::downcast
|
||||
|
|
@ -92,11 +315,11 @@ For a `&PyAny` object reference `any` where the underlying object is a `#[pyclas
|
|||
# use pyo3::prelude::*;
|
||||
# #[pyclass] #[derive(Clone)] struct MyClass { }
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
# #[allow(deprecated)]
|
||||
#[allow(deprecated)] // into_ref is part of the deprecated GIL Refs API
|
||||
let obj: &PyAny = Py::new(py, MyClass {})?.into_ref(py);
|
||||
|
||||
// To &PyCell<MyClass> with PyAny::downcast
|
||||
# #[allow(deprecated)]
|
||||
#[allow(deprecated)] // &PyCell is part of the deprecated GIL Refs API
|
||||
let _: &PyCell<MyClass> = obj.downcast()?;
|
||||
|
||||
// To Py<PyAny> (aka PyObject) with .into()
|
||||
|
|
@ -117,18 +340,13 @@ let _: PyRefMut<'_, MyClass> = obj.extract()?;
|
|||
|
||||
### `PyTuple`, `PyDict`, and many more
|
||||
|
||||
**Represents:** a native Python object of known type, restricted to a GIL
|
||||
lifetime just like `PyAny`.
|
||||
**Represented:** a native Python object of known type. In the GIL Refs API, they were only accessed as the GIL Refs `&'py PyTuple`, `&'py PyDict`.
|
||||
|
||||
**Used:** Whenever you want to operate with native Python types while holding
|
||||
the GIL. Like `PyAny`, this is the most convenient form to use for function
|
||||
arguments and intermediate values.
|
||||
**Used:** `&'py PyTuple` and similar were used to operate with native Python types while holding the GIL. Like `PyAny`, this is the most convenient form to use for function arguments and intermediate values.
|
||||
|
||||
These types all implement `Deref<Target = PyAny>`, so they all expose the same
|
||||
methods which can be found on `PyAny`.
|
||||
These GIL Refs implement `Deref<Target = PyAny>`, so they all expose the same methods which can be found on `PyAny`.
|
||||
|
||||
To see all Python types exposed by `PyO3` you should consult the
|
||||
[`pyo3::types`][pyo3::types] module.
|
||||
To see all Python types exposed by `PyO3` consult the [`pyo3::types`][pyo3::types] module.
|
||||
|
||||
**Conversions:**
|
||||
|
||||
|
|
@ -136,7 +354,7 @@ To see all Python types exposed by `PyO3` you should consult the
|
|||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
#[allow(deprecated)] // PyList::empty is part of the deprecated "GIL Refs" API.
|
||||
#[allow(deprecated)] // PyList::empty is part of the deprecated "GIL Refs" API.
|
||||
let list = PyList::empty(py);
|
||||
|
||||
// Use methods from PyAny on all Python types with Deref implementation
|
||||
|
|
@ -146,7 +364,7 @@ let _ = list.repr()?;
|
|||
let _: &PyAny = list;
|
||||
|
||||
// To &PyAny explicitly with .as_ref()
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let _: &PyAny = list.as_ref();
|
||||
|
||||
// To Py<T> with .into() or Py::from()
|
||||
|
|
@ -160,7 +378,7 @@ let _: PyObject = list.into();
|
|||
|
||||
### `Py<T>` and `PyObject`
|
||||
|
||||
**Represents:** a GIL-independent reference to a Python object. This can be a Python native type
|
||||
**Represented:** a GIL-independent reference to a Python object. This can be a Python native type
|
||||
(like `PyTuple`), or a `pyclass` type implemented in Rust. The most commonly-used variant,
|
||||
`Py<PyAny>`, is also known as `PyObject`.
|
||||
|
||||
|
|
@ -170,86 +388,23 @@ Python-Rust FFI boundary, or returning objects from functions implemented in Rus
|
|||
|
||||
Can be cloned using Python reference counts with `.clone()`.
|
||||
|
||||
**Conversions:**
|
||||
|
||||
For a `Py<PyList>`, the conversions are as below:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# use pyo3::types::PyList;
|
||||
# Python::with_gil(|py| {
|
||||
let list: Py<PyList> = PyList::empty_bound(py).unbind();
|
||||
|
||||
// To &PyList with Py::as_ref() (borrows from the Py)
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let _: &PyList = list.as_ref(py);
|
||||
|
||||
# let list_clone = list.clone(); // Because `.into_ref()` will consume `list`.
|
||||
// To &PyList with Py::into_ref() (moves the pointer into PyO3's object storage)
|
||||
# #[allow(deprecated)]
|
||||
let _: &PyList = list.into_ref(py);
|
||||
|
||||
# let list = list_clone;
|
||||
// To Py<PyAny> (aka PyObject) with .into()
|
||||
let _: Py<PyAny> = list.into();
|
||||
# })
|
||||
```
|
||||
|
||||
For a `#[pyclass] struct MyClass`, the conversions for `Py<MyClass>` are below:
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# Python::with_gil(|py| {
|
||||
# #[pyclass] struct MyClass { }
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
let my_class: Py<MyClass> = Py::new(py, MyClass { })?;
|
||||
|
||||
// To &PyCell<MyClass> with Py::as_ref() (borrows from the Py)
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let _: &PyCell<MyClass> = my_class.as_ref(py);
|
||||
|
||||
# let my_class_clone = my_class.clone(); // Because `.into_ref()` will consume `my_class`.
|
||||
// To &PyCell<MyClass> with Py::into_ref() (moves the pointer into PyO3's object storage)
|
||||
# #[allow(deprecated)]
|
||||
let _: &PyCell<MyClass> = my_class.into_ref(py);
|
||||
|
||||
# let my_class = my_class_clone.clone();
|
||||
// To Py<PyAny> (aka PyObject) with .into_py(py)
|
||||
let _: Py<PyAny> = my_class.into_py(py);
|
||||
|
||||
# let my_class = my_class_clone;
|
||||
// To PyRef<'_, MyClass> with Py::borrow or Py::try_borrow
|
||||
let _: PyRef<'_, MyClass> = my_class.try_borrow(py)?;
|
||||
|
||||
// To PyRefMut<'_, MyClass> with Py::borrow_mut or Py::try_borrow_mut
|
||||
let _: PyRefMut<'_, MyClass> = my_class.try_borrow_mut(py)?;
|
||||
# Ok(())
|
||||
# }).unwrap();
|
||||
# });
|
||||
```
|
||||
|
||||
### `PyCell<SomeType>`
|
||||
|
||||
**Represents:** a reference to a Rust object (instance of `PyClass`) which is
|
||||
wrapped in a Python object. The cell part is an analog to stdlib's
|
||||
[`RefCell`][RefCell] to allow access to `&mut` references.
|
||||
**Represented:** a reference to a Rust object (instance of `PyClass`) wrapped in a Python object. The cell part is an analog to stdlib's [`RefCell`][RefCell] to allow access to `&mut` references.
|
||||
|
||||
**Used:** for accessing pure-Rust API of the instance (members and functions
|
||||
taking `&SomeType` or `&mut SomeType`) while maintaining the aliasing rules of
|
||||
Rust references.
|
||||
**Used:** for accessing pure-Rust API of the instance (members and functions taking `&SomeType` or `&mut SomeType`) while maintaining the aliasing rules of Rust references.
|
||||
|
||||
Like PyO3's Python native types, `PyCell<T>` implements `Deref<Target = PyAny>`,
|
||||
so it also exposes all of the methods on `PyAny`.
|
||||
Like PyO3's Python native types, the GIL Ref `&PyCell<T>` implements `Deref<Target = PyAny>`, so it also exposed all of the methods on `PyAny`.
|
||||
|
||||
**Conversions:**
|
||||
|
||||
`PyCell<T>` can be used to access `&T` and `&mut T` via `PyRef<T>` and `PyRefMut<T>` respectively.
|
||||
`PyCell<T>` was used to access `&T` and `&mut T` via `PyRef<T>` and `PyRefMut<T>` respectively.
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# #[pyclass] struct MyClass { }
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
# #[allow(deprecated)]
|
||||
#[allow(deprecated)] // &PyCell is part of the deprecated GIL Refs API
|
||||
let cell: &PyCell<MyClass> = PyCell::new(py, MyClass {})?;
|
||||
|
||||
// To PyRef<T> with .borrow() or .try_borrow()
|
||||
|
|
@ -264,13 +419,13 @@ let _: &mut MyClass = &mut *py_ref_mut;
|
|||
# }).unwrap();
|
||||
```
|
||||
|
||||
`PyCell<T>` can also be accessed like a Python-native type.
|
||||
`PyCell<T>` was also accessed like a Python-native type.
|
||||
|
||||
```rust
|
||||
# use pyo3::prelude::*;
|
||||
# #[pyclass] struct MyClass { }
|
||||
# Python::with_gil(|py| -> PyResult<()> {
|
||||
# #[allow(deprecated)]
|
||||
#[allow(deprecated)] // &PyCell is part of the deprecate GIL Refs API
|
||||
let cell: &PyCell<MyClass> = PyCell::new(py, MyClass {})?;
|
||||
|
||||
// Use methods from PyAny on PyCell<T> with Deref implementation
|
||||
|
|
@ -280,37 +435,30 @@ let _ = cell.repr()?;
|
|||
let _: &PyAny = cell;
|
||||
|
||||
// To &PyAny explicitly with .as_ref()
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
#[allow(deprecated)] // as_ref is part of the deprecated "GIL Refs" API.
|
||||
let _: &PyAny = cell.as_ref();
|
||||
# Ok(())
|
||||
# }).unwrap();
|
||||
```
|
||||
|
||||
### `PyRef<SomeType>` and `PyRefMut<SomeType>`
|
||||
|
||||
**Represents:** reference wrapper types employed by `PyCell` to keep track of
|
||||
borrows, analog to `Ref` and `RefMut` used by `RefCell`.
|
||||
|
||||
**Used:** while borrowing a `PyCell`. They can also be used with `.extract()`
|
||||
on types like `Py<T>` and `PyAny` to get a reference quickly.
|
||||
|
||||
|
||||
## Related traits and types
|
||||
|
||||
### `PyClass`
|
||||
|
||||
This trait marks structs defined in Rust that are also usable as Python classes,
|
||||
usually defined using the `#[pyclass]` macro.
|
||||
|
||||
### `PyNativeType`
|
||||
|
||||
This trait marks structs that mirror native Python types, such as `PyList`.
|
||||
|
||||
|
||||
[Bound]: {{#PYO3_DOCS_URL}}/pyo3/struct.Bound.html
|
||||
[`Bound::unbind`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Bound.html#method.unbind
|
||||
[Py]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html
|
||||
[PyAnyMethods::add]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.add
|
||||
[PyAnyMethods::extract]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.extract
|
||||
[PyAnyMethods::downcast]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.downcast
|
||||
[PyAnyMethods::downcast_into]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html#tymethod.downcast_into
|
||||
[`PyTypeCheck`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeCheck.html
|
||||
[`PyAnyMethods`]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyAnyMethods.html
|
||||
[`PyDictMethods`]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyDictMethods.html
|
||||
[`PyTupleMethods`]: {{#PYO3_DOCS_URL}}/pyo3/types/trait.PyTupleMethods.html
|
||||
[pyclass]: class.md
|
||||
[Borrowed]: {{#PYO3_DOCS_URL}}/pyo3/struct.Borrowed.html
|
||||
[Drop]: https://doc.rust-lang.org/std/ops/trait.Drop.html
|
||||
[eval]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.eval
|
||||
[clone_ref]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html#method.clone_ref
|
||||
[pyo3::types]: {{#PYO3_DOCS_URL}}/pyo3/types/index.html
|
||||
[PyAny]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html
|
||||
[PyList_append]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyList.html#method.append
|
||||
[RefCell]: https://doc.rust-lang.org/std/cell/struct.RefCell.html
|
||||
[obtaining-py]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#obtaining-a-python-token
|
||||
[smart-pointers]: https://doc.rust-lang.org/book/ch15-00-smart-pointers.html
|
||||
|
|
|
|||
46
src/lib.rs
46
src/lib.rs
|
|
@ -30,29 +30,38 @@
|
|||
//!
|
||||
//! PyO3 has several core types that you should familiarize yourself with:
|
||||
//!
|
||||
//! ## The Python<'py> object
|
||||
//! ## The `Python<'py>` object, and the `'py` lifetime
|
||||
//!
|
||||
//! Holding the [global interpreter lock] (GIL) is modeled with the [`Python<'py>`](crate::Python)
|
||||
//! token. All APIs that require that the GIL is held require this token as proof that you really
|
||||
//! are holding the GIL. It can be explicitly acquired and is also implicitly acquired by PyO3 as
|
||||
//! it wraps Rust functions and structs into Python functions and objects.
|
||||
//! Holding the [global interpreter lock] (GIL) is modeled with the [`Python<'py>`](Python) token. Many
|
||||
//! Python APIs require that the GIL is held, and PyO3 uses this token as proof that these APIs
|
||||
//! can be called safely. It can be explicitly acquired and is also implicitly acquired by PyO3
|
||||
//! as it wraps Rust functions and structs into Python functions and objects.
|
||||
//!
|
||||
//! ## The GIL-dependent types
|
||||
//! The [`Python<'py>`](Python) token's lifetime `'py` is common to many PyO3 APIs:
|
||||
//! - Types that also have the `'py` lifetime, such as the [`Bound<'py, T>`](Bound) smart pointer, are
|
||||
//! bound to the Python GIL and rely on this to offer their functionality. These types often
|
||||
//! have a [`.py()`](Bound::py) method to get the associated [`Python<'py>`](Python) token.
|
||||
//! - Functions which depend on the `'py` lifetime, such as [`PyList::new_bound`](types::PyList::new_bound),
|
||||
//! require a [`Python<'py>`](Python) token as an input. Sometimes the token is passed implicitly by
|
||||
//! taking a [`Bound<'py, T>`](Bound) or other type which is bound to the `'py` lifetime.
|
||||
//! - Traits which depend on the `'py` lifetime, such as [`FromPyObject<'py>`](FromPyObject), usually have
|
||||
//! inputs or outputs which depend on the lifetime. Adding the lifetime to the trait allows
|
||||
//! these inputs and outputs to express their binding to the GIL in the Rust type system.
|
||||
//!
|
||||
//! For example `&`[`PyAny`]. These are only ever seen as references, with a lifetime that is only
|
||||
//! valid for as long as the GIL is held, which is why using them doesn't require a
|
||||
//! [`Python<'py>`](crate::Python) token. The underlying Python object, if mutable, can be mutated
|
||||
//! through any reference.
|
||||
//! ## Python object smart pointers
|
||||
//!
|
||||
//! PyO3 has two core smart pointers to refer to Python objects, [`Py<T>`](Py) and its GIL-bound
|
||||
//! form [`Bound<'py, T>`](Bound) which carries the `'py` lifetime. (There is also
|
||||
//! [`Borrowed<'a, 'py, T>`](instance::Borrowed), but it is used much more rarely).
|
||||
//!
|
||||
//! The type parameter `T` in these smart pointers can be filled by:
|
||||
//! - [`PyAny`], e.g. `Py<PyAny>` or `Bound<'py, PyAny>`, where the Python object type is not
|
||||
//! known. `Py<PyAny>` is so common it has a type alias [`PyObject`].
|
||||
//! - Concrete Python types like [`PyList`](types::PyList) or [`PyTuple`](types::PyTuple).
|
||||
//! - Rust types which are exposed to Python using the [`#[pyclass]`](macro@pyclass) macro.
|
||||
//!
|
||||
//! See the [guide][types] for an explanation of the different Python object types.
|
||||
//!
|
||||
//! ## The GIL-independent types
|
||||
//!
|
||||
//! When wrapped in [`Py`]`<...>`, like with [`Py`]`<`[`PyAny`]`>` or [`Py`]`<SomePyClass>`, Python
|
||||
//! objects no longer have a limited lifetime which makes them easier to store in structs and pass
|
||||
//! between functions. However, you cannot do much with them without a
|
||||
//! [`Python<'py>`](crate::Python) token, for which you’d need to reacquire the GIL.
|
||||
//!
|
||||
//! ## PyErr
|
||||
//!
|
||||
//! The vast majority of operations in this library will return [`PyResult<...>`](PyResult).
|
||||
|
|
@ -512,7 +521,10 @@ pub mod doc_test {
|
|||
"guide/src/parallelism.md" => guide_parallelism_md,
|
||||
"guide/src/performance.md" => guide_performance_md,
|
||||
"guide/src/python-from-rust.md" => guide_python_from_rust_md,
|
||||
"guide/src/python-from-rust/calling-existing-code.md" => guide_pfr_calling_existing_code_md,
|
||||
"guide/src/python-from-rust/function-calls.md" => guide_pfr_function_calls_md,
|
||||
"guide/src/python-typing-hints.md" => guide_python_typing_hints_md,
|
||||
"guide/src/rust-from-python.md" => guide_rust_from_python_md,
|
||||
"guide/src/trait-bounds.md" => guide_trait_bounds_md,
|
||||
"guide/src/types.md" => guide_types_md,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,9 @@
|
|||
//! Synchronization mechanisms based on the Python GIL.
|
||||
//!
|
||||
//! With the acceptance of [PEP 703] (aka a "freethreaded Python") for Python 3.13, these
|
||||
//! are likely to undergo significant developments in the future.
|
||||
//!
|
||||
//! [PEP 703]: https://peps.python.org/pep-703/
|
||||
use crate::{
|
||||
types::{any::PyAnyMethods, PyString, PyType},
|
||||
Bound, Py, PyResult, PyVisit, Python,
|
||||
|
|
|
|||
Loading…
Reference in New Issue