Apply suggestions from birkenfeld
Co-authored-by: Georg Brandl <georg@python.org>
This commit is contained in:
parent
a591b55219
commit
7c1cc29612
|
@ -1,18 +1,18 @@
|
|||
<!-- This file contains a rough overview of PyO3 codebase. -->
|
||||
<!-- This file contains a rough overview of the PyO3 codebase. -->
|
||||
<!-- Please do not make descriptions too specific, so that we can easily -->
|
||||
<!-- keep this file in sync with the codebase. -->
|
||||
|
||||
# PyO3: Architecture.md
|
||||
|
||||
This document roughly describes the high-level architecture of PyO3.
|
||||
If you are to become familiar with the codebase, you are in the right place!
|
||||
If you want to become familiar with the codebase, you are in the right place!
|
||||
|
||||
## Overview
|
||||
|
||||
PyO3 provides a bridge between Rust and Python, based on the [Python C/API].
|
||||
Thus, PyO3 has low-level bindings of these API as its core.
|
||||
On top of that, we have higher-level bindings to operate Python objects safely.
|
||||
Also, to define Python class and functions in Rust code, we have `trait PyClass<T>` and a set of
|
||||
Also, to define Python classes and functions in Rust code, we have `trait PyClass<T>` and a set of
|
||||
protocol traits (e.g., `PyIterProtocol`) for supporting object protocols (i.e., `__dunder__` methods).
|
||||
Since implementing `PyClass` requires lots of boilerplates, we have a proc-macro `#[pyclass]`.
|
||||
|
||||
|
@ -25,18 +25,16 @@ To summarize, we have mainly four parts in the PyO3 codebase.
|
|||
- [`src/pycell.rs`], [`src/pyclass.rs`]
|
||||
4. Protocol methods like `__getitem__`.
|
||||
- [`src/class`]
|
||||
5. Defining a Python class requires lots of glue codes, so we provide proc-macros to simplify the procedure.
|
||||
5. Defining a Python class requires lots of glue code, so we provide proc-macros to simplify the procedure.
|
||||
- [`src/derive_utils.rs`]
|
||||
- [`pyo3-macros`], [`pyo3-macros-backend`]
|
||||
|
||||
## Low-level bindings of CPython API
|
||||
[`src/ffi`] contains wrappers of [Python C/API](https://docs.python.org/3/c-api/).
|
||||
|
||||
We aim to provide straight-forward Rust wrappers and resemble the file structure of the
|
||||
We aim to provide straight-forward Rust wrappers resembling the file structure of
|
||||
[`cpython/Include`](https://github.com/python/cpython/tree/v3.9.2/Include).
|
||||
|
||||
However, we still lack some API and continue to refactor the module to resemble the CPython's
|
||||
file structure completely.
|
||||
However, we still lack some API and continue to refactor the module to completely resemble
|
||||
the CPython's file structure.
|
||||
The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and contribution is welcome.
|
||||
|
@ -44,13 +42,12 @@ The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and con
|
|||
## Bindings to Python Objects
|
||||
[`src/types`] contains bindings to [built-in types](https://docs.python.org/3/library/stdtypes.html)
|
||||
of Python, such as `dict` and `list`.
|
||||
Due to a historical reason, Python's `object` is called `PyAny` and placed in [`src/types/any.rs`].
|
||||
Due to historical reasons, Python's `object` is called `PyAny` and placed in [`src/types/any.rs`].
|
||||
Currently, `PyAny` is a straight-forward wrapper of `ffi::PyObject`, like:
|
||||
```rust
|
||||
#[repr(transparent)]
|
||||
pub struct PyAny(UnsafeCell<ffi::PyObject>);
|
||||
```
|
||||
.
|
||||
|
||||
All built-in types are defined as a C struct.
|
||||
For example, `dict` is defined as:
|
||||
|
@ -66,7 +63,6 @@ typedef struct {
|
|||
PyObject **ma_values;
|
||||
} PyDictObject;
|
||||
```
|
||||
.
|
||||
|
||||
However, we cannot access such a specific data structure with `#[cfg(Py_LIMITED_API)]` set.
|
||||
Thus, all builtin objects are implemented as opaque types by wrapping `PyAny`, like:
|
||||
|
@ -80,7 +76,7 @@ Python heap, as `&PyAny`.
|
|||
This design choice can be changed
|
||||
(see the discussion in [#1056](https://github.com/PyO3/pyo3/issues/1056)).
|
||||
|
||||
Since we need lots of boilerplates for implementing common traits for these types
|
||||
Since we need lots of boilerplate for implementing common traits for these types
|
||||
(e.g., `AsPyPointer`, `AsRef<PyAny>`, and `Debug`), we have some macros in
|
||||
[`src/types/mod.rs`].
|
||||
|
||||
|
@ -99,9 +95,9 @@ pub struct PyObject {
|
|||
...
|
||||
}
|
||||
```
|
||||
Thanks to this gurantee, casting `*mut A` to `*mut PyObject` is valid if `A` is a Python object.
|
||||
Thanks to this guarantee, casting `*mut A` to `*mut PyObject` is valid if `A` is a Python object.
|
||||
|
||||
To ensure this gurantee, we have a wrapper struct `PyCell<T>` in [`src/pycell.rs`] which is roughly:
|
||||
To ensure this guarantee, we have a wrapper struct `PyCell<T>` in [`src/pycell.rs`] which is roughly:
|
||||
```rust
|
||||
#[repr(C)]
|
||||
pub struct PyCell<T: PyClass> {
|
||||
|
@ -109,9 +105,8 @@ pub struct PyCell<T: PyClass> {
|
|||
inner: T,
|
||||
}
|
||||
```
|
||||
.
|
||||
Thus, when copying a Rust struct to a Python object, we first allocate `PyCell` on a Python heap and then
|
||||
copies `T`.
|
||||
Thus, when copying a Rust struct to a Python object, we first allocate `PyCell` on the Python heap and then
|
||||
copy `T`.
|
||||
Also, `PyCell` provides [RefCell](https://doc.rust-lang.org/std/cell/struct.RefCell.html)-like methods
|
||||
to ensure Rust's borrow rules.
|
||||
See [the document](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) for more.
|
||||
|
@ -120,9 +115,9 @@ See [the document](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) f
|
|||
This trait is somewhat complex and derives many traits, but the most important one is `PyTypeObject`
|
||||
in [`src/type_object.rs`].
|
||||
`PyTypeObject` is also implemented for built-in types.
|
||||
Type objects are singleton, and all Python types have their unique type objects.
|
||||
Type objects are singletons, and all Python types have their unique type objects.
|
||||
For example, you can see `type({})` shows `dict` and `type(type({}))` shows `type` in Python REPL.
|
||||
`T: PyTypeObject` implies that `T` has a corresponding type object
|
||||
`T: PyTypeObject` implies that `T` has a corresponding type object.
|
||||
|
||||
## Protocol methods
|
||||
Python has some built-in special methods called dunder, such as `__iter__`.
|
||||
|
@ -132,10 +127,10 @@ We provide a way to implement those protocols by using `#[pyproto]` and specific
|
|||
as `PyIterProtocol`.
|
||||
[`src/class`] defines these traits.
|
||||
Each protocol method has a corresponding FFI function.
|
||||
For example, when `PyIterProtocol::__iter__` has
|
||||
For example, `PyIterProtocol::__iter__` has
|
||||
`pub unsafe extern "C" fn iter<T>(slf: *mut PyObject) -> *mut PyObject`.
|
||||
When `#[pyproto]` finds that `T` implements `PyIterProtocol::__iter__`, it automatically
|
||||
set `iter<T>` to the type object of `T`.
|
||||
sets `iter<T>` on the type object of `T`.
|
||||
|
||||
Also, [`src/class/methods.rs`] has utilities for `#[pyfunction]` and [`src/class/impl_.rs`] has
|
||||
some internal tricks for making `#[pyproto]` flexible.
|
||||
|
@ -143,7 +138,7 @@ some internal tricks for making `#[pyproto]` flexible.
|
|||
## Proc-macros
|
||||
[`pyo3-macros`] provides six proc-macro APIs: `pymodule`, `pyproto`, `pyfunction`, `pyclass`,
|
||||
`pymethods`, and `#[derive(FromPyObject)]`.
|
||||
[`pyo3-macros-backend`] has actual implementations of these APIs.
|
||||
[`pyo3-macros-backend`] has the actual implementations of these APIs.
|
||||
[`src/derive_utils.rs`] contains some utilities used in codes generated by these proc-macros,
|
||||
such as parsing function arguments.
|
||||
|
||||
|
|
|
@ -14,7 +14,7 @@
|
|||
|
||||
* Contributing Notes: [github](https://github.com/PyO3/pyo3/blob/master/Contributing.md)
|
||||
|
||||
* Berif Architecture Guide: [github](https://github.com/PyO3/pyo3/blob/master/Architecture.md)
|
||||
* Brief Architecture Guide: [github](https://github.com/PyO3/pyo3/blob/master/Architecture.md)
|
||||
|
||||
A comparison with rust-cpython can be found [in the guide](https://pyo3.rs/master/rust_cpython.html).
|
||||
|
||||
|
|
Loading…
Reference in New Issue