From 7c1cc29612edecac979292584fd41c298691e4de Mon Sep 17 00:00:00 2001 From: Yuji Kanagawa Date: Sat, 6 Mar 2021 23:59:46 +0900 Subject: [PATCH] Apply suggestions from birkenfeld Co-authored-by: Georg Brandl --- Architecture.md | 37 ++++++++++++++++--------------------- README.md | 2 +- 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/Architecture.md b/Architecture.md index 315b8f00..b1205e89 100644 --- a/Architecture.md +++ b/Architecture.md @@ -1,18 +1,18 @@ - + # PyO3: Architecture.md This document roughly describes the high-level architecture of PyO3. -If you are to become familiar with the codebase, you are in the right place! +If you want to become familiar with the codebase, you are in the right place! ## Overview PyO3 provides a bridge between Rust and Python, based on the [Python C/API]. Thus, PyO3 has low-level bindings of these API as its core. On top of that, we have higher-level bindings to operate Python objects safely. -Also, to define Python class and functions in Rust code, we have `trait PyClass` and a set of +Also, to define Python classes and functions in Rust code, we have `trait PyClass` and a set of protocol traits (e.g., `PyIterProtocol`) for supporting object protocols (i.e., `__dunder__` methods). Since implementing `PyClass` requires lots of boilerplates, we have a proc-macro `#[pyclass]`. @@ -25,18 +25,16 @@ To summarize, we have mainly four parts in the PyO3 codebase. - [`src/pycell.rs`], [`src/pyclass.rs`] 4. Protocol methods like `__getitem__`. - [`src/class`] -5. Defining a Python class requires lots of glue codes, so we provide proc-macros to simplify the procedure. +5. Defining a Python class requires lots of glue code, so we provide proc-macros to simplify the procedure. - [`src/derive_utils.rs`] - [`pyo3-macros`], [`pyo3-macros-backend`] ## Low-level bindings of CPython API [`src/ffi`] contains wrappers of [Python C/API](https://docs.python.org/3/c-api/). -We aim to provide straight-forward Rust wrappers and resemble the file structure of the +We aim to provide straight-forward Rust wrappers resembling the file structure of [`cpython/Include`](https://github.com/python/cpython/tree/v3.9.2/Include). -However, we still lack some API and continue to refactor the module to resemble the CPython's -file structure completely. However, we still lack some API and continue to refactor the module to completely resemble the CPython's file structure. The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and contribution is welcome. @@ -44,13 +42,12 @@ The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and con ## Bindings to Python Objects [`src/types`] contains bindings to [built-in types](https://docs.python.org/3/library/stdtypes.html) of Python, such as `dict` and `list`. -Due to a historical reason, Python's `object` is called `PyAny` and placed in [`src/types/any.rs`]. +Due to historical reasons, Python's `object` is called `PyAny` and placed in [`src/types/any.rs`]. Currently, `PyAny` is a straight-forward wrapper of `ffi::PyObject`, like: ```rust #[repr(transparent)] pub struct PyAny(UnsafeCell); ``` -. All built-in types are defined as a C struct. For example, `dict` is defined as: @@ -66,7 +63,6 @@ typedef struct { PyObject **ma_values; } PyDictObject; ``` -. However, we cannot access such a specific data structure with `#[cfg(Py_LIMITED_API)]` set. Thus, all builtin objects are implemented as opaque types by wrapping `PyAny`, like: @@ -80,7 +76,7 @@ Python heap, as `&PyAny`. This design choice can be changed (see the discussion in [#1056](https://github.com/PyO3/pyo3/issues/1056)). -Since we need lots of boilerplates for implementing common traits for these types +Since we need lots of boilerplate for implementing common traits for these types (e.g., `AsPyPointer`, `AsRef`, and `Debug`), we have some macros in [`src/types/mod.rs`]. @@ -99,9 +95,9 @@ pub struct PyObject { ... } ``` -Thanks to this gurantee, casting `*mut A` to `*mut PyObject` is valid if `A` is a Python object. +Thanks to this guarantee, casting `*mut A` to `*mut PyObject` is valid if `A` is a Python object. -To ensure this gurantee, we have a wrapper struct `PyCell` in [`src/pycell.rs`] which is roughly: +To ensure this guarantee, we have a wrapper struct `PyCell` in [`src/pycell.rs`] which is roughly: ```rust #[repr(C)] pub struct PyCell { @@ -109,9 +105,8 @@ pub struct PyCell { inner: T, } ``` -. -Thus, when copying a Rust struct to a Python object, we first allocate `PyCell` on a Python heap and then -copies `T`. +Thus, when copying a Rust struct to a Python object, we first allocate `PyCell` on the Python heap and then +copy `T`. Also, `PyCell` provides [RefCell](https://doc.rust-lang.org/std/cell/struct.RefCell.html)-like methods to ensure Rust's borrow rules. See [the document](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) for more. @@ -120,9 +115,9 @@ See [the document](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) f This trait is somewhat complex and derives many traits, but the most important one is `PyTypeObject` in [`src/type_object.rs`]. `PyTypeObject` is also implemented for built-in types. -Type objects are singleton, and all Python types have their unique type objects. +Type objects are singletons, and all Python types have their unique type objects. For example, you can see `type({})` shows `dict` and `type(type({}))` shows `type` in Python REPL. -`T: PyTypeObject` implies that `T` has a corresponding type object +`T: PyTypeObject` implies that `T` has a corresponding type object. ## Protocol methods Python has some built-in special methods called dunder, such as `__iter__`. @@ -132,10 +127,10 @@ We provide a way to implement those protocols by using `#[pyproto]` and specific as `PyIterProtocol`. [`src/class`] defines these traits. Each protocol method has a corresponding FFI function. -For example, when `PyIterProtocol::__iter__` has +For example, `PyIterProtocol::__iter__` has `pub unsafe extern "C" fn iter(slf: *mut PyObject) -> *mut PyObject`. When `#[pyproto]` finds that `T` implements `PyIterProtocol::__iter__`, it automatically -set `iter` to the type object of `T`. +sets `iter` on the type object of `T`. Also, [`src/class/methods.rs`] has utilities for `#[pyfunction]` and [`src/class/impl_.rs`] has some internal tricks for making `#[pyproto]` flexible. @@ -143,7 +138,7 @@ some internal tricks for making `#[pyproto]` flexible. ## Proc-macros [`pyo3-macros`] provides six proc-macro APIs: `pymodule`, `pyproto`, `pyfunction`, `pyclass`, `pymethods`, and `#[derive(FromPyObject)]`. -[`pyo3-macros-backend`] has actual implementations of these APIs. +[`pyo3-macros-backend`] has the actual implementations of these APIs. [`src/derive_utils.rs`] contains some utilities used in codes generated by these proc-macros, such as parsing function arguments. diff --git a/README.md b/README.md index 9657e724..fe349ddc 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ * Contributing Notes: [github](https://github.com/PyO3/pyo3/blob/master/Contributing.md) -* Berif Architecture Guide: [github](https://github.com/PyO3/pyo3/blob/master/Architecture.md) +* Brief Architecture Guide: [github](https://github.com/PyO3/pyo3/blob/master/Architecture.md) A comparison with rust-cpython can be found [in the guide](https://pyo3.rs/master/rust_cpython.html).