guide: use sentence case, move function signatures to own section

This commit is contained in:
David Hewitt 2022-08-21 14:10:49 +01:00
parent af60a359c5
commit 580e747521
15 changed files with 177 additions and 125 deletions

View File

@ -4,35 +4,36 @@
--- ---
- [Getting Started](getting_started.md) - [Getting started](getting_started.md)
- [Python Modules](module.md) - [Python modules](module.md)
- [Python Functions](function.md) - [Python functions](function.md)
- [Python Classes](class.md) - [Function signatures](function/signature.md)
- [Python classes](class.md)
- [Class customizations](class/protocols.md) - [Class customizations](class/protocols.md)
- [Basic object customization](class/object.md) - [Basic object customization](class/object.md)
- [Emulating numeric types](class/numeric.md) - [Emulating numeric types](class/numeric.md)
- [Emulating callable objects](class/call.md) - [Emulating callable objects](class/call.md)
- [Type Conversions](conversions.md) - [Type conversions](conversions.md)
- [Mapping of Rust types to Python types](conversions/tables.md)] - [Mapping of Rust types to Python types](conversions/tables.md)]
- [Conversion traits](conversions/traits.md)] - [Conversion traits](conversions/traits.md)]
- [Python Exceptions](exception.md) - [Python exceptions](exception.md)
- [Calling Python from Rust](python_from_rust.md) - [Calling Python from Rust](python_from_rust.md)
- [GIL, mutability and object types](types.md) - [GIL, mutability and object types](types.md)
- [Parallelism](parallelism.md) - [Parallelism](parallelism.md)
- [Debugging](debugging.md) - [Debugging](debugging.md)
- [Features Reference](features.md) - [Features reference](features.md)
- [Memory Management](memory.md) - [Memory management](memory.md)
- [Advanced Topics](advanced.md) - [Advanced topics](advanced.md)
- [Building and Distribution](building_and_distribution.md) - [Building and distribution](building_and_distribution.md)
- [Supporting multiple Python versions](building_and_distribution/multiple_python_versions.md) - [Supporting multiple Python versions](building_and_distribution/multiple_python_versions.md)
- [Useful Crates](ecosystem.md) - [Useful crates](ecosystem.md)
- [Logging](ecosystem/logging.md) - [Logging](ecosystem/logging.md)
- [Async / Await](ecosystem/async-await.md) - [Using `async` and `await`](ecosystem/async-await.md)
- [FAQ & Troubleshooting](faq.md) - [FAQ and troubleshooting](faq.md)
--- ---
[Appendix A: Migration Guide](migration.md) [Appendix A: Migration guide](migration.md)
[Appendix B: PyO3 and rust-cpython](rust_cpython.md) [Appendix B: PyO3 and rust-cpython](rust_cpython.md)
[Appendix C: Trait bounds](trait_bounds.md) [Appendix C: Trait bounds](trait_bounds.md)
[Appendix D: Python typing hints](python_typing_hints.md) [Appendix D: Python typing hints](python_typing_hints.md)

View File

@ -6,7 +6,7 @@ PyO3 exposes much of Python's C API through the `ffi` module.
The C API is naturally unsafe and requires you to manage reference counts, errors and specific invariants yourself. Please refer to the [C API Reference Manual](https://docs.python.org/3/c-api/) and [The Rustonomicon](https://doc.rust-lang.org/nightly/nomicon/ffi.html) before using any function from that API. The C API is naturally unsafe and requires you to manage reference counts, errors and specific invariants yourself. Please refer to the [C API Reference Manual](https://docs.python.org/3/c-api/) and [The Rustonomicon](https://doc.rust-lang.org/nightly/nomicon/ffi.html) before using any function from that API.
## Memory Management ## Memory management
PyO3's `&PyAny` "owned references" and `Py<PyAny>` smart pointers are used to PyO3's `&PyAny` "owned references" and `Py<PyAny>` smart pointers are used to
access memory stored in Python's heap. This memory sometimes lives for longer access memory stored in Python's heap. This memory sometimes lives for longer

View File

@ -1,4 +1,4 @@
# Building and Distribution # Building and distribution
This chapter of the guide goes into detail on how to build and distribute projects using PyO3. The way to achieve this is very different depending on whether the project is a Python module implemented in Rust, or a Rust binary embedding Python. For both types of project there are also common problems such as the Python version to build for and the [linker](https://en.wikipedia.org/wiki/Linker_(computing)) arguments to use. This chapter of the guide goes into detail on how to build and distribute projects using PyO3. The way to achieve this is very different depending on whether the project is a Python module implemented in Rust, or a Rust binary embedding Python. For both types of project there are also common problems such as the Python version to build for and the [linker](https://en.wikipedia.org/wiki/Linker_(computing)) arguments to use.

View File

@ -1,4 +1,4 @@
# Python Classes # Python classes
PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs. PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs.
@ -614,29 +614,10 @@ impl MyClass {
## Method arguments ## Method arguments
By default, PyO3 uses function signatures to determine which arguments are required. Then it scans Similar to `#[pyfunction]`, the `#[args]` attribute can be used to specify the way that `#[pymethods]` accept arguments. Consult the documentation for [`function signatures`](./function/signature.md) to see the parameters this attribute accepts.
the incoming `args` and `kwargs` parameters. If it can not find all required
parameters, it raises a `TypeError` exception. It is possible to override the default behavior
with the `#[args(...)]` attribute. This attribute accepts a comma separated list of parameters in
the form of `attr_name="default value"`. Each parameter has to match the method parameter by name.
Each parameter can be one of the following types: The following example defines a class `MyClass` with a method `method`. This method has an `#[args]` attribute which sets default values for `num` and `name`, and indicates that `py_args` should collect all extra positional arguments and `py_kwargs` all extra keyword arguments:
* `"/"`: positional-only arguments separator, each parameter defined before `"/"` is a
positional-only parameter.
Corresponds to python's `def meth(arg1, arg2, ..., /, argN..)`.
* `"*"`: var arguments separator, each parameter defined after `"*"` is a keyword-only parameter.
Corresponds to python's `def meth(*, arg1.., arg2=..)`.
* `args="*"`: "args" is var args, corresponds to Python's `def meth(*args)`. Type of the `args`
parameter has to be `&PyTuple`.
* `kwargs="**"`: "kwargs" receives keyword arguments, corresponds to Python's `def meth(**kwargs)`.
The type of the `kwargs` parameter has to be `Option<&PyDict>`.
* `arg="Value"`: arguments with default value. Corresponds to Python's `def meth(arg=Value)`.
If the `arg` argument is defined after var arguments, it is treated as a keyword-only argument.
Note that `Value` has to be valid rust code, PyO3 just inserts it into the generated
code unmodified.
Example:
```rust ```rust
# use pyo3::prelude::*; # use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple}; use pyo3::types::{PyDict, PyTuple};
@ -665,35 +646,26 @@ impl MyClass {
name: &str, name: &str,
py_args: &PyTuple, py_args: &PyTuple,
py_kwargs: Option<&PyDict>, py_kwargs: Option<&PyDict>,
) -> PyResult<String> { ) -> String {
let num_before = self.num;
self.num = num; self.num = num;
Ok(format!( format!(
"py_args={:?}, py_kwargs={:?}, name={}, num={}", "py_args={:?}, py_kwargs={:?}, name={}, num={} num_before={}",
py_args, py_kwargs, name, self.num py_args, py_kwargs, name, self.num, num_before,
)) )
}
fn make_change(&mut self, num: i32) -> PyResult<String> {
self.num = num;
Ok(format!("num={}", self.num))
} }
} }
``` ```
N.B. the position of the `"/"` and `"*"` arguments (if included) control the system of handling positional and keyword arguments. In Python:
In Python this might be used like:
```python ```python
import mymodule >>> import mymodule
>>> mc = mymodule.MyClass()
mc = mymodule.MyClass() >>> print(mc.method(44, False, "World", 666, x=44, y=55))
print(mc.method(44, False, "World", 666, x=44, y=55)) py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44, num_before=-1
print(mc.method(num=-1, name="World")) >>> print(mc.method(num=-1, name="World"))
print(mc.make_change(44, False)) py_args=(), py_kwargs=None, name=World, num=-1, num_before=44
```
Produces output:
```text
py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44
py_args=(), py_kwargs=None, name=World, num=-1
num=44
num=-1
``` ```
## Making class method signatures available to Python ## Making class method signatures available to Python

View File

@ -1,3 +1,3 @@
# Type Conversions # Type conversions
In this portion of the guide we'll talk about the mapping of Python types to Rust types offered by PyO3, as well as the traits available to perform conversions between them. In this portion of the guide we'll talk about the mapping of Python types to Rust types offered by PyO3, as well as the traits available to perform conversions between them.

View File

@ -1,4 +1,4 @@
# The PyO3 Ecosystem # The PyO3 ecosystem
This portion of the guide is dedicated to crates which are external to the main PyO3 project and provide additional functionality you might find useful. This portion of the guide is dedicated to crates which are external to the main PyO3 project and provide additional functionality you might find useful.

View File

@ -1,4 +1,4 @@
# Async / Await # Using `async` and `await`
If you are working with a Python library that makes use of async functions or wish to provide If you are working with a Python library that makes use of async functions or wish to provide
Python bindings for an async Rust library, [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio) Python bindings for an async Rust library, [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio)

View File

@ -1,4 +1,4 @@
# Python Exceptions # Python exceptions
## Defining a new exception ## Defining a new exception

View File

@ -1,4 +1,4 @@
# Frequently Asked Questions / Troubleshooting # Frequently Asked Questions and troubleshooting
## I'm experiencing deadlocks using PyO3 with lazy_static or once_cell! ## I'm experiencing deadlocks using PyO3 with lazy_static or once_cell!

View File

@ -1,4 +1,4 @@
# Features Reference # Features reference
PyO3 provides a number of Cargo features to customise functionality. This chapter of the guide provides detail on each of them. PyO3 provides a number of Cargo features to customise functionality. This chapter of the guide provides detail on each of them.

View File

@ -1,4 +1,4 @@
# Python Functions # Python functions
The `#[pyfunction]` attribute is used to define a Python function from a Rust function. Once defined, the function needs to be added to a [module](./module.md) using the `wrap_pyfunction!` macro. The `#[pyfunction]` attribute is used to define a Python function from a Rust function. Once defined, the function needs to be added to a [module](./module.md) using the `wrap_pyfunction!` macro.
@ -19,17 +19,20 @@ fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> {
} }
``` ```
This chapter of the guide explains full usage of the `#[pyfunction]` attribute. The following topics are covered: This chapter of the guide explains full usage of the `#[pyfunction]` attribute. In this first section, the following topics are covered:
- [Function options](#function-options) - [Function options](#function-options)
- [`#[pyo3(name = "...")]`](#name) - [`#[pyo3(name = "...")]`](#name)
- [`#[pyo3(text_signature = "...")]`](#text_signature) - [`#[pyo3(text_signature = "...")]`](#text_signature)
- [`#[pyo3(pass_module)]`](#pass_module) - [`#[pyo3(pass_module)]`](#pass_module)
- [Argument parsing](#argument-parsing) - [Per-argument options](#per-argument-options)
- [`#[pyo3(from_py_with = "...")]`](#from_py_with)
- [Advanced function patterns](#advanced-function-patterns) - [Advanced function patterns](#advanced-function-patterns)
- [`#[pyfn]` shorthand](#pyfn-shorthand) - [`#[pyfn]` shorthand](#pyfn-shorthand)
There are also additional sections on the following topics:
- [Function Signatures](./function/signature.md)
## Function options ## Function options
The `#[pyo3]` attribute can be used to modify properties of the generated Python function. It can take any combination of the following options: The `#[pyo3]` attribute can be used to modify properties of the generated Python function. It can take any combination of the following options:
@ -118,27 +121,7 @@ The `#[pyo3]` attribute can be used to modify properties of the generated Python
} }
``` ```
## Argument parsing ## Per-argument options
The `#[pyfunction]` attribute supports specifying details of argument parsing. The details are given in the section ["Method arguments" of the Classes chapter](class.md#method-arguments). Here is an example for a function that accepts arbitrary keyword arguments (`**kwargs` in Python syntax) and returns the number that was passed:
```rust
use pyo3::prelude::*;
use pyo3::types::PyDict;
#[pyfunction(kwds="**")]
fn num_kwds(kwds: Option<&PyDict>) -> usize {
kwds.map_or(0, |dict| dict.len())
}
#[pymodule]
fn module_with_functions(py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(num_kwds, m)?).unwrap();
Ok(())
}
```
### Per-argument options
The `#[pyo3]` attribute can be used on individual arguments to modify properties of them in the generated function. It can take any combination of the following options: The `#[pyo3]` attribute can be used on individual arguments to modify properties of them in the generated function. It can take any combination of the following options:

View File

@ -0,0 +1,97 @@
# Function signatures
The `#[pyfunction]` attribute also accepts parameters to control how the generated Python function accepts arguments. Just like in Python, arguments can be positional-only, keyword-only, or accept either. `*args` lists and `**kwargs` dicts can also be accepted. These parameters also work for `#[pymethods]` which will be introduced in the [Python Classes](../class.md) section of the guide.
Like Python, by default PyO3 accepts all arguments as either positional or keyword arguments. The extra arguments to `#[pyfunction]` modify this behaviour. For example, below is a function that accepts arbitrary keyword arguments (`**kwargs` in Python syntax) and returns the number that was passed:
```rust
use pyo3::prelude::*;
use pyo3::types::PyDict;
#[pyfunction(kwds="**")]
fn num_kwds(kwds: Option<&PyDict>) -> usize {
kwds.map_or(0, |dict| dict.len())
}
#[pymodule]
fn module_with_functions(py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(num_kwds, m)?).unwrap();
Ok(())
}
```
The following parameters can be passed to the `#[pyfunction]` attribute:
* `"/"`: positional-only arguments separator, each parameter defined before `"/"` is a
positional-only parameter.
Corresponds to python's `def meth(arg1, arg2, ..., /, argN..)`.
* `"*"`: var arguments separator, each parameter defined after `"*"` is a keyword-only parameter.
Corresponds to python's `def meth(*, arg1.., arg2=..)`.
* `args="*"`: "args" is var args, corresponds to Python's `def meth(*args)`. Type of the `args`
parameter has to be `&PyTuple`.
* `kwargs="**"`: "kwargs" receives keyword arguments, corresponds to Python's `def meth(**kwargs)`.
The type of the `kwargs` parameter has to be `Option<&PyDict>`.
* `arg="Value"`: arguments with default value. Corresponds to Python's `def meth(arg=Value)`.
If the `arg` argument is defined after var arguments, it is treated as a keyword-only argument.
Note that `Value` has to be valid rust code, PyO3 just inserts it into the generated
code unmodified.
Example:
```rust
# use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};
#
# #[pyclass]
# struct MyClass {
# num: i32,
# }
#[pymethods]
impl MyClass {
#[new]
#[args(num = "-1")]
fn new(num: i32) -> Self {
MyClass { num }
}
#[args(
num = "10",
py_args = "*",
name = "\"Hello\"",
py_kwargs = "**"
)]
fn method(
&mut self,
num: i32,
name: &str,
py_args: &PyTuple,
py_kwargs: Option<&PyDict>,
) -> PyResult<String> {
self.num = num;
Ok(format!(
"py_args={:?}, py_kwargs={:?}, name={}, num={}",
py_args, py_kwargs, name, self.num
))
}
fn make_change(&mut self, num: i32) -> PyResult<String> {
self.num = num;
Ok(format!("num={}", self.num))
}
}
```
N.B. the position of the `"/"` and `"*"` arguments (if included) control the system of handling positional and keyword arguments. In Python:
```python
import mymodule
mc = mymodule.MyClass()
print(mc.method(44, False, "World", 666, x=44, y=55))
print(mc.method(num=-1, name="World"))
print(mc.make_change(44, False))
```
Produces output:
```text
py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44
py_args=(), py_kwargs=None, name=World, num=-1
num=44
num=-1
```

View File

@ -1,5 +1,5 @@
# Instalation # Installation
To get started using PyO3 you will need three things: a rust toolchain, a python environment, and a way to build. We'll cover each of these below. To get started using PyO3 you will need three things: a rust toolchain, a python environment, and a way to build. We'll cover each of these below.
@ -13,7 +13,6 @@ if you can run `rustc --version` and the version is high enough you're good to g
To use PyO3 you need at least Python 3.7. While you can simply use the default Python version on your system, it is recommended to use a virtual environment. To use PyO3 you need at least Python 3.7. While you can simply use the default Python version on your system, it is recommended to use a virtual environment.
## Virtualenvs ## Virtualenvs
While you can use any virtualenv manager you like, we recommend the use of `pyenv` especially if you want to develop or test for multiple different python versions, so that is what the examples in this book will use. The installation instructions for `pyenv` can be found [here](https://github.com/pyenv/pyenv#getting-pyenv). While you can use any virtualenv manager you like, we recommend the use of `pyenv` especially if you want to develop or test for multiple different python versions, so that is what the examples in this book will use. The installation instructions for `pyenv` can be found [here](https://github.com/pyenv/pyenv#getting-pyenv).
@ -22,8 +21,8 @@ Note that when using `pyenv` you should also set the following environment varia
```bash ```bash
PYTHON_CONFIGURE_OPTS="--enable-shared" PYTHON_CONFIGURE_OPTS="--enable-shared"
``` ```
### Building
### Building
There are a number of build and python package management systems such as [`setuptools-rust`](https://github.com/PyO3/setuptools-rust) or [manually](https://pyo3.rs/latest/building_and_distribution.html#manual-builds) we recommend the use of `maturin` which you can install [here](https://maturin.rs/installation.html). It is developed to work with PyO3 and is the most "batteries included" experience. `maturin` is just a python package so you can add it in any way that you install python packages. There are a number of build and python package management systems such as [`setuptools-rust`](https://github.com/PyO3/setuptools-rust) or [manually](https://pyo3.rs/latest/building_and_distribution.html#manual-builds) we recommend the use of `maturin` which you can install [here](https://maturin.rs/installation.html). It is developed to work with PyO3 and is the most "batteries included" experience. `maturin` is just a python package so you can add it in any way that you install python packages.
@ -50,7 +49,6 @@ poetry add -D maturin
after installation, you can run `maturin --version` to check that you have correctly installed it. after installation, you can run `maturin --version` to check that you have correctly installed it.
# Starting a new project # Starting a new project
Firstly you should create the folder and virtual environment that are going to contain your new project. Here we will use the recommended `pyenv`: Firstly you should create the folder and virtual environment that are going to contain your new project. Here we will use the recommended `pyenv`:
@ -111,6 +109,7 @@ pyo3 = { version = "0.16.5", features = ["extension-module"] }
``` ```
## pyproject.toml ## pyproject.toml
You should also create a `pyproject.toml` with the following contents: You should also create a `pyproject.toml` with the following contents:
```toml ```toml

View File

@ -1,4 +1,4 @@
# Memory Management # Memory management
Rust and Python have very different notions of memory management. Rust has Rust and Python have very different notions of memory management. Rust has
a strict memory model with concepts of ownership, borrowing, and lifetimes, a strict memory model with concepts of ownership, borrowing, and lifetimes,
@ -13,7 +13,7 @@ PyO3 bridges the Rust and Python memory models with two different strategies for
accessing memory allocated on Python's heap from inside Rust. These are accessing memory allocated on Python's heap from inside Rust. These are
GIL-bound, or "owned" references, and GIL-independent `Py<Any>` smart pointers. GIL-bound, or "owned" references, and GIL-independent `Py<Any>` smart pointers.
## GIL-bound Memory ## GIL-bound memory
PyO3's GIL-bound, "owned references" (`&PyAny` etc.) make PyO3 more ergonomic to PyO3's GIL-bound, "owned references" (`&PyAny` etc.) make PyO3 more ergonomic to
use by ensuring that their lifetime can never be longer than the duration the use by ensuring that their lifetime can never be longer than the duration the
@ -119,7 +119,7 @@ dropped you do not retain access to any owned references created after the
[documentation for `Python::new_pool()`]({{#PYO3_DOCS_URL}}/pyo3/prelude/struct.Python.html#method.new_pool) [documentation for `Python::new_pool()`]({{#PYO3_DOCS_URL}}/pyo3/prelude/struct.Python.html#method.new_pool)
for more information on safety. for more information on safety.
## GIL-independent Memory ## GIL-independent memory
Sometimes we need a reference to memory on Python's heap that can outlive the Sometimes we need a reference to memory on Python's heap that can outlive the
GIL. Python's `Py<PyAny>` is analogous to `Rc<T>`, but for variables whose GIL. Python's `Py<PyAny>` is analogous to `Rc<T>`, but for variables whose

View File

@ -1,4 +1,4 @@
# Python Modules # Python modules
You can create a module using `#[pymodule]`: You can create a module using `#[pymodule]`: