Merge pull request #1772 from davidhewitt/async-guide-fixes

guide: don't bother doctesting async guide
This commit is contained in:
David Hewitt 2021-08-11 00:52:25 +01:00 committed by GitHub
commit 76e79d615c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 40 additions and 37 deletions

View File

@ -1,14 +1,14 @@
# Async / Await
If you are working with a Python library that makes use of async functions or wish to provide
If you are working with a Python library that makes use of async functions or wish to provide
Python bindings for an async Rust library, [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio)
likely has the tools you need. It provides conversions between async functions in both Python and
Rust and was designed with first-class support for popular Rust runtimes such as
[`tokio`](https://tokio.rs/) and [`async-std`](https://async.rs/). In addition, all async Python
code runs on the default `asyncio` event loop, so `pyo3-asyncio` should work just fine with existing
likely has the tools you need. It provides conversions between async functions in both Python and
Rust and was designed with first-class support for popular Rust runtimes such as
[`tokio`](https://tokio.rs/) and [`async-std`](https://async.rs/). In addition, all async Python
code runs on the default `asyncio` event loop, so `pyo3-asyncio` should work just fine with existing
Python libraries.
In the following sections, we'll give a general overview of `pyo3-asyncio` explaining how to call
In the following sections, we'll give a general overview of `pyo3-asyncio` explaining how to call
async Python functions with PyO3, how to call async Rust functions from Python, and how to configure
your codebase to manage the runtimes of both.
@ -163,13 +163,13 @@ maturin develop && python3
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at python3
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>>
>>> from my_async_module import rust_sleep
>>>
>>>
>>> async def main():
>>> await rust_sleep()
>>>
@ -188,19 +188,19 @@ async def py_sleep():
await asyncio.sleep(1)
```
**Async functions in Python are simply functions that return a `coroutine` object**. For our purposes,
**Async functions in Python are simply functions that return a `coroutine` object**. For our purposes,
we really don't need to know much about these `coroutine` objects. The key factor here is that calling
an `async` function is _just like calling a regular function_, the only difference is that we have
to do something special with the object that it returns.
Normally in Python, that something special is the `await` keyword, but in order to await this
coroutine in Rust, we first need to convert it into Rust's version of a `coroutine`: a `Future`.
That's where `pyo3-asyncio` comes in.
[`pyo3_asyncio::into_future`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/fn.into_future.html)
Normally in Python, that something special is the `await` keyword, but in order to await this
coroutine in Rust, we first need to convert it into Rust's version of a `coroutine`: a `Future`.
That's where `pyo3-asyncio` comes in.
[`pyo3_asyncio::into_future`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/fn.into_future.html)
performs this conversion for us:
```rust no_run
```rust
use pyo3::prelude::*;
#[pyo3_asyncio::tokio::main]
@ -209,11 +209,11 @@ async fn main() -> PyResult<()> {
// import the module containing the py_sleep function
let example = py.import("example")?;
// calling the py_sleep method like a normal function
// calling the py_sleep method like a normal function
// returns a coroutine
let coroutine = example.call_method0("py_sleep")?;
// convert the coroutine into a Rust future using the
// convert the coroutine into a Rust future using the
// tokio runtime
pyo3_asyncio::tokio::into_future(coroutine)
})?;
@ -225,12 +225,12 @@ async fn main() -> PyResult<()> {
}
```
> If you're interested in learning more about `coroutines` and `awaitables` in general, check out the
> If you're interested in learning more about `coroutines` and `awaitables` in general, check out the
> [Python 3 `asyncio` docs](https://docs.python.org/3/library/asyncio-task.html) for more information.
## Awaiting a Rust Future in Python
Here we have the same async function as before written in Rust using the
Here we have the same async function as before written in Rust using the
[`async-std`](https://async.rs/) runtime:
```rust
@ -243,12 +243,12 @@ async fn rust_sleep() {
Similar to Python, Rust's async functions also return a special object called a
`Future`:
```rust compile_fail
```rust
let future = rust_sleep();
```
We can convert this `Future` object into Python to make it `awaitable`. This tells Python that you
can use the `await` keyword with it. In order to do this, we'll call
We can convert this `Future` object into Python to make it `awaitable`. This tells Python that you
can use the `await` keyword with it. In order to do this, we'll call
[`pyo3_asyncio::async_std::future_into_py`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/fn.future_into_py.html):
```rust
@ -284,38 +284,38 @@ doesn't always play well with Rust.
Luckily, Rust's event loops are pretty flexible and don't _need_ control over the main thread, so in
`pyo3-asyncio`, we decided the best way to handle Rust/Python interop was to just surrender the main
thread to Python and run Rust's event loops in the background. Unfortunately, since most event loop
thread to Python and run Rust's event loops in the background. Unfortunately, since most event loop
implementations _prefer_ control over the main thread, this can still make some things awkward.
### PyO3 Asyncio Initialization
Because Python needs to control the main thread, we can't use the convenient proc macros from Rust
runtimes to handle the `main` function or `#[test]` functions. Instead, the initialization for PyO3 has to be done from the `main` function and the main
runtimes to handle the `main` function or `#[test]` functions. Instead, the initialization for PyO3 has to be done from the `main` function and the main
thread must block on [`pyo3_asyncio::run_forever`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/fn.run_forever.html) or [`pyo3_asyncio::async_std::run_until_complete`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/fn.run_until_complete.html).
Because we have to block on one of those functions, we can't use [`#[async_std::main]`](https://docs.rs/async-std/latest/async_std/attr.main.html) or [`#[tokio::main]`](https://docs.rs/tokio/1.1.0/tokio/attr.main.html)
since it's not a good idea to make long blocking calls during an async function.
> Internally, these `#[main]` proc macros are expanded to something like this:
> ```rust compile_fail
> ```rust
> fn main() {
> // your async main fn
> async fn _main_impl() { /* ... */ }
> Runtime::new().block_on(_main_impl());
> Runtime::new().block_on(_main_impl());
> }
> ```
> Making a long blocking call inside the `Future` that's being driven by `block_on` prevents that
> thread from doing anything else and can spell trouble for some runtimes (also this will actually
> deadlock a single-threaded runtime!). Many runtimes have some sort of `spawn_blocking` mechanism
> that can avoid this problem, but again that's not something we can use here since we need it to
> thread from doing anything else and can spell trouble for some runtimes (also this will actually
> deadlock a single-threaded runtime!). Many runtimes have some sort of `spawn_blocking` mechanism
> that can avoid this problem, but again that's not something we can use here since we need it to
> block on the _main_ thread.
For this reason, `pyo3-asyncio` provides its own set of proc macros to provide you with this
initialization. These macros are intended to mirror the initialization of `async-std` and `tokio`
For this reason, `pyo3-asyncio` provides its own set of proc macros to provide you with this
initialization. These macros are intended to mirror the initialization of `async-std` and `tokio`
while also satisfying the Python runtime's needs.
Here's a full example of PyO3 initialization with the `async-std` runtime:
```rust no_run
```rust
use pyo3::prelude::*;
#[pyo3_asyncio::async_std::main]
@ -429,19 +429,19 @@ $ maturin develop && python3
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at python3
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Python 3.8.8 (default, Apr 13 2021, 19:58:26)
Python 3.8.8 (default, Apr 13 2021, 19:58:26)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import uvloop
>>>
>>>
>>> import my_async_module
>>>
>>>
>>> uvloop.install()
>>>
>>>
>>> async def main():
... await my_async_module.rust_sleep()
...
...
>>> asyncio.run(main())
>>>
```
@ -498,4 +498,4 @@ fn main() -> PyResult<()> {
## Additional Information
- Managing event loop references can be tricky with pyo3-asyncio. See [Event Loop References](https://docs.rs/pyo3-asyncio/#event-loop-references) in the API docs to get a better intuition for how event loop references are managed in this library.
- Testing pyo3-asyncio libraries and applications requires a custom test harness since Python requires control over the main thread. You can find a testing guide in the [API docs for the `testing` module](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/testing)
- Testing pyo3-asyncio libraries and applications requires a custom test harness since Python requires control over the main thread. You can find a testing guide in the [API docs for the `testing` module](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/testing)

View File

@ -545,4 +545,7 @@ pub mod doc_test {
doctest!("guide/src/trait_bounds.md", guide_trait_bounds_md);
doctest!("guide/src/types.md", guide_types_md);
doctest!("guide/src/faq.md", faq);
// deliberate choice not to test guide/ecosystem because those pages depend on external crates
// such as pyo3_asyncio.
}