luxolus
/
pyo3
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'main' into immutable

This commit is contained in:
Bruno Kolenbrander 2022-04-12 14:22:33 +02:00 committed by GitHub
parent b7745dffc8 cb48557601
commit bc6bd6099c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
55 changed files with 1618 additions and 450 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
.github/workflows/ci.yml vendored
View File

@ -271,6 +271,23 @@ jobs:
target: aarch64-apple-darwin
args: --release -i python3.9 --no-sdist -m examples/maturin-starter/Cargo.toml
- name: Test cross compile to Windows
if: ${{ matrix.platform.os == 'ubuntu-latest' && matrix.python-version == '3.8' }}
env:
XWIN_ARCH: x86_64
run: |
sudo apt-get install -y mingw-w64 llvm
rustup target add x86_64-pc-windows-gnu x86_64-pc-windows-msvc
which cargo-xwin > /dev/null || cargo install cargo-xwin
cargo build --manifest-path examples/maturin-starter/Cargo.toml --features abi3 --target x86_64-pc-windows-gnu
cargo xwin build --manifest-path examples/maturin-starter/Cargo.toml --features abi3 --target x86_64-pc-windows-msvc
- name: Test cross compile to Windows with maturin
if: ${{ matrix.platform.os == 'ubuntu-latest' && matrix.python-version == '3.8' }}
uses: messense/maturin-action@v1
with:
target: x86_64-pc-windows-gnu
args: -i python3.8 --no-sdist -m examples/maturin-starter/Cargo.toml --cargo-extra-args="--features abi3"
env:
CARGO_TERM_VERBOSE: true
CARGO_BUILD_TARGET: ${{ matrix.platform.rust-target }}
@ -303,7 +320,7 @@ jobs:
continue-on-error: true
- uses: actions-rs/toolchain@v1
with:
toolchain: nightly
toolchain: stable
override: true
profile: minimal
components: llvm-tools-preview

12
Architecture.md
View File

@ -188,12 +188,20 @@ Some of the functionality of `pyo3-build-config`:
Currently we use the `extension-module` feature for this purpose. This may change in the future.
See [#1123](https://github.com/PyO3/pyo3/pull/1123).
- Cross-compiling configuration
- If `TARGET` architecture and `HOST` architecture differ, we find cross compile information
from environment variables (`PYO3_CROSS_LIB_DIR` and `PYO3_CROSS_PYTHON_VERSION`) or system files.
- If `TARGET` architecture and `HOST` architecture differ, we can find cross compile information
from environment variables (`PYO3_CROSS_LIB_DIR`, `PYO3_CROSS_PYTHON_VERSION` and
`PYO3_CROSS_PYTHON_IMPLEMENTATION`) or system files.
When cross compiling extension modules it is often possible to make it work without any
additional user input.
- When an experimental feature `generate-abi3-import-lib` is enabled, the `pyo3-ffi` build script can
generate `python3.dll` import libraries for Windows targets automatically via an external
[`python3-dll-a`] crate. This enables the users to cross compile abi3 extensions for Windows without
having to install any Windows Python libraries.
<!-- External Links -->
[python/c api]: https://docs.python.org/3/c-api/
[`python3-dll-a`]: https://docs.rs/python3-dll-a/latest/python3_dll_a/
<!-- Crates -->

48
CHANGELOG.md
View File

@ -6,34 +6,53 @@ PyO3 versions, please see the [migration guide](https://pyo3.rs/latest/migration
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## Unreleased
### Added
- Allow dependent crates to access config values from `pyo3-build-config` via cargo link dep env vars. [#2092](https://github.com/PyO3/pyo3/pull/2092)
- Added methods on `InterpreterConfig` to run Python scripts using the configured executable. [#2092](https://github.com/PyO3/pyo3/pull/2092)
- Added FFI definitions for `PyType_FromModuleAndSpec`, `PyType_GetModule`, `PyType_GetModuleState` and `PyModule_AddType`. [#2250](https://github.com/PyO3/pyo3/pull/2250)
- Add an experimental `generate-abi3-import-lib` feature to auto-generate `python3.dll` import libraries for Windows. [#2282](https://github.com/PyO3/pyo3/pull/2282)
- Add FFI definitions for `PyDateTime_BaseTime` and `PyDateTime_BaseDateTime`. [#2294](https://github.com/PyO3/pyo3/pull/2294)
### Changed
- Allow `#[pyo3(crate = "...", text_signature = "...")]` options to be used directly in `#[pyclass(crate = "...", text_signature = "...")]`. [#2234](https://github.com/PyO3/pyo3/pull/2234)
- Mark `METH_FASTCALL` calling convention as limited API on Python 3.10. [#2250](https://github.com/PyO3/pyo3/pull/2250)
- Default to "m" ABI tag when choosing `libpython` link name for CPython 3.7 on Unix. [#2288](https://github.com/PyO3/pyo3/pull/2288)
- Improved performance of failing calls to `FromPyObject::extract` which is common when functions accept multiple distinct types. [#2279](https://github.com/PyO3/pyo3/pull/2279)
### Fixed
- Fix `abi3-py310` feature: use Python 3.10 ABI when available instead of silently falling back to the 3.9 ABI. [#2242](https://github.com/PyO3/pyo3/pull/2242)
- Considered `PYTHONFRAMEWORK` when cross compiling in order that on macos cross compiling against a [Framework bundle](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkAnatomy.html) is considered shared. [#2233](https://github.com/PyO3/pyo3/pull/2233)
- Panic during compilation when `PYO3_CROSS_LIB_DIR` is set for some host/target combinations. [#2232](https://github.com/PyO3/pyo3/pull/2232)
- Correct dependency version for `syn` to require correct minimal patch version 1.0.56. [#2240](https://github.com/PyO3/pyo3/pull/2240)
- Fix segfault when calling FFI methods `PyDateTime_DATE_GET_TZINFO` or `PyDateTime_TIME_GET_TZINFO` on `datetime` or `time` without a tzinfo. [#2289](https://github.com/PyO3/pyo3/pull/2289)
### Added
- Added `as_bytes` on `Py<PyBytes>`. [#2235](https://github.com/PyO3/pyo3/pull/2235)
## [0.16.3] - 2022-04-05
### Packaging
- Extend `parking_lot` dependency supported versions to include 0.12. [#2239](https://github.com/PyO3/pyo3/pull/2239)
### Added
- Add methods to `pyo3_build_config::InterpreterConfig` to run Python scripts using the configured executable. [#2092](https://github.com/PyO3/pyo3/pull/2092)
- Add `as_bytes` method to `Py<PyBytes>`. [#2235](https://github.com/PyO3/pyo3/pull/2235)
- Add FFI definitions for `PyType_FromModuleAndSpec`, `PyType_GetModule`, `PyType_GetModuleState` and `PyModule_AddType`. [#2250](https://github.com/PyO3/pyo3/pull/2250)
- Add `pyo3_build_config::cross_compiling_from_to` as a helper to detect when PyO3 is cross-compiling. [#2253](https://github.com/PyO3/pyo3/pull/2253)
- Add `#[pyclass(mapping)]` option to leave sequence slots empty in container implementations. [#2265](https://github.com/PyO3/pyo3/pull/2265)
- Add `PyString::intern` to enable usage of the Python's built-in string interning. [#2268](https://github.com/PyO3/pyo3/pull/2268)
- Add `intern!` macro which can be used to amortize the cost of creating Python strings by storing them inside a `GILOnceCell`. [#2269](https://github.com/PyO3/pyo3/pull/2269)
- Add `PYO3_CROSS_PYTHON_IMPLEMENTATION` environment variable for selecting the default cross Python implementation. [#2272](https://github.com/PyO3/pyo3/pull/2272)
### Changed
- Allow `#[pyo3(crate = "...", text_signature = "...")]` options to be used directly in `#[pyclass(crate = "...", text_signature = "...")]`. [#2234](https://github.com/PyO3/pyo3/pull/2234)
- Make `PYO3_CROSS_LIB_DIR` environment variable optional when cross compiling. [#2241](https://github.com/PyO3/pyo3/pull/2241)
- Mark `METH_FASTCALL` calling convention as limited API on Python 3.10. [#2250](https://github.com/PyO3/pyo3/pull/2250)
- Deprecate `pyo3_build_config::cross_compiling` in favour of `pyo3_build_config::cross_compiling_from_to`. [#2253](https://github.com/PyO3/pyo3/pull/2253)
### Fixed
- Fix `abi3-py310` feature: use Python 3.10 ABI when available instead of silently falling back to the 3.9 ABI. [#2242](https://github.com/PyO3/pyo3/pull/2242)
- Use shared linking mode when cross compiling against a [Framework bundle](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkAnatomy.html) for macOS. [#2233](https://github.com/PyO3/pyo3/pull/2233)
- Fix panic during compilation when `PYO3_CROSS_LIB_DIR` is set for some host/target combinations. [#2232](https://github.com/PyO3/pyo3/pull/2232)
- Correct dependency version for `syn` to require minimal patch version 1.0.56. [#2240](https://github.com/PyO3/pyo3/pull/2240)
## [0.16.2] - 2022-03-15
### Packaging
@ -1126,7 +1145,8 @@ Yanked
- Initial release
[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.16.2...HEAD
[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.16.3...HEAD
[0.16.3]: https://github.com/pyo3/pyo3/compare/v0.16.2...v0.16.3
[0.16.2]: https://github.com/pyo3/pyo3/compare/v0.16.1...v0.16.2
[0.16.1]: https://github.com/pyo3/pyo3/compare/v0.16.0...v0.16.1
[0.16.0]: https://github.com/pyo3/pyo3/compare/v0.15.1...v0.16.0

15
Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "pyo3"
version = "0.16.2"
version = "0.16.3"
description = "Bindings to Python interpreter"
authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
readme = "README.md"
@ -19,10 +19,10 @@ libc = "0.2.62"
parking_lot = ">= 0.11, < 0.13"
# ffi bindings to the python interpreter, split into a seperate crate so they can be used independently
pyo3-ffi = { path = "pyo3-ffi", version = "=0.16.2" }
pyo3-ffi = { path = "pyo3-ffi", version = "=0.16.3" }
# support crates for macros feature
pyo3-macros = { path = "pyo3-macros", version = "=0.16.2", optional = true }
pyo3-macros = { path = "pyo3-macros", version = "=0.16.3", optional = true }
indoc = { version = "1.0.3", optional = true }
unindent = { version = "0.1.4", optional = true }
@ -50,7 +50,7 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.61"
[build-dependencies]
pyo3-build-config = { path = "pyo3-build-config", version = "0.16.2", features = ["resolve-config"] }
pyo3-build-config = { path = "pyo3-build-config", version = "0.16.3", features = ["resolve-config"] }
[features]
default = ["macros", "pyproto"]
@ -78,6 +78,9 @@ abi3-py38 = ["abi3-py39", "pyo3-build-config/abi3-py38", "pyo3-ffi/abi3-py38"]
abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39", "pyo3-ffi/abi3-py39"]
abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310", "pyo3-ffi/abi3-py310"]
# Automatically generates `python3.dll` import libraries for Windows targets.
generate-abi3-import-lib = ["pyo3-build-config/python3-dll-a"]
# Changes `Python::with_gil` and `Python::acquire_gil` to automatically initialize the
# Python interpreter if needed.
auto-initialize = []
@ -142,6 +145,10 @@ harness = false
name = "bench_tuple"
harness = false
[[bench]]
name = "bench_intern"
harness = false
[workspace]
members = [
"pyo3-ffi",

4
README.md
View File

@ -68,7 +68,7 @@ name = "string_sum"
crate-type = ["cdylib"]
[dependencies]
pyo3 = { version = "0.16.2", features = ["extension-module"] }
pyo3 = { version = "0.16.3", features = ["extension-module"] }
```
**`src/lib.rs`**
@ -132,7 +132,7 @@ Start a new project with `cargo new` and add `pyo3` to the `Cargo.toml` like th
```toml
[dependencies.pyo3]
version = "0.16.2"
version = "0.16.3"
features = ["auto-initialize"]
```

70
benches/bench_frompyobject.rs
View File

@ -1,6 +1,9 @@
use criterion::{criterion_group, criterion_main, Bencher, Criterion};
use criterion::{black_box, criterion_group, criterion_main, Bencher, Criterion};
use pyo3::{prelude::*, types::PyString};
use pyo3::{
prelude::*,
types::{PyList, PyString},
};
#[derive(FromPyObject)]
enum ManyTypes {
@ -18,8 +21,71 @@ fn enum_from_pyobject(b: &mut Bencher<'_>) {
})
}
fn list_via_cast_as(b: &mut Bencher<'_>) {
Python::with_gil(|py| {
let any: &PyAny = PyList::empty(py).into();
b.iter(|| {
let _list: &PyList = black_box(any).cast_as().unwrap();
});
})
}
fn list_via_extract(b: &mut Bencher<'_>) {
Python::with_gil(|py| {
let any: &PyAny = PyList::empty(py).into();
b.iter(|| {
let _list: &PyList = black_box(any).extract().unwrap();
});
})
}
fn not_a_list_via_cast_as(b: &mut Bencher<'_>) {
Python::with_gil(|py| {
let any: &PyAny = PyString::new(py, "foobar").into();
b.iter(|| {
black_box(any).cast_as::<PyList>().unwrap_err();
});
})
}
fn not_a_list_via_extract(b: &mut Bencher<'_>) {
Python::with_gil(|py| {
let any: &PyAny = PyString::new(py, "foobar").into();
b.iter(|| {
black_box(any).extract::<&PyList>().unwrap_err();
});
})
}
#[derive(FromPyObject)]
enum ListOrNotList<'a> {
List(&'a PyList),
NotList(&'a PyAny),
}
fn not_a_list_via_extract_enum(b: &mut Bencher<'_>) {
Python::with_gil(|py| {
let any: &PyAny = PyString::new(py, "foobar").into();
b.iter(|| match black_box(any).extract::<ListOrNotList<'_>>() {
Ok(ListOrNotList::List(_list)) => panic!(),
Ok(ListOrNotList::NotList(_any)) => (),
Err(_) => panic!(),
});
})
}
fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("enum_from_pyobject", enum_from_pyobject);
c.bench_function("list_via_cast_as", list_via_cast_as);
c.bench_function("list_via_extract", list_via_extract);
c.bench_function("not_a_list_via_cast_as", not_a_list_via_cast_as);
c.bench_function("not_a_list_via_extract", not_a_list_via_extract);
c.bench_function("not_a_list_via_extract_enum", not_a_list_via_extract_enum);
}
criterion_group!(benches, criterion_benchmark);

33
benches/bench_intern.rs Normal file
View File

@ -0,0 +1,33 @@
use criterion::{criterion_group, criterion_main, Bencher, Criterion};
use pyo3::prelude::*;
use pyo3::{intern, prepare_freethreaded_python};
fn getattr_direct(b: &mut Bencher<'_>) {
prepare_freethreaded_python();
Python::with_gil(|py| {
let sys = py.import("sys").unwrap();
b.iter(|| sys.getattr("version").unwrap());
});
}
fn getattr_intern(b: &mut Bencher<'_>) {
prepare_freethreaded_python();
Python::with_gil(|py| {
let sys = py.import("sys").unwrap();
b.iter(|| sys.getattr(intern!(py, "version")).unwrap());
});
}
fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("getattr_direct", getattr_direct);
c.bench_function("getattr_intern", getattr_intern);
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

2
examples/Cargo.toml
View File

@ -5,7 +5,7 @@ publish = false
edition = "2018"
[dev-dependencies]
pyo3 = { version = "0.16.2", path = "..", features = ["auto-initialize", "extension-module"] }
pyo3 = { version = "0.16.3", path = "..", features = ["auto-initialize", "extension-module"] }
[[example]]
name = "decorator"

2
examples/decorator/.template/pre-script.rhai
View File

@ -1,4 +1,4 @@
variable::set("PYO3_VERSION", "0.16.2");
variable::set("PYO3_VERSION", "0.16.3");
file::rename(".template/Cargo.toml", "Cargo.toml");
file::rename(".template/pyproject.toml", "pyproject.toml");
file::rename(".template/tox.ini", "tox.ini");

2
examples/maturin-starter/.template/pre-script.rhai
View File

@ -1,4 +1,4 @@
variable::set("PYO3_VERSION", "0.16.2");
variable::set("PYO3_VERSION", "0.16.3");
file::rename(".template/Cargo.toml", "Cargo.toml");
file::rename(".template/pyproject.toml", "pyproject.toml");
file::rename(".template/tox.ini", "tox.ini");

3
examples/maturin-starter/Cargo.toml
View File

@ -10,4 +10,7 @@ crate-type = ["cdylib"]
[dependencies]
pyo3 = { path = "../../", features = ["extension-module"] }
[features]
abi3 = ["pyo3/abi3-py37", "pyo3/generate-abi3-import-lib"]
[workspace]

2
examples/setuptools-rust-starter/.template/pre-script.rhai
View File

@ -1,4 +1,4 @@
variable::set("PYO3_VERSION", "0.16.2");
variable::set("PYO3_VERSION", "0.16.3");
file::rename(".template/Cargo.toml", "Cargo.toml");
file::rename(".template/setup.cfg", "setup.cfg");
file::rename(".template/tox.ini", "tox.ini");

2
examples/word-count/.template/pre-script.rhai
View File

@ -1,4 +1,4 @@
variable::set("PYO3_VERSION", "0.16.2");
variable::set("PYO3_VERSION", "0.16.3");
file::rename(".template/Cargo.toml", "Cargo.toml");
file::rename(".template/tox.ini", "tox.ini");
file::delete(".template");

36
guide/src/building_and_distribution.md
View File

@ -152,10 +152,20 @@ As your extension module may be run with multiple different Python versions you
PyO3 is only able to link your extension module to api3 version up to and including your host Python version. E.g., if you set `abi3-py38` and try to compile the crate with a host of Python 3.7, the build will fail.
As an advanced feature, you can build PyO3 wheel without calling Python interpreter with the environment variable `PYO3_NO_PYTHON` set. On unix systems this works unconditionally; on Windows you must also set the `RUSTFLAGS` evironment variable to contain `-L native=/path/to/python/libs` so that the linker can find `python3.lib`.
> Note: If you set more that one of these api version feature flags the lowest version always wins. For example, with both `abi3-py37` and `abi3-py38` set, PyO3 would build a wheel which supports Python 3.7 and up.
#### Building `abi3` extensions without a Python interpreter
As an advanced feature, you can build PyO3 wheel without calling Python interpreter with the environment variable `PYO3_NO_PYTHON` set.
On Unix-like systems this works unconditionally; on Windows you must also set the `RUSTFLAGS` environment variable
to contain `-L native=/path/to/python/libs` so that the linker can find `python3.lib`.
If the `python3.dll` import library is not available, an experimental `generate-abi3-import-lib` crate
feature may be enabled, and the required library will be created and used by PyO3 automatically.
*Note*: MSVC targets require LLVM binutils (`llvm-dlltool`) to be available in `PATH` for
the automatic import library generation feature to work.
#### Missing features
Due to limitations in the Python API, there are a few `pyo3` features that do
@ -218,16 +228,25 @@ Thanks to Rust's great cross-compilation support, cross-compiling using PyO3 is
* A toolchain for your target.
* The appropriate options in your Cargo `.config` for the platform you're targeting and the toolchain you are using.
* A Python interpreter that's already been compiled for your target.
* A Python interpreter that is built for your host and available through the `PATH` or setting the [`PYO3_PYTHON`](#python-version) variable.
* A Python interpreter that's already been compiled for your target (optional when building "abi3" extension modules).
* A Python interpreter that is built for your host and available through the `PATH` or setting the [`PYO3_PYTHON`](#python-version) variable (optional when building "abi3" extension modules).
After you've obtained the above, you can build a cross-compiled PyO3 module by using Cargo's `--target` flag. PyO3's build script will detect that you are attempting a cross-compile based on your host machine and the desired target.
When cross-compiling, PyO3's build script cannot execute the target Python interpreter to query the configuration, so there are a few additional environment variables you may need to set:
* `PYO3_CROSS`: If present this variable forces PyO3 to configure as a cross-compilation.
* `PYO3_CROSS_LIB_DIR`: This variable must be set to the directory containing the target's libpython DSO and the associated `_sysconfigdata*.py` file for Unix-like targets, or the Python DLL import libraries for the Windows target.
* `PYO3_CROSS_PYTHON_VERSION`: Major and minor version (e.g. 3.9) of the target Python installation. This variable is only needed if PyO3 cannot determine the version to target from `abi3-py3*` features, or if there are multiple versions of Python present in `PYO3_CROSS_LIB_DIR`.
* `PYO3_CROSS_LIB_DIR`: This variable can be set to the directory containing the target's libpython DSO and the associated `_sysconfigdata*.py` file for Unix-like targets, or the Python DLL import libraries for the Windows target. This variable is only needed when the output binary must link to libpython explicitly (e.g. when targeting Windows and Android or embedding a Python interpreter), or when it is absolutely required to get the interpreter configuration from `_sysconfigdata*.py`.
* `PYO3_CROSS_PYTHON_VERSION`: Major and minor version (e.g. 3.9) of the target Python installation. This variable is only needed if PyO3 cannot determine the version to target from `abi3-py3*` features, or if `PYO3_CROSS_LIB_DIR` is not set, or if there are multiple versions of Python present in `PYO3_CROSS_LIB_DIR`.
* `PYO3_CROSS_PYTHON_IMPLEMENTATION`: Python implementation name ("CPython" or "PyPy") of the target Python installation. CPython is assumed by default when this variable is not set, unless `PYO3_CROSS_LIB_DIR` is set for a Unix-like target and PyO3 can get the interpreter configuration from `_sysconfigdata*.py`.
An experimental `pyo3` crate feature `generate-abi3-import-lib` enables the user to cross-compile
"abi3" extension modules for Windows targets without setting the `PYO3_CROSS_LIB_DIR` environment
variable or providing any Windows Python library files. It uses an external [`python3-dll-a`] crate
to generate import libraries for the Stable ABI Python DLL for MinGW-w64 and MSVC compile targets.
*Note*: MSVC targets require LLVM binutils to be available on the host system.
More specifically, `python3-dll-a` requires `llvm-dlltool` executable to be present in `PATH` when
targeting `*-pc-windows-msvc`.
An example might look like the following (assuming your target's sysroot is at `/home/pyo3/cross/sysroot` and that your target is `armv7`):
@ -255,6 +274,10 @@ cargo build --target x86_64-pc-windows-gnu
Any of the `abi3-py3*` features can be enabled instead of setting `PYO3_CROSS_PYTHON_VERSION` in the above examples.
`PYO3_CROSS_LIB_DIR` can often be omitted when cross compiling extension modules for Unix and macOS targets,
or when cross compiling "abi3" extension modules for Windows and the experimental `generate-abi3-import-lib`
crate feature is enabled.
The following resources may also be useful for cross-compiling:
- [github.com/japaric/rust-cross](https://github.com/japaric/rust-cross) is a primer on cross compiling Rust.
- [github.com/rust-embedded/cross](https://github.com/rust-embedded/cross) uses Docker to make Rust cross-compilation easier.
@ -265,3 +288,4 @@ The following resources may also be useful for cross-compiling:
[`maturin`]: https://github.com/PyO3/maturin
[`setuptools-rust`]: https://github.com/PyO3/setuptools-rust
[PyOxidizer]: https://github.com/indygreg/PyOxidizer
[`python3-dll-a`]: https://docs.rs/python3-dll-a/latest/python3_dll_a/

1
guide/src/class.md
View File

@ -203,6 +203,7 @@ Python::with_gil(|py|{
[params-4]: https://doc.rust-lang.org/stable/std/rc/struct.Rc.html
[params-5]: https://doc.rust-lang.org/stable/std/sync/struct.Rc.html
[params-6]: https://docs.python.org/3/library/weakref.html
[params-mapping]: ./class/protocols.md#mapping--sequence-types
These parameters are covered in various sections of this guide.

14
guide/src/class/protocols.md
View File

@ -192,6 +192,18 @@ and `Return` a final value - see its docs for further details and an example.
### Mapping & Sequence types
The magic methods in this section can be used to implement Python container types. They are two main categories of container in Python: "mappings" such as `dict`, with arbitrary keys, and "sequences" such as `list` and `tuple`, with integer keys.
The Python C-API which PyO3 is built upon has separate "slots" for sequences and mappings. When writing a `class` in pure Python, there is no such distinction in the implementation - a `__getitem__` implementation will fill the slots for both the mapping and sequence forms, for example.
By default PyO3 reproduces the Python behaviour of filling both mapping and sequence slots. This makes sense for the "simple" case which matches Python, and also for sequences, where the mapping slot is used anyway to implement slice indexing.
Mapping types usually will not want the sequence slots filled. Having them filled will lead to outcomes which may be unwanted, such as:
- The mapping type will successfully cast to [`PySequence`]. This may lead to consumers of the type handling it incorrectly.
- Python provides a default implementation of `__iter__` for sequences, which calls `__getitem__` with consecutive positive integers starting from 0 until an `IndexError` is returned. Unless the mapping only contains consecutive positive integer keys, this `__iter__` implementation will likely not be the intended behavior.
Use the `#[pyclass(mapping)]` annotation to instruct PyO3 to only fill the mapping slots, leaving the sequence ones empty. This will apply to `__getitem__`, `__setitem__`, and `__delitem__`.
- `__len__(<self>) -> usize`
Implements the built-in function `len()` for the sequence.
@ -265,7 +277,6 @@ and `Return` a final value - see its docs for further details and an example.
Used by the `*=` operator, after trying the numeric multiplication via
the `__imul__` method.
### Descriptors
- `__get__(<self>, object, object) -> object`
@ -599,3 +610,4 @@ For details, look at the `#[pymethods]` regarding GC methods.
[`PyObjectProtocol`]: {{#PYO3_DOCS_URL}}/pyo3/class/basic/trait.PyObjectProtocol.html
[`PySequenceProtocol`]: {{#PYO3_DOCS_URL}}/pyo3/class/sequence/trait.PySequenceProtocol.html
[`PyIterProtocol`]: {{#PYO3_DOCS_URL}}/pyo3/class/iter/trait.PyIterProtocol.html
[`PySequence`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PySequence.html

11
guide/src/features.md
View File

@ -30,6 +30,17 @@ These features are extensions of the `abi3` feature to specify the exact minimum
See the [building and distribution](building_and_distribution.md#minimum-python-version-for-abi3) section for further detail.
### `generate-abi3-import-lib`
This experimental feature is used to generate import libraries for the Stable ABI Python DLL
for MinGW-w64 and MSVC (cross-)compile targets.
Enabling it allows to (cross-)compile `abi3` extension modules to any Windows targets
without having to install the Windows Python distribution files for the target.
See the [building and distribution](building_and_distribution.md#building-abi3-extensions-without-a-python-interpreter)
section for further detail.
## Features for embedding Python in Rust
### `auto-initialize`

42
noxfile.py
View File

@ -86,3 +86,45 @@ def publish(session: nox.Session) -> None:
)
time.sleep(10)
session.run("cargo", "publish", external=True)
@nox.session(venv_backend="none")
def contributors(session: nox.Session) -> None:
import requests
if len(session.posargs) != 1:
raise Exception("base commit positional argument missing")
base = session.posargs[0]
page = 1
authors = set()
while True:
resp = requests.get(
f"https://api.github.com/repos/PyO3/pyo3/compare/{base}...HEAD",
params={"page": page, "per_page": 100},
)
body = resp.json()
if resp.status_code != 200:
raise Exception(
f"failed to retrieve commits: {resp.status_code} {body['message']}"
)
for commit in body["commits"]:
try:
authors.add(commit["author"]["login"])
except:
continue
if "next" in resp.links:
page += 1
else:
break
authors = sorted(list(authors))
for author in authors:
print(f"@{author}")

8
pyo3-build-config/Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "pyo3-build-config"
version = "0.16.2"
version = "0.16.3"
description = "Build configuration for the PyO3 ecosystem"
authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
keywords = ["pyo3", "python", "cpython", "ffi"]
@ -12,6 +12,12 @@ edition = "2018"
[dependencies]
once_cell = "1"
python3-dll-a = { version = "0.2", optional = true }
target-lexicon = "0.12"
[build-dependencies]
python3-dll-a = { version = "0.2", optional = true }
target-lexicon = "0.12"
[features]
default = []

36
pyo3-build-config/build.rs
View File

@ -12,10 +12,7 @@ mod errors;
use std::{env, path::Path};
use errors::{Context, Result};
use impl_::{
env_var, get_abi3_version, make_interpreter_config, BuildFlags, InterpreterConfig,
PythonImplementation,
};
use impl_::{env_var, make_interpreter_config, InterpreterConfig};
fn configure(interpreter_config: Option<InterpreterConfig>, name: &str) -> Result<bool> {
let target = Path::new(&env::var_os("OUT_DIR").unwrap()).join(name);
@ -52,37 +49,12 @@ fn config_file() -> Result<Option<InterpreterConfig>> {
}
}
/// If PYO3_NO_PYTHON is set with abi3, use standard abi3 settings.
pub fn abi3_config() -> Option<InterpreterConfig> {
if let Some(version) = get_abi3_version() {
if env_var("PYO3_NO_PYTHON").is_some() {
return Some(InterpreterConfig {
version,
// NB PyPy doesn't support abi3 yet
implementation: PythonImplementation::CPython,
abi3: true,
lib_name: None,
lib_dir: None,
build_flags: BuildFlags::default(),
pointer_width: None,
executable: None,
shared: true,
suppress_build_script_link_lines: false,
extra_build_script_lines: vec![],
});
}
}
None
}
fn generate_build_configs() -> Result<()> {
let mut configured = false;
configured |= configure(config_file()?, "pyo3-build-config-file.txt")?;
configured |= configure(abi3_config(), "pyo3-build-config-abi3.txt")?;
let configured = configure(config_file()?, "pyo3-build-config-file.txt")?;
if configured {
// Don't bother trying to find an interpreter on the host system if at least one of the
// config file or abi3 settings are present
// Don't bother trying to find an interpreter on the host system
// if the user-provided config file is present.
configure(None, "pyo3-build-config.txt")?;
} else {
configure(Some(make_interpreter_config()?), "pyo3-build-config.txt")?;

48
pyo3-build-config/src/abi3_import_lib.rs Normal file
View File

@ -0,0 +1,48 @@
//! Optional `python3.dll` import library generator for Windows
use std::env;
use std::path::PathBuf;
use python3_dll_a::generate_implib_for_target;
use crate::errors::{Context, Result};
use super::{Architecture, OperatingSystem, Triple};
/// Generates the `python3.dll` import library for Windows targets.
///
/// Places the generated import library into the build script output directory
/// and returns the full library directory path.
///
/// Does nothing if the target OS is not Windows.
pub(super) fn generate_abi3_import_lib(target: &Triple) -> Result<Option<String>> {
if target.operating_system != OperatingSystem::Windows {
return Ok(None);
}
let out_dir = env::var_os("OUT_DIR")
.expect("generate_abi3_import_lib() must be called from a build script");
// Put the newly created import library into the build script output directory.
let mut out_lib_dir = PathBuf::from(out_dir);
out_lib_dir.push("lib");
// Convert `Architecture` enum to rustc `target_arch` option format.
let arch = match target.architecture {
// i686, i586, etc.
Architecture::X86_32(_) => "x86".to_string(),
other => other.to_string(),
};
let env = target.environment.to_string();
generate_implib_for_target(&out_lib_dir, &arch, &env)
.context("failed to generate python3.dll import library")?;
let out_lib_dir_string = out_lib_dir
.to_str()
.ok_or("build directory is not a valid UTF-8 string")?
.to_owned();
Ok(Some(out_lib_dir_string))
}

967
pyo3-build-config/src/impl_.rs
View File

File diff suppressed because it is too large Load Diff

73
pyo3-build-config/src/lib.rs
View File

@ -11,15 +11,21 @@ mod errors;
mod impl_;
#[cfg(feature = "resolve-config")]
use std::io::Cursor;
use std::{
io::Cursor,
path::{Path, PathBuf},
};
use std::{env, process::Command};
#[cfg(feature = "resolve-config")]
use once_cell::sync::OnceCell;
#[allow(deprecated)]
pub use impl_::{
cross_compiling, find_all_sysconfigdata, parse_sysconfigdata, BuildFlag, BuildFlags,
CrossCompileConfig, InterpreterConfig, PythonImplementation, PythonVersion,
cross_compiling, cross_compiling_from_to, find_all_sysconfigdata, parse_sysconfigdata,
BuildFlag, BuildFlags, CrossCompileConfig, InterpreterConfig, PythonImplementation,
PythonVersion, Triple,
};
/// Adds all the [`#[cfg]` flags](index.html) to the current compilation.
@ -67,14 +73,19 @@ fn _add_extension_module_link_args(target_os: &str, mut writer: impl std::io::Wr
pub fn get() -> &'static InterpreterConfig {
static CONFIG: OnceCell<InterpreterConfig> = OnceCell::new();
CONFIG.get_or_init(|| {
// Check if we are in a build script and cross compiling to a different target.
let cross_compile_config_path = resolve_cross_compile_config_path();
let cross_compiling = cross_compile_config_path
.as_ref()
.map(|path| path.exists())
.unwrap_or(false);
if let Some(interpreter_config) = InterpreterConfig::from_cargo_dep_env() {
interpreter_config
} else if !CONFIG_FILE.is_empty() {
InterpreterConfig::from_reader(Cursor::new(CONFIG_FILE))
} else if !ABI3_CONFIG.is_empty() {
Ok(abi3_config())
} else if impl_::cross_compile_env_vars().any() {
InterpreterConfig::from_path(DEFAULT_CROSS_COMPILE_CONFIG_PATH)
} else if cross_compiling {
InterpreterConfig::from_path(cross_compile_config_path.as_ref().unwrap())
} else {
InterpreterConfig::from_reader(Cursor::new(HOST_CONFIG))
}
@ -82,41 +93,31 @@ pub fn get() -> &'static InterpreterConfig {
})
}
/// Path where PyO3's build.rs will write configuration by default.
#[doc(hidden)]
#[cfg(feature = "resolve-config")]
const DEFAULT_CROSS_COMPILE_CONFIG_PATH: &str =
concat!(env!("OUT_DIR"), "/pyo3-cross-compile-config.txt");
/// Build configuration provided by `PYO3_CONFIG_FILE`. May be empty if env var not set.
#[doc(hidden)]
#[cfg(feature = "resolve-config")]
const CONFIG_FILE: &str = include_str!(concat!(env!("OUT_DIR"), "/pyo3-build-config-file.txt"));
/// Build configuration set if abi3 features enabled and `PYO3_NO_PYTHON` env var present. Empty if
/// not both present.
#[doc(hidden)]
#[cfg(feature = "resolve-config")]
const ABI3_CONFIG: &str = include_str!(concat!(env!("OUT_DIR"), "/pyo3-build-config-abi3.txt"));
/// Build configuration discovered by `pyo3-build-config` build script. Not aware of
/// cross-compilation settings.
#[doc(hidden)]
#[cfg(feature = "resolve-config")]
const HOST_CONFIG: &str = include_str!(concat!(env!("OUT_DIR"), "/pyo3-build-config.txt"));
/// Returns the path where PyO3's build.rs writes its cross compile configuration.
///
/// The config file will be named `$OUT_DIR/<triple>/pyo3-build-config.txt`.
///
/// Must be called from a build script, returns `None` if not.
#[doc(hidden)]
#[cfg(feature = "resolve-config")]
fn abi3_config() -> InterpreterConfig {
let mut interpreter_config = InterpreterConfig::from_reader(Cursor::new(ABI3_CONFIG))
.expect("failed to parse hardcoded PyO3 abi3 config");
// If running from a build script on Windows, tweak the hardcoded abi3 config to contain
// the standard lib name (this is necessary so that abi3 extension modules using
// PYO3_NO_PYTHON on Windows can link)
if std::env::var("CARGO_CFG_TARGET_OS").map_or(false, |target_os| target_os == "windows") {
assert_eq!(interpreter_config.lib_name, None);
interpreter_config.lib_name = Some("python3".to_owned())
}
interpreter_config
fn resolve_cross_compile_config_path() -> Option<PathBuf> {
env::var_os("TARGET").map(|target| {
let mut path = PathBuf::from(env!("OUT_DIR"));
path.push(Path::new(&target));
path.push("pyo3-build-config.txt");
path
})
}
/// Use certain features if we detect the compiler being used supports them.
@ -156,8 +157,6 @@ pub fn print_feature_cfgs() {
pub mod pyo3_build_script_impl {
#[cfg(feature = "resolve-config")]
use crate::errors::{Context, Result};
#[cfg(feature = "resolve-config")]
use std::path::Path;
#[cfg(feature = "resolve-config")]
use super::*;
@ -166,7 +165,8 @@ pub mod pyo3_build_script_impl {
pub use crate::errors::*;
}
pub use crate::impl_::{
cargo_env_var, env_var, make_cross_compile_config, InterpreterConfig, PythonVersion,
cargo_env_var, env_var, is_linking_libpython, make_cross_compile_config, InterpreterConfig,
PythonVersion,
};
/// Gets the configuration for use from PyO3's build script.
@ -178,11 +178,10 @@ pub mod pyo3_build_script_impl {
pub fn resolve_interpreter_config() -> Result<InterpreterConfig> {
if !CONFIG_FILE.is_empty() {
InterpreterConfig::from_reader(Cursor::new(CONFIG_FILE))
} else if !ABI3_CONFIG.is_empty() {
Ok(abi3_config())
} else if let Some(interpreter_config) = impl_::make_cross_compile_config()? {
} else if let Some(interpreter_config) = make_cross_compile_config()? {
// This is a cross compile and need to write the config file.
let path = Path::new(DEFAULT_CROSS_COMPILE_CONFIG_PATH);
let path = resolve_cross_compile_config_path()
.expect("resolve_interpreter_config() must be called from a build script");
let parent_dir = path.parent().ok_or_else(|| {
format!(
"failed to resolve parent directory of config file {}",

4
pyo3-ffi/Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "pyo3-ffi"
version = "0.16.2"
version = "0.16.3"
description = "Python-API bindings for the PyO3 ecosystem"
authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
keywords = ["pyo3", "python", "cpython", "ffi"]
@ -35,4 +35,4 @@ abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310"]
[build-dependencies]
pyo3-build-config = { path = "../pyo3-build-config", version = "0.16.2", features = ["resolve-config"] }
pyo3-build-config = { path = "../pyo3-build-config", version = "0.16.3", features = ["resolve-config"] }

56
pyo3-ffi/build.rs
View File

@ -1,8 +1,8 @@
use pyo3_build_config::{
bail, ensure, print_feature_cfgs,
pyo3_build_script_impl::{
cargo_env_var, env_var, errors::Result, resolve_interpreter_config, InterpreterConfig,
PythonVersion,
cargo_env_var, env_var, errors::Result, is_linking_libpython, resolve_interpreter_config,
InterpreterConfig, PythonVersion,
},
};
@ -44,33 +44,27 @@ fn ensure_target_pointer_width(interpreter_config: &InterpreterConfig) -> Result
fn emit_link_config(interpreter_config: &InterpreterConfig) -> Result<()> {
let target_os = cargo_env_var("CARGO_CFG_TARGET_OS").unwrap();
let is_extension_module = cargo_env_var("CARGO_FEATURE_EXTENSION_MODULE").is_some();
if target_os == "windows" || target_os == "android" || !is_extension_module {
// windows and android - always link
// other systems - only link if not extension module
println!(
"cargo:rustc-link-lib={link_model}{alias}{lib_name}",
link_model = if interpreter_config.shared {
""
} else {
"static="
},
alias = if target_os == "windows" {
"pythonXY:"
} else {
""
},
lib_name = interpreter_config.lib_name.as_ref().ok_or(
"attempted to link to Python shared library but config does not contain lib_name"
)?,
);
if let Some(lib_dir) = &interpreter_config.lib_dir {
println!("cargo:rustc-link-search=native={}", lib_dir);
}
}
// serialize the whole interpreter config in DEP_PYTHON_PYO3_CONFIG
interpreter_config.to_cargo_dep_env()?;
println!(
"cargo:rustc-link-lib={link_model}{alias}{lib_name}",
link_model = if interpreter_config.shared {
""
} else {
"static="
},
alias = if target_os == "windows" {
"pythonXY:"
} else {
""
},
lib_name = interpreter_config.lib_name.as_ref().ok_or(
"attempted to link to Python shared library but config does not contain lib_name"
)?,
);
if let Some(lib_dir) = &interpreter_config.lib_dir {
println!("cargo:rustc-link-search=native={}", lib_dir);
}
Ok(())
}
@ -92,7 +86,10 @@ fn configure_pyo3() -> Result<()> {
ensure_python_version(&interpreter_config)?;
ensure_target_pointer_width(&interpreter_config)?;
if !interpreter_config.suppress_build_script_link_lines {
// Serialize the whole interpreter config into DEP_PYTHON_PYO3_CONFIG env var.
interpreter_config.to_cargo_dep_env()?;
if is_linking_libpython() && !interpreter_config.suppress_build_script_link_lines {
emit_link_config(&interpreter_config)?;
}
@ -114,6 +111,7 @@ fn print_config_and_exit(config: &InterpreterConfig) {
config
.to_writer(&mut std::io::stdout())
.expect("failed to print config to stdout");
println!("\nnote: unset the PYO3_PRINT_CONFIG environment variable and retry to compile with the above config");
std::process::exit(101);
}

47
pyo3-ffi/src/datetime.rs
View File

@ -38,11 +38,29 @@ pub struct PyDateTime_Delta {
// skipped non-limited PyDateTime_TZInfo
// skipped non-limited _PyDateTime_BaseTZInfo
// skipped non-limited _PyDateTime_BaseTime
#[repr(C)]
#[derive(Debug, Copy, Clone)]
/// Structure representing a `datetime.time` without a `tzinfo` member.
pub struct PyDateTime_BaseTime {
pub ob_base: PyObject,
#[cfg(not(PyPy))]
pub hashcode: Py_hash_t,
pub hastzinfo: c_char,
#[cfg(not(PyPy))]
pub data: [c_uchar; _PyDateTime_TIME_DATASIZE],
#[cfg(not(PyPy))]
pub fold: c_uchar,
}
#[repr(C)]
#[derive(Debug, Copy, Clone)]
/// Structure representing a `datetime.time`.
///
/// # Safety
///
/// Care should be taken when reading the `tzinfo` field of this type. If the time does not have a
/// tzinfo then the Python interpreter is free to allocate it as a [PyDateTime_BaseTime].
pub struct PyDateTime_Time {
pub ob_base: PyObject,
#[cfg(not(PyPy))]
@ -66,11 +84,28 @@ pub struct PyDateTime_Date {
pub data: [c_uchar; _PyDateTime_DATE_DATASIZE],
}
// skipped non-limited _PyDateTime_BaseDateTime
#[repr(C)]
#[derive(Debug, Copy, Clone)]
/// Structure representing a `datetime.datetime` without a `tzinfo` member.
pub struct PyDateTime_BaseDateTime {
pub ob_base: PyObject,
#[cfg(not(PyPy))]
pub hashcode: Py_hash_t,
pub hastzinfo: c_char,
#[cfg(not(PyPy))]
pub data: [c_uchar; _PyDateTime_DATETIME_DATASIZE],
#[cfg(not(PyPy))]
pub fold: c_uchar,
}
#[repr(C)]
#[derive(Debug, Copy, Clone)]
/// Structure representing a `datetime.datetime`
/// Structure representing a `datetime.datetime`.
///
/// # Safety
///
/// Care should be taken when reading the `tzinfo` field of this type. If the datetime does not have a
/// tzinfo then the Python interpreter is free to allocate it as a [PyDateTime_BaseDateTime].
pub struct PyDateTime_DateTime {
pub ob_base: PyObject,
#[cfg(not(PyPy))]
@ -155,7 +190,11 @@ macro_rules! _PyDateTime_GET_FOLD {
#[cfg(not(PyPy))]
macro_rules! _PyDateTime_GET_TZINFO {
($o: expr) => {
(*$o).tzinfo
if (*$o).hastzinfo != 0 {
(*$o).tzinfo
} else {
$crate::Py_None()
}
};
}

6
pyo3-ffi/src/object.rs
View File

@ -337,6 +337,12 @@ extern "C" {
// Flag bits for printing:
pub const Py_PRINT_RAW: c_int = 1; // No string quotes etc.
#[cfg(all(Py_3_10, not(Py_LIMITED_API)))]
pub const Py_TPFLAGS_SEQUENCE: c_ulong = 1 << 5;
#[cfg(all(Py_3_10, not(Py_LIMITED_API)))]
pub const Py_TPFLAGS_MAPPING: c_ulong = 1 << 6;
#[cfg(Py_3_10)]
pub const Py_TPFLAGS_DISALLOW_INSTANTIATION: c_ulong = 1 << 7;

2
pyo3-macros-backend/Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "pyo3-macros-backend"
version = "0.16.2"
version = "0.16.3"
description = "Code generation for PyO3 package"
authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
keywords = ["pyo3", "python", "cpython", "ffi"]

1
pyo3-macros-backend/src/attributes.rs
View File

@ -18,6 +18,7 @@ pub mod kw {
syn::custom_keyword!(gc);
syn::custom_keyword!(get);
syn::custom_keyword!(item);
syn::custom_keyword!(mapping);
syn::custom_keyword!(module);
syn::custom_keyword!(name);
syn::custom_keyword!(pass_module);

19
pyo3-macros-backend/src/frompyobject.rs
View File

@ -294,8 +294,10 @@ impl<'a> Container<'a> {
let mut fields: Punctuated<TokenStream, syn::Token![,]> = Punctuated::new();
for (ident, attrs) in tups {
let getter = match &attrs.getter {
FieldGetter::GetAttr(Some(name)) => quote!(getattr(#name)),
FieldGetter::GetAttr(None) => quote!(getattr(stringify!(#ident))),
FieldGetter::GetAttr(Some(name)) => quote!(getattr(_pyo3::intern!(py, #name))),
FieldGetter::GetAttr(None) => {
quote!(getattr(_pyo3::intern!(py, stringify!(#ident))))
}
FieldGetter::GetItem(Some(key)) => quote!(get_item(#key)),
FieldGetter::GetItem(None) => quote!(get_item(stringify!(#ident))),
};
@ -303,13 +305,14 @@ impl<'a> Container<'a> {
format!("failed to extract field {}.{}", quote!(#self_ty), ident);
let get_field = quote!(obj.#getter?);
let extractor = match &attrs.from_py_with {
None => quote!(
#get_field.extract().map_err(|inner| {
None => quote!({
let py = _pyo3::PyNativeType::py(obj);
let new_err = _pyo3::exceptions::PyTypeError::new_err(#conversion_error_msg);
new_err.set_cause(py, ::std::option::Option::Some(inner));
new_err
})?),
#get_field.extract().map_err(|inner| {
let new_err = _pyo3::exceptions::PyTypeError::new_err(#conversion_error_msg);
new_err.set_cause(py, ::std::option::Option::Some(inner));
new_err
})?
}),
Some(FromPyWithAttribute {
value: expr_path, ..
}) => quote! (

7
pyo3-macros-backend/src/pyclass.rs
View File

@ -55,6 +55,7 @@ pub struct PyClassPyO3Options {
pub extends: Option<ExtendsAttribute>,
pub freelist: Option<FreelistAttribute>,
pub immutable: Option<kw::immutable>,
pub mapping: Option<kw::mapping>,
pub module: Option<ModuleAttribute>,
pub name: Option<NameAttribute>,
pub subclass: Option<kw::subclass>,
@ -71,6 +72,7 @@ enum PyClassPyO3Option {
Extends(ExtendsAttribute),
Freelist(FreelistAttribute),
Immutable(kw::immutable),
Mapping(kw::mapping),
Module(ModuleAttribute),
Name(NameAttribute),
Subclass(kw::subclass),
@ -94,6 +96,8 @@ impl Parse for PyClassPyO3Option {
input.parse().map(PyClassPyO3Option::Freelist)
} else if lookahead.peek(attributes::kw::immutable) {
input.parse().map(PyClassPyO3Option::Immutable)
} else if lookahead.peek(attributes::kw::mapping) {
input.parse().map(PyClassPyO3Option::Mapping)
} else if lookahead.peek(attributes::kw::module) {
input.parse().map(PyClassPyO3Option::Module)
} else if lookahead.peek(kw::name) {
@ -150,6 +154,7 @@ impl PyClassPyO3Options {
PyClassPyO3Option::Extends(extends) => set_option!(extends),
PyClassPyO3Option::Freelist(freelist) => set_option!(freelist),
PyClassPyO3Option::Immutable(immutable) => set_option!(immutable),
PyClassPyO3Option::Mapping(mapping) => set_option!(mapping),
PyClassPyO3Option::Module(module) => set_option!(module),
PyClassPyO3Option::Name(name) => set_option!(name),
PyClassPyO3Option::Subclass(subclass) => set_option!(subclass),
@ -764,6 +769,7 @@ impl<'a> PyClassImplsBuilder<'a> {
.map(|extends_attr| extends_attr.value.clone())
.unwrap_or_else(|| parse_quote! { _pyo3::PyAny });
let is_subclass = self.attr.options.extends.is_some();
let is_mapping: bool = self.attr.options.mapping.is_some();
let dict_offset = if self.attr.options.dict.is_some() {
quote! {
@ -855,6 +861,7 @@ impl<'a> PyClassImplsBuilder<'a> {
const DOC: &'static str = #doc;
const IS_BASETYPE: bool = #is_basetype;
const IS_SUBCLASS: bool = #is_subclass;
const IS_MAPPING: bool = #is_mapping;
type Layout = _pyo3::PyCell<Self>;
type BaseType = #base;

2
pyo3-macros-backend/src/pyimpl.rs
View File

@ -88,7 +88,7 @@ pub fn build_py_methods(
pub fn impl_methods(
ty: &syn::Type,
impls: &mut Vec<syn::ImplItem>,
impls: &mut [syn::ImplItem],
methods_type: PyClassMethodsType,
options: PyImplOptions,
) -> syn::Result<TokenStream> {

2
pyo3-macros-backend/src/pyproto.rs
View File

@ -44,7 +44,7 @@ pub fn build_py_proto(ast: &mut syn::ItemImpl) -> syn::Result<TokenStream> {
fn impl_proto_impl(
ty: &syn::Type,
impls: &mut Vec<syn::ImplItem>,
impls: &mut [syn::ImplItem],
proto: &defs::Proto,
) -> syn::Result<TokenStream> {
let mut trait_impls = TokenStream::new();

4
pyo3-macros/Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "pyo3-macros"
version = "0.16.2"
version = "0.16.3"
description = "Proc macros for PyO3 package"
authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
keywords = ["pyo3", "python", "cpython", "ffi"]
@ -23,4 +23,4 @@ abi3 = ["pyo3-macros-backend/abi3"]
proc-macro2 = { version = "1", default-features = false }
quote = "1"
syn = { version = "1.0.56", features = ["full", "extra-traits"] }
pyo3-macros-backend = { path = "../pyo3-macros-backend", version = "=0.16.2" }
pyo3-macros-backend = { path = "../pyo3-macros-backend", version = "=0.16.3" }

1
pyo3-macros/docs/pyclass_parameters.md
View File

@ -6,6 +6,7 @@
| `dict` | Gives instances of this class an empty `__dict__` to store custom attributes. |
| <span style="white-space: pre">`extends = BaseType`</span> | Use a custom baseclass. Defaults to [`PyAny`][params-1] |
| <span style="white-space: pre">`freelist = N`</span> | Implements a [free list][params-2] of size N. This can improve performance for types that are often created and deleted in quick succession. Profile your code to see whether `freelist` is right for you. |
| `mapping` | Inform PyO3 that this class is a [`Mapping`][params-mapping], and so leave its implementation of sequence C-API slots empty. |
| <span style="white-space: pre">`module = "module_name"`</span> | Python code will see the class as being defined in this module. Defaults to `builtins`. |
| <span style="white-space: pre">`name = "python_name"`</span> | Sets the name that Python sees this class as. Defaults to the name of the Rust struct. |
| <span style="white-space: pre">`text_signature = "(arg1, arg2, ...)"`</span> | Sets the text signature for the Python class' `__new__` method. |

1
pyo3-macros/src/lib.rs
View File

@ -93,6 +93,7 @@ pub fn pyproto(_: TokenStream, input: TokenStream) -> TokenStream {
/// [params-4]: std::rc::Rc
/// [params-5]: std::sync::Arc
/// [params-6]: https://docs.python.org/3/library/weakref.html
/// [params-mapping]: https://pyo3.rs/latest/class/protocols.html#mapping--sequence-types
#[proc_macro_attribute]
pub fn pyclass(attr: TokenStream, input: TokenStream) -> TokenStream {
use syn::Item;

12
pytests/src/buf_and_str.rs
View File

@ -16,22 +16,26 @@ impl BytesExtractor {
BytesExtractor {}
}
pub fn from_bytes(&mut self, bytes: &PyBytes) -> PyResult<usize> {
#[staticmethod]
pub fn from_bytes(bytes: &PyBytes) -> PyResult<usize> {
let byte_vec: Vec<u8> = bytes.extract()?;
Ok(byte_vec.len())
}
pub fn from_str(&mut self, string: &PyString) -> PyResult<usize> {
#[staticmethod]
pub fn from_str(string: &PyString) -> PyResult<usize> {
let rust_string: String = string.extract()?;
Ok(rust_string.len())
}
pub fn from_str_lossy(&mut self, string: &PyString) -> usize {
#[staticmethod]
pub fn from_str_lossy(string: &PyString) -> usize {
let rust_string_lossy: String = string.to_string_lossy().to_string();
rust_string_lossy.len()
}
pub fn from_buffer(&mut self, buf: &PyAny) -> PyResult<usize> {
#[staticmethod]
pub fn from_buffer(buf: &PyAny) -> PyResult<usize> {
let buf = PyBuffer::<u8>::get(buf)?;
Ok(buf.item_count())
}

26
src/err/mod.rs
View File

@ -663,10 +663,34 @@ impl<'a> IntoPy<PyObject> for &'a PyErr {
}
}
struct PyDowncastErrorArguments {
from: Py<PyType>,
to: Cow<'static, str>,
}
impl PyErrArguments for PyDowncastErrorArguments {
fn arguments(self, py: Python<'_>) -> PyObject {
format!(
"'{}' object cannot be converted to '{}'",
self.from
.as_ref(py)
.name()
.unwrap_or("<failed to extract type name>"),
self.to
)
.to_object(py)
}
}
/// Convert `PyDowncastError` to Python `TypeError`.
impl<'a> std::convert::From<PyDowncastError<'a>> for PyErr {
fn from(err: PyDowncastError<'_>) -> PyErr {
exceptions::PyTypeError::new_err(err.to_string())
let args = PyDowncastErrorArguments {
from: err.from.get_type().into(),
to: err.to,
};
exceptions::PyTypeError::new_err(args)
}
}

41
src/ffi/tests.rs
View File

@ -180,3 +180,44 @@ fn ucs4() {
}
})
}
#[test]
#[cfg(not(PyPy))]
fn test_get_tzinfo() {
crate::Python::with_gil(|py| {
use crate::types::{PyDateTime, PyTime};
use crate::{AsPyPointer, PyAny, ToPyObject};
let datetime = py.import("datetime").map_err(|e| e.print(py)).unwrap();
let timezone = datetime.getattr("timezone").unwrap();
let utc = timezone.getattr("utc").unwrap().to_object(py);
let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, Some(&utc)).unwrap();
assert!(
unsafe { py.from_borrowed_ptr::<PyAny>(PyDateTime_DATE_GET_TZINFO(dt.as_ptr())) }
.is(&utc)
);
let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, None).unwrap();
assert!(
unsafe { py.from_borrowed_ptr::<PyAny>(PyDateTime_DATE_GET_TZINFO(dt.as_ptr())) }
.is_none()
);
let t = PyTime::new(py, 0, 0, 0, 0, Some(&utc)).unwrap();
assert!(
unsafe { py.from_borrowed_ptr::<PyAny>(PyDateTime_TIME_GET_TZINFO(t.as_ptr())) }
.is(&utc)
);
let t = PyTime::new(py, 0, 0, 0, 0, None).unwrap();
assert!(
unsafe { py.from_borrowed_ptr::<PyAny>(PyDateTime_TIME_GET_TZINFO(t.as_ptr())) }
.is_none()
);
})
}

61
src/gil.rs
View File

@ -495,6 +495,7 @@ impl EnsureGIL {
mod tests {
use super::{gil_is_acquired, GILPool, GIL_COUNT, OWNED_OBJECTS, POOL};
use crate::{ffi, gil, AsPyPointer, IntoPyPointer, PyObject, Python, ToPyObject};
use parking_lot::{const_mutex, Condvar, Mutex};
use std::ptr::NonNull;
fn get_object(py: Python<'_>) -> PyObject {
@ -699,16 +700,41 @@ mod tests {
assert_eq!(count + 1, c.get_refcnt(py));
}
struct Event {
set: Mutex<bool>,
wait: Condvar,
}
impl Event {
const fn new() -> Self {
Self {
set: const_mutex(false),
wait: Condvar::new(),
}
}
fn set(&self) {
*self.set.lock() = true;
self.wait.notify_all();
}
fn wait(&self) {
let mut set = self.set.lock();
while !*set {
self.wait.wait(&mut set);
}
}
}
#[test]
fn test_clone_without_gil() {
use crate::{Py, PyAny};
use std::sync::atomic::{AtomicBool, Ordering};
use std::{sync::Arc, thread};
// Some spinlocks for synchronizing
static GIL_ACQUIRED: AtomicBool = AtomicBool::new(false);
static OBJECT_CLONED: AtomicBool = AtomicBool::new(false);
static REFCNT_CHECKED: AtomicBool = AtomicBool::new(false);
// Some events for synchronizing
static GIL_ACQUIRED: Event = Event::new();
static OBJECT_CLONED: Event = Event::new();
static REFCNT_CHECKED: Event = Event::new();
Python::with_gil(|py| {
let obj: Arc<Py<PyAny>> = Arc::new(get_object(py));
@ -723,31 +749,31 @@ mod tests {
let handle = thread::spawn(move || {
Python::with_gil(move |py| {
println!("3. The GIL has been acquired on another thread.");
GIL_ACQUIRED.store(true, Ordering::Release);
GIL_ACQUIRED.set();
// Spin a bit while the main thread registers obj in POOL
while !OBJECT_CLONED.load(Ordering::Acquire) {}
// Wait while the main thread registers obj in POOL
OBJECT_CLONED.wait();
println!("5. Checking refcnt");
assert_eq!(thread_obj.get_refcnt(py), count);
REFCNT_CHECKED.store(true, Ordering::Release);
REFCNT_CHECKED.set();
})
});
let cloned = py.allow_threads(|| {
println!("2. The GIL has been released.");
// spin until the gil has been acquired on the thread.
while !GIL_ACQUIRED.load(Ordering::Acquire) {}
// Wait until the GIL has been acquired on the thread.
GIL_ACQUIRED.wait();
println!("4. The other thread is now hogging the GIL, we clone without it held");
// Cloning without GIL should not update reference count
let cloned = Py::clone(&*obj);
OBJECT_CLONED.store(true, Ordering::Release);
OBJECT_CLONED.set();
cloned
});
while !REFCNT_CHECKED.load(Ordering::Acquire) {}
REFCNT_CHECKED.wait();
// Returning from allow_threads doesn't clear the pool
py.allow_threads(|| {
@ -773,11 +799,10 @@ mod tests {
#[test]
fn test_clone_in_other_thread() {
use crate::Py;
use std::sync::atomic::{AtomicBool, Ordering};
use std::{sync::Arc, thread};
// Some spinlocks for synchronizing
static OBJECT_CLONED: AtomicBool = AtomicBool::new(false);
// Some events for synchronizing
static OBJECT_CLONED: Event = Event::new();
let (obj, count, ptr) = Python::with_gil(|py| {
let obj = Arc::new(get_object(py));
@ -789,10 +814,10 @@ mod tests {
// Cloning without GIL should not update reference count
#[allow(clippy::redundant_clone)]
let _ = Py::clone(&*thread_obj);
OBJECT_CLONED.store(true, Ordering::Release);
OBJECT_CLONED.set();
});
while !OBJECT_CLONED.load(Ordering::Acquire) {}
OBJECT_CLONED.wait();
assert_eq!(count, obj.get_refcnt(py));
t.join().unwrap();

3
src/impl_/pyclass.rs
View File

@ -156,6 +156,9 @@ pub trait PyClassImpl: Sized {
/// #[pyclass(extends=...)]
const IS_SUBCLASS: bool = false;
/// #[pyclass(mapping)]
const IS_MAPPING: bool = false;
/// Layout
type Layout: PyLayout<Self>;

38
src/instance.rs
View File

@ -540,6 +540,25 @@ impl<T> Py<T> {
/// Retrieves an attribute value.
///
/// This is equivalent to the Python expression `self.attr_name`.
///
/// If calling this method becomes performance-critical, the [`intern!`] macro can be used
/// to intern `attr_name`, thereby avoiding repeated temporary allocations of Python strings.
///
/// # Example: `intern!`ing the attribute name
///
/// ```
/// # use pyo3::{intern, pyfunction, types::PyModule, IntoPy, Py, Python, PyObject, PyResult};
/// #
/// #[pyfunction]
/// fn version(sys: Py<PyModule>, py: Python<'_>) -> PyResult<PyObject> {
/// sys.getattr(py, intern!(py, "version"))
/// }
/// #
/// # Python::with_gil(|py| {
/// # let sys = py.import("sys").unwrap().into_py(py);
/// # version(sys, py).unwrap();
/// # });
/// ```
pub fn getattr<N>(&self, py: Python<'_>, attr_name: N) -> PyResult<PyObject>
where
N: ToPyObject,
@ -552,6 +571,25 @@ impl<T> Py<T> {
/// Sets an attribute value.
///
/// This is equivalent to the Python expression `self.attr_name = value`.
///
/// If calling this method becomes performance-critical, the [`intern!`] macro can be used
/// to intern `attr_name`, thereby avoiding repeated temporary allocations of Python strings.
///
/// # Example: `intern!`ing the attribute name
///
/// ```
/// # use pyo3::{intern, pyfunction, types::PyModule, IntoPy, PyObject, Python, PyResult};
/// #
/// #[pyfunction]
/// fn set_answer(ob: PyObject, py: Python<'_>) -> PyResult<()> {
/// ob.setattr(py, intern!(py, "answer"), 42)
/// }
/// #
/// # Python::with_gil(|py| {
/// # let ob = PyModule::new(py, "empty").unwrap().into_py(py);
/// # set_answer(ob, py).unwrap();
/// # });
/// ```
pub fn setattr<N, V>(&self, py: Python<'_>, attr_name: N, value: V) -> PyResult<()>
where
N: ToPyObject,

2
src/lib.rs
View File

@ -20,7 +20,6 @@
),
allow(unused_variables, unused_assignments)
)))]
#![cfg_attr(coverage, feature(no_coverage))] // used in src/test_hygiene.rs
//! Rust bindings to the Python interpreter.
//!
@ -375,6 +374,7 @@ pub mod impl_;
mod instance;
pub mod marker;
pub mod marshal;
#[macro_use]
pub mod once_cell;
pub mod panic;
pub mod prelude;

89
src/once_cell.rs
View File

@ -42,6 +42,7 @@ impl<T> GILOnceCell<T> {
}
/// Get a reference to the contained value, or `None` if the cell has not yet been written.
#[inline]
pub fn get(&self, _py: Python<'_>) -> Option<&T> {
// Safe because if the cell has not yet been written, None is returned.
unsafe { &*self.0.get() }.as_ref()
@ -61,15 +62,23 @@ impl<T> GILOnceCell<T> {
/// exactly once, even if multiple threads attempt to call `get_or_init`
/// 4) if f() can panic but still does not release the GIL, it may be called multiple times,
/// but it is guaranteed that f() will never be called concurrently
#[inline]
pub fn get_or_init<F>(&self, py: Python<'_>, f: F) -> &T
where
F: FnOnce() -> T,
{
let inner = unsafe { &*self.0.get() }.as_ref();
if let Some(value) = inner {
if let Some(value) = self.get(py) {
return value;
}
self.init(py, f)
}
#[cold]
fn init<F>(&self, py: Python<'_>, f: F) -> &T
where
F: FnOnce() -> T,
{
// Note that f() could temporarily release the GIL, so it's possible that another thread
// writes to this GILOnceCell before f() finishes. That's fine; we'll just have to discard
// the value computed here and accept a bit of wasted computation.
@ -101,3 +110,79 @@ impl<T> GILOnceCell<T> {
Ok(())
}
}
/// Interns `text` as a Python string and stores a reference to it in static storage.
///
/// A reference to the same Python string is returned on each invocation.
///
/// # Example: Using `intern!` to avoid needlessly recreating the same Python string
///
/// ```
/// use pyo3::intern;
/// # use pyo3::{pyfunction, types::PyDict, wrap_pyfunction, PyResult, Python};
///
/// #[pyfunction]
/// fn create_dict(py: Python<'_>) -> PyResult<&PyDict> {
/// let dict = PyDict::new(py);
/// // 👇 A new `PyString` is created
/// // for every call of this function.
/// dict.set_item("foo", 42)?;
/// Ok(dict)
/// }
///
/// #[pyfunction]
/// fn create_dict_faster(py: Python<'_>) -> PyResult<&PyDict> {
/// let dict = PyDict::new(py);
/// // 👇 A `PyString` is created once and reused
/// // for the lifetime of the program.
/// dict.set_item(intern!(py, "foo"), 42)?;
/// Ok(dict)
/// }
/// #
/// # Python::with_gil(|py| {
/// # let fun = wrap_pyfunction!(create_dict_faster, py).unwrap();
/// # let dict = fun.call0().unwrap();
/// # assert!(dict.contains("foo").unwrap());
/// # });
/// ```
#[macro_export]
macro_rules! intern {
($py: expr, $text: expr) => {{
fn isolate_from_dyn_env(py: $crate::Python<'_>) -> &$crate::types::PyString {
static INTERNED: $crate::once_cell::GILOnceCell<$crate::Py<$crate::types::PyString>> =
$crate::once_cell::GILOnceCell::new();
INTERNED
.get_or_init(py, || {
$crate::conversion::IntoPy::into_py(
$crate::types::PyString::intern(py, $text),
py,
)
})
.as_ref(py)
}
isolate_from_dyn_env($py)
}};
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::PyDict;
#[test]
fn test_intern() {
Python::with_gil(|py| {
let foo1 = "foo";
let foo2 = intern!(py, "foo");
let foo3 = intern!(py, stringify!(foo));
let dict = PyDict::new(py);
dict.set_item(foo1, 42_usize).unwrap();
assert!(dict.contains(foo2).unwrap());
assert_eq!(dict.get_item(foo3).unwrap().extract::<usize>().unwrap(), 42);
});
}
}

40
src/pyclass.rs
View File

@ -62,6 +62,7 @@ where
T::weaklist_offset(),
&T::for_all_items,
T::IS_BASETYPE,
T::IS_MAPPING,
)
} {
Ok(type_object) => type_object,
@ -82,6 +83,7 @@ unsafe fn create_type_object_impl(
weaklist_offset: Option<ffi::Py_ssize_t>,
for_all_items: &dyn Fn(&mut dyn FnMut(&PyClassItems)),
is_basetype: bool,
is_mapping: bool,
) -> PyResult<*mut ffi::PyTypeObject> {
let mut slots = Vec::new();
@ -154,26 +156,30 @@ unsafe fn create_type_object_impl(
slots.extend_from_slice(items.slots);
});
// If mapping methods implemented, define sequence methods get implemented too.
// CPython does the same for Python `class` statements.
if !is_mapping {
// If mapping methods implemented, define sequence methods get implemented too.
// CPython does the same for Python `class` statements.
// NB we don't implement sq_length to avoid annoying CPython behaviour of automatically adding
// the length to negative indices.
// NB we don't implement sq_length to avoid annoying CPython behaviour of automatically adding
// the length to negative indices.
if has_getitem {
push_slot(
&mut slots,
ffi::Py_sq_item,
get_sequence_item_from_mapping as _,
);
}
// Don't add these methods for "pure" mappings.
if has_setitem {
push_slot(
&mut slots,
ffi::Py_sq_ass_item,
assign_sequence_item_from_mapping as _,
);
if has_getitem {
push_slot(
&mut slots,
ffi::Py_sq_item,
get_sequence_item_from_mapping as _,
);
}
if has_setitem {
push_slot(
&mut slots,
ffi::Py_sq_ass_item,
assign_sequence_item_from_mapping as _,
);
}
}
if !has_new {

6
src/test_hygiene/misc.rs
View File

@ -27,3 +27,9 @@ enum Derive4 {
crate::create_exception!(mymodule, CustomError, crate::exceptions::PyException);
crate::import_exception!(socket, gaierror);
#[allow(dead_code)]
fn intern(py: crate::Python<'_>) {
let _foo = crate::intern!(py, "foo");
let _bar = crate::intern!(py, stringify!(bar));
}

3
src/test_hygiene/mod.rs
View File

@ -1,9 +1,6 @@
// The modules in this test are used to check PyO3 macro expansion is hygienic. By locating the test
// inside the crate the global `::pyo3` namespace is not available, so in combination with
// #[pyo3(crate = "crate")] this validates that all macro expansion respects the setting.
//
// The generated code is never executed (these tests are checking compile time correctness.)
#![cfg_attr(coverage, no_coverage)]
mod misc;
mod pyclass;

38
src/types/any.rs
View File

@ -111,6 +111,25 @@ impl PyAny {
/// Retrieves an attribute value.
///
/// This is equivalent to the Python expression `self.attr_name`.
///
/// If calling this method becomes performance-critical, the [`intern!`] macro can be used
/// to intern `attr_name`, thereby avoiding repeated temporary allocations of Python strings.
///
/// # Example: `intern!`ing the attribute name
///
/// ```
/// # use pyo3::{intern, pyfunction, types::PyModule, PyAny, Python, PyResult};
/// #
/// #[pyfunction]
/// fn version(sys: &PyModule) -> PyResult<&PyAny> {
/// sys.getattr(intern!(sys.py(), "version"))
/// }
/// #
/// # Python::with_gil(|py| {
/// # let sys = py.import("sys").unwrap();
/// # version(sys).unwrap();
/// # });
/// ```
pub fn getattr<N>(&self, attr_name: N) -> PyResult<&PyAny>
where
N: ToPyObject,
@ -124,6 +143,25 @@ impl PyAny {
/// Sets an attribute value.
///
/// This is equivalent to the Python expression `self.attr_name = value`.
///
/// If calling this method becomes performance-critical, the [`intern!`] macro can be used
/// to intern `attr_name`, thereby avoiding repeated temporary allocations of Python strings.
///
/// # Example: `intern!`ing the attribute name
///
/// ```
/// # use pyo3::{intern, pyfunction, types::PyModule, PyAny, Python, PyResult};
/// #
/// #[pyfunction]
/// fn set_answer(ob: &PyAny) -> PyResult<()> {
/// ob.setattr(intern!(ob.py(), "answer"), 42)
/// }
/// #
/// # Python::with_gil(|py| {
/// # let ob = PyModule::new(py, "empty").unwrap();
/// # set_answer(ob).unwrap();
/// # });
/// ```
pub fn setattr<N, V>(&self, attr_name: N, value: V) -> PyResult<()>
where
N: ToBorrowedObject,

21
src/types/module.rs
View File

@ -8,8 +8,7 @@ use crate::exceptions;
use crate::ffi;
use crate::pyclass::PyClass;
use crate::type_object::PyTypeObject;
use crate::types::PyCFunction;
use crate::types::{PyAny, PyDict, PyList};
use crate::types::{PyAny, PyCFunction, PyDict, PyList, PyString};
use crate::{AsPyPointer, IntoPy, PyObject, Python};
use std::ffi::{CStr, CString};
use std::str;
@ -168,12 +167,13 @@ impl PyModule {
///
/// `__all__` declares the items that will be imported with `from my_module import *`.
pub fn index(&self) -> PyResult<&PyList> {
match self.getattr("__all__") {
let __all__ = __all__(self.py());
match self.getattr(__all__) {
Ok(idx) => idx.downcast().map_err(PyErr::from),
Err(err) => {
if err.is_instance_of::<exceptions::PyAttributeError>(self.py()) {
let l = PyList::empty(self.py());
self.setattr("__all__", l).map_err(PyErr::from)?;
self.setattr(__all__, l).map_err(PyErr::from)?;
Ok(l)
} else {
Err(err)
@ -202,7 +202,6 @@ impl PyModule {
/// May fail if the module does not have a `__file__` attribute.
#[cfg(not(all(windows, PyPy)))]
pub fn filename(&self) -> PyResult<&str> {
use crate::types::PyString;
unsafe {
self.py()
.from_owned_ptr_or_err::<PyString>(ffi::PyModule_GetFilenameObject(self.as_ptr()))?
@ -304,7 +303,7 @@ impl PyModule {
{
let py = self.py();
let function = wrapper(py).convert(py)?;
let name = function.getattr(py, "__name__")?;
let name = function.getattr(py, __name__(py))?;
let name = name.extract(py)?;
self.add(name, function)
}
@ -389,11 +388,19 @@ impl PyModule {
/// [1]: crate::prelude::pyfunction
/// [2]: crate::wrap_pyfunction
pub fn add_function<'a>(&'a self, fun: &'a PyCFunction) -> PyResult<()> {
let name = fun.getattr("__name__")?.extract()?;
let name = fun.getattr(__name__(self.py()))?.extract()?;
self.add(name, fun)
}
}
fn __all__(py: Python<'_>) -> &PyString {
intern!(py, "__all__")
}
fn __name__(py: Python<'_>) -> &PyString {
intern!(py, "__name__")
}
#[cfg(test)]
mod tests {
use crate::{types::PyModule, Python};

38
src/types/string.rs
View File

@ -144,6 +144,26 @@ impl PyString {
unsafe { py.from_owned_ptr(ffi::PyUnicode_FromStringAndSize(ptr, len)) }
}
/// Intern the given string
///
/// This will return a reference to the same Python string object if called repeatedly with the same string.
///
/// Note that while this is more memory efficient than [`PyString::new`], it unconditionally allocates a
/// temporary Python string object and is thereby slower than [`PyString::new`].
///
/// Panics if out of memory.
pub fn intern<'p>(py: Python<'p>, s: &str) -> &'p PyString {
let ptr = s.as_ptr() as *const c_char;
let len = s.len() as ffi::Py_ssize_t;
unsafe {
let mut ob = ffi::PyUnicode_FromStringAndSize(ptr, len);
if !ob.is_null() {
ffi::PyUnicode_InternInPlace(&mut ob);
}
py.from_owned_ptr(ob)
}
}
/// Attempts to create a Python string from a Python [bytes-like object].
///
/// [bytes-like object]: (https://docs.python.org/3/glossary.html#term-bytes-like-object).
@ -592,4 +612,22 @@ mod tests {
assert_eq!(data.to_string_lossy(), Cow::Owned::<str>("𠀀<EFBFBD>".into()));
});
}
#[test]
fn test_intern_string() {
Python::with_gil(|py| {
let py_string1 = PyString::intern(py, "foo");
assert_eq!(py_string1.to_str().unwrap(), "foo");
let py_string2 = PyString::intern(py, "foo");
assert_eq!(py_string2.to_str().unwrap(), "foo");
assert_eq!(py_string1.as_ptr(), py_string2.as_ptr());
let py_string3 = PyString::intern(py, "bar");
assert_eq!(py_string3.to_str().unwrap(), "bar");
assert_ne!(py_string1.as_ptr(), py_string3.as_ptr());
});
}
}

2
src/types/typeobject.rs
View File

@ -37,7 +37,7 @@ impl PyType {
/// Gets the name of the `PyType`.
pub fn name(&self) -> PyResult<&str> {
self.getattr("__qualname__")?.extract()
self.getattr(intern!(self.py(), "__qualname__"))?.extract()
}
/// Checks whether `self` is a subclass of `other`.

10
tests/test_compile_error.rs
View File

@ -50,6 +50,7 @@ fn _test_compile_errors() {
tests_rust_1_56(&t);
tests_rust_1_57(&t);
tests_rust_1_58(&t);
tests_rust_1_60(&t);
#[rustversion::since(1.49)]
fn tests_rust_1_49(t: &trybuild::TestCases) {
@ -82,7 +83,6 @@ fn _test_compile_errors() {
#[rustversion::since(1.58)]
fn tests_rust_1_58(t: &trybuild::TestCases) {
t.compile_fail("tests/ui/invalid_pyfunctions.rs");
t.compile_fail("tests/ui/invalid_pymethod_receiver.rs");
t.compile_fail("tests/ui/invalid_pymethods.rs");
t.compile_fail("tests/ui/missing_clone.rs");
t.compile_fail("tests/ui/not_send.rs");
@ -94,6 +94,14 @@ fn _test_compile_errors() {
#[rustversion::before(1.58)]
fn tests_rust_1_58(_t: &trybuild::TestCases) {}
#[rustversion::since(1.60)]
fn tests_rust_1_60(t: &trybuild::TestCases) {
t.compile_fail("tests/ui/invalid_pymethod_receiver.rs");
}
#[rustversion::before(1.60)]
fn tests_rust_1_60(_t: &trybuild::TestCases) {}
}
#[cfg(feature = "nightly")]

23
tests/test_mapping.rs
View File

@ -7,10 +7,12 @@ use pyo3::prelude::*;
use pyo3::py_run;
use pyo3::types::IntoPyDict;
use pyo3::types::PyList;
use pyo3::types::PyMapping;
use pyo3::types::PySequence;
mod common;
#[pyclass]
#[pyclass(mapping)]
struct Mapping {
index: HashMap<String, usize>,
}
@ -55,6 +57,13 @@ impl Mapping {
Ok(())
}
}
fn get(&self, py: Python<'_>, key: &str, default: Option<PyObject>) -> Option<PyObject> {
self.index
.get(key)
.map(|value| value.into_py(py))
.or(default)
}
}
/// Return a dict with `m = Mapping(['1', '2', '3'])`.
@ -103,3 +112,15 @@ fn test_delitem() {
py_expect_exception!(py, *d, "del m[-1]", PyTypeError);
py_expect_exception!(py, *d, "del m['4']", PyKeyError);
}
#[test]
fn mapping_is_not_sequence() {
Python::with_gil(|py| {
let mut index = HashMap::new();
index.insert("Foo".into(), 1);
index.insert("Bar".into(), 2);
let m = Py::new(py, Mapping { index }).unwrap();
assert!(m.as_ref(py).downcast::<PyMapping>().is_ok());
assert!(m.as_ref(py).downcast::<PySequence>().is_err());
});
}

5
tests/ui/invalid_pyclass_args.stderr
View File

@ -1,4 +1,5 @@
error: expected one of: `crate`, `dict`, `extends`, `freelist`, `immutable`, `module`, `name`, `subclass`, `text_signature`, `unsendable`, `weakref`, `gc`
error: expected one of: `crate`, `dict`, `extends`, `freelist`, `immutable`, 'mapping', `module`, `name`, `subclass`, `text_signature`, `unsendable`, `weakref`, `gc`
--> tests/ui/invalid_pyclass_args.rs:3:11
|
3 | #[pyclass(extend=pyo3::types::PyDict)]
@ -34,7 +35,7 @@ error: expected string literal
18 | #[pyclass(module = my_module)]
| ^^^^^^^^^
error: expected one of: `crate`, `dict`, `extends`, `freelist`, `immutable`, `module`, `name`, `subclass`, `text_signature`, `unsendable`, `weakref`, `gc`
error: expected one of: `crate`, `dict`, `extends`, `freelist`, `immutable`, 'mapping', `module`, `name`, `subclass`, `text_signature`, `unsendable`, `weakref`, `gc`
--> tests/ui/invalid_pyclass_args.rs:21:11
|
21 | #[pyclass(weakrev)]

8
xtask/src/llvm_cov.rs
View File

@ -88,13 +88,5 @@ fn get_coverage_env() -> Result<HashMap<String, String>> {
env.get("CARGO_LLVM_COV_TARGET_DIR").unwrap().to_owned(),
);
// Coverage only works on nightly.
let rustc_version =
String::from_utf8(get_output(Command::new("rustc").arg("--version"))?.stdout)
.context("failed to parse rust version as utf8")?;
if !rustc_version.contains("nightly") {
env.insert("RUSTUP_TOOLCHAIN".to_owned(), "nightly".to_owned());
}
Ok(env)
}