luxolus
/
pyo3
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

pyclass: refactor initialization, call native type dealloc

This commit is contained in:
David Hewitt 2021-05-17 00:31:05 +01:00
parent ab8925572b
commit f916867375
15 changed files with 637 additions and 390 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

148
CHANGELOG.md
View File

@ -1,4 +1,5 @@
# Changelog
All notable changes to this project will be documented in this file. For help with updating to new
PyO3 versions, please see the [migration guide](https://pyo3.rs/main/migration.html).
@ -6,7 +7,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Packaging
- Update `num-bigint` optional dependency to 0.4. [#1481](https://github.com/PyO3/pyo3/pull/1481)
- Update `num-complex` optional dependency to 0.4. [#1482](https://github.com/PyO3/pyo3/pull/1482)
- Extend `hashbrown` optional dependency supported versions to include 0.11. [#1496](https://github.com/PyO3/pyo3/pull/1496)
@ -14,6 +17,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Try harder to filter sysconfigdata candidates on arch.config
### Added
- Add conversions for `[T; N]` for all `N` on Rust 1.51 and up. [#1128](https://github.com/PyO3/pyo3/pull/1128)
- Add conversions between `OsStr`/`OsString`/`Path`/`PathBuf` and Python strings. [#1379](https://github.com/PyO3/pyo3/pull/1379)
- Add `#[pyo3(from_py_with = "...")]` attribute for function arguments and struct fields to override the default from-Python conversion. [#1411](https://github.com/PyO3/pyo3/pull/1411)
@ -29,6 +33,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `#[pyo3(text_signature = "...")]` syntax for setting text signature. [#1658](https://github.com/PyO3/pyo3/pull/1658)
### Changed
- Allow only one `#[pymethods]` block per `#[pyclass]` by default, to simplify the proc macro implementations. Add `multiple-pymethods` feature to opt-in to the more complex full behavior. [#1457](https://github.com/PyO3/pyo3/pull/1457)
- Change `PyTimeAcces::get_fold()` to return a `bool` instead of a `u8`. [#1397](https://github.com/PyO3/pyo3/pull/1397)
- Deprecate FFI definition `PyCFunction_Call` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425)
@ -55,6 +60,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
[#1619](https://github.com/PyO3/pyo3/pull/1619), [#1660](https://github.com/PyO3/pyo3/pull/1660)
### Removed
- Remove deprecated exception names `BaseException` etc. [#1426](https://github.com/PyO3/pyo3/pull/1426)
- Remove deprecated redundant methods `Python::is_instance`, `Python::is_subclass`, `Python::release`, `Python::xdecref`, and `Py::from_owned_ptr_or_panic`. [#1426](https://github.com/PyO3/pyo3/pull/1426)
- Remove many ffi definitions which never existed in the Python C-API:
@ -69,9 +75,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove `__doc__` from module's `__all__`. [#1509](https://github.com/PyO3/pyo3/pull/1509)
- Remove `PYO3_CROSS_INCLUDE_DIR` environment variable and the associated C header parsing functionality. [#1521](https://github.com/PyO3/pyo3/pull/1521)
- Remove `raw_pycfunction!` macro. [#1619](https://github.com/PyO3/pyo3/pull/1619)
- Remove `PyClassAlloc` trait. [#1657](https://github.com/PyO3/pyo3/pull/1657)
- Remove `PyList::get_parked_item`. [#1664](https://github.com/PyO3/pyo3/pull/1664)
### Fixed
- Remove FFI definition `PyCFunction_ClearFreeList` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425)
- `PYO3_CROSS_LIB_DIR` enviroment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. [#1428](https://github.com/PyO3/pyo3/pull/1428)
- Fix FFI definition `_PyEval_RequestCodeExtraIndex` which took an argument of the wrong type. [#1429](https://github.com/PyO3/pyo3/pull/1429)
@ -85,23 +93,29 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix unneccessary rebuilds when cycling between `cargo check` and `cargo clippy` in a Python virtualenv. [#1557](https://github.com/PyO3/pyo3/pull/1557)
- Fix segfault when dereferencing `ffi::PyDateTimeAPI` without the GIL. [#1563](https://github.com/PyO3/pyo3/pull/1563)
- Fix memory leak when converting to u128 and i128. [#1638](https://github.com/PyO3/pyo3/pull/1638)
- Fix `#[pyclass(extends=PyDict)]` leaking the dict contents on drop. [#1657](https://github.com/PyO3/pyo3/pull/1657)
- Fix segfault when calling `PyList::get_item` with negative indices. [#1668](https://github.com/PyO3/pyo3/pull/1668)
## [0.13.2] - 2021-02-12
### Packaging
- Lower minimum supported Rust version to 1.41. [#1421](https://github.com/PyO3/pyo3/pull/1421)
### Added
- Add unsafe API `with_embedded_python_interpreter` to initalize a Python interpreter, execute a closure, and finalize the interpreter. [#1355](https://github.com/PyO3/pyo3/pull/1355)
- Add `serde` feature which provides implementations of `Serialize` and `Deserialize` for `Py<T>`. [#1366](https://github.com/PyO3/pyo3/pull/1366)
- Add FFI definition `_PyCFunctionFastWithKeywords` on Python 3.7 and up. [#1384](https://github.com/PyO3/pyo3/pull/1384)
- Add `PyDateTime::new_with_fold()` method. [#1398](https://github.com/PyO3/pyo3/pull/1398)
### Changed
- `prepare_freethreaded_python` will no longer register an `atexit` handler to call `Py_Finalize`. This resolves a number of issues with incompatible C extensions causing crashes at finalization. [#1355](https://github.com/PyO3/pyo3/pull/1355)
- Mark `PyLayout::py_init`, `PyClassDict::clear_dict`, and `opt_to_pyobj` safe, as they do not perform any unsafe operations. [#1404](https://github.com/PyO3/pyo3/pull/1404)
### Fixed
- Fix support for using `r#raw_idents` as argument names in pyfunctions. [#1383](https://github.com/PyO3/pyo3/pull/1383)
- Fix typo in FFI definition for `PyFunction_GetCode` (was incorrectly `PyFunction_Code`). [#1387](https://github.com/PyO3/pyo3/pull/1387)
- Fix FFI definitions `PyMarshal_WriteObjectToString` and `PyMarshal_ReadObjectFromString` as available in limited API. [#1387](https://github.com/PyO3/pyo3/pull/1387)
@ -111,13 +125,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix build on mingw / MSYS2. [#1423](https://github.com/PyO3/pyo3/pull/1423)
## [0.13.1] - 2021-01-10
### Added
- Add support for `#[pyclass(dict)]` and `#[pyclass(weakref)]` with the `abi3` feature on Python 3.9 and up. [#1342](https://github.com/PyO3/pyo3/pull/1342)
- Add FFI definitions `PyOS_BeforeFork`, `PyOS_AfterFork_Parent`, `PyOS_AfterFork_Child` for Python 3.7 and up. [#1348](https://github.com/PyO3/pyo3/pull/1348)
- Add an `auto-initialize` feature to control whether PyO3 should automatically initialize an embedded Python interpreter. For compatibility this feature is enabled by default in PyO3 0.13.1, but is planned to become opt-in from PyO3 0.14.0. [#1347](https://github.com/PyO3/pyo3/pull/1347)
- Add support for cross-compiling to Windows without needing `PYO3_CROSS_INCLUDE_DIR`. [#1350](https://github.com/PyO3/pyo3/pull/1350)
### Deprecated
- Deprecate FFI definitions `PyEval_CallObjectWithKeywords`, `PyEval_CallObject`, `PyEval_CallFunction`, `PyEval_CallMethod` when building for Python 3.9. [#1338](https://github.com/PyO3/pyo3/pull/1338)
- Deprecate FFI definitions `PyGetSetDef_DICT` and `PyGetSetDef_INIT` which have never been in the Python API. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Deprecate FFI definitions `PyGen_NeedsFinalizing`, `PyImport_Cleanup` (removed in 3.9), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348)
@ -126,11 +143,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Deprecate FFI definitions for `PyUnicode_FromUnicode`, `PyUnicode_AsUnicode` and `PyUnicode_AsUnicodeAndSize`, which will be removed from 3.12 and up due to [PEP 613](https://www.python.org/dev/peps/pep-0623/). [#1370](https://github.com/PyO3/pyo3/pull/1370)
### Removed
- Remove FFI definition `PyFrame_ClearFreeList` when building for Python 3.9. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Remove FFI definition `_PyDict_Contains` when building for Python 3.10. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Remove FFI definitions `PyGen_NeedsFinalizing` and `PyImport_Cleanup` (for 3.9 and up), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348)
### Fixed
- Stop including `Py_TRACE_REFS` config setting automatically if `Py_DEBUG` is set on Python 3.8 and up. [#1334](https://github.com/PyO3/pyo3/pull/1334)
- Remove `#[deny(warnings)]` attribute (and instead refuse warnings only in CI). [#1340](https://github.com/PyO3/pyo3/pull/1340)
- Fix deprecation warning for missing `__module__` with `#[pyclass]`. [#1343](https://github.com/PyO3/pyo3/pull/1343)
@ -138,7 +157,9 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix missing `Py_INCREF` on heap type objects on Python versions before 3.8. [#1365](https://github.com/PyO3/pyo3/pull/1365)
## [0.13.0] - 2020-12-22
### Packaging
- Drop support for Python 3.5 (as it is now end-of-life). [#1250](https://github.com/PyO3/pyo3/pull/1250)
- Bump minimum supported Rust version to 1.45. [#1272](https://github.com/PyO3/pyo3/pull/1272)
- Bump indoc dependency to 1.0. [#1272](https://github.com/PyO3/pyo3/pull/1272)
@ -146,6 +167,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Rename internal crates `pyo3cls` and `pyo3-derive-backend` to `pyo3-macros` and `pyo3-macros-backend` respectively. [#1317](https://github.com/PyO3/pyo3/pull/1317)
### Added
- Add support for building for CPython limited API. Opting-in to the limited API enables a single extension wheel built with PyO3 to be installable on multiple Python versions. This required a few minor changes to runtime behaviour of of PyO3 `#[pyclass]` types. See the migration guide for full details. [#1152](https://github.com/PyO3/pyo3/pull/1152)
- Add feature flags `abi3-py36`, `abi3-py37`, `abi3-py38` etc. to set the minimum Python version when using the limited API. [#1263](https://github.com/PyO3/pyo3/pull/1263)
- Add argument names to `TypeError` messages generated by pymethod wrappers. [#1212](https://github.com/PyO3/pyo3/pull/1212)
@ -160,6 +182,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add conversions for tuples of length 10, 11, and 12. [#1454](https://github.com/PyO3/pyo3/pull/1454)
### Changed
- Change return type of `PyType::name()` from `Cow<str>` to `PyResult<&str>`. [#1152](https://github.com/PyO3/pyo3/pull/1152)
- `#[pyclass(subclass)]` is now required for subclassing from Rust (was previously just required for subclassing from Python). [#1152](https://github.com/PyO3/pyo3/pull/1152)
- Change `PyIterator` to be consistent with other native types: it is now used as `&PyIterator` instead of `PyIterator<'a>`. [#1176](https://github.com/PyO3/pyo3/pull/1176)
@ -171,13 +194,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Require double-quotes for pyclass name argument e.g `#[pyclass(name = "MyClass")]`. [#1303](https://github.com/PyO3/pyo3/pull/1303)
### Deprecated
- Deprecate `Python::is_instance`, `Python::is_subclass`, `Python::release`, and `Python::xdecref`. [#1292](https://github.com/PyO3/pyo3/pull/1292)
### Removed
- Remove deprecated ffi definitions `PyUnicode_AsUnicodeCopy`, `PyUnicode_GetMax`, `_Py_CheckRecursionLimit`, `PyObject_AsCharBuffer`, `PyObject_AsReadBuffer`, `PyObject_CheckReadBuffer` and `PyObject_AsWriteBuffer`, which will be removed in Python 3.10. [#1217](https://github.com/PyO3/pyo3/pull/1217)
- Remove unused `python3` feature. [#1235](https://github.com/PyO3/pyo3/pull/1235)
### Fixed
- Fix missing field in `PyCodeObject` struct (`co_posonlyargcount`) - caused invalid access to other fields in Python >3.7. [#1260](https://github.com/PyO3/pyo3/pull/1260)
- Fix building for `x86_64-unknown-linux-musl` target from `x86_64-unknown-linux-gnu` host. [#1267](https://github.com/PyO3/pyo3/pull/1267)
- Fix `#[text_signature]` interacting badly with rust `r#raw_identifiers`. [#1286](https://github.com/PyO3/pyo3/pull/1286)
@ -187,30 +213,41 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix using custom error type in pyclass `#[new]` methods. [#1319](https://github.com/PyO3/pyo3/pull/1319)
## [0.12.4] - 2020-11-28
### Fixed
- Fix reference count bug in implementation of `From<Py<T>>` for `PyObject`, a regression introduced in PyO3 0.12. [#1297](https://github.com/PyO3/pyo3/pull/1297)
## [0.12.3] - 2020-10-12
### Fixed
- Fix support for Rust versions 1.39 to 1.44, broken by an incorrect internal update to paste 1.0 which was done in PyO3 0.12.2. [#1234](https://github.com/PyO3/pyo3/pull/1234)
## [0.12.2] - 2020-10-12
### Added
- Add support for keyword-only arguments without default values in `#[pyfunction]`. [#1209](https://github.com/PyO3/pyo3/pull/1209)
- Add `Python::check_signals()` as a safe a wrapper for `PyErr_CheckSignals()`. [#1214](https://github.com/PyO3/pyo3/pull/1214)
### Fixed
- Fix invalid document for protocol methods. [#1169](https://github.com/PyO3/pyo3/pull/1169)
- Hide docs of PyO3 private implementation details in `pyo3::class::methods`. [#1169](https://github.com/PyO3/pyo3/pull/1169)
- Fix unnecessary rebuild on PATH changes when the python interpreter is provided by PYO3_PYTHON. [#1231](https://github.com/PyO3/pyo3/pull/1231)
## [0.12.1] - 2020-09-16
### Fixed
- Fix building for a 32-bit Python on 64-bit Windows with a 64-bit Rust toolchain. [#1179](https://github.com/PyO3/pyo3/pull/1179)
- Fix building on platforms where `c_char` is `u8`. [#1182](https://github.com/PyO3/pyo3/pull/1182)
## [0.12.0] - 2020-09-12
### Added
- Add FFI definitions `Py_FinalizeEx`, `PyOS_getsig`, and `PyOS_setsig`. [#1021](https://github.com/PyO3/pyo3/pull/1021)
- Add `PyString::to_str` for accessing `PyString` as `&str`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
- Add `Python::with_gil` for executing a closure with the Python GIL. [#1037](https://github.com/PyO3/pyo3/pull/1037)
@ -226,6 +263,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add native `PyCFunction` and `PyFunction` types. [#1163](https://github.com/PyO3/pyo3/pull/1163)
### Changed
- Rework exception types: [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1115](https://github.com/PyO3/pyo3/pull/1115)
- Rename exception types from e.g. `RuntimeError` to `PyRuntimeError`. The old names continue to exist but are deprecated.
- Exception objects are now accessible as `&T` or `Py<T>`, just like other Python-native types.
@ -250,6 +288,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Change argument to `PyModule::add` to `impl IntoPy<PyObject>` (was `impl ToPyObject`). #[1124](https://github.com/PyO3/pyo3/pull/1124)
### Removed
- Remove many exception and `PyErr` APIs; see the "changed" section above. [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1067](https://github.com/PyO3/pyo3/pull/1067) [#1115](https://github.com/PyO3/pyo3/pull/1115)
- Remove `PyString::to_string` (use new `PyString::to_str`). [#1023](https://github.com/PyO3/pyo3/pull/1023)
- Remove `PyString::as_bytes`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
@ -259,6 +298,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove the `AsPyRef` trait. [#1098](https://github.com/PyO3/pyo3/pull/1098)
### Fixed
- Correct FFI definitions `Py_SetProgramName` and `Py_SetPythonHome` to take `*const` arguments (was `*mut`). [#1021](https://github.com/PyO3/pyo3/pull/1021)
- Fix `FromPyObject` for `num_bigint::BigInt` for Python objects with an `__index__` method. [#1027](https://github.com/PyO3/pyo3/pull/1027)
- Correct FFI definition `_PyLong_AsByteArray` to take `*mut c_uchar` argument (was `*const c_uchar`). [#1029](https://github.com/PyO3/pyo3/pull/1029)
@ -272,14 +312,19 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix building with Anaconda python. [#1175](https://github.com/PyO3/pyo3/pull/1175)
## [0.11.1] - 2020-06-30
### Added
- `#[pyclass(unsendable)]`. [#1009](https://github.com/PyO3/pyo3/pull/1009)
### Changed
- Update `parking_lot` dependency to `0.11`. [#1010](https://github.com/PyO3/pyo3/pull/1010)
## [0.11.0] - 2020-06-28
### Added
- Support stable versions of Rust (>=1.39). [#969](https://github.com/PyO3/pyo3/pull/969)
- Add FFI definition `PyObject_AsFileDescriptor`. [#938](https://github.com/PyO3/pyo3/pull/938)
- Add `PyByteArray::data`, `PyByteArray::as_bytes`, and `PyByteArray::as_bytes_mut`. [#967](https://github.com/PyO3/pyo3/pull/967)
@ -288,6 +333,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `IterNextOutput` and `IterANextOutput` for returning from `__next__` / `__anext__`. [#997](https://github.com/PyO3/pyo3/pull/997)
### Changed
- Simplify internals of `#[pyo3(get)]` attribute. (Remove the hidden API `GetPropertyValue`.) [#934](https://github.com/PyO3/pyo3/pull/934)
- Call `Py_Finalize` at exit to flush buffers, etc. [#943](https://github.com/PyO3/pyo3/pull/943)
- Add type parameter to PyBuffer. #[951](https://github.com/PyO3/pyo3/pull/951)
@ -305,20 +351,26 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `#[pyproto]` methods can now skip annotating the return type if it is `()`. [#998](https://github.com/PyO3/pyo3/pull/998)
### Removed
- Remove `ManagedPyRef` (unused, and needs specialization) [#930](https://github.com/PyO3/pyo3/pull/930)
### Fixed
- Fix passing explicit `None` to `Option<T>` argument `#[pyfunction]` with a default value. [#936](https://github.com/PyO3/pyo3/pull/936)
- Fix `PyClass.__new__`'s not respecting subclasses when inherited by a Python class. [#990](https://github.com/PyO3/pyo3/pull/990)
- Fix returning `Option<T>` from `#[pyproto]` methods. [#996](https://github.com/PyO3/pyo3/pull/996)
- Fix accepting `PyRef<Self>` and `PyRefMut<Self>` to `#[getter]` and `#[setter]` methods. [#999](https://github.com/PyO3/pyo3/pull/999)
## [0.10.1] - 2020-05-14
### Fixed
- Fix deadlock in `Python::acquire_gil()` after dropping a `PyObject` or `Py<T>`. [#924](https://github.com/PyO3/pyo3/pull/924)
## [0.10.0] - 2020-05-13
### Added
- Add FFI definition `_PyDict_NewPresized`. [#849](https://github.com/PyO3/pyo3/pull/849)
- Implement `IntoPy<PyObject>` for `HashSet` and `BTreeSet`. [#864](https://github.com/PyO3/pyo3/pull/864)
- Add `PyAny::dir` method. [#886](https://github.com/PyO3/pyo3/pull/886)
@ -330,6 +382,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `#[classattr]` support for associated constants in `#[pymethods]`. [#914](https://github.com/PyO3/pyo3/pull/914)
### Changed
- Panics will now be raised as a Python `PanicException`. [#797](https://github.com/PyO3/pyo3/pull/797)
- Change `PyObject` and `Py<T>` reference counts to decrement immediately upon drop when the GIL is held. [#851](https://github.com/PyO3/pyo3/pull/851)
- Allow `PyIterProtocol` methods to use either `PyRef` or `PyRefMut` as the receiver type. [#856](https://github.com/PyO3/pyo3/pull/856)
@ -338,6 +391,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove need for `#![feature(specialization)]` in crates depending on PyO3. [#917](https://github.com/PyO3/pyo3/pull/917)
### Removed
- Remove `PyMethodsProtocol` trait. [#889](https://github.com/PyO3/pyo3/pull/889)
- Remove `num-traits` dependency. [#895](https://github.com/PyO3/pyo3/pull/895)
- Remove `ObjectProtocol` trait. [#911](https://github.com/PyO3/pyo3/pull/911)
@ -345,6 +399,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove all `*ProtocolImpl` traits. [#917](https://github.com/PyO3/pyo3/pull/917)
### Fixed
- Fix support for `__radd__` and other `__r*__` methods as implementations for Python mathematical operators. [#839](https://github.com/PyO3/pyo3/pull/839)
- Fix panics during garbage collection when traversing objects that were already mutably borrowed. [#855](https://github.com/PyO3/pyo3/pull/855)
- Prevent `&'static` references to Python objects as arguments to `#[pyfunction]` and `#[pymethods]`. [#869](https://github.com/PyO3/pyo3/pull/869)
@ -355,19 +410,26 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix segmentatation faults when a panic occurs during a call to `Python::allow_threads`. [#912](https://github.com/PyO3/pyo3/pull/912)
## [0.9.2] - 2020-04-09
### Added
- `FromPyObject` implementations for `HashSet` and `BTreeSet`. [#842](https://github.com/PyO3/pyo3/pull/842)
### Fixed
- Correctly detect 32bit architecture. [#830](https://github.com/PyO3/pyo3/pull/830)
## [0.9.1] - 2020-03-23
### Fixed
- Error messages for `#[pyclass]`. [#826](https://github.com/PyO3/pyo3/pull/826)
- `FromPyObject` implementation for `PySequence`. [#827](https://github.com/PyO3/pyo3/pull/827)
## [0.9.0] - 2020-03-19
### Added
- `PyCell`, which has RefCell-like features. [#770](https://github.com/PyO3/pyo3/pull/770)
- `PyClass`, `PyLayout`, `PyClassInitializer`. [#683](https://github.com/PyO3/pyo3/pull/683)
- Implemented `IntoIterator` for `PySet` and `PyFrozenSet`. [#716](https://github.com/PyO3/pyo3/pull/716)
@ -381,6 +443,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `ffi::{_PyBytes_Resize, _PyDict_Next, _PyDict_Contains, _PyDict_GetDictPtr}`. #[820](https://github.com/PyO3/pyo3/pull/820)
### Changed
- `#[new]` does not take `PyRawObject` and can return `Self`. [#683](https://github.com/PyO3/pyo3/pull/683)
- The blanket implementations for `FromPyObject` for `&T` and `&mut T` are no longer specializable. Implement `PyTryFrom` for your type to control the behavior of `FromPyObject::extract()` for your types. [#713](https://github.com/PyO3/pyo3/pull/713)
- The implementation for `IntoPy<U> for T` where `U: FromPy<T>` is no longer specializable. Control the behavior of this via the implementation of `FromPy`. [#713](https://github.com/PyO3/pyo3/pull/713)
@ -391,6 +454,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `PyAny` is now on the top level module and prelude. [#816](https://github.com/PyO3/pyo3/pull/816)
### Removed
- `PyRawObject`. [#683](https://github.com/PyO3/pyo3/pull/683)
- `PyNoArgsFunction`. [#741](https://github.com/PyO3/pyo3/pull/741)
- `initialize_type()`. To set the module name for a `#[pyclass]`, use the `module` argument to the macro. #[751](https://github.com/PyO3/pyo3/pull/751)
@ -400,6 +464,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `ObjectProtocol::get_base/get_mut_base`. [#770](https://github.com/PyO3/pyo3/pull/770)
### Fixed
- Fixed unsoundness of subclassing. [#683](https://github.com/PyO3/pyo3/pull/683).
- Clear error indicator when the exception is handled on the Rust side. [#719](https://github.com/PyO3/pyo3/pull/719)
- Usage of raw identifiers with `#[pyo3(set)]`. [#745](https://github.com/PyO3/pyo3/pull/745)
@ -410,67 +475,87 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix the case where `DESCRIPTION` is not null-terminated. #[822](https://github.com/PyO3/pyo3/pull/822)
## [0.8.5] - 2020-01-05
### Added
- Implemented `FromPyObject` for `HashMap` and `BTreeMap`
- Support for `#[name = "foo"]` attribute for `#[pyfunction]` and in `#[pymethods]`. [#692](https://github.com/PyO3/pyo3/pull/692)
## [0.8.4] - 2019-12-14
### Added
- Support for `#[text_signature]` attribute. [#675](https://github.com/PyO3/pyo3/pull/675)
## [0.8.3] - 2019-11-23
### Removed
- `#[init]` is removed. [#658](https://github.com/PyO3/pyo3/pull/658)
### Fixed
- Now all `&Py~` types have `!Send` bound. [#655](https://github.com/PyO3/pyo3/pull/655)
- Fix a compile error raised by the stabilization of `!` type. [#672](https://github.com/PyO3/pyo3/issues/672).
## [0.8.2] - 2019-10-27
### Added
- FFI compatibility for PEP 590 Vectorcall. [#641](https://github.com/PyO3/pyo3/pull/641)
### Fixed
- Fix PySequenceProtocol::set_item. [#624](https://github.com/PyO3/pyo3/pull/624)
- Fix a corner case of BigInt::FromPyObject. [#630](https://github.com/PyO3/pyo3/pull/630)
- Fix index errors in parameter conversion. [#631](https://github.com/PyO3/pyo3/pull/631)
- Fix handling of invalid utf-8 sequences in `PyString::as_bytes`. [#639](https://github.com/PyO3/pyo3/pull/639)
and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
- Remove `__contains__` and `__iter__` from PyMappingProtocol. [#644](https://github.com/PyO3/pyo3/pull/644)
- Fix proc-macro definition of PySetAttrProtocol. [#645](https://github.com/PyO3/pyo3/pull/645)
## [0.8.1] - 2019-10-08
### Added
- Conversion between [num-bigint](https://github.com/rust-num/num-bigint) and Python int. [#608](https://github.com/PyO3/pyo3/pull/608)
### Fixed
- Make sure the right Python interpreter is used in OSX builds. [#604](https://github.com/PyO3/pyo3/pull/604)
- Patch specialization being broken by Rust 1.40. [#614](https://github.com/PyO3/pyo3/issues/614)
- Fix a segfault around PyErr. [#597](https://github.com/PyO3/pyo3/pull/597)
## [0.8.0] - 2019-09-16
### Added
- `module` argument to `pyclass` macro. [#499](https://github.com/PyO3/pyo3/pull/499)
- `py_run!` macro [#512](https://github.com/PyO3/pyo3/pull/512)
- Use existing fields and methods before calling custom __getattr__. [#505](https://github.com/PyO3/pyo3/pull/505)
- Use existing fields and methods before calling custom **getattr**. [#505](https://github.com/PyO3/pyo3/pull/505)
- `PyBytes` can now be indexed just like `Vec<u8>`
- Implement `IntoPy<PyObject>` for `PyRef` and `PyRefMut`.
### Changed
- Implementing the Using the `gc` parameter for `pyclass` (e.g. `#[pyclass(gc)]`) without implementing the `class::PyGCProtocol` trait is now a compile-time error. Failing to implement this trait could lead to segfaults. [#532](https://github.com/PyO3/pyo3/pull/532)
- `PyByteArray::data` has been replaced with `PyDataArray::to_vec` because returning a `&[u8]` is unsound. (See [this comment](https://github.com/PyO3/pyo3/issues/373#issuecomment-512332696) for a great write-up for why that was unsound)
- Replace `mashup` with `paste`.
- `GILPool` gained a `Python` marker to prevent it from being misused to release Python objects without the GIL held.
### Removed
- `IntoPyObject` was replaced with `IntoPy<PyObject>`
- `#[pyclass(subclass)]` is hidden a `unsound-subclass` feature because it's causing segmentation faults.
### Fixed
- More readable error message for generics in pyclass [#503](https://github.com/PyO3/pyo3/pull/503)
## [0.7.0] - 2019-05-26
### Added
- PyPy support by omerbenamram in [#393](https://github.com/PyO3/pyo3/pull/393)
- Have `PyModule` generate an index of its members (`__all__` list).
- Allow `slf: PyRef<T>` for pyclass(#419)
@ -478,11 +563,13 @@ and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
- Add `marshal` module. [#460](https://github.com/PyO3/pyo3/pull/460)
### Changed
- `Python::run` returns `PyResult<()>` instead of `PyResult<&PyAny>`.
- Methods decorated with `#[getter]` and `#[setter]` can now omit wrapping the
result type in `PyResult` if they don't raise exceptions.
### Fixed
- `type_object::PyTypeObject` has been marked unsafe because breaking the contract `type_object::PyTypeObject::init_type` can lead to UB.
- Fixed automatic derive of `PySequenceProtocol` implementation in [#423](https://github.com/PyO3/pyo3/pull/423).
- Capitalization & better wording to README.md.
@ -496,7 +583,9 @@ and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
tuple/dict now doesn't contain arguments that are otherwise assigned to parameters.
## [0.6.0] - 2019-03-28
### Regressions
- Currently, [#341](https://github.com/PyO3/pyo3/issues/341) causes `cargo test` to fail with weird linking errors when the `extension-module` feature is activated. For now you can work around this by making the `extension-module` feature optional and running the tests with `cargo test --no-default-features`:
```toml
@ -509,6 +598,7 @@ default = ["extension-module"]
```
### Added
- Added a `wrap_pymodule!` macro similar to the existing `wrap_pyfunction!` macro. Only available on python 3
- Added support for cross compiling (e.g. to arm v7) by mtp401 in [#327](https://github.com/PyO3/pyo3/pull/327). See the "Cross Compiling" section in the "Building and Distribution" chapter of the guide for more details.
- The `PyRef` and `PyRefMut` types, which allow to differentiate between an instance of a rust struct on the rust heap and an instance that is embedded inside a python object. By kngwyu in [#335](https://github.com/PyO3/pyo3/pull/335)
@ -516,6 +606,7 @@ default = ["extension-module"]
- Added `ManagedPyRef`, which should eventually replace `ToBorrowedObject`.
### Changed
- Renamed `PyObjectRef` to `PyAny` in #388
- Renamed `add_function` to `add_wrapped` as it now also supports modules.
- Renamed `#[pymodinit]` to `#[pymodule]`
@ -538,29 +629,38 @@ default = ["extension-module"]
- Renamed the `typeob` module to `type_object`
### Removed
- `PyToken` was removed due to unsoundness (See [#94](https://github.com/PyO3/pyo3/issues/94)).
- Removed the unnecessary type parameter from `PyObjectAlloc`
- `NoArgs`. Just use an empty tuple
- `PyObjectWithGIL`. `PyNativeType` is sufficient now that PyToken is removed.
### Fixed
- A soudness hole where every instances of a `#[pyclass]` struct was considered to be part of a python object, even though you can create instances that are not part of the python heap. This was fixed through `PyRef` and `PyRefMut`.
- Fix kwargs support in [#328](https://github.com/PyO3/pyo3/pull/328).
- Add full support for `__dict__` in [#403](https://github.com/PyO3/pyo3/pull/403).
## [0.5.3] - 2019-01-04
### Fixed
- Fix memory leak in ArrayList by kngwyu [#316](https://github.com/PyO3/pyo3/pull/316)
## [0.5.2] - 2018-11-25
### Fixed
- Fix undeterministic segfaults when creating many objects by kngwyu in [#281](https://github.com/PyO3/pyo3/pull/281)
## [0.5.1] - 2018-11-24
Yanked
## [0.5.0] - 2018-11-11
### Added
- `#[pyclass]` objects can now be returned from rust functions
- `PyComplex` by kngwyu in [#226](https://github.com/PyO3/pyo3/pull/226)
- `PyDict::from_sequence()`, equivalent to `dict([(key, val), ...])`
@ -569,6 +669,7 @@ Yanked
- `PyObjectProtocol::get_type_ptr()` by ijl in [#242](https://github.com/PyO3/pyo3/pull/242)
### Changed
- Removes the types from the root module and the prelude. They now live in `pyo3::types` instead.
- All exceptions are consturcted with `py_err` instead of `new`, as they return `PyErr` and not `Self`.
- `as_mut` and friends take and `&mut self` instead of `&self`
@ -582,6 +683,7 @@ Yanked
- Starting to use `NonNull<*mut PyObject>` for Py and PyObject by ijl [#260](https://github.com/PyO3/pyo3/pull/260)
### Removed
- Removed most entries from the prelude. The new prelude is small and clear.
- Slowly removing specialization uses
- `PyString`, `PyUnicode`, and `PyBytes` no longer have a `data()` method
@ -589,6 +691,7 @@ Yanked
- The pyobject_extract macro
### Fixed
- Added an explanation that the GIL can temporarily be released even while holding a GILGuard.
- Lots of clippy errors
- Fix segfault on calling an unknown method on a PyObject
@ -596,38 +699,52 @@ Yanked
- Fixed a segfault with subclassing pyo3 create classes and using `__class__` by kngwyu [#263](https://github.com/PyO3/pyo3/pull/263)
## [0.4.1] - 2018-08-20
### Changed
- PyTryFrom's error is always to `PyDowncastError`
### Fixed
- Fixed compilation on nightly since `use_extern_macros` was stabilized
### Removed
- The pyobject_downcast macro
## [0.4.0] - 2018-07-30
### Changed
- Merged both examples into one
- Rustfmt all the things :heavy_check_mark:
- Switched to [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
### Removed
- Conversions from tuples to PyDict due to [rust-lang/rust#52050](https://github.com/rust-lang/rust/issues/52050)
## [0.3.2] - 2018-07-22
### Changed
- Replaced `concat_idents` with mashup
## [0.3.1] - 2018-07-18
### Fixed
- Fixed scoping bug in pyobject_native_type that would break rust-numpy
## [0.3.0] - 2018-07-18
### Added
- A few internal macros became part of the public api ([#155](https://github.com/PyO3/pyo3/pull/155), [#186](https://github.com/PyO3/pyo3/pull/186))
- Always clone in getters. This allows using the get-annotation on all Clone-Types
### Changed
- Upgraded to syn 0.14 which means much better error messages :tada:
- 128 bit integer support by [kngwyu](https://github.com/kngwyu) ([#137](https://github.com/PyO3/pyo3/pull/173))
- `proc_macro` has been stabilized on nightly ([rust-lang/rust#52081](https://github.com/rust-lang/rust/pull/52081)). This means that we can remove the `proc_macro` feature, but now we need the `use_extern_macros` from the 2018 edition instead.
@ -637,53 +754,73 @@ Yanked
- The guide is now properly versioned.
## [0.2.7] - 2018-05-18
### Fixed
- Fix nightly breakage with proc_macro_path
## [0.2.6] - 2018-04-03
### Fixed
- Fix compatibility with TryFrom trait #137
## [0.2.5] - 2018-02-21
### Added
- CPython 3.7 support
### Fixed
- Embedded CPython 3.7b1 crashes on initialization #110
- Generated extension functions are weakly typed #108
- call_method*() crashes when the method does not exist #113
- call_method\*() crashes when the method does not exist #113
- Allow importing exceptions from nested modules #116
## [0.2.4] - 2018-01-19
### Added
- Allow to get mutable ref from PyObject #106
- Drop `RefFromPyObject` trait
- Add Python::register_any() method
### Fixed
- Fix impl `FromPyObject` for `Py<T>`
- Mark method that work with raw pointers as unsafe #95
## [0.2.3] - 11-27-2017
### Changed
- Rustup to 1.23.0-nightly 2017-11-07
### Fixed
- Proper `c_char` usage #93
### Removed
- Remove use of now unneeded 'AsciiExt' trait
## [0.2.2] - 09-26-2017
### Changed
- Rustup to 1.22.0-nightly 2017-09-30
## [0.2.1] - 09-26-2017
### Fixed
- Fix rustc const_fn nightly breakage
## [0.2.0] - 08-12-2017
### Added
- Added inheritance support #15
- Added weakref support #56
- Added subclass support #64
@ -693,14 +830,17 @@ Yanked
- Introduce IntoPyDictPointer similar to IntoPyTuple #69
### Changed
- Allow to add gc support without implementing PyGCProtocol #57
- Refactor `PyErr` implementation. Drop `py` parameter from constructor.
## [0.1.0] - 07-23-2017
### Added
- Initial release
[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.13.2...HEAD
[unreleased]: https://github.com/pyo3/pyo3/compare/v0.13.2...HEAD
[0.13.2]: https://github.com/pyo3/pyo3/compare/v0.13.1...v0.13.2
[0.13.1]: https://github.com/pyo3/pyo3/compare/v0.13.0...v0.13.1
[0.13.0]: https://github.com/pyo3/pyo3/compare/v0.12.4...v0.13.0

12
guide/src/class.md
View File

@ -720,8 +720,6 @@ struct MyClass {
debug: bool,
}
impl pyo3::pyclass::PyClassAlloc for MyClass {}
unsafe impl pyo3::PyTypeInfo for MyClass {
type AsRefTarget = PyCell<Self>;
@ -774,6 +772,16 @@ impl pyo3::class::impl_::PyClassImpl for MyClass {
let collector = PyClassImplCollector::<Self>::new();
collector.new_impl()
}
fn get_alloc() -> Option<pyo3::ffi::allocfunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.alloc_impl()
}
fn get_free() -> Option<pyo3::ffi::freefunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.free_impl()
}
fn get_call() -> Option<pyo3::ffi::PyCFunctionWithKeywords> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();

40
pyo3-macros-backend/src/pyclass.rs
View File

@ -378,29 +378,37 @@ fn impl_class(
) -> syn::Result<TokenStream> {
let cls_name = get_class_python_name(cls, attr).to_string();
let alloc = {
if let Some(freelist) = &attr.freelist {
let alloc = attr.freelist.as_ref().map(|freelist| {
quote! {
impl pyo3::freelist::PyClassWithFreeList for #cls {
impl pyo3::class::impl_::PyClassWithFreeList for #cls {
#[inline]
fn get_free_list(_py: pyo3::Python) -> &mut pyo3::freelist::FreeList<*mut pyo3::ffi::PyObject> {
static mut FREELIST: *mut pyo3::freelist::FreeList<*mut pyo3::ffi::PyObject> = 0 as *mut _;
fn get_free_list(_py: pyo3::Python) -> &mut pyo3::impl_::freelist::FreeList<*mut pyo3::ffi::PyObject> {
static mut FREELIST: *mut pyo3::impl_::freelist::FreeList<*mut pyo3::ffi::PyObject> = 0 as *mut _;
unsafe {
if FREELIST.is_null() {
FREELIST = Box::into_raw(Box::new(
pyo3::freelist::FreeList::with_capacity(#freelist)));
pyo3::impl_::freelist::FreeList::with_capacity(#freelist)));
}
&mut *FREELIST
}
}
}
}
} else {
quote! {
impl pyo3::pyclass::PyClassAlloc for #cls {}
impl pyo3::class::impl_::PyClassAllocImpl<#cls> for pyo3::class::impl_::PyClassImplCollector<#cls> {
#[inline]
fn alloc_impl(self) -> Option<pyo3::ffi::allocfunc> {
Some(pyo3::class::impl_::alloc_with_freelist::<#cls>)
}
}
};
impl pyo3::class::impl_::PyClassFreeImpl<#cls> for pyo3::class::impl_::PyClassImplCollector<#cls> {
#[inline]
fn free_impl(self) -> Option<pyo3::ffi::freefunc> {
Some(pyo3::class::impl_::free_with_freelist::<#cls>)
}
}
}
});
let descriptors = impl_descriptors(cls, field_options)?;
@ -551,6 +559,16 @@ fn impl_class(
let collector = PyClassImplCollector::<Self>::new();
collector.new_impl()
}
fn get_alloc() -> Option<pyo3::ffi::allocfunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.alloc_impl()
}
fn get_free() -> Option<pyo3::ffi::freefunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.free_impl()
}
fn get_call() -> Option<pyo3::ffi::PyCFunctionWithKeywords> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();

135
src/class/impl_.rs
View File

@ -2,12 +2,13 @@
use crate::{
ffi,
impl_::freelist::FreeList,
pycell::PyCellLayout,
pyclass_init::PyObjectInit,
type_object::{PyLayout, PyTypeObject},
PyClass, PyMethodDefType, PyNativeType, PyTypeInfo,
PyCell, PyClass, PyMethodDefType, PyNativeType, PyTypeInfo, Python,
};
use std::{marker::PhantomData, thread};
use std::{marker::PhantomData, os::raw::c_void, thread};
/// This type is used as a "dummy" type on which dtolnay specializations are
/// applied to apply implementations from `#[pymethods]` & `#[pyproto]`
@ -72,6 +73,12 @@ pub trait PyClassImpl: Sized {
fn get_call() -> Option<ffi::PyCFunctionWithKeywords> {
None
}
fn get_alloc() -> Option<ffi::allocfunc> {
None
}
fn get_free() -> Option<ffi::freefunc> {
None
}
fn for_each_proto_slot(_visitor: &mut dyn FnMut(&[ffi::PyType_Slot])) {}
fn get_buffer() -> Option<&'static PyBufferProcs> {
None
@ -100,6 +107,104 @@ impl<T> PyClassCallImpl<T> for &'_ PyClassImplCollector<T> {
}
}
pub trait PyClassAllocImpl<T> {
fn alloc_impl(self) -> Option<ffi::allocfunc>;
}
impl<T> PyClassAllocImpl<T> for &'_ PyClassImplCollector<T> {
fn alloc_impl(self) -> Option<ffi::allocfunc> {
None
}
}
pub trait PyClassFreeImpl<T> {
fn free_impl(self) -> Option<ffi::freefunc>;
}
impl<T> PyClassFreeImpl<T> for &'_ PyClassImplCollector<T> {
fn free_impl(self) -> Option<ffi::freefunc> {
None
}
}
/// Implements a freelist.
///
/// Do not implement this trait manually. Instead, use `#[pyclass(freelist = N)]`
/// on a Rust struct to implement it.
pub trait PyClassWithFreeList: PyClass {
fn get_free_list(py: Python) -> &mut FreeList<*mut ffi::PyObject>;
}
/// Implementation of tp_alloc for `freelist` classes.
pub unsafe extern "C" fn alloc_with_freelist<T: PyClassWithFreeList>(
subtype: *mut ffi::PyTypeObject,
nitems: ffi::Py_ssize_t,
) -> *mut ffi::PyObject {
let py = Python::assume_gil_acquired();
#[cfg(not(Py_3_8))]
bpo_35810_workaround(py, subtype);
let self_type = T::type_object_raw(py);
// If this type is a variable type or the subtype is not equal to this type, we cannot use the
// freelist
if nitems == 0 && subtype == self_type {
if let Some(obj) = T::get_free_list(py).pop() {
ffi::PyObject_Init(obj, subtype);
return obj as _;
}
}
ffi::PyType_GenericAlloc(subtype, nitems)
}
/// Implementation of tp_free for `freelist` classes.
#[allow(clippy::collapsible_if)] // for if cfg!
pub unsafe extern "C" fn free_with_freelist<T: PyClassWithFreeList>(obj: *mut c_void) {
let obj = obj as *mut ffi::PyObject;
debug_assert_eq!(
T::type_object_raw(Python::assume_gil_acquired()),
ffi::Py_TYPE(obj)
);
if let Some(obj) = T::get_free_list(Python::assume_gil_acquired()).insert(obj) {
let ty = ffi::Py_TYPE(obj);
// Deduce appropriate inverse of PyType_GenericAlloc
let free = if ffi::PyType_IS_GC(ty) != 0 {
ffi::PyObject_GC_Del
} else {
ffi::PyObject_Free
};
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
/// Workaround for Python issue 35810; no longer necessary in Python 3.8
#[inline]
#[cfg(not(Py_3_8))]
unsafe fn bpo_35810_workaround(_py: Python, ty: *mut ffi::PyTypeObject) {
#[cfg(Py_LIMITED_API)]
{
// Must check version at runtime for abi3 wheels - they could run against a higher version
// than the build config suggests.
use crate::once_cell::GILOnceCell;
static IS_PYTHON_3_8: GILOnceCell<bool> = GILOnceCell::new();
if *IS_PYTHON_3_8.get_or_init(_py, || _py.version_info() >= (3, 8)) {
// No fix needed - the wheel is running on a sufficiently new interpreter.
return;
}
}
ffi::Py_INCREF(ty as *mut ffi::PyObject);
}
// General methods implementation: either dtolnay specialization trait or inventory if
// multiple-pymethods feature is enabled.
@ -216,6 +321,7 @@ pub struct ThreadCheckerStub<T: Send>(PhantomData<T>);
impl<T: Send> PyClassThreadChecker<T> for ThreadCheckerStub<T> {
fn ensure(&self) {}
#[inline]
fn new() -> Self {
ThreadCheckerStub(PhantomData)
}
@ -224,6 +330,7 @@ impl<T: Send> PyClassThreadChecker<T> for ThreadCheckerStub<T> {
impl<T: PyNativeType> PyClassThreadChecker<T> for ThreadCheckerStub<crate::PyObject> {
fn ensure(&self) {}
#[inline]
fn new() -> Self {
ThreadCheckerStub(PhantomData)
}
@ -279,8 +386,30 @@ pub trait PyClassBaseType: Sized {
impl<T: PyClass> PyClassBaseType for T {
type Dict = T::Dict;
type WeakRef = T::WeakRef;
type LayoutAsBase = crate::pycell::PyCellInner<T>;
type LayoutAsBase = crate::pycell::PyCell<T>;
type BaseNativeType = T::BaseNativeType;
type ThreadChecker = T::ThreadChecker;
type Initializer = crate::pyclass_init::PyClassInitializer<Self>;
}
// Default new implementation
pub(crate) unsafe extern "C" fn fallback_new(
_subtype: *mut ffi::PyTypeObject,
_args: *mut ffi::PyObject,
_kwds: *mut ffi::PyObject,
) -> *mut ffi::PyObject {
crate::callback_body!(py, {
Err::<(), _>(crate::exceptions::PyTypeError::new_err(
"No constructor defined",
))
})
}
pub(crate) unsafe extern "C" fn tp_dealloc<T: PyClass>(obj: *mut ffi::PyObject) {
crate::callback_body!(py, {
// Safety: Python will only call tp_dealloc when no references to the object remain.
let cell: &mut PyCell<T> = &mut *(obj as *mut _);
cell.tp_dealloc(py);
})
}

1
src/ffi/datetime.rs
View File

@ -650,7 +650,6 @@ mod tests {
fn test_date_fromtimestamp() {
Python::with_gil(|py| {
let args: Py<PyAny> = (100,).into_py(py);
dbg!(args.as_ref(py));
unsafe { PyDateTime_IMPORT() };
let dt: &PyAny = unsafe { py.from_owned_ptr(PyDate_FromTimestamp(args.as_ptr())) };
py_run!(

115
src/freelist.rs
View File

@ -1,115 +0,0 @@
// Copyright (c) 2017-present PyO3 Project and Contributors
//! Support for [free allocation lists][1].
//!
//! This can improve performance for types that are often created and deleted in quick succession.
//!
//! Rather than implementing this manually,
//! implement it by annotating a struct with `#[pyclass(freelist = N)]`,
//! where `N` is the size of the freelist.
//!
//! [1]: https://en.wikipedia.org/wiki/Free_list
use crate::class::impl_::PyClassImpl;
use crate::pyclass::{get_type_free, tp_free_fallback, PyClassAlloc};
use crate::type_object::{PyLayout, PyTypeInfo};
use crate::{ffi, AsPyPointer, FromPyPointer, PyAny, Python};
use std::mem;
use std::os::raw::c_void;
/// Implements a freelist.
///
/// Do not implement this trait manually. Instead, use `#[pyclass(freelist = N)]`
/// on a Rust struct to implement it.
pub trait PyClassWithFreeList {
fn get_free_list(py: Python) -> &mut FreeList<*mut ffi::PyObject>;
}
/// Represents a slot of a [`FreeList`].
pub enum Slot<T> {
Empty,
Filled(T),
}
pub struct FreeList<T> {
entries: Vec<Slot<T>>,
split: usize,
capacity: usize,
}
impl<T> FreeList<T> {
/// Creates a new `FreeList` instance with specified capacity.
pub fn with_capacity(capacity: usize) -> FreeList<T> {
let entries = (0..capacity).map(|_| Slot::Empty).collect::<Vec<_>>();
FreeList {
entries,
split: 0,
capacity,
}
}
/// Pops the first non empty item.
pub fn pop(&mut self) -> Option<T> {
let idx = self.split;
if idx == 0 {
None
} else {
match mem::replace(&mut self.entries[idx - 1], Slot::Empty) {
Slot::Filled(v) => {
self.split = idx - 1;
Some(v)
}
_ => panic!("FreeList is corrupt"),
}
}
}
/// Inserts a value into the list. Returns `None` if the `FreeList` is full.
pub fn insert(&mut self, val: T) -> Option<T> {
let next = self.split + 1;
if next < self.capacity {
self.entries[self.split] = Slot::Filled(val);
self.split = next;
None
} else {
Some(val)
}
}
}
impl<T> PyClassAlloc for T
where
T: PyTypeInfo + PyClassImpl + PyClassWithFreeList,
{
unsafe fn new(py: Python, subtype: *mut ffi::PyTypeObject) -> *mut Self::Layout {
// if subtype is not equal to this type, we cannot use the freelist
if subtype == Self::type_object_raw(py) {
if let Some(obj) = <Self as PyClassWithFreeList>::get_free_list(py).pop() {
ffi::PyObject_Init(obj, subtype);
#[cfg(not(Py_3_8))]
crate::pyclass::bpo_35810_workaround(py, subtype);
return obj as _;
}
}
crate::pyclass::default_new::<Self>(py, subtype) as _
}
#[allow(clippy::collapsible_if)] // for if cfg!
unsafe fn dealloc(py: Python, self_: *mut Self::Layout) {
(*self_).py_drop(py);
let obj = PyAny::from_borrowed_ptr_or_panic(py, self_ as _);
if let Some(obj) = <Self as PyClassWithFreeList>::get_free_list(py).insert(obj.as_ptr()) {
let ty = ffi::Py_TYPE(obj);
let free = get_type_free(ty).unwrap_or_else(|| tp_free_fallback(ty));
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
}

1
src/impl_.rs
View File

@ -3,3 +3,4 @@
//! any time without documentation in the CHANGELOG and without breaking semver guarantees.
pub mod deprecations;
pub mod freelist;

66
src/impl_/freelist.rs Normal file
View File

@ -0,0 +1,66 @@
// Copyright (c) 2017-present PyO3 Project and Contributors
//! Support for [free allocation lists][1].
//!
//! This can improve performance for types that are often created and deleted in quick succession.
//!
//! Rather than implementing this manually,
//! implement it by annotating a struct with `#[pyclass(freelist = N)]`,
//! where `N` is the size of the freelist.
//!
//! [1]: https://en.wikipedia.org/wiki/Free_list
use std::mem;
/// Represents a slot of a [`FreeList`].
pub enum Slot<T> {
Empty,
Filled(T),
}
pub struct FreeList<T> {
entries: Vec<Slot<T>>,
split: usize,
capacity: usize,
}
impl<T> FreeList<T> {
/// Creates a new `FreeList` instance with specified capacity.
pub fn with_capacity(capacity: usize) -> FreeList<T> {
let entries = (0..capacity).map(|_| Slot::Empty).collect::<Vec<_>>();
FreeList {
entries,
split: 0,
capacity,
}
}
/// Pops the first non empty item.
pub fn pop(&mut self) -> Option<T> {
let idx = self.split;
if idx == 0 {
None
} else {
match mem::replace(&mut self.entries[idx - 1], Slot::Empty) {
Slot::Filled(v) => {
self.split = idx - 1;
Some(v)
}
_ => panic!("FreeList is corrupt"),
}
}
}
/// Inserts a value into the list. Returns `None` if the `FreeList` is full.
pub fn insert(&mut self, val: T) -> Option<T> {
let next = self.split + 1;
if next < self.capacity {
self.entries[self.split] = Slot::Filled(val);
self.split = next;
None
} else {
Some(val)
}
}
}

1
src/lib.rs
View File

@ -277,7 +277,6 @@ pub mod derive_utils;
mod err;
pub mod exceptions;
pub mod ffi;
pub mod freelist;
mod gil;
pub mod impl_;
mod instance;

157
src/pycell.rs
View File

@ -1,5 +1,4 @@
//! Includes `PyCell` implementation.
use crate::conversion::{AsPyPointer, FromPyPointer, ToPyObject};
use crate::exceptions::PyRuntimeError;
use crate::pyclass::PyClass;
use crate::pyclass_init::PyClassInitializer;
@ -7,6 +6,12 @@ use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::type_object::{PyLayout, PySizedLayout};
use crate::types::PyAny;
use crate::{class::impl_::PyClassBaseType, class::impl_::PyClassThreadChecker};
use crate::{
conversion::{AsPyPointer, FromPyPointer, ToPyObject},
ffi::PyBaseObject_Type,
type_object::get_tp_free,
PyTypeInfo,
};
use crate::{ffi, IntoPy, PyErr, PyNativeType, PyObject, PyResult, Python};
use std::cell::{Cell, UnsafeCell};
use std::fmt;
@ -22,49 +27,7 @@ pub struct PyCellBase<T> {
borrow_flag: Cell<BorrowFlag>,
}
unsafe impl<T, U> PyLayout<T> for PyCellBase<U>
where
U: PySizedLayout<T>,
{
const IS_NATIVE_TYPE: bool = true;
}
/// Inner type of `PyCell` without dict slots and reference counter.
/// This struct has two usages:
/// 1. As an inner type of `PyRef` and `PyRefMut`.
/// 2. When `#[pyclass(extends=Base)]` is specified, `PyCellInner<Base>` is used as a base layout.
#[doc(hidden)]
#[repr(C)]
pub struct PyCellInner<T: PyClass> {
ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
value: ManuallyDrop<UnsafeCell<T>>,
}
impl<T: PyClass> AsPyPointer for PyCellInner<T> {
fn as_ptr(&self) -> *mut ffi::PyObject {
(self as *const _) as *mut _
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCellInner<T> {
const IS_NATIVE_TYPE: bool = false;
fn py_init(&mut self, value: T) {
self.value = ManuallyDrop::new(UnsafeCell::new(value));
}
unsafe fn py_drop(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.value);
self.ob_base.py_drop(py);
}
}
// These impls ensures `PyCellInner` can be a base type.
impl<T: PyClass> PySizedLayout<T> for PyCellInner<T> {}
impl<T: PyClass> PyCellInner<T> {
fn get_ptr(&self) -> *mut T {
self.value.get()
}
}
unsafe impl<T, U> PyLayout<T> for PyCellBase<U> where U: PySizedLayout<T> {}
/// `PyCell` is the container type for [`PyClass`](../pyclass/trait.PyClass.html).
///
@ -133,12 +96,18 @@ impl<T: PyClass> PyCellInner<T> {
/// ```
#[repr(C)]
pub struct PyCell<T: PyClass> {
inner: PyCellInner<T>,
thread_checker: T::ThreadChecker,
ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
contents: PyCellContents<T>,
}
#[repr(C)]
pub(crate) struct PyCellContents<T: PyClass> {
pub(crate) value: ManuallyDrop<UnsafeCell<T>>,
pub(crate) thread_checker: T::ThreadChecker,
// DO NOT CHANGE THE ORDER OF THESE FIELDS WITHOUT CHANGING PyCell::dict_offset()
// AND PyCell::weakref_offset()
dict: T::Dict,
weakref: T::WeakRef,
pub(crate) dict: T::Dict,
pub(crate) weakref: T::WeakRef,
}
impl<T: PyClass> PyCell<T> {
@ -228,13 +197,12 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub fn try_borrow(&self) -> Result<PyRef<'_, T>, PyBorrowError> {
self.thread_checker.ensure();
let flag = self.inner.get_borrow_flag();
let flag = self.get_borrow_flag();
if flag == BorrowFlag::HAS_MUTABLE_BORROW {
Err(PyBorrowError { _private: () })
} else {
self.inner.set_borrow_flag(flag.increment());
Ok(PyRef { inner: &self.inner })
self.set_borrow_flag(flag.increment());
Ok(PyRef { inner: self })
}
}
@ -260,12 +228,11 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub fn try_borrow_mut(&self) -> Result<PyRefMut<'_, T>, PyBorrowMutError> {
self.thread_checker.ensure();
if self.inner.get_borrow_flag() != BorrowFlag::UNUSED {
if self.get_borrow_flag() != BorrowFlag::UNUSED {
Err(PyBorrowMutError { _private: () })
} else {
self.inner.set_borrow_flag(BorrowFlag::HAS_MUTABLE_BORROW);
Ok(PyRefMut { inner: &self.inner })
self.set_borrow_flag(BorrowFlag::HAS_MUTABLE_BORROW);
Ok(PyRefMut { inner: self })
}
}
@ -299,11 +266,10 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub unsafe fn try_borrow_unguarded(&self) -> Result<&T, PyBorrowError> {
self.thread_checker.ensure();
if self.inner.get_borrow_flag() == BorrowFlag::HAS_MUTABLE_BORROW {
if self.get_borrow_flag() == BorrowFlag::HAS_MUTABLE_BORROW {
Err(PyBorrowError { _private: () })
} else {
Ok(&*self.inner.value.get())
Ok(&*self.contents.value.get())
}
}
@ -338,41 +304,17 @@ impl<T: PyClass> PyCell<T> {
std::mem::swap(&mut *self.borrow_mut(), &mut *other.borrow_mut())
}
/// Allocates a new PyCell given a type object `subtype`. Used by our `tp_new` implementation.
pub(crate) unsafe fn internal_new(
py: Python,
subtype: *mut ffi::PyTypeObject,
) -> PyResult<*mut Self> {
let base = T::new(py, subtype);
if base.is_null() {
return Err(PyErr::fetch(py));
}
let base = base as *mut PyCellBase<T::BaseNativeType>;
(*base).borrow_flag = Cell::new(BorrowFlag::UNUSED);
let self_ = base as *mut Self;
(*self_).dict = T::Dict::new();
(*self_).weakref = T::WeakRef::new();
(*self_).thread_checker = T::ThreadChecker::new();
Ok(self_)
fn get_ptr(&self) -> *mut T {
self.contents.value.get()
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCell<T> {
const IS_NATIVE_TYPE: bool = false;
fn py_init(&mut self, value: T) {
self.inner.value = ManuallyDrop::new(UnsafeCell::new(value));
}
unsafe fn py_drop(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.inner.value);
self.dict.clear_dict(py);
self.weakref.clear_weakrefs(self.as_ptr(), py);
self.inner.ob_base.py_drop(py);
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCell<T> {}
impl<T: PyClass> PySizedLayout<T> for PyCell<T> {}
impl<T: PyClass> AsPyPointer for PyCell<T> {
fn as_ptr(&self) -> *mut ffi::PyObject {
self.inner.as_ptr()
(self as *const _) as *mut _
}
}
@ -453,7 +395,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyCell<T> {
/// # });
/// ```
pub struct PyRef<'p, T: PyClass> {
inner: &'p PyCellInner<T>,
inner: &'p PyCell<T>,
}
impl<'p, T: PyClass> PyRef<'p, T> {
@ -570,7 +512,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyRef<'_, T> {
///
/// See the [`PyCell`](struct.PyCell.html) and [`PyRef`](struct.PyRef.html) documentations for more.
pub struct PyRefMut<'p, T: PyClass> {
inner: &'p PyCellInner<T>,
inner: &'p PyCell<T>,
}
impl<'p, T: PyClass> PyRefMut<'p, T> {
@ -669,7 +611,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyRefMut<'_, T> {
pub struct BorrowFlag(usize);
impl BorrowFlag {
const UNUSED: BorrowFlag = BorrowFlag(0);
pub(crate) const UNUSED: BorrowFlag = BorrowFlag(0);
const HAS_MUTABLE_BORROW: BorrowFlag = BorrowFlag(usize::max_value());
const fn increment(self) -> Self {
Self(self.0 + 1)
@ -733,11 +675,14 @@ impl From<PyBorrowMutError> for PyErr {
pub trait PyCellLayout<T>: PyLayout<T> {
fn get_borrow_flag(&self) -> BorrowFlag;
fn set_borrow_flag(&self, flag: BorrowFlag);
/// Implementation of tp_dealloc. Do not attempt to use &self after calling this!
unsafe fn tp_dealloc(&mut self, py: Python);
}
impl<T, U> PyCellLayout<T> for PyCellBase<U>
where
U: PySizedLayout<T>,
T: PyTypeInfo,
{
fn get_borrow_flag(&self) -> BorrowFlag {
self.borrow_flag.get()
@ -745,16 +690,44 @@ where
fn set_borrow_flag(&self, flag: BorrowFlag) {
self.borrow_flag.set(flag)
}
unsafe fn tp_dealloc(&mut self, py: Python) {
let obj: *mut ffi::PyObject = self as *mut _ as *mut _;
// For `#[pyclass]` types which inherit from PyAny, we can just call tp_free
if T::type_object_raw(py) == &mut PyBaseObject_Type {
return get_tp_free(ffi::Py_TYPE(obj))(obj as _);
}
// More complex native types (e.g. `extends=PyDict`) require calling the base's dealloc.
#[cfg(not(Py_LIMITED_API))]
{
if let Some(dealloc) = (*T::type_object_raw(py)).tp_dealloc {
dealloc(obj as _);
} else {
get_tp_free(ffi::Py_TYPE(obj))(obj as _);
}
}
#[cfg(Py_LIMITED_API)]
unreachable!("subclassing native types is not possible with the `abi3` feature");
}
}
impl<T: PyClass> PyCellLayout<T> for PyCellInner<T>
impl<T: PyClass> PyCellLayout<T> for PyCell<T>
where
<T::BaseType as PyClassBaseType>::LayoutAsBase: PyCellLayout<T::BaseType>,
{
fn get_borrow_flag(&self) -> BorrowFlag {
self.contents.thread_checker.ensure();
self.ob_base.get_borrow_flag()
}
fn set_borrow_flag(&self, flag: BorrowFlag) {
self.ob_base.set_borrow_flag(flag)
}
unsafe fn tp_dealloc(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.contents.value);
self.contents.dict.clear_dict(py);
self.contents.weakref.clear_weakrefs(self.as_ptr(), py);
self.ob_base.tp_dealloc(py);
}
}

155
src/pyclass.rs
View File

@ -1,135 +1,16 @@
//! `PyClass` and related traits.
use crate::class::methods::PyMethodDefType;
use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::type_object::PyLayout;
use crate::{class::impl_::PyClassBaseType, class::impl_::PyClassImpl};
use crate::{ffi, PyCell, PyErr, PyNativeType, PyResult, PyTypeInfo, Python};
use std::convert::TryInto;
use std::ffi::CString;
use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::{mem, ptr};
#[inline]
unsafe fn get_type_alloc(tp: *mut ffi::PyTypeObject) -> Option<ffi::allocfunc> {
mem::transmute(ffi::PyType_GetSlot(tp, ffi::Py_tp_alloc))
}
#[inline]
pub(crate) unsafe fn get_type_free(tp: *mut ffi::PyTypeObject) -> Option<ffi::freefunc> {
mem::transmute(ffi::PyType_GetSlot(tp, ffi::Py_tp_free))
}
/// Workaround for Python issue 35810; no longer necessary in Python 3.8
#[inline]
#[cfg(not(Py_3_8))]
pub(crate) unsafe fn bpo_35810_workaround(_py: Python, ty: *mut ffi::PyTypeObject) {
#[cfg(Py_LIMITED_API)]
{
// Must check version at runtime for abi3 wheels - they could run against a higher version
// than the build config suggests.
use crate::once_cell::GILOnceCell;
static IS_PYTHON_3_8: GILOnceCell<bool> = GILOnceCell::new();
if *IS_PYTHON_3_8.get_or_init(_py, || _py.version_info() >= (3, 8)) {
// No fix needed - the wheel is running on a sufficiently new interpreter.
return;
}
}
ffi::Py_INCREF(ty as *mut ffi::PyObject);
}
#[inline]
pub(crate) unsafe fn default_new<T: PyTypeInfo + PyClassImpl>(
py: Python,
subtype: *mut ffi::PyTypeObject,
) -> *mut ffi::PyObject {
// if the class derives native types(e.g., PyDict), call special new
if T::IS_SUBCLASS && <T::BaseType as PyClassBaseType>::LayoutAsBase::IS_NATIVE_TYPE {
#[cfg(not(Py_LIMITED_API))]
{
let base_tp = T::BaseType::type_object_raw(py);
if let Some(base_new) = (*base_tp).tp_new {
return base_new(subtype, ptr::null_mut(), ptr::null_mut());
}
}
#[cfg(Py_LIMITED_API)]
{
// Silence unused parameter warning.
let _ = py;
unreachable!("Subclassing native types isn't support in limited API mode");
}
}
let alloc = get_type_alloc(subtype).unwrap_or(ffi::PyType_GenericAlloc);
#[cfg(not(Py_3_8))]
bpo_35810_workaround(py, subtype);
alloc(subtype, 0)
}
/// This trait enables custom `tp_new`/`tp_dealloc` implementations for `T: PyClass`.
pub trait PyClassAlloc: PyTypeInfo + PyClassImpl {
/// Allocate the actual field for `#[pyclass]`.
///
/// # Safety
/// This function must return a valid pointer to the Python heap.
unsafe fn new(py: Python, subtype: *mut ffi::PyTypeObject) -> *mut Self::Layout {
default_new::<Self>(py, subtype) as _
}
/// Deallocate `#[pyclass]` on the Python heap.
///
/// # Safety
/// `self_` must be a valid pointer to the Python heap.
#[allow(clippy::collapsible_if)] // for if cfg!
unsafe fn dealloc(py: Python, self_: *mut Self::Layout) {
(*self_).py_drop(py);
let obj = self_ as *mut ffi::PyObject;
let ty = ffi::Py_TYPE(obj);
let free = get_type_free(ty).unwrap_or_else(|| tp_free_fallback(ty));
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
// Default new implementation
unsafe extern "C" fn fallback_new(
_subtype: *mut ffi::PyTypeObject,
_args: *mut ffi::PyObject,
_kwds: *mut ffi::PyObject,
) -> *mut ffi::PyObject {
crate::callback_body!(py, {
Err::<(), _>(crate::exceptions::PyTypeError::new_err(
"No constructor defined",
))
})
}
unsafe extern "C" fn tp_dealloc<T>(obj: *mut ffi::PyObject)
where
T: PyClassAlloc,
{
let pool = crate::GILPool::new();
let py = pool.python();
<T as PyClassAlloc>::dealloc(py, (obj as *mut T::Layout) as _)
}
pub(crate) unsafe fn tp_free_fallback(ty: *mut ffi::PyTypeObject) -> ffi::freefunc {
if ffi::PyType_IS_GC(ty) != 0 {
ffi::PyObject_GC_Del
} else {
ffi::PyObject_Free
}
}
use crate::{
class::impl_::{fallback_new, tp_dealloc, PyClassImpl},
ffi,
pyclass_slots::{PyClassDict, PyClassWeakRef},
PyCell, PyErr, PyMethodDefType, PyNativeType, PyResult, PyTypeInfo, Python,
};
use std::{
convert::TryInto,
ffi::CString,
os::raw::{c_char, c_int, c_uint, c_void},
ptr,
};
/// If `PyClass` is implemented for `T`, then we can use `T` in the Python world,
/// via `PyCell`.
@ -137,7 +18,7 @@ pub(crate) unsafe fn tp_free_fallback(ty: *mut ffi::PyTypeObject) -> ffi::freefu
/// The `#[pyclass]` attribute automatically implements this trait for your Rust struct,
/// so you don't have to use this trait directly.
pub trait PyClass:
PyTypeInfo<AsRefTarget = PyCell<Self>> + Sized + PyClassAlloc + PyClassImpl<Layout = PyCell<Self>>
PyTypeInfo<AsRefTarget = PyCell<Self>> + PyClassImpl<Layout = PyCell<Self>>
{
/// Specify this class has `#[pyclass(dict)]` or not.
type Dict: PyClassDict;
@ -188,12 +69,20 @@ where
let mut slots = TypeSlots::default();
slots.push(ffi::Py_tp_base, T::BaseType::type_object_raw(py) as _);
slots.push(ffi::Py_tp_dealloc, tp_dealloc::<T> as _);
if let Some(doc) = tp_doc::<T>()? {
slots.push(ffi::Py_tp_doc, doc);
}
slots.push(ffi::Py_tp_new, T::get_new().unwrap_or(fallback_new) as _);
slots.push(ffi::Py_tp_dealloc, tp_dealloc::<T> as _);
if let Some(alloc) = T::get_alloc() {
slots.push(ffi::Py_tp_alloc, alloc as _);
}
if let Some(free) = T::get_free() {
slots.push(ffi::Py_tp_free, free as _);
}
if let Some(call_meth) = T::get_call() {
slots.push(ffi::Py_tp_call, call_meth as _);
}

121
src/pyclass_init.rs
View File

@ -1,15 +1,29 @@
//! Initialization utilities for `#[pyclass]`.
use crate::type_object::{PyLayout, PyTypeInfo};
use crate::class::impl_::PyClassThreadChecker;
use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::{callback::IntoPyCallbackOutput, class::impl_::PyClassBaseType};
use crate::{PyCell, PyClass, PyResult, Python};
use std::marker::PhantomData;
use crate::{ffi, PyCell, PyClass, PyErr, PyResult, Python};
use crate::{
ffi::PyTypeObject,
pycell::{BorrowFlag, PyCellContents},
type_object::{get_tp_alloc, PyTypeInfo},
};
use std::{
cell::{Cell, UnsafeCell},
marker::PhantomData,
mem::{ManuallyDrop, MaybeUninit},
};
/// Initializer for Python types.
///
/// This trait is intended to use internally for distinguishing `#[pyclass]` and
/// Python native types.
pub trait PyObjectInit<T>: Sized {
fn init_class<L: PyLayout<T>>(self, layout: &mut L);
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject>;
private_decl! {}
}
@ -17,7 +31,46 @@ pub trait PyObjectInit<T>: Sized {
pub struct PyNativeTypeInitializer<T: PyTypeInfo>(PhantomData<T>);
impl<T: PyTypeInfo> PyObjectInit<T> for PyNativeTypeInitializer<T> {
fn init_class<L: PyLayout<T>>(self, _layout: &mut L) {}
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject> {
let type_object = T::type_object_raw(py);
// HACK (due to FIXME below): PyBaseObject_Type's tp_new isn't happy with NULL arguments
if type_object == (&ffi::PyBaseObject_Type as *const _ as *mut _) {
let alloc = get_tp_alloc(subtype).unwrap_or(ffi::PyType_GenericAlloc);
let obj = alloc(subtype, 0);
return if obj.is_null() {
Err(PyErr::fetch(py))
} else {
Ok(obj)
};
}
#[cfg(Py_LIMITED_API)]
unreachable!("subclassing native types is not possible with the `abi3` feature");
#[cfg(not(Py_LIMITED_API))]
{
match (*type_object).tp_new {
// FIXME: Call __new__ with actual arguments
Some(newfunc) => {
let obj = newfunc(subtype, std::ptr::null_mut(), std::ptr::null_mut());
if obj.is_null() {
Err(PyErr::fetch(py))
} else {
Ok(obj)
}
}
None => Err(crate::exceptions::PyTypeError::new_err(
"base type without tp_new",
)),
}
}
}
private_impl! {}
}
@ -142,22 +195,56 @@ impl<T: PyClass> PyClassInitializer<T> {
where
T: PyClass,
{
let cell = PyCell::internal_new(py, subtype)?;
self.init_class(&mut *cell);
Ok(cell)
self.into_new_object(py, subtype).map(|obj| obj as _)
}
}
impl<T: PyClass> PyObjectInit<T> for PyClassInitializer<T> {
fn init_class<L: PyLayout<T>>(self, layout: &mut L) {
let Self { init, super_init } = self;
// Safety: A valid PyLayout must contain the base layout as the first entry, so casting L to
// T::BaseType::LayoutAsBase is ok.
super_init.init_class(unsafe {
&mut *(layout as *mut _ as *mut <T::BaseType as PyClassBaseType>::LayoutAsBase)
});
layout.py_init(init);
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject> {
/// Layout of a PyCellBase after base new has been called, but borrow flag has not yet been
/// initialized.
#[repr(C)]
struct PartiallyInitializedPyCellBase<T> {
_ob_base: T,
borrow_flag: MaybeUninit<Cell<BorrowFlag>>,
}
/// Layout of a PyCell after base new has been called, but contents have not yet been
/// written.
#[repr(C)]
struct PartiallyInitializedPyCell<T: PyClass> {
_ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
contents: MaybeUninit<PyCellContents<T>>,
}
let Self { init, super_init } = self;
let obj = super_init.into_new_object(py, subtype)?;
// FIXME: Only need to initialize borrow flag once per whole hierarchy
let base: *mut PartiallyInitializedPyCellBase<T::BaseNativeType> = obj as _;
std::ptr::write(
(*base).borrow_flag.as_mut_ptr(),
Cell::new(BorrowFlag::UNUSED),
);
// FIXME: Initialize borrow flag if necessary??
let cell: *mut PartiallyInitializedPyCell<T> = obj as _;
std::ptr::write(
(*cell).contents.as_mut_ptr(),
PyCellContents {
value: ManuallyDrop::new(UnsafeCell::new(init)),
thread_checker: T::ThreadChecker::new(),
dict: T::Dict::new(),
weakref: T::WeakRef::new(),
},
);
Ok(obj)
}
private_impl! {}
}
@ -166,6 +253,7 @@ where
T: PyClass,
T::BaseType: PyClassBaseType<Initializer = PyNativeTypeInitializer<T::BaseType>>,
{
#[inline]
fn from(value: T) -> PyClassInitializer<T> {
Self::new(value, PyNativeTypeInitializer(PhantomData))
}
@ -190,6 +278,7 @@ where
T: PyClass,
U: Into<PyClassInitializer<T>>,
{
#[inline]
fn convert(self, _py: Python) -> PyResult<PyClassInitializer<T>> {
Ok(self.into())
}

8
src/pyclass_slots.rs
View File

@ -6,6 +6,7 @@ use crate::{ffi, Python};
pub trait PyClassDict {
const IS_DUMMY: bool = true;
fn new() -> Self;
#[inline]
fn clear_dict(&mut self, _py: Python) {}
private_decl! {}
}
@ -14,6 +15,7 @@ pub trait PyClassDict {
pub trait PyClassWeakRef {
const IS_DUMMY: bool = true;
fn new() -> Self;
#[inline]
unsafe fn clear_weakrefs(&mut self, _obj: *mut ffi::PyObject, _py: Python) {}
private_decl! {}
}
@ -23,6 +25,7 @@ pub struct PyClassDummySlot;
impl PyClassDict for PyClassDummySlot {
private_impl! {}
#[inline]
fn new() -> Self {
PyClassDummySlot
}
@ -30,6 +33,7 @@ impl PyClassDict for PyClassDummySlot {
impl PyClassWeakRef for PyClassDummySlot {
private_impl! {}
#[inline]
fn new() -> Self {
PyClassDummySlot
}
@ -44,9 +48,11 @@ pub struct PyClassDictSlot(*mut ffi::PyObject);
impl PyClassDict for PyClassDictSlot {
private_impl! {}
const IS_DUMMY: bool = false;
#[inline]
fn new() -> Self {
Self(std::ptr::null_mut())
}
#[inline]
fn clear_dict(&mut self, _py: Python) {
if !self.0.is_null() {
unsafe { ffi::PyDict_Clear(self.0) }
@ -63,9 +69,11 @@ pub struct PyClassWeakRefSlot(*mut ffi::PyObject);
impl PyClassWeakRef for PyClassWeakRefSlot {
private_impl! {}
const IS_DUMMY: bool = false;
#[inline]
fn new() -> Self {
Self(std::ptr::null_mut())
}
#[inline]
unsafe fn clear_weakrefs(&mut self, obj: *mut ffi::PyObject, _py: Python) {
if !self.0.is_null() {
ffi::PyObject_ClearWeakRefs(obj)

38
src/type_object.rs
View File

@ -3,7 +3,8 @@
use crate::internal_tricks::extract_cstr_or_leak_cstring;
use crate::once_cell::GILOnceCell;
use crate::pyclass::{create_type_object, PyClass};
use crate::pyclass::create_type_object;
use crate::pyclass::PyClass;
use crate::types::{PyAny, PyType};
use crate::{conversion::IntoPyPointer, PyMethodDefType};
use crate::{ffi, AsPyPointer, PyErr, PyNativeType, PyObject, PyResult, Python};
@ -15,11 +16,7 @@ use std::thread::{self, ThreadId};
/// is of `PyAny`.
///
/// This trait is intended to be used internally.
pub unsafe trait PyLayout<T> {
const IS_NATIVE_TYPE: bool = true;
fn py_init(&mut self, _value: T) {}
unsafe fn py_drop(&mut self, _py: Python) {}
}
pub unsafe trait PyLayout<T> {}
/// `T: PySizedLayout<U>` represents `T` is not a instance of
/// [`PyVarObject`](https://docs.python.org/3.8/c-api/structures.html?highlight=pyvarobject#c.PyVarObject).
@ -200,3 +197,32 @@ fn initialize_tp_dict(
// This is necessary for making static `LazyStaticType`s
unsafe impl Sync for LazyStaticType {}
#[inline]
pub(crate) unsafe fn get_tp_alloc(tp: *mut ffi::PyTypeObject) -> Option<ffi::allocfunc> {
#[cfg(not(Py_LIMITED_API))]
{
(*tp).tp_alloc
}
#[cfg(Py_LIMITED_API)]
{
let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_alloc);
std::mem::transmute(ptr)
}
}
#[inline]
pub(crate) unsafe fn get_tp_free(tp: *mut ffi::PyTypeObject) -> ffi::freefunc {
#[cfg(not(Py_LIMITED_API))]
{
(*tp).tp_free.unwrap()
}
#[cfg(Py_LIMITED_API)]
{
let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_free);
debug_assert_ne!(ptr, std::ptr::null_mut());
std::mem::transmute(ptr)
}
}

17
tests/test_inheritance.rs
View File

@ -210,6 +210,23 @@ mod inheriting_native_type {
);
}
#[test]
fn inherit_dict_drop() {
Python::with_gil(|py| {
let dict_sub = pyo3::Py::new(py, DictWithName::new()).unwrap();
assert_eq!(dict_sub.get_refcnt(py), 1);
let item = py.eval("object()", None, None).unwrap();
assert_eq!(item.get_refcnt(), 1);
dict_sub.as_ref(py).set_item("foo", item).unwrap();
assert_eq!(item.get_refcnt(), 2);
drop(dict_sub);
assert_eq!(item.get_refcnt(), 1);
})
}
#[pyclass(extends=PyException)]
struct CustomException {
#[pyo3(get)]