luxolus
/
pyo3
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #1657 from davidhewitt/pycell-init 

pyclass: refactor `tp_new` / `tp_dealloc` / remove `PyCellInner`
This commit is contained in:
David Hewitt 2021-06-23 08:22:23 +01:00 committed by GitHub
parent 64a0391fb1 f916867375
commit 2cba18f176
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 661 additions and 390 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
CHANGELOG.md
View File

@ -1,4 +1,5 @@
# Changelog
All notable changes to this project will be documented in this file. For help with updating to new
PyO3 versions, please see the [migration guide](https://pyo3.rs/main/migration.html).
@ -6,7 +7,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Packaging
- Update `num-bigint` optional dependency to 0.4. [#1481](https://github.com/PyO3/pyo3/pull/1481)
- Update `num-complex` optional dependency to 0.4. [#1482](https://github.com/PyO3/pyo3/pull/1482)
- Extend `hashbrown` optional dependency supported versions to include 0.11. [#1496](https://github.com/PyO3/pyo3/pull/1496)
@ -14,6 +17,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Try harder to filter sysconfigdata candidates on arch.config
### Added
- Add conversions for `[T; N]` for all `N` on Rust 1.51 and up. [#1128](https://github.com/PyO3/pyo3/pull/1128)
- Add conversions between `OsStr`/`OsString`/`Path`/`PathBuf` and Python strings. [#1379](https://github.com/PyO3/pyo3/pull/1379)
- Add `#[pyo3(from_py_with = "...")]` attribute for function arguments and struct fields to override the default from-Python conversion. [#1411](https://github.com/PyO3/pyo3/pull/1411)
@ -29,6 +33,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `#[pyo3(text_signature = "...")]` syntax for setting text signature. [#1658](https://github.com/PyO3/pyo3/pull/1658)
### Changed
- Allow only one `#[pymethods]` block per `#[pyclass]` by default, to simplify the proc macro implementations. Add `multiple-pymethods` feature to opt-in to the more complex full behavior. [#1457](https://github.com/PyO3/pyo3/pull/1457)
- Change `PyTimeAcces::get_fold()` to return a `bool` instead of a `u8`. [#1397](https://github.com/PyO3/pyo3/pull/1397)
- Deprecate FFI definition `PyCFunction_Call` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425)
@ -55,6 +60,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
[#1619](https://github.com/PyO3/pyo3/pull/1619), [#1660](https://github.com/PyO3/pyo3/pull/1660)
### Removed
- Remove deprecated exception names `BaseException` etc. [#1426](https://github.com/PyO3/pyo3/pull/1426)
- Remove deprecated redundant methods `Python::is_instance`, `Python::is_subclass`, `Python::release`, `Python::xdecref`, and `Py::from_owned_ptr_or_panic`. [#1426](https://github.com/PyO3/pyo3/pull/1426)
- Remove many ffi definitions which never existed in the Python C-API:
@ -69,9 +75,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove `__doc__` from module's `__all__`. [#1509](https://github.com/PyO3/pyo3/pull/1509)
- Remove `PYO3_CROSS_INCLUDE_DIR` environment variable and the associated C header parsing functionality. [#1521](https://github.com/PyO3/pyo3/pull/1521)
- Remove `raw_pycfunction!` macro. [#1619](https://github.com/PyO3/pyo3/pull/1619)
- Remove `PyClassAlloc` trait. [#1657](https://github.com/PyO3/pyo3/pull/1657)
- Remove `PyList::get_parked_item`. [#1664](https://github.com/PyO3/pyo3/pull/1664)
### Fixed
- Remove FFI definition `PyCFunction_ClearFreeList` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425)
- `PYO3_CROSS_LIB_DIR` enviroment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. [#1428](https://github.com/PyO3/pyo3/pull/1428)
- Fix FFI definition `_PyEval_RequestCodeExtraIndex` which took an argument of the wrong type. [#1429](https://github.com/PyO3/pyo3/pull/1429)
@ -85,23 +93,29 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix unneccessary rebuilds when cycling between `cargo check` and `cargo clippy` in a Python virtualenv. [#1557](https://github.com/PyO3/pyo3/pull/1557)
- Fix segfault when dereferencing `ffi::PyDateTimeAPI` without the GIL. [#1563](https://github.com/PyO3/pyo3/pull/1563)
- Fix memory leak when converting to u128 and i128. [#1638](https://github.com/PyO3/pyo3/pull/1638)
- Fix `#[pyclass(extends=PyDict)]` leaking the dict contents on drop. [#1657](https://github.com/PyO3/pyo3/pull/1657)
- Fix segfault when calling `PyList::get_item` with negative indices. [#1668](https://github.com/PyO3/pyo3/pull/1668)
## [0.13.2] - 2021-02-12
### Packaging
- Lower minimum supported Rust version to 1.41. [#1421](https://github.com/PyO3/pyo3/pull/1421)
### Added
- Add unsafe API `with_embedded_python_interpreter` to initalize a Python interpreter, execute a closure, and finalize the interpreter. [#1355](https://github.com/PyO3/pyo3/pull/1355)
- Add unsafe API `with_embedded_python_interpreter` to initalize a Python interpreter, execute a closure, and finalize the interpreter. [#1355](https://github.com/PyO3/pyo3/pull/1355)
- Add `serde` feature which provides implementations of `Serialize` and `Deserialize` for `Py<T>`. [#1366](https://github.com/PyO3/pyo3/pull/1366)
- Add FFI definition `_PyCFunctionFastWithKeywords` on Python 3.7 and up. [#1384](https://github.com/PyO3/pyo3/pull/1384)
- Add `PyDateTime::new_with_fold()` method. [#1398](https://github.com/PyO3/pyo3/pull/1398)
### Changed
- `prepare_freethreaded_python` will no longer register an `atexit` handler to call `Py_Finalize`. This resolves a number of issues with incompatible C extensions causing crashes at finalization. [#1355](https://github.com/PyO3/pyo3/pull/1355)
- Mark `PyLayout::py_init`, `PyClassDict::clear_dict`, and `opt_to_pyobj` safe, as they do not perform any unsafe operations. [#1404](https://github.com/PyO3/pyo3/pull/1404)
### Fixed
- Fix support for using `r#raw_idents` as argument names in pyfunctions. [#1383](https://github.com/PyO3/pyo3/pull/1383)
- Fix typo in FFI definition for `PyFunction_GetCode` (was incorrectly `PyFunction_Code`). [#1387](https://github.com/PyO3/pyo3/pull/1387)
- Fix FFI definitions `PyMarshal_WriteObjectToString` and `PyMarshal_ReadObjectFromString` as available in limited API. [#1387](https://github.com/PyO3/pyo3/pull/1387)
@ -111,13 +125,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix build on mingw / MSYS2. [#1423](https://github.com/PyO3/pyo3/pull/1423)
## [0.13.1] - 2021-01-10
### Added
- Add support for `#[pyclass(dict)]` and `#[pyclass(weakref)]` with the `abi3` feature on Python 3.9 and up. [#1342](https://github.com/PyO3/pyo3/pull/1342)
- Add FFI definitions `PyOS_BeforeFork`, `PyOS_AfterFork_Parent`, `PyOS_AfterFork_Child` for Python 3.7 and up. [#1348](https://github.com/PyO3/pyo3/pull/1348)
- Add an `auto-initialize` feature to control whether PyO3 should automatically initialize an embedded Python interpreter. For compatibility this feature is enabled by default in PyO3 0.13.1, but is planned to become opt-in from PyO3 0.14.0. [#1347](https://github.com/PyO3/pyo3/pull/1347)
- Add support for cross-compiling to Windows without needing `PYO3_CROSS_INCLUDE_DIR`. [#1350](https://github.com/PyO3/pyo3/pull/1350)
### Deprecated
- Deprecate FFI definitions `PyEval_CallObjectWithKeywords`, `PyEval_CallObject`, `PyEval_CallFunction`, `PyEval_CallMethod` when building for Python 3.9. [#1338](https://github.com/PyO3/pyo3/pull/1338)
- Deprecate FFI definitions `PyGetSetDef_DICT` and `PyGetSetDef_INIT` which have never been in the Python API. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Deprecate FFI definitions `PyGen_NeedsFinalizing`, `PyImport_Cleanup` (removed in 3.9), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348)
@ -126,11 +143,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Deprecate FFI definitions for `PyUnicode_FromUnicode`, `PyUnicode_AsUnicode` and `PyUnicode_AsUnicodeAndSize`, which will be removed from 3.12 and up due to [PEP 613](https://www.python.org/dev/peps/pep-0623/). [#1370](https://github.com/PyO3/pyo3/pull/1370)
### Removed
- Remove FFI definition `PyFrame_ClearFreeList` when building for Python 3.9. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Remove FFI definition `_PyDict_Contains` when building for Python 3.10. [#1341](https://github.com/PyO3/pyo3/pull/1341)
- Remove FFI definitions `PyGen_NeedsFinalizing` and `PyImport_Cleanup` (for 3.9 and up), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348)
### Fixed
- Stop including `Py_TRACE_REFS` config setting automatically if `Py_DEBUG` is set on Python 3.8 and up. [#1334](https://github.com/PyO3/pyo3/pull/1334)
- Remove `#[deny(warnings)]` attribute (and instead refuse warnings only in CI). [#1340](https://github.com/PyO3/pyo3/pull/1340)
- Fix deprecation warning for missing `__module__` with `#[pyclass]`. [#1343](https://github.com/PyO3/pyo3/pull/1343)
@ -138,7 +157,9 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix missing `Py_INCREF` on heap type objects on Python versions before 3.8. [#1365](https://github.com/PyO3/pyo3/pull/1365)
## [0.13.0] - 2020-12-22
### Packaging
- Drop support for Python 3.5 (as it is now end-of-life). [#1250](https://github.com/PyO3/pyo3/pull/1250)
- Bump minimum supported Rust version to 1.45. [#1272](https://github.com/PyO3/pyo3/pull/1272)
- Bump indoc dependency to 1.0. [#1272](https://github.com/PyO3/pyo3/pull/1272)
@ -146,6 +167,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Rename internal crates `pyo3cls` and `pyo3-derive-backend` to `pyo3-macros` and `pyo3-macros-backend` respectively. [#1317](https://github.com/PyO3/pyo3/pull/1317)
### Added
- Add support for building for CPython limited API. Opting-in to the limited API enables a single extension wheel built with PyO3 to be installable on multiple Python versions. This required a few minor changes to runtime behaviour of of PyO3 `#[pyclass]` types. See the migration guide for full details. [#1152](https://github.com/PyO3/pyo3/pull/1152)
- Add feature flags `abi3-py36`, `abi3-py37`, `abi3-py38` etc. to set the minimum Python version when using the limited API. [#1263](https://github.com/PyO3/pyo3/pull/1263)
- Add argument names to `TypeError` messages generated by pymethod wrappers. [#1212](https://github.com/PyO3/pyo3/pull/1212)
@ -160,6 +182,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add conversions for tuples of length 10, 11, and 12. [#1454](https://github.com/PyO3/pyo3/pull/1454)
### Changed
- Change return type of `PyType::name()` from `Cow<str>` to `PyResult<&str>`. [#1152](https://github.com/PyO3/pyo3/pull/1152)
- `#[pyclass(subclass)]` is now required for subclassing from Rust (was previously just required for subclassing from Python). [#1152](https://github.com/PyO3/pyo3/pull/1152)
- Change `PyIterator` to be consistent with other native types: it is now used as `&PyIterator` instead of `PyIterator<'a>`. [#1176](https://github.com/PyO3/pyo3/pull/1176)
@ -171,13 +194,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Require double-quotes for pyclass name argument e.g `#[pyclass(name = "MyClass")]`. [#1303](https://github.com/PyO3/pyo3/pull/1303)
### Deprecated
- Deprecate `Python::is_instance`, `Python::is_subclass`, `Python::release`, and `Python::xdecref`. [#1292](https://github.com/PyO3/pyo3/pull/1292)
### Removed
- Remove deprecated ffi definitions `PyUnicode_AsUnicodeCopy`, `PyUnicode_GetMax`, `_Py_CheckRecursionLimit`, `PyObject_AsCharBuffer`, `PyObject_AsReadBuffer`, `PyObject_CheckReadBuffer` and `PyObject_AsWriteBuffer`, which will be removed in Python 3.10. [#1217](https://github.com/PyO3/pyo3/pull/1217)
- Remove unused `python3` feature. [#1235](https://github.com/PyO3/pyo3/pull/1235)
### Fixed
- Fix missing field in `PyCodeObject` struct (`co_posonlyargcount`) - caused invalid access to other fields in Python >3.7. [#1260](https://github.com/PyO3/pyo3/pull/1260)
- Fix building for `x86_64-unknown-linux-musl` target from `x86_64-unknown-linux-gnu` host. [#1267](https://github.com/PyO3/pyo3/pull/1267)
- Fix `#[text_signature]` interacting badly with rust `r#raw_identifiers`. [#1286](https://github.com/PyO3/pyo3/pull/1286)
@ -187,30 +213,41 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix using custom error type in pyclass `#[new]` methods. [#1319](https://github.com/PyO3/pyo3/pull/1319)
## [0.12.4] - 2020-11-28
### Fixed
- Fix reference count bug in implementation of `From<Py<T>>` for `PyObject`, a regression introduced in PyO3 0.12. [#1297](https://github.com/PyO3/pyo3/pull/1297)
## [0.12.3] - 2020-10-12
### Fixed
- Fix support for Rust versions 1.39 to 1.44, broken by an incorrect internal update to paste 1.0 which was done in PyO3 0.12.2. [#1234](https://github.com/PyO3/pyo3/pull/1234)
## [0.12.2] - 2020-10-12
### Added
- Add support for keyword-only arguments without default values in `#[pyfunction]`. [#1209](https://github.com/PyO3/pyo3/pull/1209)
- Add `Python::check_signals()` as a safe a wrapper for `PyErr_CheckSignals()`. [#1214](https://github.com/PyO3/pyo3/pull/1214)
### Fixed
- Fix invalid document for protocol methods. [#1169](https://github.com/PyO3/pyo3/pull/1169)
- Hide docs of PyO3 private implementation details in `pyo3::class::methods`. [#1169](https://github.com/PyO3/pyo3/pull/1169)
- Fix unnecessary rebuild on PATH changes when the python interpreter is provided by PYO3_PYTHON. [#1231](https://github.com/PyO3/pyo3/pull/1231)
## [0.12.1] - 2020-09-16
### Fixed
- Fix building for a 32-bit Python on 64-bit Windows with a 64-bit Rust toolchain. [#1179](https://github.com/PyO3/pyo3/pull/1179)
- Fix building on platforms where `c_char` is `u8`. [#1182](https://github.com/PyO3/pyo3/pull/1182)
## [0.12.0] - 2020-09-12
### Added
- Add FFI definitions `Py_FinalizeEx`, `PyOS_getsig`, and `PyOS_setsig`. [#1021](https://github.com/PyO3/pyo3/pull/1021)
- Add `PyString::to_str` for accessing `PyString` as `&str`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
- Add `Python::with_gil` for executing a closure with the Python GIL. [#1037](https://github.com/PyO3/pyo3/pull/1037)
@ -226,6 +263,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add native `PyCFunction` and `PyFunction` types. [#1163](https://github.com/PyO3/pyo3/pull/1163)
### Changed
- Rework exception types: [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1115](https://github.com/PyO3/pyo3/pull/1115)
- Rename exception types from e.g. `RuntimeError` to `PyRuntimeError`. The old names continue to exist but are deprecated.
- Exception objects are now accessible as `&T` or `Py<T>`, just like other Python-native types.
@ -250,6 +288,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Change argument to `PyModule::add` to `impl IntoPy<PyObject>` (was `impl ToPyObject`). #[1124](https://github.com/PyO3/pyo3/pull/1124)
### Removed
- Remove many exception and `PyErr` APIs; see the "changed" section above. [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1067](https://github.com/PyO3/pyo3/pull/1067) [#1115](https://github.com/PyO3/pyo3/pull/1115)
- Remove `PyString::to_string` (use new `PyString::to_str`). [#1023](https://github.com/PyO3/pyo3/pull/1023)
- Remove `PyString::as_bytes`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
@ -259,6 +298,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove the `AsPyRef` trait. [#1098](https://github.com/PyO3/pyo3/pull/1098)
### Fixed
- Correct FFI definitions `Py_SetProgramName` and `Py_SetPythonHome` to take `*const` arguments (was `*mut`). [#1021](https://github.com/PyO3/pyo3/pull/1021)
- Fix `FromPyObject` for `num_bigint::BigInt` for Python objects with an `__index__` method. [#1027](https://github.com/PyO3/pyo3/pull/1027)
- Correct FFI definition `_PyLong_AsByteArray` to take `*mut c_uchar` argument (was `*const c_uchar`). [#1029](https://github.com/PyO3/pyo3/pull/1029)
@ -268,18 +308,23 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix many cases of lifetime elision in `#[pyproto]` implementations. [#1093](https://github.com/PyO3/pyo3/pull/1093)
- Fix detection of Python build configuration when cross-compiling. [#1095](https://github.com/PyO3/pyo3/pull/1095)
- Always link against libpython on android with the `extension-module` feature. [#1095](https://github.com/PyO3/pyo3/pull/1095)
- Fix the `+` operator not trying `__radd__` when both `__add__` and `__radd__` are defined in `PyNumberProtocol` (and similar for all other reversible operators). [#1107](https://github.com/PyO3/pyo3/pull/1107)
- Fix the `+` operator not trying `__radd__` when both `__add__` and `__radd__` are defined in `PyNumberProtocol` (and similar for all other reversible operators). [#1107](https://github.com/PyO3/pyo3/pull/1107)
- Fix building with Anaconda python. [#1175](https://github.com/PyO3/pyo3/pull/1175)
## [0.11.1] - 2020-06-30
### Added
- `#[pyclass(unsendable)]`. [#1009](https://github.com/PyO3/pyo3/pull/1009)
### Changed
- Update `parking_lot` dependency to `0.11`. [#1010](https://github.com/PyO3/pyo3/pull/1010)
## [0.11.0] - 2020-06-28
### Added
- Support stable versions of Rust (>=1.39). [#969](https://github.com/PyO3/pyo3/pull/969)
- Add FFI definition `PyObject_AsFileDescriptor`. [#938](https://github.com/PyO3/pyo3/pull/938)
- Add `PyByteArray::data`, `PyByteArray::as_bytes`, and `PyByteArray::as_bytes_mut`. [#967](https://github.com/PyO3/pyo3/pull/967)
@ -288,6 +333,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `IterNextOutput` and `IterANextOutput` for returning from `__next__` / `__anext__`. [#997](https://github.com/PyO3/pyo3/pull/997)
### Changed
- Simplify internals of `#[pyo3(get)]` attribute. (Remove the hidden API `GetPropertyValue`.) [#934](https://github.com/PyO3/pyo3/pull/934)
- Call `Py_Finalize` at exit to flush buffers, etc. [#943](https://github.com/PyO3/pyo3/pull/943)
- Add type parameter to PyBuffer. #[951](https://github.com/PyO3/pyo3/pull/951)
@ -305,20 +351,26 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `#[pyproto]` methods can now skip annotating the return type if it is `()`. [#998](https://github.com/PyO3/pyo3/pull/998)
### Removed
- Remove `ManagedPyRef` (unused, and needs specialization) [#930](https://github.com/PyO3/pyo3/pull/930)
### Fixed
- Fix passing explicit `None` to `Option<T>` argument `#[pyfunction]` with a default value. [#936](https://github.com/PyO3/pyo3/pull/936)
- Fix `PyClass.__new__`'s not respecting subclasses when inherited by a Python class. [#990](https://github.com/PyO3/pyo3/pull/990)
- Fix returning `Option<T>` from `#[pyproto]` methods. [#996](https://github.com/PyO3/pyo3/pull/996)
- Fix accepting `PyRef<Self>` and `PyRefMut<Self>` to `#[getter]` and `#[setter]` methods. [#999](https://github.com/PyO3/pyo3/pull/999)
## [0.10.1] - 2020-05-14
### Fixed
- Fix deadlock in `Python::acquire_gil()` after dropping a `PyObject` or `Py<T>`. [#924](https://github.com/PyO3/pyo3/pull/924)
## [0.10.0] - 2020-05-13
### Added
- Add FFI definition `_PyDict_NewPresized`. [#849](https://github.com/PyO3/pyo3/pull/849)
- Implement `IntoPy<PyObject>` for `HashSet` and `BTreeSet`. [#864](https://github.com/PyO3/pyo3/pull/864)
- Add `PyAny::dir` method. [#886](https://github.com/PyO3/pyo3/pull/886)
@ -330,6 +382,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Add `#[classattr]` support for associated constants in `#[pymethods]`. [#914](https://github.com/PyO3/pyo3/pull/914)
### Changed
- Panics will now be raised as a Python `PanicException`. [#797](https://github.com/PyO3/pyo3/pull/797)
- Change `PyObject` and `Py<T>` reference counts to decrement immediately upon drop when the GIL is held. [#851](https://github.com/PyO3/pyo3/pull/851)
- Allow `PyIterProtocol` methods to use either `PyRef` or `PyRefMut` as the receiver type. [#856](https://github.com/PyO3/pyo3/pull/856)
@ -338,6 +391,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove need for `#![feature(specialization)]` in crates depending on PyO3. [#917](https://github.com/PyO3/pyo3/pull/917)
### Removed
- Remove `PyMethodsProtocol` trait. [#889](https://github.com/PyO3/pyo3/pull/889)
- Remove `num-traits` dependency. [#895](https://github.com/PyO3/pyo3/pull/895)
- Remove `ObjectProtocol` trait. [#911](https://github.com/PyO3/pyo3/pull/911)
@ -345,6 +399,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Remove all `*ProtocolImpl` traits. [#917](https://github.com/PyO3/pyo3/pull/917)
### Fixed
- Fix support for `__radd__` and other `__r*__` methods as implementations for Python mathematical operators. [#839](https://github.com/PyO3/pyo3/pull/839)
- Fix panics during garbage collection when traversing objects that were already mutably borrowed. [#855](https://github.com/PyO3/pyo3/pull/855)
- Prevent `&'static` references to Python objects as arguments to `#[pyfunction]` and `#[pymethods]`. [#869](https://github.com/PyO3/pyo3/pull/869)
@ -355,19 +410,26 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix segmentatation faults when a panic occurs during a call to `Python::allow_threads`. [#912](https://github.com/PyO3/pyo3/pull/912)
## [0.9.2] - 2020-04-09
### Added
- `FromPyObject` implementations for `HashSet` and `BTreeSet`. [#842](https://github.com/PyO3/pyo3/pull/842)
### Fixed
- Correctly detect 32bit architecture. [#830](https://github.com/PyO3/pyo3/pull/830)
## [0.9.1] - 2020-03-23
### Fixed
- Error messages for `#[pyclass]`. [#826](https://github.com/PyO3/pyo3/pull/826)
- `FromPyObject` implementation for `PySequence`. [#827](https://github.com/PyO3/pyo3/pull/827)
## [0.9.0] - 2020-03-19
### Added
- `PyCell`, which has RefCell-like features. [#770](https://github.com/PyO3/pyo3/pull/770)
- `PyClass`, `PyLayout`, `PyClassInitializer`. [#683](https://github.com/PyO3/pyo3/pull/683)
- Implemented `IntoIterator` for `PySet` and `PyFrozenSet`. [#716](https://github.com/PyO3/pyo3/pull/716)
@ -381,6 +443,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `ffi::{_PyBytes_Resize, _PyDict_Next, _PyDict_Contains, _PyDict_GetDictPtr}`. #[820](https://github.com/PyO3/pyo3/pull/820)
### Changed
- `#[new]` does not take `PyRawObject` and can return `Self`. [#683](https://github.com/PyO3/pyo3/pull/683)
- The blanket implementations for `FromPyObject` for `&T` and `&mut T` are no longer specializable. Implement `PyTryFrom` for your type to control the behavior of `FromPyObject::extract()` for your types. [#713](https://github.com/PyO3/pyo3/pull/713)
- The implementation for `IntoPy<U> for T` where `U: FromPy<T>` is no longer specializable. Control the behavior of this via the implementation of `FromPy`. [#713](https://github.com/PyO3/pyo3/pull/713)
@ -391,6 +454,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `PyAny` is now on the top level module and prelude. [#816](https://github.com/PyO3/pyo3/pull/816)
### Removed
- `PyRawObject`. [#683](https://github.com/PyO3/pyo3/pull/683)
- `PyNoArgsFunction`. [#741](https://github.com/PyO3/pyo3/pull/741)
- `initialize_type()`. To set the module name for a `#[pyclass]`, use the `module` argument to the macro. #[751](https://github.com/PyO3/pyo3/pull/751)
@ -400,6 +464,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- `ObjectProtocol::get_base/get_mut_base`. [#770](https://github.com/PyO3/pyo3/pull/770)
### Fixed
- Fixed unsoundness of subclassing. [#683](https://github.com/PyO3/pyo3/pull/683).
- Clear error indicator when the exception is handled on the Rust side. [#719](https://github.com/PyO3/pyo3/pull/719)
- Usage of raw identifiers with `#[pyo3(set)]`. [#745](https://github.com/PyO3/pyo3/pull/745)
@ -410,67 +475,87 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Fix the case where `DESCRIPTION` is not null-terminated. #[822](https://github.com/PyO3/pyo3/pull/822)
## [0.8.5] - 2020-01-05
### Added
- Implemented `FromPyObject` for `HashMap` and `BTreeMap`
- Support for `#[name = "foo"]` attribute for `#[pyfunction]` and in `#[pymethods]`. [#692](https://github.com/PyO3/pyo3/pull/692)
## [0.8.4] - 2019-12-14
### Added
- Support for `#[text_signature]` attribute. [#675](https://github.com/PyO3/pyo3/pull/675)
## [0.8.3] - 2019-11-23
### Removed
- `#[init]` is removed. [#658](https://github.com/PyO3/pyo3/pull/658)
### Fixed
- Now all `&Py~` types have `!Send` bound. [#655](https://github.com/PyO3/pyo3/pull/655)
- Fix a compile error raised by the stabilization of `!` type. [#672](https://github.com/PyO3/pyo3/issues/672).
## [0.8.2] - 2019-10-27
### Added
- FFI compatibility for PEP 590 Vectorcall. [#641](https://github.com/PyO3/pyo3/pull/641)
### Fixed
- Fix PySequenceProtocol::set_item. [#624](https://github.com/PyO3/pyo3/pull/624)
- Fix a corner case of BigInt::FromPyObject. [#630](https://github.com/PyO3/pyo3/pull/630)
- Fix index errors in parameter conversion. [#631](https://github.com/PyO3/pyo3/pull/631)
- Fix handling of invalid utf-8 sequences in `PyString::as_bytes`. [#639](https://github.com/PyO3/pyo3/pull/639)
and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
- Remove `__contains__` and `__iter__` from PyMappingProtocol. [#644](https://github.com/PyO3/pyo3/pull/644)
- Fix proc-macro definition of PySetAttrProtocol. [#645](https://github.com/PyO3/pyo3/pull/645)
## [0.8.1] - 2019-10-08
### Added
- Conversion between [num-bigint](https://github.com/rust-num/num-bigint) and Python int. [#608](https://github.com/PyO3/pyo3/pull/608)
### Fixed
- Make sure the right Python interpreter is used in OSX builds. [#604](https://github.com/PyO3/pyo3/pull/604)
- Patch specialization being broken by Rust 1.40. [#614](https://github.com/PyO3/pyo3/issues/614)
- Fix a segfault around PyErr. [#597](https://github.com/PyO3/pyo3/pull/597)
## [0.8.0] - 2019-09-16
### Added
- `module` argument to `pyclass` macro. [#499](https://github.com/PyO3/pyo3/pull/499)
- `py_run!` macro [#512](https://github.com/PyO3/pyo3/pull/512)
- Use existing fields and methods before calling custom __getattr__. [#505](https://github.com/PyO3/pyo3/pull/505)
- Use existing fields and methods before calling custom **getattr**. [#505](https://github.com/PyO3/pyo3/pull/505)
- `PyBytes` can now be indexed just like `Vec<u8>`
- Implement `IntoPy<PyObject>` for `PyRef` and `PyRefMut`.
### Changed
- Implementing the Using the `gc` parameter for `pyclass` (e.g. `#[pyclass(gc)]`) without implementing the `class::PyGCProtocol` trait is now a compile-time error. Failing to implement this trait could lead to segfaults. [#532](https://github.com/PyO3/pyo3/pull/532)
- `PyByteArray::data` has been replaced with `PyDataArray::to_vec` because returning a `&[u8]` is unsound. (See [this comment](https://github.com/PyO3/pyo3/issues/373#issuecomment-512332696) for a great write-up for why that was unsound)
- Replace `mashup` with `paste`.
- `GILPool` gained a `Python` marker to prevent it from being misused to release Python objects without the GIL held.
### Removed
- `IntoPyObject` was replaced with `IntoPy<PyObject>`
- `#[pyclass(subclass)]` is hidden a `unsound-subclass` feature because it's causing segmentation faults.
### Fixed
- More readable error message for generics in pyclass [#503](https://github.com/PyO3/pyo3/pull/503)
## [0.7.0] - 2019-05-26
### Added
- PyPy support by omerbenamram in [#393](https://github.com/PyO3/pyo3/pull/393)
- Have `PyModule` generate an index of its members (`__all__` list).
- Allow `slf: PyRef<T>` for pyclass(#419)
@ -478,25 +563,29 @@ and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642).
- Add `marshal` module. [#460](https://github.com/PyO3/pyo3/pull/460)
### Changed
- `Python::run` returns `PyResult<()>` instead of `PyResult<&PyAny>`.
- Methods decorated with `#[getter]` and `#[setter]` can now omit wrapping the
result type in `PyResult` if they don't raise exceptions.
result type in `PyResult` if they don't raise exceptions.
### Fixed
- `type_object::PyTypeObject` has been marked unsafe because breaking the contract `type_object::PyTypeObject::init_type` can lead to UB.
- Fixed automatic derive of `PySequenceProtocol` implementation in [#423](https://github.com/PyO3/pyo3/pull/423).
- Capitalization & better wording to README.md.
- Docstrings of properties is now properly set using the doc of the `#[getter]` method.
- Fixed issues with `pymethods` crashing on doc comments containing double quotes.
- `PySet::new` and `PyFrozenSet::new` now return `PyResult<&Py[Frozen]Set>`; exceptions are raised if
the items are not hashable.
the items are not hashable.
- Fixed building using `venv` on Windows.
- `PyTuple::new` now returns `&PyTuple` instead of `Py<PyTuple>`.
- Fixed several issues with argument parsing; notable, the `*args` and `**kwargs`
tuple/dict now doesn't contain arguments that are otherwise assigned to parameters.
tuple/dict now doesn't contain arguments that are otherwise assigned to parameters.
## [0.6.0] - 2019-03-28
### Regressions
- Currently, [#341](https://github.com/PyO3/pyo3/issues/341) causes `cargo test` to fail with weird linking errors when the `extension-module` feature is activated. For now you can work around this by making the `extension-module` feature optional and running the tests with `cargo test --no-default-features`:
```toml
@ -509,6 +598,7 @@ default = ["extension-module"]
```
### Added
- Added a `wrap_pymodule!` macro similar to the existing `wrap_pyfunction!` macro. Only available on python 3
- Added support for cross compiling (e.g. to arm v7) by mtp401 in [#327](https://github.com/PyO3/pyo3/pull/327). See the "Cross Compiling" section in the "Building and Distribution" chapter of the guide for more details.
- The `PyRef` and `PyRefMut` types, which allow to differentiate between an instance of a rust struct on the rust heap and an instance that is embedded inside a python object. By kngwyu in [#335](https://github.com/PyO3/pyo3/pull/335)
@ -516,6 +606,7 @@ default = ["extension-module"]
- Added `ManagedPyRef`, which should eventually replace `ToBorrowedObject`.
### Changed
- Renamed `PyObjectRef` to `PyAny` in #388
- Renamed `add_function` to `add_wrapped` as it now also supports modules.
- Renamed `#[pymodinit]` to `#[pymodule]`
@ -538,29 +629,38 @@ default = ["extension-module"]
- Renamed the `typeob` module to `type_object`
### Removed
- `PyToken` was removed due to unsoundness (See [#94](https://github.com/PyO3/pyo3/issues/94)).
- Removed the unnecessary type parameter from `PyObjectAlloc`
- `NoArgs`. Just use an empty tuple
- `PyObjectWithGIL`. `PyNativeType` is sufficient now that PyToken is removed.
### Fixed
- A soudness hole where every instances of a `#[pyclass]` struct was considered to be part of a python object, even though you can create instances that are not part of the python heap. This was fixed through `PyRef` and `PyRefMut`.
- Fix kwargs support in [#328](https://github.com/PyO3/pyo3/pull/328).
- Add full support for `__dict__` in [#403](https://github.com/PyO3/pyo3/pull/403).
## [0.5.3] - 2019-01-04
### Fixed
- Fix memory leak in ArrayList by kngwyu [#316](https://github.com/PyO3/pyo3/pull/316)
## [0.5.2] - 2018-11-25
### Fixed
- Fix undeterministic segfaults when creating many objects by kngwyu in [#281](https://github.com/PyO3/pyo3/pull/281)
## [0.5.1] - 2018-11-24
Yanked
## [0.5.0] - 2018-11-11
### Added
- `#[pyclass]` objects can now be returned from rust functions
- `PyComplex` by kngwyu in [#226](https://github.com/PyO3/pyo3/pull/226)
- `PyDict::from_sequence()`, equivalent to `dict([(key, val), ...])`
@ -569,6 +669,7 @@ Yanked
- `PyObjectProtocol::get_type_ptr()` by ijl in [#242](https://github.com/PyO3/pyo3/pull/242)
### Changed
- Removes the types from the root module and the prelude. They now live in `pyo3::types` instead.
- All exceptions are consturcted with `py_err` instead of `new`, as they return `PyErr` and not `Self`.
- `as_mut` and friends take and `&mut self` instead of `&self`
@ -582,13 +683,15 @@ Yanked
- Starting to use `NonNull<*mut PyObject>` for Py and PyObject by ijl [#260](https://github.com/PyO3/pyo3/pull/260)
### Removed
- Removed most entries from the prelude. The new prelude is small and clear.
- Slowly removing specialization uses
- `PyString`, `PyUnicode`, and `PyBytes` no longer have a `data()` method
(replaced by `as_bytes()`) and `PyStringData` has been removed.
(replaced by `as_bytes()`) and `PyStringData` has been removed.
- The pyobject_extract macro
### Fixed
- Added an explanation that the GIL can temporarily be released even while holding a GILGuard.
- Lots of clippy errors
- Fix segfault on calling an unknown method on a PyObject
@ -596,38 +699,52 @@ Yanked
- Fixed a segfault with subclassing pyo3 create classes and using `__class__` by kngwyu [#263](https://github.com/PyO3/pyo3/pull/263)
## [0.4.1] - 2018-08-20
### Changed
- PyTryFrom's error is always to `PyDowncastError`
### Fixed
- Fixed compilation on nightly since `use_extern_macros` was stabilized
### Removed
- The pyobject_downcast macro
## [0.4.0] - 2018-07-30
### Changed
- Merged both examples into one
- Rustfmt all the things :heavy_check_mark:
- Switched to [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
### Removed
- Conversions from tuples to PyDict due to [rust-lang/rust#52050](https://github.com/rust-lang/rust/issues/52050)
## [0.3.2] - 2018-07-22
### Changed
- Replaced `concat_idents` with mashup
## [0.3.1] - 2018-07-18
### Fixed
- Fixed scoping bug in pyobject_native_type that would break rust-numpy
## [0.3.0] - 2018-07-18
### Added
- A few internal macros became part of the public api ([#155](https://github.com/PyO3/pyo3/pull/155), [#186](https://github.com/PyO3/pyo3/pull/186))
- Always clone in getters. This allows using the get-annotation on all Clone-Types
### Changed
- Upgraded to syn 0.14 which means much better error messages :tada:
- 128 bit integer support by [kngwyu](https://github.com/kngwyu) ([#137](https://github.com/PyO3/pyo3/pull/173))
- `proc_macro` has been stabilized on nightly ([rust-lang/rust#52081](https://github.com/rust-lang/rust/pull/52081)). This means that we can remove the `proc_macro` feature, but now we need the `use_extern_macros` from the 2018 edition instead.
@ -637,53 +754,73 @@ Yanked
- The guide is now properly versioned.
## [0.2.7] - 2018-05-18
### Fixed
- Fix nightly breakage with proc_macro_path
## [0.2.6] - 2018-04-03
### Fixed
- Fix compatibility with TryFrom trait #137
## [0.2.5] - 2018-02-21
### Added
- CPython 3.7 support
### Fixed
- Embedded CPython 3.7b1 crashes on initialization #110
- Generated extension functions are weakly typed #108
- call_method*() crashes when the method does not exist #113
- call_method\*() crashes when the method does not exist #113
- Allow importing exceptions from nested modules #116
## [0.2.4] - 2018-01-19
### Added
- Allow to get mutable ref from PyObject #106
- Drop `RefFromPyObject` trait
- Add Python::register_any() method
### Fixed
- Fix impl `FromPyObject` for `Py<T>`
- Mark method that work with raw pointers as unsafe #95
## [0.2.3] - 11-27-2017
### Changed
- Rustup to 1.23.0-nightly 2017-11-07
### Fixed
- Proper `c_char` usage #93
### Removed
- Remove use of now unneeded 'AsciiExt' trait
## [0.2.2] - 09-26-2017
### Changed
- Rustup to 1.22.0-nightly 2017-09-30
## [0.2.1] - 09-26-2017
### Fixed
- Fix rustc const_fn nightly breakage
## [0.2.0] - 08-12-2017
### Added
- Added inheritance support #15
- Added weakref support #56
- Added subclass support #64
@ -693,14 +830,17 @@ Yanked
- Introduce IntoPyDictPointer similar to IntoPyTuple #69
### Changed
- Allow to add gc support without implementing PyGCProtocol #57
- Refactor `PyErr` implementation. Drop `py` parameter from constructor.
## [0.1.0] - 07-23-2017
### Added
- Initial release
[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.13.2...HEAD
[unreleased]: https://github.com/pyo3/pyo3/compare/v0.13.2...HEAD
[0.13.2]: https://github.com/pyo3/pyo3/compare/v0.13.1...v0.13.2
[0.13.1]: https://github.com/pyo3/pyo3/compare/v0.13.0...v0.13.1
[0.13.0]: https://github.com/pyo3/pyo3/compare/v0.12.4...v0.13.0

12
examples/pyo3-benchmarks/src/lib.rs
View File

@ -55,6 +55,17 @@ fn args_kwargs<'a>(
(args, kwargs)
}
#[pyclass]
struct EmptyClass {}
#[pymethods]
impl EmptyClass {
#[new]
fn new() -> Self {
EmptyClass {}
}
}
#[pymodule]
fn _pyo3_benchmarks(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(none, m)?)?;
@ -63,5 +74,6 @@ fn _pyo3_benchmarks(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(simple_kwargs, m)?)?;
m.add_function(wrap_pyfunction!(simple_args_kwargs, m)?)?;
m.add_function(wrap_pyfunction!(args_kwargs, m)?)?;
m.add_class::<EmptyClass>()?;
Ok(())
}

12
examples/pyo3-benchmarks/tests/test_benchmarks.py
View File

@ -89,3 +89,15 @@ def test_args_kwargs_rs(benchmark):
py = args_kwargs_py(1, "foo", {1: 2}, bar=4, foo=10)
assert rust == py
benchmark(pyo3_benchmarks.args_kwargs, 1, "foo", {1: 2}, a=4, foo=10)
def test_empty_class_init(benchmark):
benchmark(pyo3_benchmarks.EmptyClass)
class EmptyClassPy:
pass
def test_empty_class_init_py(benchmark):
benchmark(EmptyClassPy)

12
guide/src/class.md
View File

@ -720,8 +720,6 @@ struct MyClass {
debug: bool,
}
impl pyo3::pyclass::PyClassAlloc for MyClass {}
unsafe impl pyo3::PyTypeInfo for MyClass {
type AsRefTarget = PyCell<Self>;
@ -774,6 +772,16 @@ impl pyo3::class::impl_::PyClassImpl for MyClass {
let collector = PyClassImplCollector::<Self>::new();
collector.new_impl()
}
fn get_alloc() -> Option<pyo3::ffi::allocfunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.alloc_impl()
}
fn get_free() -> Option<pyo3::ffi::freefunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.free_impl()
}
fn get_call() -> Option<pyo3::ffi::PyCFunctionWithKeywords> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();

42
pyo3-macros-backend/src/pyclass.rs
View File

@ -378,29 +378,37 @@ fn impl_class(
) -> syn::Result<TokenStream> {
let cls_name = get_class_python_name(cls, attr).to_string();
let alloc = {
if let Some(freelist) = &attr.freelist {
let alloc = attr.freelist.as_ref().map(|freelist| {
quote! {
impl pyo3::freelist::PyClassWithFreeList for #cls {
impl pyo3::class::impl_::PyClassWithFreeList for #cls {
#[inline]
fn get_free_list(_py: pyo3::Python) -> &mut pyo3::freelist::FreeList<*mut pyo3::ffi::PyObject> {
static mut FREELIST: *mut pyo3::freelist::FreeList<*mut pyo3::ffi::PyObject> = 0 as *mut _;
fn get_free_list(_py: pyo3::Python) -> &mut pyo3::impl_::freelist::FreeList<*mut pyo3::ffi::PyObject> {
static mut FREELIST: *mut pyo3::impl_::freelist::FreeList<*mut pyo3::ffi::PyObject> = 0 as *mut _;
unsafe {
if FREELIST.is_null() {
FREELIST = Box::into_raw(Box::new(
pyo3::freelist::FreeList::with_capacity(#freelist)));
pyo3::impl_::freelist::FreeList::with_capacity(#freelist)));
}
&mut *FREELIST
}
}
}
impl pyo3::class::impl_::PyClassAllocImpl<#cls> for pyo3::class::impl_::PyClassImplCollector<#cls> {
#[inline]
fn alloc_impl(self) -> Option<pyo3::ffi::allocfunc> {
Some(pyo3::class::impl_::alloc_with_freelist::<#cls>)
}
}
impl pyo3::class::impl_::PyClassFreeImpl<#cls> for pyo3::class::impl_::PyClassImplCollector<#cls> {
#[inline]
fn free_impl(self) -> Option<pyo3::ffi::freefunc> {
Some(pyo3::class::impl_::free_with_freelist::<#cls>)
}
}
}
} else {
quote! {
impl pyo3::pyclass::PyClassAlloc for #cls {}
}
}
};
});
let descriptors = impl_descriptors(cls, field_options)?;
@ -551,6 +559,16 @@ fn impl_class(
let collector = PyClassImplCollector::<Self>::new();
collector.new_impl()
}
fn get_alloc() -> Option<pyo3::ffi::allocfunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.alloc_impl()
}
fn get_free() -> Option<pyo3::ffi::freefunc> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();
collector.free_impl()
}
fn get_call() -> Option<pyo3::ffi::PyCFunctionWithKeywords> {
use pyo3::class::impl_::*;
let collector = PyClassImplCollector::<Self>::new();

135
src/class/impl_.rs
View File

@ -2,12 +2,13 @@
use crate::{
ffi,
impl_::freelist::FreeList,
pycell::PyCellLayout,
pyclass_init::PyObjectInit,
type_object::{PyLayout, PyTypeObject},
PyClass, PyMethodDefType, PyNativeType, PyTypeInfo,
PyCell, PyClass, PyMethodDefType, PyNativeType, PyTypeInfo, Python,
};
use std::{marker::PhantomData, thread};
use std::{marker::PhantomData, os::raw::c_void, thread};
/// This type is used as a "dummy" type on which dtolnay specializations are
/// applied to apply implementations from `#[pymethods]` & `#[pyproto]`
@ -72,6 +73,12 @@ pub trait PyClassImpl: Sized {
fn get_call() -> Option<ffi::PyCFunctionWithKeywords> {
None
}
fn get_alloc() -> Option<ffi::allocfunc> {
None
}
fn get_free() -> Option<ffi::freefunc> {
None
}
fn for_each_proto_slot(_visitor: &mut dyn FnMut(&[ffi::PyType_Slot])) {}
fn get_buffer() -> Option<&'static PyBufferProcs> {
None
@ -100,6 +107,104 @@ impl<T> PyClassCallImpl<T> for &'_ PyClassImplCollector<T> {
}
}
pub trait PyClassAllocImpl<T> {
fn alloc_impl(self) -> Option<ffi::allocfunc>;
}
impl<T> PyClassAllocImpl<T> for &'_ PyClassImplCollector<T> {
fn alloc_impl(self) -> Option<ffi::allocfunc> {
None
}
}
pub trait PyClassFreeImpl<T> {
fn free_impl(self) -> Option<ffi::freefunc>;
}
impl<T> PyClassFreeImpl<T> for &'_ PyClassImplCollector<T> {
fn free_impl(self) -> Option<ffi::freefunc> {
None
}
}
/// Implements a freelist.
///
/// Do not implement this trait manually. Instead, use `#[pyclass(freelist = N)]`
/// on a Rust struct to implement it.
pub trait PyClassWithFreeList: PyClass {
fn get_free_list(py: Python) -> &mut FreeList<*mut ffi::PyObject>;
}
/// Implementation of tp_alloc for `freelist` classes.
pub unsafe extern "C" fn alloc_with_freelist<T: PyClassWithFreeList>(
subtype: *mut ffi::PyTypeObject,
nitems: ffi::Py_ssize_t,
) -> *mut ffi::PyObject {
let py = Python::assume_gil_acquired();
#[cfg(not(Py_3_8))]
bpo_35810_workaround(py, subtype);
let self_type = T::type_object_raw(py);
// If this type is a variable type or the subtype is not equal to this type, we cannot use the
// freelist
if nitems == 0 && subtype == self_type {
if let Some(obj) = T::get_free_list(py).pop() {
ffi::PyObject_Init(obj, subtype);
return obj as _;
}
}
ffi::PyType_GenericAlloc(subtype, nitems)
}
/// Implementation of tp_free for `freelist` classes.
#[allow(clippy::collapsible_if)] // for if cfg!
pub unsafe extern "C" fn free_with_freelist<T: PyClassWithFreeList>(obj: *mut c_void) {
let obj = obj as *mut ffi::PyObject;
debug_assert_eq!(
T::type_object_raw(Python::assume_gil_acquired()),
ffi::Py_TYPE(obj)
);
if let Some(obj) = T::get_free_list(Python::assume_gil_acquired()).insert(obj) {
let ty = ffi::Py_TYPE(obj);
// Deduce appropriate inverse of PyType_GenericAlloc
let free = if ffi::PyType_IS_GC(ty) != 0 {
ffi::PyObject_GC_Del
} else {
ffi::PyObject_Free
};
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
/// Workaround for Python issue 35810; no longer necessary in Python 3.8
#[inline]
#[cfg(not(Py_3_8))]
unsafe fn bpo_35810_workaround(_py: Python, ty: *mut ffi::PyTypeObject) {
#[cfg(Py_LIMITED_API)]
{
// Must check version at runtime for abi3 wheels - they could run against a higher version
// than the build config suggests.
use crate::once_cell::GILOnceCell;
static IS_PYTHON_3_8: GILOnceCell<bool> = GILOnceCell::new();
if *IS_PYTHON_3_8.get_or_init(_py, || _py.version_info() >= (3, 8)) {
// No fix needed - the wheel is running on a sufficiently new interpreter.
return;
}
}
ffi::Py_INCREF(ty as *mut ffi::PyObject);
}
// General methods implementation: either dtolnay specialization trait or inventory if
// multiple-pymethods feature is enabled.
@ -216,6 +321,7 @@ pub struct ThreadCheckerStub<T: Send>(PhantomData<T>);
impl<T: Send> PyClassThreadChecker<T> for ThreadCheckerStub<T> {
fn ensure(&self) {}
#[inline]
fn new() -> Self {
ThreadCheckerStub(PhantomData)
}
@ -224,6 +330,7 @@ impl<T: Send> PyClassThreadChecker<T> for ThreadCheckerStub<T> {
impl<T: PyNativeType> PyClassThreadChecker<T> for ThreadCheckerStub<crate::PyObject> {
fn ensure(&self) {}
#[inline]
fn new() -> Self {
ThreadCheckerStub(PhantomData)
}
@ -279,8 +386,30 @@ pub trait PyClassBaseType: Sized {
impl<T: PyClass> PyClassBaseType for T {
type Dict = T::Dict;
type WeakRef = T::WeakRef;
type LayoutAsBase = crate::pycell::PyCellInner<T>;
type LayoutAsBase = crate::pycell::PyCell<T>;
type BaseNativeType = T::BaseNativeType;
type ThreadChecker = T::ThreadChecker;
type Initializer = crate::pyclass_init::PyClassInitializer<Self>;
}
// Default new implementation
pub(crate) unsafe extern "C" fn fallback_new(
_subtype: *mut ffi::PyTypeObject,
_args: *mut ffi::PyObject,
_kwds: *mut ffi::PyObject,
) -> *mut ffi::PyObject {
crate::callback_body!(py, {
Err::<(), _>(crate::exceptions::PyTypeError::new_err(
"No constructor defined",
))
})
}
pub(crate) unsafe extern "C" fn tp_dealloc<T: PyClass>(obj: *mut ffi::PyObject) {
crate::callback_body!(py, {
// Safety: Python will only call tp_dealloc when no references to the object remain.
let cell: &mut PyCell<T> = &mut *(obj as *mut _);
cell.tp_dealloc(py);
})
}

1
src/ffi/datetime.rs
View File

@ -650,7 +650,6 @@ mod tests {
fn test_date_fromtimestamp() {
Python::with_gil(|py| {
let args: Py<PyAny> = (100,).into_py(py);
dbg!(args.as_ref(py));
unsafe { PyDateTime_IMPORT() };
let dt: &PyAny = unsafe { py.from_owned_ptr(PyDate_FromTimestamp(args.as_ptr())) };
py_run!(

115
src/freelist.rs
View File

@ -1,115 +0,0 @@
// Copyright (c) 2017-present PyO3 Project and Contributors
//! Support for [free allocation lists][1].
//!
//! This can improve performance for types that are often created and deleted in quick succession.
//!
//! Rather than implementing this manually,
//! implement it by annotating a struct with `#[pyclass(freelist = N)]`,
//! where `N` is the size of the freelist.
//!
//! [1]: https://en.wikipedia.org/wiki/Free_list
use crate::class::impl_::PyClassImpl;
use crate::pyclass::{get_type_free, tp_free_fallback, PyClassAlloc};
use crate::type_object::{PyLayout, PyTypeInfo};
use crate::{ffi, AsPyPointer, FromPyPointer, PyAny, Python};
use std::mem;
use std::os::raw::c_void;
/// Implements a freelist.
///
/// Do not implement this trait manually. Instead, use `#[pyclass(freelist = N)]`
/// on a Rust struct to implement it.
pub trait PyClassWithFreeList {
fn get_free_list(py: Python) -> &mut FreeList<*mut ffi::PyObject>;
}
/// Represents a slot of a [`FreeList`].
pub enum Slot<T> {
Empty,
Filled(T),
}
pub struct FreeList<T> {
entries: Vec<Slot<T>>,
split: usize,
capacity: usize,
}
impl<T> FreeList<T> {
/// Creates a new `FreeList` instance with specified capacity.
pub fn with_capacity(capacity: usize) -> FreeList<T> {
let entries = (0..capacity).map(|_| Slot::Empty).collect::<Vec<_>>();
FreeList {
entries,
split: 0,
capacity,
}
}
/// Pops the first non empty item.
pub fn pop(&mut self) -> Option<T> {
let idx = self.split;
if idx == 0 {
None
} else {
match mem::replace(&mut self.entries[idx - 1], Slot::Empty) {
Slot::Filled(v) => {
self.split = idx - 1;
Some(v)
}
_ => panic!("FreeList is corrupt"),
}
}
}
/// Inserts a value into the list. Returns `None` if the `FreeList` is full.
pub fn insert(&mut self, val: T) -> Option<T> {
let next = self.split + 1;
if next < self.capacity {
self.entries[self.split] = Slot::Filled(val);
self.split = next;
None
} else {
Some(val)
}
}
}
impl<T> PyClassAlloc for T
where
T: PyTypeInfo + PyClassImpl + PyClassWithFreeList,
{
unsafe fn new(py: Python, subtype: *mut ffi::PyTypeObject) -> *mut Self::Layout {
// if subtype is not equal to this type, we cannot use the freelist
if subtype == Self::type_object_raw(py) {
if let Some(obj) = <Self as PyClassWithFreeList>::get_free_list(py).pop() {
ffi::PyObject_Init(obj, subtype);
#[cfg(not(Py_3_8))]
crate::pyclass::bpo_35810_workaround(py, subtype);
return obj as _;
}
}
crate::pyclass::default_new::<Self>(py, subtype) as _
}
#[allow(clippy::collapsible_if)] // for if cfg!
unsafe fn dealloc(py: Python, self_: *mut Self::Layout) {
(*self_).py_drop(py);
let obj = PyAny::from_borrowed_ptr_or_panic(py, self_ as _);
if let Some(obj) = <Self as PyClassWithFreeList>::get_free_list(py).insert(obj.as_ptr()) {
let ty = ffi::Py_TYPE(obj);
let free = get_type_free(ty).unwrap_or_else(|| tp_free_fallback(ty));
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
}

1
src/impl_.rs
View File

@ -3,3 +3,4 @@
//! any time without documentation in the CHANGELOG and without breaking semver guarantees.
pub mod deprecations;
pub mod freelist;

66
src/impl_/freelist.rs Normal file
View File

@ -0,0 +1,66 @@
// Copyright (c) 2017-present PyO3 Project and Contributors
//! Support for [free allocation lists][1].
//!
//! This can improve performance for types that are often created and deleted in quick succession.
//!
//! Rather than implementing this manually,
//! implement it by annotating a struct with `#[pyclass(freelist = N)]`,
//! where `N` is the size of the freelist.
//!
//! [1]: https://en.wikipedia.org/wiki/Free_list
use std::mem;
/// Represents a slot of a [`FreeList`].
pub enum Slot<T> {
Empty,
Filled(T),
}
pub struct FreeList<T> {
entries: Vec<Slot<T>>,
split: usize,
capacity: usize,
}
impl<T> FreeList<T> {
/// Creates a new `FreeList` instance with specified capacity.
pub fn with_capacity(capacity: usize) -> FreeList<T> {
let entries = (0..capacity).map(|_| Slot::Empty).collect::<Vec<_>>();
FreeList {
entries,
split: 0,
capacity,
}
}
/// Pops the first non empty item.
pub fn pop(&mut self) -> Option<T> {
let idx = self.split;
if idx == 0 {
None
} else {
match mem::replace(&mut self.entries[idx - 1], Slot::Empty) {
Slot::Filled(v) => {
self.split = idx - 1;
Some(v)
}
_ => panic!("FreeList is corrupt"),
}
}
}
/// Inserts a value into the list. Returns `None` if the `FreeList` is full.
pub fn insert(&mut self, val: T) -> Option<T> {
let next = self.split + 1;
if next < self.capacity {
self.entries[self.split] = Slot::Filled(val);
self.split = next;
None
} else {
Some(val)
}
}
}

1
src/lib.rs
View File

@ -277,7 +277,6 @@ pub mod derive_utils;
mod err;
pub mod exceptions;
pub mod ffi;
pub mod freelist;
mod gil;
pub mod impl_;
mod instance;

157
src/pycell.rs
View File

@ -1,5 +1,4 @@
//! Includes `PyCell` implementation.
use crate::conversion::{AsPyPointer, FromPyPointer, ToPyObject};
use crate::exceptions::PyRuntimeError;
use crate::pyclass::PyClass;
use crate::pyclass_init::PyClassInitializer;
@ -7,6 +6,12 @@ use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::type_object::{PyLayout, PySizedLayout};
use crate::types::PyAny;
use crate::{class::impl_::PyClassBaseType, class::impl_::PyClassThreadChecker};
use crate::{
conversion::{AsPyPointer, FromPyPointer, ToPyObject},
ffi::PyBaseObject_Type,
type_object::get_tp_free,
PyTypeInfo,
};
use crate::{ffi, IntoPy, PyErr, PyNativeType, PyObject, PyResult, Python};
use std::cell::{Cell, UnsafeCell};
use std::fmt;
@ -22,49 +27,7 @@ pub struct PyCellBase<T> {
borrow_flag: Cell<BorrowFlag>,
}
unsafe impl<T, U> PyLayout<T> for PyCellBase<U>
where
U: PySizedLayout<T>,
{
const IS_NATIVE_TYPE: bool = true;
}
/// Inner type of `PyCell` without dict slots and reference counter.
/// This struct has two usages:
/// 1. As an inner type of `PyRef` and `PyRefMut`.
/// 2. When `#[pyclass(extends=Base)]` is specified, `PyCellInner<Base>` is used as a base layout.
#[doc(hidden)]
#[repr(C)]
pub struct PyCellInner<T: PyClass> {
ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
value: ManuallyDrop<UnsafeCell<T>>,
}
impl<T: PyClass> AsPyPointer for PyCellInner<T> {
fn as_ptr(&self) -> *mut ffi::PyObject {
(self as *const _) as *mut _
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCellInner<T> {
const IS_NATIVE_TYPE: bool = false;
fn py_init(&mut self, value: T) {
self.value = ManuallyDrop::new(UnsafeCell::new(value));
}
unsafe fn py_drop(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.value);
self.ob_base.py_drop(py);
}
}
// These impls ensures `PyCellInner` can be a base type.
impl<T: PyClass> PySizedLayout<T> for PyCellInner<T> {}
impl<T: PyClass> PyCellInner<T> {
fn get_ptr(&self) -> *mut T {
self.value.get()
}
}
unsafe impl<T, U> PyLayout<T> for PyCellBase<U> where U: PySizedLayout<T> {}
/// `PyCell` is the container type for [`PyClass`](../pyclass/trait.PyClass.html).
///
@ -133,12 +96,18 @@ impl<T: PyClass> PyCellInner<T> {
/// ```
#[repr(C)]
pub struct PyCell<T: PyClass> {
inner: PyCellInner<T>,
thread_checker: T::ThreadChecker,
ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
contents: PyCellContents<T>,
}
#[repr(C)]
pub(crate) struct PyCellContents<T: PyClass> {
pub(crate) value: ManuallyDrop<UnsafeCell<T>>,
pub(crate) thread_checker: T::ThreadChecker,
// DO NOT CHANGE THE ORDER OF THESE FIELDS WITHOUT CHANGING PyCell::dict_offset()
// AND PyCell::weakref_offset()
dict: T::Dict,
weakref: T::WeakRef,
pub(crate) dict: T::Dict,
pub(crate) weakref: T::WeakRef,
}
impl<T: PyClass> PyCell<T> {
@ -228,13 +197,12 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub fn try_borrow(&self) -> Result<PyRef<'_, T>, PyBorrowError> {
self.thread_checker.ensure();
let flag = self.inner.get_borrow_flag();
let flag = self.get_borrow_flag();
if flag == BorrowFlag::HAS_MUTABLE_BORROW {
Err(PyBorrowError { _private: () })
} else {
self.inner.set_borrow_flag(flag.increment());
Ok(PyRef { inner: &self.inner })
self.set_borrow_flag(flag.increment());
Ok(PyRef { inner: self })
}
}
@ -260,12 +228,11 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub fn try_borrow_mut(&self) -> Result<PyRefMut<'_, T>, PyBorrowMutError> {
self.thread_checker.ensure();
if self.inner.get_borrow_flag() != BorrowFlag::UNUSED {
if self.get_borrow_flag() != BorrowFlag::UNUSED {
Err(PyBorrowMutError { _private: () })
} else {
self.inner.set_borrow_flag(BorrowFlag::HAS_MUTABLE_BORROW);
Ok(PyRefMut { inner: &self.inner })
self.set_borrow_flag(BorrowFlag::HAS_MUTABLE_BORROW);
Ok(PyRefMut { inner: self })
}
}
@ -299,11 +266,10 @@ impl<T: PyClass> PyCell<T> {
/// });
/// ```
pub unsafe fn try_borrow_unguarded(&self) -> Result<&T, PyBorrowError> {
self.thread_checker.ensure();
if self.inner.get_borrow_flag() == BorrowFlag::HAS_MUTABLE_BORROW {
if self.get_borrow_flag() == BorrowFlag::HAS_MUTABLE_BORROW {
Err(PyBorrowError { _private: () })
} else {
Ok(&*self.inner.value.get())
Ok(&*self.contents.value.get())
}
}
@ -338,41 +304,17 @@ impl<T: PyClass> PyCell<T> {
std::mem::swap(&mut *self.borrow_mut(), &mut *other.borrow_mut())
}
/// Allocates a new PyCell given a type object `subtype`. Used by our `tp_new` implementation.
pub(crate) unsafe fn internal_new(
py: Python,
subtype: *mut ffi::PyTypeObject,
) -> PyResult<*mut Self> {
let base = T::new(py, subtype);
if base.is_null() {
return Err(PyErr::fetch(py));
}
let base = base as *mut PyCellBase<T::BaseNativeType>;
(*base).borrow_flag = Cell::new(BorrowFlag::UNUSED);
let self_ = base as *mut Self;
(*self_).dict = T::Dict::new();
(*self_).weakref = T::WeakRef::new();
(*self_).thread_checker = T::ThreadChecker::new();
Ok(self_)
fn get_ptr(&self) -> *mut T {
self.contents.value.get()
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCell<T> {
const IS_NATIVE_TYPE: bool = false;
fn py_init(&mut self, value: T) {
self.inner.value = ManuallyDrop::new(UnsafeCell::new(value));
}
unsafe fn py_drop(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.inner.value);
self.dict.clear_dict(py);
self.weakref.clear_weakrefs(self.as_ptr(), py);
self.inner.ob_base.py_drop(py);
}
}
unsafe impl<T: PyClass> PyLayout<T> for PyCell<T> {}
impl<T: PyClass> PySizedLayout<T> for PyCell<T> {}
impl<T: PyClass> AsPyPointer for PyCell<T> {
fn as_ptr(&self) -> *mut ffi::PyObject {
self.inner.as_ptr()
(self as *const _) as *mut _
}
}
@ -453,7 +395,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyCell<T> {
/// # });
/// ```
pub struct PyRef<'p, T: PyClass> {
inner: &'p PyCellInner<T>,
inner: &'p PyCell<T>,
}
impl<'p, T: PyClass> PyRef<'p, T> {
@ -570,7 +512,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyRef<'_, T> {
///
/// See the [`PyCell`](struct.PyCell.html) and [`PyRef`](struct.PyRef.html) documentations for more.
pub struct PyRefMut<'p, T: PyClass> {
inner: &'p PyCellInner<T>,
inner: &'p PyCell<T>,
}
impl<'p, T: PyClass> PyRefMut<'p, T> {
@ -669,7 +611,7 @@ impl<T: PyClass + fmt::Debug> fmt::Debug for PyRefMut<'_, T> {
pub struct BorrowFlag(usize);
impl BorrowFlag {
const UNUSED: BorrowFlag = BorrowFlag(0);
pub(crate) const UNUSED: BorrowFlag = BorrowFlag(0);
const HAS_MUTABLE_BORROW: BorrowFlag = BorrowFlag(usize::max_value());
const fn increment(self) -> Self {
Self(self.0 + 1)
@ -733,11 +675,14 @@ impl From<PyBorrowMutError> for PyErr {
pub trait PyCellLayout<T>: PyLayout<T> {
fn get_borrow_flag(&self) -> BorrowFlag;
fn set_borrow_flag(&self, flag: BorrowFlag);
/// Implementation of tp_dealloc. Do not attempt to use &self after calling this!
unsafe fn tp_dealloc(&mut self, py: Python);
}
impl<T, U> PyCellLayout<T> for PyCellBase<U>
where
U: PySizedLayout<T>,
T: PyTypeInfo,
{
fn get_borrow_flag(&self) -> BorrowFlag {
self.borrow_flag.get()
@ -745,16 +690,44 @@ where
fn set_borrow_flag(&self, flag: BorrowFlag) {
self.borrow_flag.set(flag)
}
unsafe fn tp_dealloc(&mut self, py: Python) {
let obj: *mut ffi::PyObject = self as *mut _ as *mut _;
// For `#[pyclass]` types which inherit from PyAny, we can just call tp_free
if T::type_object_raw(py) == &mut PyBaseObject_Type {
return get_tp_free(ffi::Py_TYPE(obj))(obj as _);
}
// More complex native types (e.g. `extends=PyDict`) require calling the base's dealloc.
#[cfg(not(Py_LIMITED_API))]
{
if let Some(dealloc) = (*T::type_object_raw(py)).tp_dealloc {
dealloc(obj as _);
} else {
get_tp_free(ffi::Py_TYPE(obj))(obj as _);
}
}
#[cfg(Py_LIMITED_API)]
unreachable!("subclassing native types is not possible with the `abi3` feature");
}
}
impl<T: PyClass> PyCellLayout<T> for PyCellInner<T>
impl<T: PyClass> PyCellLayout<T> for PyCell<T>
where
<T::BaseType as PyClassBaseType>::LayoutAsBase: PyCellLayout<T::BaseType>,
{
fn get_borrow_flag(&self) -> BorrowFlag {
self.contents.thread_checker.ensure();
self.ob_base.get_borrow_flag()
}
fn set_borrow_flag(&self, flag: BorrowFlag) {
self.ob_base.set_borrow_flag(flag)
}
unsafe fn tp_dealloc(&mut self, py: Python) {
ManuallyDrop::drop(&mut self.contents.value);
self.contents.dict.clear_dict(py);
self.contents.weakref.clear_weakrefs(self.as_ptr(), py);
self.ob_base.tp_dealloc(py);
}
}

155
src/pyclass.rs
View File

@ -1,135 +1,16 @@
//! `PyClass` and related traits.
use crate::class::methods::PyMethodDefType;
use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::type_object::PyLayout;
use crate::{class::impl_::PyClassBaseType, class::impl_::PyClassImpl};
use crate::{ffi, PyCell, PyErr, PyNativeType, PyResult, PyTypeInfo, Python};
use std::convert::TryInto;
use std::ffi::CString;
use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::{mem, ptr};
#[inline]
unsafe fn get_type_alloc(tp: *mut ffi::PyTypeObject) -> Option<ffi::allocfunc> {
mem::transmute(ffi::PyType_GetSlot(tp, ffi::Py_tp_alloc))
}
#[inline]
pub(crate) unsafe fn get_type_free(tp: *mut ffi::PyTypeObject) -> Option<ffi::freefunc> {
mem::transmute(ffi::PyType_GetSlot(tp, ffi::Py_tp_free))
}
/// Workaround for Python issue 35810; no longer necessary in Python 3.8
#[inline]
#[cfg(not(Py_3_8))]
pub(crate) unsafe fn bpo_35810_workaround(_py: Python, ty: *mut ffi::PyTypeObject) {
#[cfg(Py_LIMITED_API)]
{
// Must check version at runtime for abi3 wheels - they could run against a higher version
// than the build config suggests.
use crate::once_cell::GILOnceCell;
static IS_PYTHON_3_8: GILOnceCell<bool> = GILOnceCell::new();
if *IS_PYTHON_3_8.get_or_init(_py, || _py.version_info() >= (3, 8)) {
// No fix needed - the wheel is running on a sufficiently new interpreter.
return;
}
}
ffi::Py_INCREF(ty as *mut ffi::PyObject);
}
#[inline]
pub(crate) unsafe fn default_new<T: PyTypeInfo + PyClassImpl>(
py: Python,
subtype: *mut ffi::PyTypeObject,
) -> *mut ffi::PyObject {
// if the class derives native types(e.g., PyDict), call special new
if T::IS_SUBCLASS && <T::BaseType as PyClassBaseType>::LayoutAsBase::IS_NATIVE_TYPE {
#[cfg(not(Py_LIMITED_API))]
{
let base_tp = T::BaseType::type_object_raw(py);
if let Some(base_new) = (*base_tp).tp_new {
return base_new(subtype, ptr::null_mut(), ptr::null_mut());
}
}
#[cfg(Py_LIMITED_API)]
{
// Silence unused parameter warning.
let _ = py;
unreachable!("Subclassing native types isn't support in limited API mode");
}
}
let alloc = get_type_alloc(subtype).unwrap_or(ffi::PyType_GenericAlloc);
#[cfg(not(Py_3_8))]
bpo_35810_workaround(py, subtype);
alloc(subtype, 0)
}
/// This trait enables custom `tp_new`/`tp_dealloc` implementations for `T: PyClass`.
pub trait PyClassAlloc: PyTypeInfo + PyClassImpl {
/// Allocate the actual field for `#[pyclass]`.
///
/// # Safety
/// This function must return a valid pointer to the Python heap.
unsafe fn new(py: Python, subtype: *mut ffi::PyTypeObject) -> *mut Self::Layout {
default_new::<Self>(py, subtype) as _
}
/// Deallocate `#[pyclass]` on the Python heap.
///
/// # Safety
/// `self_` must be a valid pointer to the Python heap.
#[allow(clippy::collapsible_if)] // for if cfg!
unsafe fn dealloc(py: Python, self_: *mut Self::Layout) {
(*self_).py_drop(py);
let obj = self_ as *mut ffi::PyObject;
let ty = ffi::Py_TYPE(obj);
let free = get_type_free(ty).unwrap_or_else(|| tp_free_fallback(ty));
free(obj as *mut c_void);
if cfg!(Py_3_8) {
if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 {
ffi::Py_DECREF(ty as *mut ffi::PyObject);
}
}
}
}
// Default new implementation
unsafe extern "C" fn fallback_new(
_subtype: *mut ffi::PyTypeObject,
_args: *mut ffi::PyObject,
_kwds: *mut ffi::PyObject,
) -> *mut ffi::PyObject {
crate::callback_body!(py, {
Err::<(), _>(crate::exceptions::PyTypeError::new_err(
"No constructor defined",
))
})
}
unsafe extern "C" fn tp_dealloc<T>(obj: *mut ffi::PyObject)
where
T: PyClassAlloc,
{
let pool = crate::GILPool::new();
let py = pool.python();
<T as PyClassAlloc>::dealloc(py, (obj as *mut T::Layout) as _)
}
pub(crate) unsafe fn tp_free_fallback(ty: *mut ffi::PyTypeObject) -> ffi::freefunc {
if ffi::PyType_IS_GC(ty) != 0 {
ffi::PyObject_GC_Del
} else {
ffi::PyObject_Free
}
}
use crate::{
class::impl_::{fallback_new, tp_dealloc, PyClassImpl},
ffi,
pyclass_slots::{PyClassDict, PyClassWeakRef},
PyCell, PyErr, PyMethodDefType, PyNativeType, PyResult, PyTypeInfo, Python,
};
use std::{
convert::TryInto,
ffi::CString,
os::raw::{c_char, c_int, c_uint, c_void},
ptr,
};
/// If `PyClass` is implemented for `T`, then we can use `T` in the Python world,
/// via `PyCell`.
@ -137,7 +18,7 @@ pub(crate) unsafe fn tp_free_fallback(ty: *mut ffi::PyTypeObject) -> ffi::freefu
/// The `#[pyclass]` attribute automatically implements this trait for your Rust struct,
/// so you don't have to use this trait directly.
pub trait PyClass:
PyTypeInfo<AsRefTarget = PyCell<Self>> + Sized + PyClassAlloc + PyClassImpl<Layout = PyCell<Self>>
PyTypeInfo<AsRefTarget = PyCell<Self>> + PyClassImpl<Layout = PyCell<Self>>
{
/// Specify this class has `#[pyclass(dict)]` or not.
type Dict: PyClassDict;
@ -188,12 +69,20 @@ where
let mut slots = TypeSlots::default();
slots.push(ffi::Py_tp_base, T::BaseType::type_object_raw(py) as _);
slots.push(ffi::Py_tp_dealloc, tp_dealloc::<T> as _);
if let Some(doc) = tp_doc::<T>()? {
slots.push(ffi::Py_tp_doc, doc);
}
slots.push(ffi::Py_tp_new, T::get_new().unwrap_or(fallback_new) as _);
slots.push(ffi::Py_tp_dealloc, tp_dealloc::<T> as _);
if let Some(alloc) = T::get_alloc() {
slots.push(ffi::Py_tp_alloc, alloc as _);
}
if let Some(free) = T::get_free() {
slots.push(ffi::Py_tp_free, free as _);
}
if let Some(call_meth) = T::get_call() {
slots.push(ffi::Py_tp_call, call_meth as _);
}

119
src/pyclass_init.rs
View File

@ -1,15 +1,29 @@
//! Initialization utilities for `#[pyclass]`.
use crate::type_object::{PyLayout, PyTypeInfo};
use crate::class::impl_::PyClassThreadChecker;
use crate::pyclass_slots::{PyClassDict, PyClassWeakRef};
use crate::{callback::IntoPyCallbackOutput, class::impl_::PyClassBaseType};
use crate::{PyCell, PyClass, PyResult, Python};
use std::marker::PhantomData;
use crate::{ffi, PyCell, PyClass, PyErr, PyResult, Python};
use crate::{
ffi::PyTypeObject,
pycell::{BorrowFlag, PyCellContents},
type_object::{get_tp_alloc, PyTypeInfo},
};
use std::{
cell::{Cell, UnsafeCell},
marker::PhantomData,
mem::{ManuallyDrop, MaybeUninit},
};
/// Initializer for Python types.
///
/// This trait is intended to use internally for distinguishing `#[pyclass]` and
/// Python native types.
pub trait PyObjectInit<T>: Sized {
fn init_class<L: PyLayout<T>>(self, layout: &mut L);
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject>;
private_decl! {}
}
@ -17,7 +31,46 @@ pub trait PyObjectInit<T>: Sized {
pub struct PyNativeTypeInitializer<T: PyTypeInfo>(PhantomData<T>);
impl<T: PyTypeInfo> PyObjectInit<T> for PyNativeTypeInitializer<T> {
fn init_class<L: PyLayout<T>>(self, _layout: &mut L) {}
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject> {
let type_object = T::type_object_raw(py);
// HACK (due to FIXME below): PyBaseObject_Type's tp_new isn't happy with NULL arguments
if type_object == (&ffi::PyBaseObject_Type as *const _ as *mut _) {
let alloc = get_tp_alloc(subtype).unwrap_or(ffi::PyType_GenericAlloc);
let obj = alloc(subtype, 0);
return if obj.is_null() {
Err(PyErr::fetch(py))
} else {
Ok(obj)
};
}
#[cfg(Py_LIMITED_API)]
unreachable!("subclassing native types is not possible with the `abi3` feature");
#[cfg(not(Py_LIMITED_API))]
{
match (*type_object).tp_new {
// FIXME: Call __new__ with actual arguments
Some(newfunc) => {
let obj = newfunc(subtype, std::ptr::null_mut(), std::ptr::null_mut());
if obj.is_null() {
Err(PyErr::fetch(py))
} else {
Ok(obj)
}
}
None => Err(crate::exceptions::PyTypeError::new_err(
"base type without tp_new",
)),
}
}
}
private_impl! {}
}
@ -142,22 +195,56 @@ impl<T: PyClass> PyClassInitializer<T> {
where
T: PyClass,
{
let cell = PyCell::internal_new(py, subtype)?;
self.init_class(&mut *cell);
Ok(cell)
self.into_new_object(py, subtype).map(|obj| obj as _)
}
}
impl<T: PyClass> PyObjectInit<T> for PyClassInitializer<T> {
fn init_class<L: PyLayout<T>>(self, layout: &mut L) {
unsafe fn into_new_object(
self,
py: Python,
subtype: *mut PyTypeObject,
) -> PyResult<*mut ffi::PyObject> {
/// Layout of a PyCellBase after base new has been called, but borrow flag has not yet been
/// initialized.
#[repr(C)]
struct PartiallyInitializedPyCellBase<T> {
_ob_base: T,
borrow_flag: MaybeUninit<Cell<BorrowFlag>>,
}
/// Layout of a PyCell after base new has been called, but contents have not yet been
/// written.
#[repr(C)]
struct PartiallyInitializedPyCell<T: PyClass> {
_ob_base: <T::BaseType as PyClassBaseType>::LayoutAsBase,
contents: MaybeUninit<PyCellContents<T>>,
}
let Self { init, super_init } = self;
// Safety: A valid PyLayout must contain the base layout as the first entry, so casting L to
// T::BaseType::LayoutAsBase is ok.
super_init.init_class(unsafe {
&mut *(layout as *mut _ as *mut <T::BaseType as PyClassBaseType>::LayoutAsBase)
});
layout.py_init(init);
let obj = super_init.into_new_object(py, subtype)?;
// FIXME: Only need to initialize borrow flag once per whole hierarchy
let base: *mut PartiallyInitializedPyCellBase<T::BaseNativeType> = obj as _;
std::ptr::write(
(*base).borrow_flag.as_mut_ptr(),
Cell::new(BorrowFlag::UNUSED),
);
// FIXME: Initialize borrow flag if necessary??
let cell: *mut PartiallyInitializedPyCell<T> = obj as _;
std::ptr::write(
(*cell).contents.as_mut_ptr(),
PyCellContents {
value: ManuallyDrop::new(UnsafeCell::new(init)),
thread_checker: T::ThreadChecker::new(),
dict: T::Dict::new(),
weakref: T::WeakRef::new(),
},
);
Ok(obj)
}
private_impl! {}
}
@ -166,6 +253,7 @@ where
T: PyClass,
T::BaseType: PyClassBaseType<Initializer = PyNativeTypeInitializer<T::BaseType>>,
{
#[inline]
fn from(value: T) -> PyClassInitializer<T> {
Self::new(value, PyNativeTypeInitializer(PhantomData))
}
@ -190,6 +278,7 @@ where
T: PyClass,
U: Into<PyClassInitializer<T>>,
{
#[inline]
fn convert(self, _py: Python) -> PyResult<PyClassInitializer<T>> {
Ok(self.into())
}

8
src/pyclass_slots.rs
View File

@ -6,6 +6,7 @@ use crate::{ffi, Python};
pub trait PyClassDict {
const IS_DUMMY: bool = true;
fn new() -> Self;
#[inline]
fn clear_dict(&mut self, _py: Python) {}
private_decl! {}
}
@ -14,6 +15,7 @@ pub trait PyClassDict {
pub trait PyClassWeakRef {
const IS_DUMMY: bool = true;
fn new() -> Self;
#[inline]
unsafe fn clear_weakrefs(&mut self, _obj: *mut ffi::PyObject, _py: Python) {}
private_decl! {}
}
@ -23,6 +25,7 @@ pub struct PyClassDummySlot;
impl PyClassDict for PyClassDummySlot {
private_impl! {}
#[inline]
fn new() -> Self {
PyClassDummySlot
}
@ -30,6 +33,7 @@ impl PyClassDict for PyClassDummySlot {
impl PyClassWeakRef for PyClassDummySlot {
private_impl! {}
#[inline]
fn new() -> Self {
PyClassDummySlot
}
@ -44,9 +48,11 @@ pub struct PyClassDictSlot(*mut ffi::PyObject);
impl PyClassDict for PyClassDictSlot {
private_impl! {}
const IS_DUMMY: bool = false;
#[inline]
fn new() -> Self {
Self(std::ptr::null_mut())
}
#[inline]
fn clear_dict(&mut self, _py: Python) {
if !self.0.is_null() {
unsafe { ffi::PyDict_Clear(self.0) }
@ -63,9 +69,11 @@ pub struct PyClassWeakRefSlot(*mut ffi::PyObject);
impl PyClassWeakRef for PyClassWeakRefSlot {
private_impl! {}
const IS_DUMMY: bool = false;
#[inline]
fn new() -> Self {
Self(std::ptr::null_mut())
}
#[inline]
unsafe fn clear_weakrefs(&mut self, obj: *mut ffi::PyObject, _py: Python) {
if !self.0.is_null() {
ffi::PyObject_ClearWeakRefs(obj)

38
src/type_object.rs
View File

@ -3,7 +3,8 @@
use crate::internal_tricks::extract_cstr_or_leak_cstring;
use crate::once_cell::GILOnceCell;
use crate::pyclass::{create_type_object, PyClass};
use crate::pyclass::create_type_object;
use crate::pyclass::PyClass;
use crate::types::{PyAny, PyType};
use crate::{conversion::IntoPyPointer, PyMethodDefType};
use crate::{ffi, AsPyPointer, PyErr, PyNativeType, PyObject, PyResult, Python};
@ -15,11 +16,7 @@ use std::thread::{self, ThreadId};
/// is of `PyAny`.
///
/// This trait is intended to be used internally.
pub unsafe trait PyLayout<T> {
const IS_NATIVE_TYPE: bool = true;
fn py_init(&mut self, _value: T) {}
unsafe fn py_drop(&mut self, _py: Python) {}
}
pub unsafe trait PyLayout<T> {}
/// `T: PySizedLayout<U>` represents `T` is not a instance of
/// [`PyVarObject`](https://docs.python.org/3.8/c-api/structures.html?highlight=pyvarobject#c.PyVarObject).
@ -200,3 +197,32 @@ fn initialize_tp_dict(
// This is necessary for making static `LazyStaticType`s
unsafe impl Sync for LazyStaticType {}
#[inline]
pub(crate) unsafe fn get_tp_alloc(tp: *mut ffi::PyTypeObject) -> Option<ffi::allocfunc> {
#[cfg(not(Py_LIMITED_API))]
{
(*tp).tp_alloc
}
#[cfg(Py_LIMITED_API)]
{
let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_alloc);
std::mem::transmute(ptr)
}
}
#[inline]
pub(crate) unsafe fn get_tp_free(tp: *mut ffi::PyTypeObject) -> ffi::freefunc {
#[cfg(not(Py_LIMITED_API))]
{
(*tp).tp_free.unwrap()
}
#[cfg(Py_LIMITED_API)]
{
let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_free);
debug_assert_ne!(ptr, std::ptr::null_mut());
std::mem::transmute(ptr)
}
}

17
tests/test_inheritance.rs
View File

@ -210,6 +210,23 @@ mod inheriting_native_type {
);
}
#[test]
fn inherit_dict_drop() {
Python::with_gil(|py| {
let dict_sub = pyo3::Py::new(py, DictWithName::new()).unwrap();
assert_eq!(dict_sub.get_refcnt(py), 1);
let item = py.eval("object()", None, None).unwrap();
assert_eq!(item.get_refcnt(), 1);
dict_sub.as_ref(py).set_item("foo", item).unwrap();
assert_eq!(item.get_refcnt(), 2);
drop(dict_sub);
assert_eq!(item.get_refcnt(), 1);
})
}
#[pyclass(extends=PyException)]
struct CustomException {
#[pyo3(get)]