Python code doesn't like to run without `__builtins__`, so adding them
if missing seems to be a good idea. Also that's what CPython >3.10 does.
See https://github.com/python/cpython/pull/24564Resolves#3370
When the Python iterator backing `PyIterator` has a `__length_hint__` special
method, we can use this as a lower bound for Rust's `Iterator::size_hint` to
e.g. support pre-allocation of collections.
This is implemented using `PyObject_LengthHint` which is not available in the
stable ABI and hence so is `Iterator::size_hint`. This should be fine since this
is an optimization in any case and the stable ABI is expected to have slightly
worse performance overall.
Most native types have static type objects which just need to be turned into a
pointer using addr_of_mut!, but sometimes like for exceptions we do call a
function which lazily initializes a type object.
This makes the pyobject_native_type_info! macro expect callable and passes it
the GIL token so that we do not need to call Python::assume_gil_acquired. The
case of static type objects is handled by the
pyobject_native_static_type_object! helper macro.
3209: Remove the conditional compilation flags which are made redundant by the MSRV bump r=davidhewitt a=adamreichold
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
3203: support ordering magic methods for `#[pyclass]` r=adamreichold a=davidhewitt
Closes#2089
This adds `__lt__`, `__le__`, `__eq__`, `__ne__`, `__gt__`, and `__ge__` as per the Python implementations of what we call `__richcmp__`.
There's a UI test confirming that the user cannot implement split forms and `__richcmp__` simultaneously.
There's also a benchmark comparing implementing these split methods against using `__richcmp__`. I couldn't see a meaningful performance difference, so I'm tempted to deprecate `__richcmp__`, given that's not a magic method which exists in Python. Potentially we can provide options such as the opt-in `#[pyclass(eq, ord)]` to avoid boilerplate for people who don't want to implement six different methods.
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
3198: Add abi3 + num_bigint conversion r=davidhewitt a=youknowone
<!--
Please consider adding the following to your pull request:
- an entry for this PR in newsfragments - see [https://pyo3.rs/main/contributing.html#documenting-changes]
- docs to all new functions and / or detail in the guide
- tests for all new or changed functions
PyO3's CI pipeline will check your pull request. To run its tests
locally, you can run ```cargo xtask ci```. See its documentation
[here](https://github.com/PyO3/pyo3/tree/main/xtask#readme).
-->
Fix#3196
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
Python classes that were not `complex` but implemented the `__complex__`
magic would have that method called via `PyComplex_AsCComplex` when
running against the full API, but the limited-API version
`PyComplex_RealAsDouble` does not attempt this conversion. If the input
object is not already complex, we can call the magic before proceeding.
`PyAny::lookup_special` is an approximate equivalent to the CPython
internal `_PyObject_LookupSpecial`, which is used to resolve lookups of
"magic" methods. These are only looked up from the type, and skip the
instance dictionary during the lookup. Despite this, they are still
required to resolve the descriptor protocol.
Many magic methods have slots on the `PyTypeObject` or respective
subobjects, but these are not necessarily available when targeting the
limited API or PyPy. In these cases, the requisite logic can be worked
around using safe but likely slower APIs.
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
Fix up lookup-special
3168: Do not apply deferred ref count updates and prevent the GIL from being acquired inside of __traverse__ implementations. r=davidhewitt a=adamreichold
Closes#2301Closes#3165
3176: Prevent dropping unsendable classes on other threads. r=davidhewitt a=adamreichold
Continuing the discussed from https://github.com/PyO3/pyo3/pull/3169#issuecomment-1556571504 and https://github.com/PyO3/pyo3/pull/3169#issuecomment-1556661723:
We already have checks in place to avoid borrowing these classes on other threads but it was still possible to send them to another thread and drop them there (while holding the GIL).
This change avoids running the `Drop` implementation in such a case even though Python will still free the underlying memory. This might leak resources owned by the object, but it avoids undefined behaviour due to access the unsendable type from another thread.
This does assume that the object was not unsafely integrated into an intrusive data structures which still point to the now freed memory. In that case, the only recourse would be to abort the process as freeing the memory is unavoidable when the tp_dealloc slot is called. (And moving it elsewhere into a new allocation would still break any existing pointers.)
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
We already have checks in place to avoid borrowing these classes on other
threads but it was still possible to send them to another thread and drop them
there (while holding the GIL).
This change avoids running the `Drop` implementation in such a case even though
Python will still free the underlying memory. This might leak resources owned by
the object, but it avoids undefined behaviour due to access the unsendable type
from another thread.
This does assume that the object was not unsafely integrated into an intrusive
data structures which still point to the now freed memory. In that case, the
only recourse would be to abort the process as freeing the memory is unavoidable
when the tp_dealloc slot is called. (And moving it elsewhere into a new
allocation would still break any existing pointers.)
3177: RFC: Use a const initializer for `GIL_COUNT` if possible r=adamreichold a=lifthrasiir
Normally [`LocalKey::try_with`](https://doc.rust-lang.org/stable/std/thread/struct.LocalKey.html#method.try_with) needs to check for the initialization flag to lazily initialize the TLS. This behaves badly with a compiler optimization and it seems that multiple calls to `gil_is_acquired()` cannot be correctly eliminated. Rust 1.59 added a `const { ... }` initializer (a special form only allowed here for now) in `thread_local!` which removes the initialization flag, allowing those kind of optimizations.
I should note that the performance impact is probably minimal, because the check branch is extremely well predicted and the optimization is only possible when every PyO3 code that leads to `gil_is_acquired()` is inlined, a pretty uncommon situation. I couldn't demonstrate any consistent improvement or regression from my machine, which performance seems to be flaky as well. But at least we would have one less branch there. I'll leave this as an RFC so that someone else can prove or disprove that this is indeed beneficial.
Co-authored-by: Kang Seonghoon <public+git@mearie.org>
2593: docs: mention PyBuffer r=adamreichold a=davidhewitt
Uses PEP 688 `types.Buffer` to describe `PyBuffer<T>` in the conversion tables. Will leave as draft until PEP 688 is finalised.
Closes#954
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
Tpt