3168: Do not apply deferred ref count updates and prevent the GIL from being acquired inside of __traverse__ implementations. r=davidhewitt a=adamreichold
Closes#2301Closes#3165
3176: Prevent dropping unsendable classes on other threads. r=davidhewitt a=adamreichold
Continuing the discussed from https://github.com/PyO3/pyo3/pull/3169#issuecomment-1556571504 and https://github.com/PyO3/pyo3/pull/3169#issuecomment-1556661723:
We already have checks in place to avoid borrowing these classes on other threads but it was still possible to send them to another thread and drop them there (while holding the GIL).
This change avoids running the `Drop` implementation in such a case even though Python will still free the underlying memory. This might leak resources owned by the object, but it avoids undefined behaviour due to access the unsendable type from another thread.
This does assume that the object was not unsafely integrated into an intrusive data structures which still point to the now freed memory. In that case, the only recourse would be to abort the process as freeing the memory is unavoidable when the tp_dealloc slot is called. (And moving it elsewhere into a new allocation would still break any existing pointers.)
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
We already have checks in place to avoid borrowing these classes on other
threads but it was still possible to send them to another thread and drop them
there (while holding the GIL).
This change avoids running the `Drop` implementation in such a case even though
Python will still free the underlying memory. This might leak resources owned by
the object, but it avoids undefined behaviour due to access the unsendable type
from another thread.
This does assume that the object was not unsafely integrated into an intrusive
data structures which still point to the now freed memory. In that case, the
only recourse would be to abort the process as freeing the memory is unavoidable
when the tp_dealloc slot is called. (And moving it elsewhere into a new
allocation would still break any existing pointers.)
2981: Remove 0.17 deprecations r=adamreichold,davidhewitt a=davidhewitt
Since #2980 starts a breaking change for 0.19, let's also clean up all 0.17's deprecations.
I've removed `Python::acquire_gil` in its own commit, as that was a reasonably chunky removal.
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2980: support `text_signature` on `#[new]` r=adamreichold a=davidhewitt
Closes#2866
This is a breaking change for 0.19.0, because it starts autogenerating `text_signature` for `#[new]`. This could affect runtime behaviour if the user is relying on the class docs at runtime for some reason.
Guide & tests all updated accordingly.
`#[pyclass(text_signature = "...")]` is deprecated by this PR, however if it's set, it will be used in preference to `#[new]`.
(The signature / `text_signature` from `#[new]` will simply be ignored in this case. I figure that when users fix their deprecation warnings by removing `#[pyclass(text_signature = "...")]` then the `#[new]` signatures will start flowing properly, and this is good enough.)
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
2975: RFC: Add GILProtected synchronization primitive and use it for LazyTypeObjectInner. r=davidhewitt a=adamreichold
I would also like to use that type in rust-numpy and it seems we can avoid ~~both a manual unsafe impl and~~ a full blown mutex if we apply it to `LazyTypeObjectInner`.
One downside might be that it ties us closer to the GIL when we want to enable nogil experimentation, but on the other hand, it may also help by reifying the GIL usage. (This is currently limited to comments in unsafe code in rust-numpy for example.)
3022: Fix function name shadowing r=davidhewitt a=mejrs
Fixes https://github.com/PyO3/pyo3/issues/3017
3023: Emit a better error for bad argument names r=davidhewitt a=mejrs
This will emit a better error for code like
```rust
#[pyfunction]
fn output([a,b,c]: [u8;3]) {}
```
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
Co-authored-by: mejrs <59372212+mejrs@users.noreply.github.com>
3014: feat: add #[pyo3(get, set)] for Cell r=davidhewitt a=AntoineRR
This fixes#2659.
The types for which `#[pyo3(get, set)]` should now work are `Cell`, `Arc` and `Box`.
There is one issue regarding `Box`, the implementation of `FromPyObject` conflicts with another one. I could not find what the issue was, especially since the other implementations for `Arc` and `Cell` work as expected. The related code and test has been commented out for now. Maybe someone could help me fix this issue if I don't figure it out myself? There is also the possibility to remove the implementation for `Box` of course.
Co-authored-by: Antoine Romero-Romero <ant.romero2@orange.fr>
2930: add better error message for Python in signature r=adamreichold a=davidhewitt
Inspired by #2929, this just adds a better error message when `Python` arguments are accidentally included in the signature.
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
Since we're just testing a bug during traversal, we don't actually have
to reap the object, we just have to make a reference cycle so that it's
traverse method is called.
2796: Forward cfgs on pyclass fields to the method defs r=davidhewitt a=mejrs
With this and the cfg_attr PR, I don't need cfg_eval at all anymore :)
- [x] needs some more tests
Co-authored-by: mejrs <>
2739: error when `#[pyo3(signature = ())]` used on invalid methods r=davidhewitt a=davidhewitt
A follow-up to #2702 to reject some invalid applications of `#[pyo3(signature = (...))]` attribute, specifically on magic methods and getters / setters / class attributes.
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
- Implement __ne__ as well as __eq__.
- Return NotImplemented when types cannot be converted, rather than
throwing.
- Compare the integer ids inside the __eq__/__ne__ implementation.
Previously a match block was generated.
* cov: src/buffer.rs - add tests for debug and element from format
* cov: src/buffer.rs - add some fortran-specific calls in test_array_buffer
* fix issues in MSRV
* cov: src/types/function.rs - directly call PyCFunction::new and PyCFunction::new_with_keywords
* docs: clarify docs of PyCFunction::new and PyCFunction::new_with_keywords
* revert added rust-version for MSRV in Cargo.toml
* cov: src/types/slice.rs - simple tests for PySliceIndices::new
* fix for multi-platform
* Update src/types/function.rs
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
* cov: src/buffer.rs - a better PyBuffer Debug test
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
* ci: test on emscripten target
This adds CI to build libpython3.11 for wasm32-emscripten and
running tests against it. We need to patch instant to work
around the emscripten_get_now:
https://github.com/sebcrozet/instant/pull/47
We also have to patch emscripten to work aroung the "undefined
symbol gxx_personality_v0" error:
https://github.com/emscripten-core/emscripten/issues/17128
I set up a nox file to download and install emscripten,
download and build cpython, set appropriate environment variables
then run cargo test. The workflow just installs python, rust,
node, and nox and runs the nox session.
I xfailed all the test failures. There are problems with datetime.
iter_dict_nosegv and test_filenotfounderror should probably be
fixable. The tests that involve threads or asyncio probably can't
be fixed.
* Some cleanup
* Remove instant patch
* Add explanations for xfails
* Allow more methods to take interned arguments
* Changelog
* Unify name bounds
* Resolve merge conflict
* reduce use of py_decref
* Add some attr tests
* Update migration
`pyo3-macros-backend` is now compiled with PyO3 cfgs to enable different magic method definitions based on version.
Add check for correct number of arguments on magic methods.
* Clean up `Python` documentation
* Apply suggestion
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
* Fix some things
* Fix PyPy link
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
* Add support for positional-only args
* Update changelog. Add a few more tests. Run rust-fmt.
* Fix test.
* Fix tests again.
* Update CHANGELOG.md to link PR instead of issue
* Update guide to mention positional-only params
* Add unreachable!() per code review
* Update and expand tests for pos args.
* Fix tests, lint, add UI tests.
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
* Add 'anyhow' feature which provides simple From<anyhow::Error> for PyErr impl
This makes it possible to use anyhow::Result<T> as the return type for
methods and functions exposed to Python.
The current implementation just stringifies the anyhow::Error before
shoving it into a PyRuntimeError. Conversion back to the anyhow::Error
is not possible, but it is better than nothing.
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
* Document `anyhow` feature in the guide
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
* update changelog to document anyhow feature
* WIP adding tests
* Finish up anyhow feature
* Fix formatting
* Fix tests
* Fix tests
* Apply review suggestions
Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>
Co-authored-by: mejrs <brunokolenbrander@hotmail.com>
* chore: update rust 1.55
* move test to 1.55 only
* make ci happy
* make ci happy
* make ci happy
* make ci happy
* make clippy happy
* make ci happy
* formatting
bors[bot]