Commit Graph

3636 Commits

Author SHA1 Message Date
David Hewitt 591f44ea6d
Merge pull request #1794 from indygreg/pystringdata
string: implement API to access raw string data
2021-08-21 22:54:58 +01:00
David Hewitt 6807c8069e
Merge pull request #1818 from indygreg/interpreter-config-docs
build: document InterpreterConfig fields
2021-08-21 22:52:14 +01:00
Gregory Szorc 736b03b8cc build: document InterpreterConfig fields
I'm building functionality on top of this config and figured I'd
take the time to document the fields to make things easier to
understand.
2021-08-21 12:47:20 -07:00
David Hewitt 17077fd05e
Merge pull request #1816 from mejrs/dedup
deduplicate pyo3-macros documentation
2021-08-21 09:23:50 +01:00
Benjamin Kay ac4613eb9f
Improve documentation about when we free memory, resolves #311 (#1807)
* Improve API docs regarding when we free memory, resolves #311

* Add chapter to guide about when we free memory, resolves #311

* Fix typos in documentation

Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>

* Add links from guide to docs.rs

* Update guide/src/memory.md

Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2021-08-21 09:23:10 +01:00
Gregory Szorc 0cf7776a4d string: implement API to access raw string data
With the recent implementation of non-limited unicode APIs, we're
able to query Python's low-level state to access the raw bytes that
Python is using to store string objects.

This commit implements a safe Rust API for obtaining a view into
Python's internals and representing the raw bytes Python is using
to store strings.

Not only do we allow accessing what Python has stored internally,
but we also support coercing this data to a `Cow<str>`.

Closes #1776.
2021-08-19 18:24:17 -07:00
David Hewitt 79c7e28799
Merge pull request #1812 from tiran/datetime_use_crate
Use 'crate::' to refer to pyo3
2021-08-19 22:46:25 +01:00
mejrs e753d77e4a deduplicate documentation 2021-08-19 22:55:39 +02:00
David Hewitt 0613b5a8f8
Merge pull request #1805 from mejrs/proc_macro_hygiene
Test proc macro hygiene for `#[pyclass]` macro.
2021-08-18 19:08:41 +01:00
Christian Heimes d74c3f6cd0 Use 'crate::' to refer to pyo3
Fixes: #1811
Signed-off-by: Christian Heimes <christian@python.org>
2021-08-18 12:54:27 +02:00
Péter Leéh f72b2c8f09
Py_CompileString decref (#1810)
* update changelog

* fix memory leak in PyModule::from_code

* add PR link to changelog

* Add Py_DECREF also when PyImport_ExecCodeModuleEx fails

* Remove duplicated calls, simplify logic

Co-authored-by: messense <messense@icloud.com>

Co-authored-by: messense <messense@icloud.com>
2021-08-18 10:50:57 +00:00
messense 78b924db07
Merge pull request #1806 from pschafhalter/fix-memory-leak
Fix memory leak when calling Python from Rust
2021-08-18 14:52:25 +08:00
Peter Schafhalter 37ea57de2b Update changelog 2021-08-17 22:56:27 -07:00
Peter Schafhalter e4466549e5 Edit changelog 2021-08-17 14:37:39 -07:00
Peter Schafhalter cf39181efd Fix memory leak when calling Python from Rust 2021-08-17 14:21:56 -07:00
mejrs 21196e68e2 fully disambiguate types 2021-08-17 22:32:41 +02:00
mejrs f79420dabd remove shadowing 2021-08-17 21:38:28 +02:00
David Hewitt fc90092ef9
Merge pull request #1733 from aviramha/list_stuff
Tuple/List get_item to fallible
2021-08-17 17:13:51 +01:00
Aviram Hassan c6255e6734 - `PyList`, `PyTuple` and `PySequence`'s `get_item` now accepts only `usize` indices instead of `isize`.
- `PyList` and `PyTuple`'s `get_item` now always uses the safe API. See `get_item_unchecked` for retrieving index without checks.
2021-08-17 15:01:22 +01:00
David Hewitt 7a92e2699b
Merge pull request #1804 from davidhewitt/deprecate_split_from
pytuple: deprecate split_from
2021-08-17 15:00:21 +01:00
mejrs 3be3af4121 fix extends param 2021-08-17 15:19:19 +02:00
mejrs bce5100dc3 use macro to shadow 2021-08-17 15:16:03 +02:00
David Hewitt 37d39aa83a
Merge pull request #1751 from davidhewitt/pyany-py
pyany: add PyAny::py()
2021-08-17 14:10:16 +01:00
David Hewitt 7fd70d103e pytuple: deprecate split_from 2021-08-17 13:57:27 +01:00
David Hewitt df41903c98
Merge pull request #1802 from PyO3/pylist_insert_index
Some PyList index fixes
2021-08-17 13:54:54 +01:00
Georg Brandl 79d9741906 Add PyList::slice and fix index types of PyTuple::slice, PyList::insert and PyList::set_item.
NB: the behavior on out-of-range indices hasn't changed;
it was merely wrongly documented before.

See #1667
2021-08-17 09:45:06 +02:00
David Hewitt 029ac5b416
Merge pull request #1799 from mtreinish/loosen-num-complex
Expand supported num-complex versions
2021-08-17 08:02:35 +01:00
Matthew Treinish d575c30eb6
Update Cargo.toml
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2021-08-16 18:22:46 -04:00
Matthew Treinish 60db3a45b3
Expand supported num-complex versions
When building an extension with rust-numpy and ndarray on the MSRV of
1.41 with complex numbers. The num-complex crate version needs to be
0.2 which was the pinned version as of ndarray 0.13.1 which was the last
release of ndarray that supported building with rust 1.41. However, the
pyo3 pinned version of 0.4 is incompatible with this and will cause an
error when building because of the version mismatch. To fix this This
commit expands the supported versions for num-complex to match what
rust-numpy uses [1] so that we can build pyo3, numpy, ndarray, and
num-complex in an extension with rust 1.41.

Fixes #1798

[1] https://github.com/PyO3/rust-numpy/blob/v0.14.1/Cargo.toml#L19
2021-08-16 14:43:07 -04:00
mejrs 370968b595 test macro hygiene for pyclass 2021-08-16 16:32:11 +02:00
David Hewitt 336e87ec18
Merge pull request #1795 from davidhewitt/update-cargo-llvm-cov
ci: update cargo-llvm-cov
2021-08-16 08:17:02 +01:00
David Hewitt b1abe03984 ci: update cargo-llvm-cov 2021-08-15 23:54:43 +01:00
David Hewitt eafc75ab06
Merge pull request #1777 from indygreg/unicode-apis
ffi: define some cpython/unicodeobject bindings
2021-08-14 23:07:49 +01:00
Gregory Szorc 807c8ee33c string: enable use of PyUnicode_AsUTF8AndSize on all Python versions
This API is available in all supported Python versions and isn't
deprecated.
2021-08-14 11:19:02 -07:00
David Hewitt 974a782895 ffi: PyPy and Python 3.10 attributes for unicodeobject 2021-08-14 10:09:57 -07:00
Gregory Szorc 9690e9ece6 ffi: move limited API unicode symbols to cpython/unicodeobject.rs
All symbols which are canonically defined in cpython/unicodeobject.h
and had been defined in our unicodeobject.rs have been moved to our
corresponding cpython/unicodeobject.rs file.

This module is only present in non-limited build configurations, so
we were able to drop the cfg annotations as part of moving the
definitions.
2021-08-14 08:19:10 -07:00
Gregory Szorc b8738b0430 ffi: define some cpython/unicodeobject bindings
pyo3 doesn't currently define various Unicode bindings that allow the
retrieval of raw data from Python strings. Said bindings are a
prerequisite to possibly exposing this data in the Rust API (#1776).
Even if those high-level APIs never materialize, the FFI bindings are
necessary to enable consumers of the raw C API to utilize them.

This commit partially defines the FFI bindings as defined in
CPython's Include/cpython/unicodeobject.h file.

I used the latest CPython 3.9 Git commit for defining the order
of the symbols and the implementation of various inline preprocessor
macros. I tried to be as faithful as possible to the original
implementation, preserving intermediate `#define`s as inline functions.

Missing symbols have been annotated with `skipped` and symbols currently
defined in `src/ffi/unicodeobject.rs` have been annotated with `move`.

The `state` field of `PyASCIIObject` is a bitfield, which Rust doesn't
support. So we've provided accessor functions for retrieving these
fields' values. No accessor functions are present because you shouldn't
be touching these values from Rust code.

Tests of the bitfield APIs and macro implementations have been added.
2021-08-14 08:19:10 -07:00
David Hewitt 584de688c3
Merge pull request #1779 from indygreg/setter-handle-del
macros: raise AttributeError on property deletion requests
2021-08-14 08:57:33 +01:00
David Hewitt 78f269e946
Merge branch 'main' into setter-handle-del 2021-08-14 08:07:15 +01:00
David Hewitt 739a885f5a
Merge pull request #1791 from indygreg/fix-pystatus
ffi: fix PyStatus._type
2021-08-14 08:05:40 +01:00
David Hewitt 30e6f58080
Update CHANGELOG.md 2021-08-14 07:43:32 +01:00
David Hewitt 150f4adbba
Update CHANGELOG.md 2021-08-14 07:40:52 +01:00
Gregory Szorc 410c9f13c9 macros: raise AttributeError on property deletion requests
The setter function will receive a NULL value on deletion requests.
This wasn't properly handled before, leading to a panic.

The new code raises AttributeError in this scenario instead.

A test for the behavior has been added. Documentation has also
been updated to reflect the behavior.
2021-08-13 21:01:00 -07:00
Gregory Szorc d3762a679f ffi: fix PyStatus._type
The field wasn't defined previously. And the enum wasn't defined as
`[repr(C)]`.

This missing field could result in memory corruption if a Rust-allocated
`PyStatus` was passed to a Python API, which could perform an
out-of-bounds write. In my code, the out-of-bounds write corrupted a
variable on the stack, leading to a segfault due to illegal memory
access. However, this crash only occurred on Rust 1.54! So I initially
mis-attribted it as a compiler bug / regression. It appears that a
low-level Rust change in 1.54.0 changed the LLVM IR in such a way to
cause LLVM optimization passes to produce sufficiently different
assembly code, tickling the crash. See
https://github.com/rust-lang/rust/issues/87947 if you want to see
the wild goose chase I went on in Rust / LLVM land to potentially
pin this on a compiler bug.

Lessen learned: Rust crashes are almost certainly due to use of
`unsafe`.
2021-08-13 20:51:26 -07:00
David Hewitt 254ea53f3f
Merge pull request #1789 from davidhewitt/example-word-count
examples: make `word-count` example comparison fairer
2021-08-13 15:08:16 +01:00
David Hewitt e44920ef9c
Merge pull request #1790 from PyO3/davidhewitt-patch-1
ffi: fix PyPy symbols for `cpython/pystate.rs`
2021-08-13 15:07:29 +01:00
David Hewitt 06ff76fec2
examples: make `word-count` example comparison fairer 2021-08-13 14:19:46 +01:00
David Hewitt c439f97f1a
ffi: fix PyPy symbols for `cpython/pystate.rs` 2021-08-13 14:07:15 +01:00
David Hewitt 388c2552f8
Merge pull request #1786 from mejrs/with_gil2
tests: switch to python_with_gil
2021-08-13 12:39:17 +01:00
David Hewitt 874e7e5cab
Merge pull request #1787 from davidhewitt/ffi-pystate
ffi: cleanup pystate
2021-08-13 12:36:37 +01:00