luxolus
/
pyo3
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'master' into slot-provider

This commit is contained in:
Yuji Kanagawa 2020-06-18 15:49:56 +09:00 committed by GitHub
parent 1207c2d03a 6ea834526b
commit a044f3c322
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
29 changed files with 953 additions and 144 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
CHANGELOG.md
View File

@ -7,12 +7,20 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
## [Unreleased]
### Added
- Add FFI definition `PyObject_AsFileDescriptor` [#938](https://github.com/PyO3/pyo3/pull/938)
- Add `PyByteArray::data`, `PyByteArray::as_bytes`, and `PyByteArray::as_bytes_mut`. [#967](https://github.com/PyO3/pyo3/pull/967)
- Add `Py::borrow`, `Py::borrow_mut`, `Py::try_borrow`, and `Py::try_borrow_mut` for accessing `#[pyclass]` values. [#976](https://github.com/PyO3/pyo3/pull/976)
### Changed
- Simplify internals of `#[pyo3(get)]` attribute. (Remove the hidden API `GetPropertyValue`.) [#934](https://github.com/PyO3/pyo3/pull/934)
- Call `Py_Finalize` at exit to flush buffers, etc. [#943](https://github.com/PyO3/pyo3/pull/943)
- Add type parameter to PyBuffer. #[951](https://github.com/PyO3/pyo3/pull/951)
- Require `Send` bound for `#[pyclass]`. [#966](https://github.com/PyO3/pyo3/pull/966)
- Add `Python` argument to most methods on `PyObject` and `Py<T>` to ensure GIL safety. [#970](https://github.com/PyO3/pyo3/pull/970)
- Change signature of `PyTypeObject::type_object()` - now takes `Python` argument and returns `&PyType`. [#970](https://github.com/PyO3/pyo3/pull/970)
- Change return type of `PyTuple::slice()` and `PyTuple::split_from()` from `Py<PyTuple>` to `&PyTuple`. [#970](https://github.com/PyO3/pyo3/pull/970)
- Change return type of `PyTuple::as_slice` to `&[&PyAny]`. [#971](https://github.com/PyO3/pyo3/pull/971)
- Update `num-complex` optional dependendency from `0.2` to `0.3`. [#977](https://github.com/PyO3/pyo3/pull/977)
- Update `num-bigint` optional dependendency from `0.2` to `0.3`. [#978](https://github.com/PyO3/pyo3/pull/978)
### Removed
- Remove `ManagedPyRef` (unused, and needs specialization) [#930](https://github.com/PyO3/pyo3/pull/930)

4
Cargo.toml
View File

@ -24,8 +24,8 @@ indoc = { version = "0.3.4", optional = true }
inventory = { version = "0.1.4", optional = true }
libc = "0.2.62"
parking_lot = "0.10.2"
num-bigint = { version = "0.2", optional = true }
num-complex = { version = "0.2", optional = true }
num-bigint = { version = "0.3", optional = true }
num-complex = { version = "0.3", optional = true }
paste = { version = "0.1.6", optional = true }
pyo3cls = { path = "pyo3cls", version = "=0.10.1", optional = true }
unindent = { version = "0.1.4", optional = true }

5
README.md
View File

@ -87,6 +87,11 @@ rustflags = [
While developing, you can symlink (or copy) and rename the shared library from the target folder: On MacOS, rename `libstring_sum.dylib` to `string_sum.so`, on Windows `libstring_sum.dll` to `string_sum.pyd`, and on Linux `libstring_sum.so` to `string_sum.so`. Then open a Python shell in the same folder and you'll be able to `import string_sum`.
Adding the `cdylib` arguments in the `Cargo.toml` files changes the way your crate is compiled.
Other Rust projects using your crate will have to link against the `.so` or `.pyd` file rather than include your library directly as normal.
In order to make available your crate in the usual way for Rust user, you you might want to consider using both `crate-type = ["cdylib", "rlib"]` so that Rust users can use the `rlib` (the default lib crate type).
Another possibility is to create a new crate to perform the binding.
To build, test and publish your crate as a Python module, you can use [maturin](https://github.com/PyO3/maturin) or [setuptools-rust](https://github.com/PyO3/setuptools-rust). You can find an example for setuptools-rust in [examples/word-count](https://github.com/PyO3/pyo3/tree/master/examples/word-count), while maturin should work on your crate without any configuration.
## Using Python from Rust

1
guide/src/SUMMARY.md
View File

@ -14,3 +14,4 @@
- [PyPy support](pypy.md)
- [Appendix A: PyO3 and rust-cpython](rust_cpython.md)
- [Appendix B: Migration Guide](migration.md)
- [Appendix C: Trait bounds](trait_bounds.md)

1
guide/src/class.md
View File

@ -471,6 +471,7 @@ From the Python perspective, the `method2` in this example does not accept any a
To create a class method for a custom class, the method needs to be annotated
with the `#[classmethod]` attribute.
This is the equivalent of the Python decorator `@classmethod`.
```rust
# use pyo3::prelude::*;

2
guide/src/conversions.md
View File

@ -70,7 +70,7 @@ For most PyO3 usage the conversion cost is worth paying to get these benefits. A
When returning values from functions callable from Python, Python-native types (`&PyAny`, `&PyDict` etc.) can be used with zero cost.
Because these types are references, in some situations the Rust compiler may ask for lifetime annotations. If this is the case, you should use `Py<PyAny>`, `Py<PyDict>` etc. instead - which are also zero-cost and can be created from the native types with an `.into()` conversion.
Because these types are references, in some situations the Rust compiler may ask for lifetime annotations. If this is the case, you should use `Py<PyAny>`, `Py<PyDict>` etc. instead - which are also zero-cost. For all of these Python-native types `T`, `Py<T>` can be created from `T` with an `.into()` conversion.
If your function is fallible, it should return `PyResult<T>`, which will raise a `Python` exception if the `Err` variant is returned.

3
guide/src/module.md
View File

@ -34,6 +34,9 @@ fn sum_as_string(a: i64, b: i64) -> String {
The `#[pymodule]` procedural macro attribute takes care of exporting the initialization function of your module to Python. It can take as an argument the name of your module, which must be the name of the `.so` or `.pyd` file; the default is the Rust function's name.
If the name of the module (the default being the function name) does not match the name of the `.so` or `.pyd` file, you will get an import error in Python with the following message:
`ImportError: dynamic module does not define module export function (PyInit_name_of_your_module)`
To import the module, either copy the shared library as described in [the README](https://github.com/PyO3/pyo3) or use a tool, e.g. `maturin develop` with [maturin](https://github.com/PyO3/maturin) or `python setup.py develop` with [setuptools-rust](https://github.com/PyO3/setuptools-rust).
## Documentation

568
guide/src/trait_bounds.md Normal file
View File

@ -0,0 +1,568 @@
# Using in Python a Rust function with trait bounds
PyO3 allows for easy conversion from Rust to Python for certain functions and classes (see the [conversion table](https://pyo3.rs/master/conversions.html)).
However, it is not always straightforward to convert Rust code that requires a given trait implementation as an argument.
This tutorial explains how to convert a Rust function that takes a trait as argument for use in Python with classes implementing the same methods as the trait.
Why is this useful?
### Pros
- Make your Rust code available to Python users
- Code complex algorithms in Rust with the help of the borrow checker
### Cons
- Not as fast as native Rust (type conversion has to be performed and one part of the code runs in Python)
- You need to adapt your code to expose it
## Example
Let's work with the following basic example of an implementation of a optimization solver operating on a given model.
Let's say we have a function `solve` that operates on a model and mutates its state.
The argument of the function can be any model that implements the `Model` trait :
```rust
pub trait Model {
fn set_variables(&mut self, inputs: &Vec<f64>);
fn compute(&mut self);
fn get_results(&self) -> Vec<f64>;
}
pub fn solve<T: Model>(model: &mut T) {
println!("Magic solver that mutates the model into a resolved state");
}
```
Let's assume we have the following constraints:
- We cannot change that code as it runs on many Rust models.
- We also have many Python models that cannot be solved as this solver is not available in that language.
Rewriting it in Python would be cumbersome and error-prone, as everything is already available in Rust.
How could we expose this solver to Python thanks to PyO3 ?
## Implementation of the trait bounds for the Python class
If a Python class implements the same three methods as the `Model` trait, it seems logical it could be adapted to use the solver.
However, it is not possible to pass a `PyObject` to it as it does not implement the Rust trait (even if the Python model has the required methods).
In order to implement the trait, we must write a wrapper around the calls in Rust to the Python model.
The method signatures must be the same as the trait, keeping in mind that the Rust trait cannot be changed for the purpose of making the code available in Python.
The Python model we want to expose is the following one, which already contains all the required methods:
```python
class Model:
def set_variables(self, inputs):
self.inputs = inputs
def compute(self):
self.results = [elt**2 - 3 for elt in self.inputs]
def get_results(self):
return self.results
```
The following wrapper will call the Python model from Rust, using a struct to hold the model as a `PyAny` object:
```rust
use pyo3::prelude::*;
use pyo3::types::PyAny;
# pub trait Model {
# fn set_variables(&mut self, inputs: &Vec<f64>);
# fn compute(&mut self);
# fn get_results(&self) -> Vec<f64>;
# }
struct UserModel {
model: Py<PyAny>,
}
impl Model for UserModel {
fn set_variables(&mut self, var: &Vec<f64>) {
println!("Rust calling Python to set the variables");
let gil = Python::acquire_gil();
let py = gil.python();
let values: Vec<f64> = var.clone();
let list: PyObject = values.into_py(py);
let py_model = self.model.as_ref(py);
py_model
.call_method("set_variables", (list,), None)
.unwrap();
}
fn get_results(&self) -> Vec<f64> {
println!("Rust calling Python to get the results");
let gil = Python::acquire_gil();
let py = gil.python();
self
.model
.as_ref(py)
.call_method("get_results", (), None)
.unwrap()
.extract()
.unwrap()
}
fn compute(&mut self) {
println!("Rust calling Python to perform the computation");
let gil = Python::acquire_gil();
let py = gil.python();
self.model
.as_ref(py)
.call_method("compute", (), None)
.unwrap();
}
}
```
Now that this bit is implemented, let's expose the model wrapper to Python.
Let's add the PyO3 annotations and add a constructor:
```rust
# pub trait Model {
# fn set_variables(&mut self, inputs: &Vec<f64>);
# fn compute(&mut self);
# fn get_results(&self) -> Vec<f64>;
# }
# use pyo3::prelude::*;
# use pyo3::types::PyAny;
#[pyclass]
struct UserModel {
model: Py<PyAny>,
}
#[pymodule]
fn trait_exposure(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<UserModel>()?;
Ok(())
}
#[pymethods]
impl UserModel {
#[new]
pub fn new(model: Py<PyAny>) -> Self {
UserModel { model }
}
}
```
Now we add the PyO3 annotations to the trait implementation:
```rust,ignore
#[pymethods]
impl Model for UserModel {
// the previous trait implementation
}
However, the previous code will not compile. The compilation error is the following one:
`error: #[pymethods] cannot be used on trait impl blocks`
That's a bummer!
However, we can write a second wrapper around these functions to call them directly.
This wrapper will also perform the type conversions between Python and Rust.
```rust
# use pyo3::prelude::*;
# use pyo3::types::PyAny;
#
# pub trait Model {
# fn set_variables(&mut self, inputs: &Vec<f64>);
# fn compute(&mut self);
# fn get_results(&self) -> Vec<f64>;
# }
#
# #[pyclass]
# struct UserModel {
# model: Py<PyAny>,
# }
#
# impl Model for UserModel {
# fn set_variables(&mut self, var: &Vec<f64>) {
# println!("Rust calling Python to set the variables");
# let gil = Python::acquire_gil();
# let py = gil.python();
# let values: Vec<f64> = var.clone();
# let list: PyObject = values.into_py(py);
# let py_model = self.model.as_ref(py);
# py_model
# .call_method("set_variables", (list,), None)
# .unwrap();
# }
#
# fn get_results(&self) -> Vec<f64> {
# println!("Rust calling Python to get the results");
# let gil = Python::acquire_gil();
# let py = gil.python();
# self
# .model
# .as_ref(py)
# .call_method("get_results", (), None)
# .unwrap()
# .extract()
# .unwrap()
# }
#
# fn compute(&mut self) {
# println!("Rust calling Python to perform the computation");
# let gil = Python::acquire_gil();
# let py = gil.python();
# self.model
# .as_ref(py)
# .call_method("compute", (), None)
# .unwrap();
# }
# }
#[pymethods]
impl UserModel {
pub fn set_variables(&mut self, var: Vec<f64>) -> PyResult<()> {
println!("Set variables from Python calling Rust");
Model::set_variables(self, &var);
Ok(())
}
pub fn get_results(&mut self) -> PyResult<Vec<f64>> {
println!("Get results from Python calling Rust");
let results = Model::get_results(self);
let gil = Python::acquire_gil();
let py = gil.python();
let py_results = results.into_py(py);
Ok(py_results)
}
pub fn compute(&mut self) -> PyResult<()> {
println!("Compute from Python calling Rust");
Model::compute(self);
Ok(())
}
}
```
This wrapper handles the type conversion between the PyO3 requirements and the trait.
In order to meet PyO3 requirements, this wrapper must:
- return an object of type `PyResult`
- use only values, not references in the method signatures
Let's run the file python file:
```python
class Model:
def set_variables(self, inputs):
self.inputs = inputs
def compute(self):
self.results = [elt**2 - 3 for elt in self.inputs]
def get_results(self):
return self.results
if __name__=="__main__":
import trait_exposure
myModel = Model()
my_rust_model = trait_exposure.UserModel(myModel)
my_rust_model.set_variables([2.0])
print("Print value from Python: ", myModel.inputs)
my_rust_model.compute()
print("Print value from Python through Rust: ", my_rust_model.get_results())
print("Print value directly from Python: ", myModel.get_results())
```
This outputs:
```block
Set variables from Python calling Rust
Set variables from Rust calling Python
Print value from Python: [2.0]
Compute from Python calling Rust
Compute from Rust calling Python
Get results from Python calling Rust
Get results from Rust calling Python
Print value from Python through Rust: [1.0]
Print value directly from Python: [1.0]
```
We have now successfully exposed a Rust model that implements the `Model` trait to Python!
We will now expose the `solve` function, but before, let's talk about types errors.
## Type errors in Python
What happens if you have type errors when using Python and how can you improve the error messages?
### Wrong types in Python function arguments
Let's assume in the first case that you will use in your Python file `my_rust_model.set_variables(2.0)` instead of `my_rust_model.set_variables([2.0])`.
The Rust signature expects a vector, which corresponds to a list in Python.
What happens if instead of a vector, we pass a single value ?
At the execution of Python, we get :
```block
File "main.py", line 15, in <module>
my_rust_model.set_variables(2)
TypeError
```
It is a type error and Python points to it, so it's easy to identify and solve.
### Wrong types in Python method signatures
Let's assume now that the return type of one of the methods of our Model class is wrong, for example the `get_results` method that is expected to return a `Vec<f64>` in Rust, a list in Python.
```python
class Model:
def set_variables(self, inputs):
self.inputs = inputs
def compute(self):
self.results = [elt**2 -3 for elt in self.inputs]
def get_results(self):
return self.results[0]
#return self.results <-- this is the expected output
```
This call results in the following panic:
```block
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: PyErr { type: Py(0x10dcf79f0, PhantomData) }
```
This error code is not helpful for a Python user that does not know anything about Rust, or someone that does not know PyO3 was used to interface the Rust code.
However, as we are responsible for making the Rust code available to Python, we can do something about it.
The issue is that we called `unwrap` anywhere we could, and therefore any panic from PyO3 will be directly forwarded to the end user.
Let's modify the code performing the type conversion to give a helpful error message to the Python user:
We used in our `get_results` method the following call that performs the type conversion:
```rust
# use pyo3::prelude::*;
# use pyo3::types::PyAny;
#
# pub trait Model {
# fn set_variables(&mut self, inputs: &Vec<f64>);
# fn compute(&mut self);
# fn get_results(&self) -> Vec<f64>;
# }
#
# #[pyclass]
# struct UserModel {
# model: Py<PyAny>,
# }
impl Model for UserModel {
fn get_results(&self) -> Vec<f64> {
println!("Get results from Rust calling Python");
let gil = Python::acquire_gil();
let py = gil.python();
self
.model
.as_ref(py)
.call_method("get_results", (), None)
.unwrap()
.extract()
.unwrap()
}
# fn set_variables(&mut self, var: &Vec<f64>) {
# println!("Rust calling Python to set the variables");
# let gil = Python::acquire_gil();
# let py = gil.python();
# let values: Vec<f64> = var.clone();
# let list: PyObject = values.into_py(py);
# let py_model = self.model.as_ref(py);
# py_model
# .call_method("set_variables", (list,), None)
# .unwrap();
# }
#
# fn compute(&mut self) {
# println!("Rust calling Python to perform the computation");
# let gil = Python::acquire_gil();
# let py = gil.python();
# self.model
# .as_ref(py)
# .call_method("compute", (), None)
# .unwrap();
# }
}
```
Let's break it down in order to perform better error handling:
```rust
# use pyo3::prelude::*;
# use pyo3::types::PyAny;
#
# pub trait Model {
# fn set_variables(&mut self, inputs: &Vec<f64>);
# fn compute(&mut self);
# fn get_results(&self) -> Vec<f64>;
# }
#
# #[pyclass]
# struct UserModel {
# model: Py<PyAny>,
# }
impl Model for UserModel {
fn get_results(&self) -> Vec<f64> {
println!("Get results from Rust calling Python");
let gil = Python::acquire_gil();
let py = gil.python();
let py_result: &PyAny = self
.model
.as_ref(py)
.call_method("get_results", (), None)
.unwrap();
if py_result.get_type().name() != "list" {
panic!("Expected a list for the get_results() method signature, got {}", py_result.get_type().name());
}
py_result.extract().unwrap()
}
# fn set_variables(&mut self, var: &Vec<f64>) {
# println!("Rust calling Python to set the variables");
# let gil = Python::acquire_gil();
# let py = gil.python();
# let values: Vec<f64> = var.clone();
# let list: PyObject = values.into_py(py);
# let py_model = self.model.as_ref(py);
# py_model
# .call_method("set_variables", (list,), None)
# .unwrap();
# }
#
# fn compute(&mut self) {
# println!("Rust calling Python to perform the computation");
# let gil = Python::acquire_gil();
# let py = gil.python();
# self.model
# .as_ref(py)
# .call_method("compute", (), None)
# .unwrap();
# }
}
```
By doing so, you catch the result of the Python computation and check its type in order to be able to deliver a better error message before performing the unwrapping.
Of course, it does not cover all the possible wrong outputs:
the user could return a list of strings instead of a list of floats.
In this case, a runtime panic would still occur due to PyO3, but with an error message much more difficult to decipher for non-rust user.
It is up to the developer exposing the rust code to decide how much effort to invest into Python type error handling and improved error messages.
## The final code
Now let's expose the `solve()` function to make it available from Python.
It is not possible to directly expose the `solve` function to Python, as the type conversion cannot be performed.
It requires an object implementing the `Model` trait as input.
However, the `UserModel` already implements this trait.
Because of this, we can write a function wrapper that takes the `UserModel`--which has already been exposed to Python--as an argument in order to call the core function `solve`.
It is also required to make the struct public.
```rust
use pyo3::prelude::*;
use pyo3::wrap_pyfunction;
use pyo3::types::PyAny;
pub trait Model {
fn set_variables(&mut self, var: &Vec<f64>);
fn get_results(&self) -> Vec<f64>;
fn compute(&mut self);
}
pub fn solve<T: Model>(model: &mut T) {
println!("Magic solver that mutates the model into a resolved state");
}
#[pyfunction]
#[name = "solve"]
pub fn solve_wrapper(model: &mut UserModel) {
solve(model);
}
#[pyclass]
pub struct UserModel {
model: Py<PyAny>,
}
#[pymodule]
fn trait_exposure(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<UserModel>()?;
m.add_wrapped(wrap_pyfunction!(solve_wrapper)).unwrap();
Ok(())
}
#[pymethods]
impl UserModel {
#[new]
pub fn new(model: Py<PyAny>) -> Self {
UserModel { model }
}
pub fn set_variables(&mut self, var: Vec<f64>) -> PyResult<()> {
println!("Set variables from Python calling Rust");
Model::set_variables(self, &var);
Ok(())
}
pub fn get_results(&mut self) -> PyResult<Vec<f64>> {
println!("Get results from Python calling Rust");
let results = Model::get_results(self);
let gil = Python::acquire_gil();
let py = gil.python();
let py_results = results.into_py(py);
Ok(py_results)
}
pub fn compute(&mut self) -> PyResult<()> {
Model::compute(self);
Ok(())
}
}
impl Model for UserModel {
fn set_variables(&mut self, var: &Vec<f64>) {
println!("Set variables from Rust calling Python");
let gil = Python::acquire_gil();
let py = gil.python();
let values: Vec<f64> = var.clone();
let list: PyObject = values.into_py(py);
let py_model = self.model.as_ref(py);
py_model
.call_method("set_variables", (list,), None)
.unwrap();
}
fn get_results(&self) -> Vec<f64> {
println!("Get results from Rust calling Python");
let gil = Python::acquire_gil();
let py = gil.python();
let py_result: &PyAny = self
.model
.as_ref(py)
.call_method("get_results", (), None)
.unwrap();
if py_result.get_type().name() != "list" {
panic!("Expected a list for the get_results() method signature, got {}", py_result.get_type().name());
}
py_result.extract().unwrap()
}
fn compute(&mut self) {
println!("Compute from Rust calling Python");
let gil = Python::acquire_gil();
let py = gil.python();
self.model
.as_ref(py)
.call_method("compute", (), None)
.unwrap();
}
}
```

4
src/conversion.rs
View File

@ -419,7 +419,7 @@ pub unsafe trait FromPyPointer<'p>: Sized {
unsafe fn from_owned_ptr_or_panic(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self {
match Self::from_owned_ptr_or_opt(py, ptr) {
Some(s) => s,
None => err::panic_after_error(),
None => err::panic_after_error(py),
}
}
unsafe fn from_owned_ptr(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self {
@ -436,7 +436,7 @@ pub unsafe trait FromPyPointer<'p>: Sized {
unsafe fn from_borrowed_ptr_or_panic(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self {
match Self::from_borrowed_ptr_or_opt(py, ptr) {
Some(s) => s,
None => err::panic_after_error(),
None => err::panic_after_error(py),
}
}
unsafe fn from_borrowed_ptr(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self {

54
src/err.rs
View File

@ -1,12 +1,13 @@
// Copyright (c) 2017-present PyO3 Project and Contributors
use crate::gil::ensure_gil;
use crate::panic::PanicException;
use crate::type_object::PyTypeObject;
use crate::types::PyType;
use crate::{exceptions, ffi};
use crate::{
AsPyPointer, FromPy, FromPyPointer, IntoPy, IntoPyPointer, Py, PyAny, PyObject, Python,
ToBorrowedObject, ToPyObject,
AsPyPointer, FromPy, FromPyPointer, IntoPy, IntoPyPointer, Py, PyAny, PyNativeType, PyObject,
Python, ToBorrowedObject, ToPyObject,
};
use libc::c_int;
use std::ffi::CString;
@ -88,11 +89,14 @@ impl PyErr {
T: PyTypeObject,
V: ToPyObject + 'static,
{
let ty = T::type_object();
let gil = ensure_gil();
let py = unsafe { gil.python() };
let ty = T::type_object(py);
assert_ne!(unsafe { ffi::PyExceptionClass_Check(ty.as_ptr()) }, 0);
PyErr {
ptype: ty,
ptype: ty.into(),
pvalue: PyErrValue::ToObject(Box::new(value)),
ptraceback: None,
}
@ -103,12 +107,12 @@ impl PyErr {
/// `exc` is the exception type; usually one of the standard exceptions
/// like `exceptions::RuntimeError`.
/// `args` is the a tuple of arguments to pass to the exception constructor.
pub fn from_type<A>(exc: Py<PyType>, args: A) -> PyErr
pub fn from_type<A>(exc: &PyType, args: A) -> PyErr
where
A: ToPyObject + 'static,
{
PyErr {
ptype: exc,
ptype: exc.into(),
pvalue: PyErrValue::ToObject(Box::new(args)),
ptraceback: None,
}
@ -119,11 +123,14 @@ impl PyErr {
where
T: PyTypeObject,
{
let ty = T::type_object();
let gil = ensure_gil();
let py = unsafe { gil.python() };
let ty = T::type_object(py);
assert_ne!(unsafe { ffi::PyExceptionClass_Check(ty.as_ptr()) }, 0);
PyErr {
ptype: ty,
ptype: ty.into(),
pvalue: value,
ptraceback: None,
}
@ -140,19 +147,21 @@ impl PyErr {
if unsafe { ffi::PyExceptionInstance_Check(ptr) } != 0 {
PyErr {
ptype: unsafe { Py::from_borrowed_ptr(ffi::PyExceptionInstance_Class(ptr)) },
ptype: unsafe {
Py::from_borrowed_ptr(obj.py(), ffi::PyExceptionInstance_Class(ptr))
},
pvalue: PyErrValue::Value(obj.into()),
ptraceback: None,
}
} else if unsafe { ffi::PyExceptionClass_Check(obj.as_ptr()) } != 0 {
PyErr {
ptype: unsafe { Py::from_borrowed_ptr(ptr) },
ptype: unsafe { Py::from_borrowed_ptr(obj.py(), ptr) },
pvalue: PyErrValue::None,
ptraceback: None,
}
} else {
PyErr {
ptype: exceptions::TypeError::type_object(),
ptype: exceptions::TypeError::type_object(obj.py()).into(),
pvalue: PyErrValue::ToObject(Box::new("exceptions must derive from BaseException")),
ptraceback: None,
}
@ -179,9 +188,9 @@ impl PyErr {
let mut ptraceback: *mut ffi::PyObject = std::ptr::null_mut();
ffi::PyErr_Fetch(&mut ptype, &mut pvalue, &mut ptraceback);
let err = PyErr::new_from_ffi_tuple(ptype, pvalue, ptraceback);
let err = PyErr::new_from_ffi_tuple(py, ptype, pvalue, ptraceback);
if ptype == PanicException::type_object().as_ptr() {
if ptype == PanicException::type_object(py).as_ptr() {
let msg: String = PyAny::from_borrowed_ptr_or_opt(py, pvalue)
.and_then(|obj| obj.extract().ok())
.unwrap_or_else(|| String::from("Unwrapped panic from Python code"));
@ -233,6 +242,7 @@ impl PyErr {
}
unsafe fn new_from_ffi_tuple(
py: Python,
ptype: *mut ffi::PyObject,
pvalue: *mut ffi::PyObject,
ptraceback: *mut ffi::PyObject,
@ -240,24 +250,22 @@ impl PyErr {
// Note: must not panic to ensure all owned pointers get acquired correctly,
// and because we mustn't panic in normalize().
let pvalue = if let Some(obj) =
PyObject::from_owned_ptr_or_opt(Python::assume_gil_acquired(), pvalue)
{
let pvalue = if let Some(obj) = PyObject::from_owned_ptr_or_opt(py, pvalue) {
PyErrValue::Value(obj)
} else {
PyErrValue::None
};
let ptype = if ptype.is_null() {
<exceptions::SystemError as PyTypeObject>::type_object()
<exceptions::SystemError as PyTypeObject>::type_object(py).into()
} else {
Py::from_owned_ptr(ptype)
Py::from_owned_ptr(py, ptype)
};
PyErr {
ptype,
pvalue,
ptraceback: PyObject::from_owned_ptr_or_opt(Python::assume_gil_acquired(), ptraceback),
ptraceback: PyObject::from_owned_ptr_or_opt(py, ptraceback),
}
}
@ -288,12 +296,12 @@ impl PyErr {
}
/// Returns true if the current exception is instance of `T`.
pub fn is_instance<T>(&self, _py: Python) -> bool
pub fn is_instance<T>(&self, py: Python) -> bool
where
T: PyTypeObject,
{
unsafe {
ffi::PyErr_GivenExceptionMatches(self.ptype.as_ptr(), T::type_object().as_ptr()) != 0
ffi::PyErr_GivenExceptionMatches(self.ptype.as_ptr(), T::type_object(py).as_ptr()) != 0
}
}
@ -328,7 +336,7 @@ impl PyErr {
let mut ptraceback = ptraceback.into_ptr();
unsafe {
ffi::PyErr_NormalizeException(&mut ptype, &mut pvalue, &mut ptraceback);
PyErr::new_from_ffi_tuple(ptype, pvalue, ptraceback)
PyErr::new_from_ffi_tuple(py, ptype, pvalue, ptraceback)
}
}
@ -565,7 +573,7 @@ impl_to_pyerr!(std::string::FromUtf16Error, exceptions::UnicodeDecodeError);
impl_to_pyerr!(std::char::DecodeUtf16Error, exceptions::UnicodeDecodeError);
impl_to_pyerr!(std::net::AddrParseError, exceptions::ValueError);
pub fn panic_after_error() -> ! {
pub fn panic_after_error(_py: Python) -> ! {
unsafe {
ffi::PyErr_Print();
}

12
src/exceptions.rs
View File

@ -89,7 +89,7 @@ macro_rules! import_exception {
macro_rules! import_exception_type_object {
($module: expr, $name: ident) => {
unsafe impl $crate::type_object::PyTypeObject for $name {
fn type_object() -> $crate::Py<$crate::types::PyType> {
fn type_object(py: $crate::Python) -> &$crate::types::PyType {
use $crate::type_object::LazyHeapType;
static TYPE_OBJECT: LazyHeapType = LazyHeapType::new();
@ -111,7 +111,7 @@ macro_rules! import_exception_type_object {
}
});
unsafe { $crate::Py::from_borrowed_ptr(ptr.as_ptr() as *mut $crate::ffi::PyObject) }
unsafe { py.from_borrowed_ptr(ptr.as_ptr() as *mut $crate::ffi::PyObject) }
}
}
};
@ -173,7 +173,7 @@ macro_rules! create_exception {
macro_rules! create_exception_type_object {
($module: ident, $name: ident, $base: ty) => {
unsafe impl $crate::type_object::PyTypeObject for $name {
fn type_object() -> $crate::Py<$crate::types::PyType> {
fn type_object(py: $crate::Python) -> &$crate::types::PyType {
use $crate::type_object::LazyHeapType;
static TYPE_OBJECT: LazyHeapType = LazyHeapType::new();
@ -186,7 +186,7 @@ macro_rules! create_exception_type_object {
)
});
unsafe { $crate::Py::from_borrowed_ptr(ptr.as_ptr() as *mut $crate::ffi::PyObject) }
unsafe { py.from_borrowed_ptr(ptr.as_ptr() as *mut $crate::ffi::PyObject) }
}
}
};
@ -215,8 +215,8 @@ macro_rules! impl_native_exception (
}
}
unsafe impl PyTypeObject for $name {
fn type_object() -> $crate::Py<$crate::types::PyType> {
unsafe { $crate::Py::from_borrowed_ptr(ffi::$exc_name) }
fn type_object(py: $crate::Python) -> &$crate::types::PyType {
unsafe { py.from_borrowed_ptr(ffi::$exc_name) }
}
}
);

66
src/gil.rs
View File

@ -370,6 +370,35 @@ extern "C" fn finalize() {
}
}
/// Ensure the GIL is held, useful in implementation of APIs like PyErr::new where it's
/// inconvenient to force the user to acquire the GIL.
pub(crate) fn ensure_gil() -> EnsureGIL {
if gil_is_acquired() {
EnsureGIL(None)
} else {
EnsureGIL(Some(GILGuard::acquire()))
}
}
/// Struct used internally which avoids acquiring the GIL where it's not necessary.
pub(crate) struct EnsureGIL(Option<GILGuard>);
impl EnsureGIL {
/// Get the GIL token.
///
/// # Safety
/// If `self.0` is `None`, then this calls [Python::assume_gil_acquired].
/// Thus this method could be used to get access to a GIL token while the GIL is not held.
/// Care should be taken to only use the returned Python in contexts where it is certain the
/// GIL continues to be held.
pub unsafe fn python(&self) -> Python {
match &self.0 {
Some(gil) => gil.python(),
None => Python::assume_gil_acquired(),
}
}
}
#[cfg(test)]
mod test {
use super::{gil_is_acquired, GILPool, GIL_COUNT, OWNED_OBJECTS, POOL};
@ -569,13 +598,15 @@ mod test {
#[test]
fn test_clone_with_gil() {
let gil = Python::acquire_gil();
let obj = get_object(gil.python());
let count = obj.get_refcnt();
let py = gil.python();
let obj = get_object(py);
let count = obj.get_refcnt(py);
// Cloning with the GIL should increase reference count immediately
#[allow(clippy::redundant_clone)]
let c = obj.clone();
assert_eq!(count + 1, c.get_refcnt());
assert_eq!(count + 1, c.get_refcnt(py));
}
#[test]
@ -583,39 +614,46 @@ mod test {
let gil = Python::acquire_gil();
let py = gil.python();
let obj = get_object(py);
let count = obj.get_refcnt();
let count = obj.get_refcnt(py);
// Cloning without GIL should not update reference count
drop(gil);
let c = obj.clone();
assert_eq!(count, obj.get_refcnt());
assert_eq!(
count,
obj.get_refcnt(unsafe { Python::assume_gil_acquired() })
);
// Acquring GIL will clear this pending change
let gil = Python::acquire_gil();
let py = gil.python();
// Total reference count should be one higher
assert_eq!(count + 1, obj.get_refcnt());
assert_eq!(count + 1, obj.get_refcnt(py));
// Clone dropped then GIL released
// Clone dropped
drop(c);
drop(gil);
// Overall count is now back to the original, and should be no pending change
assert_eq!(count, obj.get_refcnt());
assert_eq!(count, obj.get_refcnt(py));
}
#[test]
fn test_clone_in_other_thread() {
let gil = Python::acquire_gil();
let obj = get_object(gil.python());
let count = obj.get_refcnt();
let py = gil.python();
let obj = get_object(py);
let count = obj.get_refcnt(py);
// Move obj to a thread which does not have the GIL, and clone it
let t = std::thread::spawn(move || {
// Cloning without GIL should not update reference count
#[allow(clippy::redundant_clone)]
let _ = obj.clone();
assert_eq!(count, obj.get_refcnt());
assert_eq!(
count,
obj.get_refcnt(unsafe { Python::assume_gil_acquired() })
);
// Return obj so original thread can continue to use
obj
@ -631,13 +669,13 @@ mod test {
// Re-acquring GIL will clear these pending changes
drop(gil);
let _gil = Python::acquire_gil();
let gil = Python::acquire_gil();
assert!(POOL.pointers_to_incref.lock().is_empty());
assert!(POOL.pointers_to_decref.lock().is_empty());
// Overall count is still unchanged
assert_eq!(count, obj.get_refcnt());
assert_eq!(count, obj.get_refcnt(gil.python()));
}
#[test]

114
src/instance.rs
View File

@ -2,10 +2,11 @@
use crate::err::{PyErr, PyResult};
use crate::gil;
use crate::object::PyObject;
use crate::pycell::{PyBorrowError, PyBorrowMutError, PyCell};
use crate::type_object::PyBorrowFlagLayout;
use crate::{
ffi, AsPyPointer, FromPyObject, IntoPy, IntoPyPointer, PyAny, PyCell, PyClass,
PyClassInitializer, PyRef, PyRefMut, PyTypeInfo, Python, ToPyObject,
ffi, AsPyPointer, FromPyObject, IntoPy, IntoPyPointer, PyAny, PyClass, PyClassInitializer,
PyRef, PyRefMut, PyTypeInfo, Python, ToPyObject,
};
use std::marker::PhantomData;
use std::mem;
@ -48,29 +49,78 @@ pub struct Py<T>(NonNull<ffi::PyObject>, PhantomData<T>);
unsafe impl<T> Send for Py<T> {}
unsafe impl<T> Sync for Py<T> {}
impl<T> Py<T> {
/// Create a new instance `Py<T>`.
///
/// This method is **soft-duplicated** since PyO3 0.9.0.
/// Use [`PyCell::new`](../pycell/struct.PyCell.html#method.new) and
/// `Py::from` instead.
impl<T> Py<T>
where
T: PyClass,
{
/// Create a new instance `Py<T>` of a `#[pyclass]` on the Python heap.
pub fn new(py: Python, value: impl Into<PyClassInitializer<T>>) -> PyResult<Py<T>>
where
T: PyClass,
T::BaseLayout: PyBorrowFlagLayout<T::BaseType>,
{
let initializer = value.into();
let obj = unsafe { initializer.create_cell(py)? };
let ob = unsafe { Py::from_owned_ptr(obj as _) };
let ob = unsafe { Py::from_owned_ptr(py, obj as _) };
Ok(ob)
}
/// Immutably borrows the value `T`. This borrow lasts untill the returned `PyRef` exists.
///
/// Equivalent to `self.as_ref(py).borrow()` -
/// see [`PyCell::borrow`](../pycell/struct.PyCell.html#method.borrow)
///
/// # Panics
/// Panics if the value is currently mutably borrowed. For a non-panicking variant, use
/// [`try_borrow`](#method.try_borrow).
pub fn borrow<'py>(&'py self, py: Python<'py>) -> PyRef<'py, T> {
self.as_ref(py).borrow()
}
/// Mutably borrows the value `T`. This borrow lasts untill the returned `PyRefMut` exists.
///
/// Equivalent to `self.as_ref(py).borrow_mut()` -
/// see [`PyCell::borrow_mut`](../pycell/struct.PyCell.html#method.borrow_mut)
///
/// # Panics
/// Panics if the value is currently mutably borrowed. For a non-panicking variant, use
/// [`try_borrow_mut`](#method.try_borrow_mut).
pub fn borrow_mut<'py>(&'py self, py: Python<'py>) -> PyRefMut<'py, T> {
self.as_ref(py).borrow_mut()
}
/// Immutably borrows the value `T`, returning an error if the value is currently
/// mutably borrowed. This borrow lasts untill the returned `PyRef` exists.
///
/// This is the non-panicking variant of [`borrow`](#method.borrow).
///
/// Equivalent to `self.as_ref(py).try_borrow()` -
/// see [`PyCell::try_borrow`](../pycell/struct.PyCell.html#method.try_borrow)
pub fn try_borrow<'py>(&'py self, py: Python<'py>) -> Result<PyRef<'py, T>, PyBorrowError> {
self.as_ref(py).try_borrow()
}
/// Mutably borrows the value `T`, returning an error if the value is currently borrowed.
/// This borrow lasts untill the returned `PyRefMut` exists.
///
/// This is the non-panicking variant of [`borrow_mut`](#method.borrow_mut).
///
/// Equivalent to `self.as_ref(py).try_borrow_mut() -
/// see [`PyCell::try_borrow_mut`](../pycell/struct.PyCell.html#method.try_borrow_mut)
pub fn try_borrow_mut<'py>(
&'py self,
py: Python<'py>,
) -> Result<PyRefMut<'py, T>, PyBorrowMutError> {
self.as_ref(py).try_borrow_mut()
}
}
impl<T> Py<T> {
/// Creates a `Py<T>` instance for the given FFI pointer.
///
/// This moves ownership over the pointer into the `Py<T>`.
/// Undefined behavior if the pointer is NULL or invalid.
#[inline]
pub unsafe fn from_owned_ptr(ptr: *mut ffi::PyObject) -> Py<T> {
pub unsafe fn from_owned_ptr(_py: Python, ptr: *mut ffi::PyObject) -> Py<T> {
debug_assert!(
!ptr.is_null() && ffi::Py_REFCNT(ptr) > 0,
format!("REFCNT: {:?} - {:?}", ptr, ffi::Py_REFCNT(ptr))
@ -83,11 +133,11 @@ impl<T> Py<T> {
/// Panics if the pointer is NULL.
/// Undefined behavior if the pointer is invalid.
#[inline]
pub unsafe fn from_owned_ptr_or_panic(ptr: *mut ffi::PyObject) -> Py<T> {
pub unsafe fn from_owned_ptr_or_panic(_py: Python, ptr: *mut ffi::PyObject) -> Py<T> {
match NonNull::new(ptr) {
Some(nonnull_ptr) => Py(nonnull_ptr, PhantomData),
None => {
crate::err::panic_after_error();
crate::err::panic_after_error(_py);
}
}
}
@ -109,7 +159,7 @@ impl<T> Py<T> {
/// Calls `Py_INCREF()` on the ptr.
/// Undefined behavior if the pointer is NULL or invalid.
#[inline]
pub unsafe fn from_borrowed_ptr(ptr: *mut ffi::PyObject) -> Py<T> {
pub unsafe fn from_borrowed_ptr(_py: Python, ptr: *mut ffi::PyObject) -> Py<T> {
debug_assert!(
!ptr.is_null() && ffi::Py_REFCNT(ptr) > 0,
format!("REFCNT: {:?} - {:?}", ptr, ffi::Py_REFCNT(ptr))
@ -120,14 +170,14 @@ impl<T> Py<T> {
/// Gets the reference count of the `ffi::PyObject` pointer.
#[inline]
pub fn get_refcnt(&self) -> isize {
pub fn get_refcnt(&self, _py: Python) -> isize {
unsafe { ffi::Py_REFCNT(self.0.as_ptr()) }
}
/// Clones self by calling `Py_INCREF()` on the ptr.
#[inline]
pub fn clone_ref(&self, _py: Python) -> Py<T> {
unsafe { Py::from_borrowed_ptr(self.0.as_ptr()) }
pub fn clone_ref(&self, py: Python) -> Py<T> {
unsafe { Py::from_borrowed_ptr(py, self.0.as_ptr()) }
}
/// Returns the inner pointer without decreasing the refcount.
@ -160,14 +210,20 @@ impl<T> Py<T> {
/// for `#[pyclass]`.
/// ```
/// # use pyo3::prelude::*;
/// let obj: PyObject = {
/// #[pyclass]
/// struct Counter {
/// count: usize,
/// }
/// let counter = {
/// let gil = Python::acquire_gil();
/// let py = gil.python();
/// py.eval("[]", None, None).unwrap().to_object(py)
/// Py::new(py, Counter { count: 0}).unwrap()
/// };
/// let gil = Python::acquire_gil();
/// let py = gil.python();
/// assert_eq!(obj.as_ref(py).len().unwrap(), 0);
/// let counter_cell: &PyCell<Counter> = counter.as_ref(py);
/// let counter_ref = counter_cell.borrow();
/// assert_eq!(counter_ref.count, 0);
/// ```
pub trait AsPyRef: Sized {
type Target;
@ -225,7 +281,7 @@ where
T: AsPyPointer + PyNativeType,
{
fn from(obj: &'a T) -> Self {
unsafe { Py::from_borrowed_ptr(obj.as_ptr()) }
unsafe { Py::from_borrowed_ptr(obj.py(), obj.as_ptr()) }
}
}
@ -235,7 +291,7 @@ where
T: PyClass,
{
fn from(cell: &PyCell<T>) -> Self {
unsafe { Py::from_borrowed_ptr(cell.as_ptr()) }
unsafe { Py::from_borrowed_ptr(cell.py(), cell.as_ptr()) }
}
}
@ -244,7 +300,7 @@ where
T: PyClass,
{
fn from(pyref: PyRef<'a, T>) -> Self {
unsafe { Py::from_borrowed_ptr(pyref.as_ptr()) }
unsafe { Py::from_borrowed_ptr(pyref.py(), pyref.as_ptr()) }
}
}
@ -253,7 +309,7 @@ where
T: PyClass,
{
fn from(pyref: PyRefMut<'a, T>) -> Self {
unsafe { Py::from_borrowed_ptr(pyref.as_ptr()) }
unsafe { Py::from_borrowed_ptr(pyref.py(), pyref.as_ptr()) }
}
}
@ -291,19 +347,19 @@ impl<T> std::convert::From<Py<T>> for PyObject {
impl<'a, T> std::convert::From<&'a T> for PyObject
where
T: AsPyPointer,
T: AsPyPointer + PyNativeType,
{
fn from(ob: &'a T) -> Self {
unsafe { Py::<T>::from_borrowed_ptr(ob.as_ptr()) }.into()
unsafe { Py::<T>::from_borrowed_ptr(ob.py(), ob.as_ptr()) }.into()
}
}
impl<'a, T> std::convert::From<&'a mut T> for PyObject
where
T: AsPyPointer,
T: AsPyPointer + PyNativeType,
{
fn from(ob: &'a mut T) -> Self {
unsafe { Py::<T>::from_borrowed_ptr(ob.as_ptr()) }.into()
unsafe { Py::<T>::from_borrowed_ptr(ob.py(), ob.as_ptr()) }.into()
}
}
@ -317,7 +373,7 @@ where
fn extract(ob: &'a PyAny) -> PyResult<Self> {
unsafe {
ob.extract::<&T::AsRefTarget>()
.map(|val| Py::from_borrowed_ptr(val.as_ptr()))
.map(|val| Py::from_borrowed_ptr(ob.py(), val.as_ptr()))
}
}
}

1
src/lib.rs
View File

@ -346,4 +346,5 @@ pub mod doc_test {
doctest!("../guide/src/pypy.md", guide_pypy_md);
doctest!("../guide/src/rust_cpython.md", guide_rust_cpython_md);
doctest!("../guide/src/types.md", guide_types_md);
doctest!("../guide/src/trait_bounds.md", guide_trait_bounds_md);
}

8
src/object.rs
View File

@ -48,11 +48,11 @@ impl PyObject {
/// Panics if the pointer is NULL.
/// Undefined behavior if the pointer is invalid.
#[inline]
pub unsafe fn from_owned_ptr_or_panic(_py: Python, ptr: *mut ffi::PyObject) -> PyObject {
pub unsafe fn from_owned_ptr_or_panic(py: Python, ptr: *mut ffi::PyObject) -> PyObject {
match NonNull::new(ptr) {
Some(nonnull_ptr) => PyObject(nonnull_ptr),
None => {
crate::err::panic_after_error();
crate::err::panic_after_error(py);
}
}
}
@ -119,7 +119,7 @@ impl PyObject {
}
/// Gets the reference count of the ffi::PyObject pointer.
pub fn get_refcnt(&self) -> isize {
pub fn get_refcnt(&self, _py: Python) -> isize {
unsafe { ffi::Py_REFCNT(self.0.as_ptr()) }
}
@ -131,7 +131,7 @@ impl PyObject {
/// Returns whether the object is considered to be None.
///
/// This is equivalent to the Python expression `self is None`.
pub fn is_none(&self) -> bool {
pub fn is_none(&self, _py: Python) -> bool {
unsafe { ffi::Py_None() == self.as_ptr() }
}

5
src/pycell.rs
View File

@ -166,8 +166,11 @@ pub struct PyCell<T: PyClass> {
unsafe impl<T: PyClass> PyNativeType for PyCell<T> {}
impl<T: PyClass> PyCell<T> {
/// Make new `PyCell` on the Python heap and returns the reference of it.
/// Make a new `PyCell` on the Python heap and return the reference to it.
///
/// In cases where the value in the cell does not need to be accessed immediately after
/// creation, consider [`Py::new`](../instance/struct.Py.html#method.new) as a more efficient
/// alternative.
pub fn new(py: Python, value: impl Into<PyClassInitializer<T>>) -> PyResult<&Self>
where
T::BaseLayout: PyBorrowFlagLayout<T::BaseType>,

7
src/pyclass.rs
View File

@ -74,7 +74,12 @@ pub unsafe fn tp_free_fallback(obj: *mut ffi::PyObject) {
/// The `#[pyclass]` attribute automatically implements this trait for your Rust struct,
/// so you don't have to use this trait directly.
pub trait PyClass:
PyTypeInfo<Layout = PyCell<Self>> + Sized + PyClassAlloc + PyMethods + PyProtoMethods + Send
PyTypeInfo<Layout = PyCell<Self>, AsRefTarget = PyCell<Self>>
+ Sized
+ PyClassAlloc
+ PyMethods
+ PyProtoMethods
+ Send
{
/// Specify this class has `#[pyclass(dict)]` or not.
type Dict: PyClassDict;

10
src/python.rs
View File

@ -6,9 +6,7 @@ use crate::err::{PyDowncastError, PyErr, PyResult};
use crate::gil::{self, GILGuard, GILPool};
use crate::type_object::{PyTypeInfo, PyTypeObject};
use crate::types::{PyAny, PyDict, PyModule, PyType};
use crate::{
ffi, AsPyPointer, AsPyRef, FromPyPointer, IntoPyPointer, PyNativeType, PyObject, PyTryFrom,
};
use crate::{ffi, AsPyPointer, FromPyPointer, IntoPyPointer, PyNativeType, PyObject, PyTryFrom};
use std::ffi::CString;
use std::marker::PhantomData;
use std::os::raw::c_int;
@ -253,7 +251,7 @@ impl<'p> Python<'p> {
where
T: PyTypeObject,
{
unsafe { self.from_borrowed_ptr(T::type_object().into_ptr()) }
T::type_object(self)
}
/// Imports the Python module with the specified name.
@ -265,7 +263,7 @@ impl<'p> Python<'p> {
///
/// This is equivalent to the Python `isinstance` function.
pub fn is_instance<T: PyTypeObject, V: AsPyPointer>(self, obj: &V) -> PyResult<bool> {
T::type_object().as_ref(self).is_instance(obj)
T::type_object(self).is_instance(obj)
}
/// Checks whether type `T` is subclass of type `U`.
@ -276,7 +274,7 @@ impl<'p> Python<'p> {
T: PyTypeObject,
U: PyTypeObject,
{
T::type_object().as_ref(self).is_subclass::<U>()
T::type_object(self).is_subclass::<U>()
}
/// Gets the Python builtin value `None`.

8
src/type_object.rs
View File

@ -4,7 +4,7 @@
use crate::pyclass::{initialize_type_object, PyClass};
use crate::pyclass_init::PyObjectInit;
use crate::types::{PyAny, PyType};
use crate::{ffi, AsPyPointer, Py, Python};
use crate::{ffi, AsPyPointer, Python};
use std::cell::UnsafeCell;
use std::ptr::NonNull;
use std::sync::atomic::{AtomicBool, Ordering};
@ -124,15 +124,15 @@ pub unsafe trait PyTypeInfo: Sized {
/// See [PyTypeInfo::type_object]
pub unsafe trait PyTypeObject {
/// Returns the safe abstraction over the type object.
fn type_object() -> Py<PyType>;
fn type_object(py: Python) -> &PyType;
}
unsafe impl<T> PyTypeObject for T
where
T: PyTypeInfo,
{
fn type_object() -> Py<PyType> {
unsafe { Py::from_borrowed_ptr(<Self as PyTypeInfo>::type_object() as *const _ as _) }
fn type_object(py: Python) -> &PyType {
unsafe { py.from_borrowed_ptr(<Self as PyTypeInfo>::type_object() as *const _ as _) }
}
}

4
src/types/any.rs
View File

@ -259,6 +259,10 @@ impl PyAny {
/// let dict = vec![("reverse", true)].into_py_dict(py);
/// list.call_method(py, "sort", (), Some(dict)).unwrap();
/// assert_eq!(list.extract::<Vec<i32>>(py).unwrap(), vec![7, 6, 5, 4, 3]);
///
/// let new_element = 1.to_object(py);
/// list.call_method(py, "append", (new_element,), None).unwrap();
/// assert_eq!(list.extract::<Vec<i32>>(py).unwrap(), vec![7, 6, 5, 4, 3, 1]);
/// ```
pub fn call_method(
&self,

138
src/types/bytearray.rs
View File

@ -42,6 +42,51 @@ impl PyByteArray {
self.len() == 0
}
/// Get the start of the buffer containing the contents of the bytearray.
///
/// Note that this bytearray object is both shared and mutable, and the backing buffer may be
/// reallocated if the bytearray is resized. This can occur from Python code as well as from
/// Rust via [PyByteArray::resize].
///
/// As a result, the returned pointer should be dereferenced only if since calling this method
/// no Python code has executed, [PyByteArray::resize] has not been called.
pub fn data(&self) -> *mut u8 {
unsafe { ffi::PyByteArray_AsString(self.as_ptr()) as *mut u8 }
}
/// Get the contents of this buffer as a slice.
///
/// # Safety
/// This bytearray must not be resized or edited while holding the slice.
///
/// ## Safety Detail
/// This method is equivalent to `std::slice::from_raw_parts(self.data(), self.len())`, and so
/// all the safety notes of `std::slice::from_raw_parts` apply here.
///
/// In particular, note that this bytearray object is both shared and mutable, and the backing
/// buffer may be reallocated if the bytearray is resized. Mutations can occur from Python
/// code as well as from Rust, via [PyByteArray::as_bytes_mut] and [PyByteArray::resize].
///
/// Extreme care should be exercised when using this slice, as the Rust compiler will
/// make optimizations based on the assumption the contents of this slice cannot change. This
/// can easily lead to undefined behavior.
///
/// As a result, this slice should only be used for short-lived operations to read this
/// bytearray without executing any Python code, such as copying into a Vec.
pub unsafe fn as_bytes(&self) -> &[u8] {
slice::from_raw_parts(self.data(), self.len())
}
/// Get the contents of this buffer as a mutable slice.
///
/// # Safety
/// This slice should only be used for short-lived operations that write to this bytearray
/// without executing any Python code. See the safety note for [PyByteArray::as_bytes].
#[allow(clippy::mut_from_ref)]
pub unsafe fn as_bytes_mut(&self) -> &mut [u8] {
slice::from_raw_parts_mut(self.data(), self.len())
}
/// Copies the contents of the bytearray to a Rust vector.
///
/// # Example
@ -64,15 +109,13 @@ impl PyByteArray {
/// py.run("assert bytearray == b'Hello World.'", None, Some(locals)).unwrap();
/// ```
pub fn to_vec(&self) -> Vec<u8> {
let slice = unsafe {
let buffer = ffi::PyByteArray_AsString(self.as_ptr()) as *mut u8;
let length = ffi::PyByteArray_Size(self.as_ptr()) as usize;
slice::from_raw_parts_mut(buffer, length)
};
slice.to_vec()
unsafe { self.as_bytes() }.to_vec()
}
/// Resizes the bytearray object to the new length `len`.
///
/// Note that this will invalidate any pointers obtained by [PyByteArray::data], as well as
/// any (unsafe) slices obtained from [PyByteArray::as_bytes] and [PyByteArray::as_bytes_mut].
pub fn resize(&self, len: usize) -> PyResult<()> {
unsafe {
let result = ffi::PyByteArray_Resize(self.as_ptr(), len as ffi::Py_ssize_t);
@ -93,30 +136,95 @@ mod test {
use crate::Python;
#[test]
fn test_bytearray() {
fn test_len() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
assert_eq!(src.len(), bytearray.len());
assert_eq!(src, bytearray.to_vec().as_slice());
}
#[test]
fn test_as_bytes() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
let slice = unsafe { bytearray.as_bytes() };
assert_eq!(src, slice);
assert_eq!(bytearray.data() as *const _, slice.as_ptr());
}
#[test]
fn test_as_bytes_mut() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
let slice = unsafe { bytearray.as_bytes_mut() };
assert_eq!(src, slice);
assert_eq!(bytearray.data(), slice.as_mut_ptr());
slice[0..5].copy_from_slice(b"Hi...");
assert_eq!(
&bytearray.str().unwrap().to_string().unwrap(),
"bytearray(b'Hi... Python')"
);
}
#[test]
fn test_to_vec() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
let vec = bytearray.to_vec();
assert_eq!(src, vec.as_slice());
}
#[test]
fn test_from() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
let ba: PyObject = bytearray.into();
let bytearray = PyByteArray::from(py, &ba).unwrap();
assert_eq!(src.len(), bytearray.len());
assert_eq!(src, bytearray.to_vec().as_slice());
assert_eq!(src, unsafe { bytearray.as_bytes() });
}
bytearray.resize(20).unwrap();
assert_eq!(20, bytearray.len());
#[test]
fn test_from_err() {
let gil = Python::acquire_gil();
let py = gil.python();
let none = py.None();
if let Err(err) = PyByteArray::from(py, &none) {
if let Err(err) = PyByteArray::from(py, &py.None()) {
assert!(err.is_instance::<exceptions::TypeError>(py));
} else {
panic!("error");
}
drop(none);
}
#[test]
fn test_resize() {
let gil = Python::acquire_gil();
let py = gil.python();
let src = b"Hello Python";
let bytearray = PyByteArray::new(py, src);
bytearray.resize(20).unwrap();
assert_eq!(20, bytearray.len());
}
}

4
src/types/dict.rs
View File

@ -479,11 +479,11 @@ mod test {
{
let _pool = unsafe { crate::GILPool::new() };
let none = py.None();
cnt = none.get_refcnt();
cnt = none.get_refcnt(py);
let _dict = [(10, none)].into_py_dict(py);
}
{
assert_eq!(cnt, py.None().get_refcnt());
assert_eq!(cnt, py.None().get_refcnt(py));
}
}

8
src/types/iterator.rs
View File

@ -118,7 +118,7 @@ mod tests {
let gil_guard = Python::acquire_gil();
let py = gil_guard.python();
obj = vec![10, 20].to_object(py);
count = obj.get_refcnt();
count = obj.get_refcnt(py);
}
{
@ -129,7 +129,7 @@ mod tests {
assert_eq!(10, it.next().unwrap().unwrap().extract().unwrap());
}
assert_eq!(count, obj.get_refcnt());
assert_eq!(count, obj.get_refcnt(Python::acquire_gil().python()));
}
#[test]
@ -146,7 +146,7 @@ mod tests {
none = py.None();
l.append(10).unwrap();
l.append(&none).unwrap();
count = none.get_refcnt();
count = none.get_refcnt(py);
obj = l.to_object(py);
}
@ -158,7 +158,7 @@ mod tests {
assert_eq!(10, it.next().unwrap().unwrap().extract().unwrap());
assert!(it.next().unwrap().unwrap().is_none());
}
assert_eq!(count, none.get_refcnt());
assert_eq!(count, none.get_refcnt(py));
}
#[test]

12
src/types/list.rs
View File

@ -303,11 +303,11 @@ mod test {
let ob = v.to_object(py);
let list = <PyList as PyTryFrom>::try_from(ob.as_ref(py)).unwrap();
let none = py.None();
cnt = none.get_refcnt();
cnt = none.get_refcnt(py);
list.set_item(0, none).unwrap();
}
assert_eq!(cnt, py.None().get_refcnt());
assert_eq!(cnt, py.None().get_refcnt(py));
}
#[test]
@ -336,11 +336,11 @@ mod test {
let _pool = unsafe { crate::GILPool::new() };
let list = PyList::empty(py);
let none = py.None();
cnt = none.get_refcnt();
cnt = none.get_refcnt(py);
list.insert(0, none).unwrap();
}
assert_eq!(cnt, py.None().get_refcnt());
assert_eq!(cnt, py.None().get_refcnt(py));
}
#[test]
@ -365,10 +365,10 @@ mod test {
let _pool = unsafe { crate::GILPool::new() };
let list = PyList::empty(py);
let none = py.None();
cnt = none.get_refcnt();
cnt = none.get_refcnt(py);
list.append(none).unwrap();
}
assert_eq!(cnt, py.None().get_refcnt());
assert_eq!(cnt, py.None().get_refcnt(py));
}
#[test]

2
src/types/module.rs
View File

@ -176,7 +176,7 @@ impl PyModule {
where
T: PyClass,
{
self.add(T::NAME, <T as PyTypeObject>::type_object())
self.add(T::NAME, <T as PyTypeObject>::type_object(self.py()))
}
/// Adds a function or a (sub)module to a module, using the functions __name__ as name.

4
src/types/sequence.rs
View File

@ -521,8 +521,8 @@ mod test {
}
{
let gil = Python::acquire_gil();
let _py = gil.python();
assert_eq!(1, obj.get_refcnt());
let py = gil.python();
assert_eq!(1, obj.get_refcnt(py));
}
}

33
src/types/tuple.rs
View File

@ -2,8 +2,8 @@
use crate::ffi::{self, Py_ssize_t};
use crate::{
exceptions, AsPyPointer, AsPyRef, FromPy, FromPyObject, IntoPy, IntoPyPointer, Py, PyAny,
PyErr, PyNativeType, PyObject, PyResult, PyTryFrom, Python, ToPyObject,
exceptions, AsPyPointer, FromPy, FromPyObject, IntoPy, IntoPyPointer, Py, PyAny, PyErr,
PyNativeType, PyObject, PyResult, PyTryFrom, Python, ToPyObject,
};
use std::slice;
@ -52,16 +52,19 @@ impl PyTuple {
}
/// Takes a slice of the tuple pointed from `low` to `high` and returns it as a new tuple.
pub fn slice(&self, low: isize, high: isize) -> Py<PyTuple> {
unsafe { Py::from_owned_ptr_or_panic(ffi::PyTuple_GetSlice(self.as_ptr(), low, high)) }
pub fn slice(&self, low: isize, high: isize) -> &PyTuple {
unsafe {
self.py()
.from_owned_ptr(ffi::PyTuple_GetSlice(self.as_ptr(), low, high))
}
}
/// Takes a slice of the tuple from `low` to the end and returns it as a new tuple.
pub fn split_from(&self, low: isize) -> Py<PyTuple> {
pub fn split_from(&self, low: isize) -> &PyTuple {
unsafe {
let ptr =
ffi::PyTuple_GetSlice(self.as_ptr(), low, ffi::PyTuple_GET_SIZE(self.as_ptr()));
Py::from_owned_ptr_or_panic(ptr)
self.py().from_owned_ptr(ptr)
}
}
@ -77,20 +80,19 @@ impl PyTuple {
}
/// Returns `self` as a slice of objects.
pub fn as_slice(&self) -> &[PyObject] {
// This is safe because PyObject has the same memory layout as *mut ffi::PyObject,
pub fn as_slice(&self) -> &[&PyAny] {
// This is safe because &PyAny has the same memory layout as *mut ffi::PyObject,
// and because tuples are immutable.
unsafe {
let ptr = self.as_ptr() as *mut ffi::PyTupleObject;
let slice = slice::from_raw_parts((*ptr).ob_item.as_ptr(), self.len());
&*(slice as *const [*mut ffi::PyObject] as *const [PyObject])
&*(slice as *const [*mut ffi::PyObject] as *const [&PyAny])
}
}
/// Returns an iterator over the tuple items.
pub fn iter(&self) -> PyTupleIterator {
PyTupleIterator {
py: self.py(),
slice: self.as_slice(),
index: 0,
}
@ -99,8 +101,7 @@ impl PyTuple {
/// Used by `PyTuple::iter()`.
pub struct PyTupleIterator<'a> {
py: Python<'a>,
slice: &'a [PyObject],
slice: &'a [&'a PyAny],
index: usize,
}
@ -110,7 +111,7 @@ impl<'a> Iterator for PyTupleIterator<'a> {
#[inline]
fn next(&mut self) -> Option<&'a PyAny> {
if self.index < self.slice.len() {
let item = self.slice[self.index].as_ref(self.py);
let item = self.slice[self.index];
self.index += 1;
Some(item)
} else {
@ -130,7 +131,7 @@ impl<'a> IntoIterator for &'a PyTuple {
impl<'a> FromPy<&'a PyTuple> for Py<PyTuple> {
fn from_py(tuple: &'a PyTuple, _py: Python) -> Py<PyTuple> {
unsafe { Py::from_borrowed_ptr(tuple.as_ptr()) }
unsafe { Py::from_borrowed_ptr(tuple.py(), tuple.as_ptr()) }
}
}
@ -168,7 +169,7 @@ macro_rules! tuple_conversion ({$length:expr,$(($refN:ident, $n:tt, $T:ident)),+
unsafe {
let ptr = ffi::PyTuple_New($length);
$(ffi::PyTuple_SetItem(ptr, $n, self.$n.into_py(py).into_ptr());)+
Py::from_owned_ptr_or_panic(ptr)
Py::from_owned_ptr_or_panic(py, ptr)
}
}
}
@ -180,7 +181,7 @@ macro_rules! tuple_conversion ({$length:expr,$(($refN:ident, $n:tt, $T:ident)),+
let slice = t.as_slice();
if t.len() == $length {
Ok((
$(slice[$n].extract::<$T>(obj.py())?,)+
$(slice[$n].extract::<$T>()?,)+
))
} else {
Err(wrong_tuple_length(t, $length))

9
src/types/typeobject.rs
View File

@ -3,7 +3,7 @@
// based on Daniel Grunwald's https://github.com/dgrunwald/rust-cpython
use crate::err::{PyErr, PyResult};
use crate::instance::{Py, PyNativeType};
use crate::instance::PyNativeType;
use crate::type_object::PyTypeObject;
use crate::{ffi, AsPyPointer, PyAny, Python};
use std::borrow::Cow;
@ -18,8 +18,8 @@ pyobject_native_var_type!(PyType, ffi::PyType_Type, ffi::PyType_Check);
impl PyType {
/// Creates a new type object.
#[inline]
pub fn new<T: PyTypeObject>() -> Py<PyType> {
T::type_object()
pub fn new<T: PyTypeObject>(py: Python) -> &PyType {
T::type_object(py)
}
/// Retrieves the underlying FFI pointer associated with this Python object.
@ -48,7 +48,8 @@ impl PyType {
where
T: PyTypeObject,
{
let result = unsafe { ffi::PyObject_IsSubclass(self.as_ptr(), T::type_object().as_ptr()) };
let result =
unsafe { ffi::PyObject_IsSubclass(self.as_ptr(), T::type_object(self.py()).as_ptr()) };
if result == -1 {
Err(PyErr::fetch(self.py()))
} else if result == 1 {

2
tests/test_gc.rs
View File

@ -284,7 +284,7 @@ fn gc_during_borrow() {
}
// get the traverse function
let ty = TraversableClass::type_object().as_ref(py).as_type_ptr();
let ty = TraversableClass::type_object(py).as_type_ptr();
let traverse = (*ty).tp_traverse.unwrap();
// create an object and check that traversing it works normally