Refactoring

This commit is contained in:
konstin 2018-07-30 22:52:22 +02:00
parent d9d1650fc4
commit 83db765889
33 changed files with 298 additions and 461 deletions

View File

@ -23,10 +23,7 @@ matrix:
env:
global:
- TRAVIS_RUST_VERSION=nightly
- RUSTFLAGS="-C link-dead-code"
- RUST_BACKTRACE=1
- SCCACHE_DIR="$HOME/.cargo/sccache"
- RUSTC_WRAPPER=sccache
addons:
apt:

View File

@ -1,14 +1,34 @@
# Changelog
All notable changes to this project will be documented in this file.
## 0.3.2
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Removed
* Conversions from tuples to PyDict.
### Changed
* Merged both examples into one
## [0.3.2] - 2018-07-22
### Changed
* Replaced `concat_idents` with mashup
## 0.3.1
## [0.3.1] - 2018-07-18
### Fixed
* Fixed scoping bug in pyobject_native_type that would break rust-numpy
## 0.3.0
## [0.3.0] - 2018-07-18
### Changed
* Upgraded to syn 0.14 which means much better error messages :tada:
* 128 bit integer support by [kngwyu](https://github.com/kngwyu) ([#137](https://github.com/PyO3/pyo3/pull/173))
@ -17,60 +37,109 @@
* Renamed the `base` option in the `pyclass` macro to `extends`.
* `#[pymodinit]` uses the function name as module name, unless the name is overrriden with `#[pymodinit(name)]`
* The guide is now properly versioned.
### Added
* A few internal macros became part of the public api ([#155](https://github.com/PyO3/pyo3/pull/155), [#186](https://github.com/PyO3/pyo3/pull/186))
* Always clone in getters. This allows using the get-annotation on all Clone-Types
## 0.2.7 (2018-05-18)
## [0.2.7] - 2018-05-18
### Fixed
* Fix nightly breakage with proc_macro_path
## 0.2.6 (2018-04-03)
## [0.2.6] - 2018-04-03
### Fixed
* Fix compatibility with TryFrom trait #137
## 0.2.5 (2018-02-21)
## [0.2.5] - 2018-02-21
### Added
* CPython 3.7 support
### Fixed
* Embedded CPython 3.7b1 crashes on initialization #110
* Generated extension functions are weakly typed #108
* call_method*() crashes when the method does not exist #113
* Allow importing exceptions from nested modules #116
## 0.2.4 (2018-01-19)
## [0.2.4] - 2018-01-19
### Added
* Allow to get mutable ref from PyObject #106
* Drop `RefFromPyObject` trait
* Add Python::register_any() method
### Fixed
* Fix impl `FromPyObject` for `Py<T>`
* Mark method that work with raw pointers as unsafe #95
## [0.2.3] - 11-27-2017
## 0.2.3 (11-27-2017)
### Fixed
* Proper `c_char` usage #93
* Remove use of now unneeded 'AsciiExt' trait
### Changed
* Rustup to 1.23.0-nightly 2017-11-07
## 0.2.2 (09-26-2017)
### Removed
* Remove use of now unneeded 'AsciiExt' trait
## [0.2.2] - 09-26-2017
### Changed
* Rustup to 1.22.0-nightly 2017-09-30
## 0.2.1 (09-26-2017)
## [0.2.1] - 09-26-2017
### Fixed
* Fix rustc const_fn nightly breakage
## 0.2.0 (08-12-2017)
## [0.2.0] - 08-12-2017
### Changed
* Allow to add gc support without implementing PyGCProtocol #57
* Refactor `PyErr` implementation. Drop `py` parameter from constructor.
### Added
* Added inheritance support #15
* Added weakref support #56
* Allow to add gc support without implementing PyGCProtocol #57
* Refactor `PyErr` implementation. Drop `py` parameter from constructor.
* Added subclass support #64
* Added `self.__dict__` supoort #68
* Added `pyo3::prelude` module #70
* Better `Iterator` support for PyTuple, PyList, PyDict #75
* Introduce IntoPyDictPointer similar to IntoPyTuple #69
## 0.1.0 (07-23-2017)
## [0.1.0] - 07-23-2017
### Added
* Initial release
[Unreleased]: https://github.com/olivierlacan/keep-a-changelog/compare/v0.3.2...HEAD
[0.3.2]: https://github.com/pyo3/pyo3/compare/v0.3.1...v0.3.2
[0.3.1]: https://github.com/pyo3/pyo3/compare/v0.3.0...v0.3.1
[0.3.0]: https://github.com/pyo3/pyo3/compare/v0.2.7...v0.3.0
[0.2.7]: https://github.com/pyo3/pyo3/compare/v0.2.6...v0.2.7
[0.2.6]: https://github.com/pyo3/pyo3/compare/v0.2.5...v0.2.6
[0.2.5]: https://github.com/pyo3/pyo3/compare/v0.2.4...v0.2.5
[0.2.4]: https://github.com/pyo3/pyo3/compare/v0.2.3...v0.2.4
[0.2.3]: https://github.com/pyo3/pyo3/compare/v0.2.2...v0.2.3
[0.2.2]: https://github.com/pyo3/pyo3/compare/v0.2.1...v0.2.2
[0.2.1]: https://github.com/pyo3/pyo3/compare/v0.2.0...v0.2.1
[0.2.0]: https://github.com/pyo3/pyo3/compare/v0.1.0...v0.2.0

145
README.md
View File

@ -1,8 +1,12 @@
# PyO3
[![Build Status](https://travis-ci.org/PyO3/pyo3.svg?branch=master)](https://travis-ci.org/PyO3/pyo3) [![Build Status](https://ci.appveyor.com/api/projects/status/github/PyO3/pyo3?branch=master&svg=true)](https://ci.appveyor.com/project/fafhrd91/pyo3) [![codecov](https://codecov.io/gh/PyO3/pyo3/branch/master/graph/badge.svg)](https://codecov.io/gh/PyO3/pyo3) [![crates.io](http://meritbadge.herokuapp.com/pyo3)](https://crates.io/crates/pyo3) [![Join the dev chat](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/PyO3/Lobby)
[![Build Status](https://travis-ci.org/PyO3/pyo3.svg?branch=master)](https://travis-ci.org/PyO3/pyo3)
[![Build Status](https://ci.appveyor.com/api/projects/status/github/PyO3/pyo3?branch=master&svg=true)](https://ci.appveyor.com/project/fafhrd91/pyo3)
[![codecov](https://codecov.io/gh/PyO3/pyo3/branch/master/graph/badge.svg)](https://codecov.io/gh/PyO3/pyo3)
[![crates.io](http://meritbadge.herokuapp.com/pyo3)](https://crates.io/crates/pyo3)
[![Join the dev chat](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/PyO3/Lobby)
[Rust](http://www.rust-lang.org/) bindings for the [Python](https://www.python.org/) interpreter. This includes running and interacting with python code from a rust binaries as well as writing native python modules.
[Rust](http://www.rust-lang.org/) bindings for [Python](https://www.python.org/). This includes running and interacting with python code from a rust binaries as well as writing native python modules.
* User Guide: [stable](https://pyo3.rs) | [master](https://pyo3.rs/master)
* [API Documentation](https://docs.rs/crate/pyo3/)
@ -11,11 +15,72 @@ A comparison with rust-cpython can be found [in the guide](https://pyo3.rs/maste
## Usage
Pyo3 supports python 2.7 as well as python 3.5 and up. The minimum required rust version is 1.27.0-nightly 2018-05-01.
Pyo3 supports python 2.7 as well as python 3.5 and up. The minimum required rust version is 1.29.0-nightly 2018-07-16.
### From a rust binary
You can either write a native python module in rust or use python from a rust binary.
To use `pyo3`, add this to your `Cargo.toml`:
### Using rust from python
Pyo3 can be used to generate a native python module.
**`Cargo.toml`:**
```toml
[package]
name = "rust-py"
version = "0.1.0"
[lib]
name = "rust_py"
crate-type = ["cdylib"]
[dependencies.pyo3]
version = "0.3"
features = ["extension-module"]
```
**`src/lib.rs`**
```rust
#![feature(use_extern_macros, specialization)]
#[macro_use]
extern crate pyo3;
use pyo3::prelude::*;
#[pyfunction]
/// Formats the sum of two numbers as string
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
/// This module is a python moudle implemented in Rust.
#[pymodinit]
fn rust_py(py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_function!(sum_as_string))?;
Ok(())
}
```
On windows and linux, you can build normally with `cargo build --release`. On Mac Os, you need to set additional linker arguments. One option is to compile with `cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup`, the other is to create a `.cargo/config` with the following content:
```toml
[target.x86_64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
```
Also on macOS, you will need to rename the output from \*.dylib to \*.so. On Windows, you will need to rename the output from \*.dll to \*.pyd.
[`setuptools-rust`](https://github.com/PyO3/setuptools-rust) can be used to generate a python package and includes the commands above by default. See [examples/word-count](examples/word-count) and the associated setup.py.
### Using python from rust
Add `pyo3` this to your `Cargo.toml`:
```toml
[dependencies]
@ -46,72 +111,12 @@ fn main() -> PyResult<()> {
}
```
### As native module
## Examples and tooling
Pyo3 can be used to generate a python-compatible library.
**`Cargo.toml`:**
```toml
[package]
name = "rust2py"
version = "0.1.0"
[lib]
name = "rust2py"
crate-type = ["cdylib"]
[dependencies.pyo3]
version = "0.3"
features = ["extension-module"]
```
**`src/lib.rs`**
```rust
#![feature(use_extern_macros, specialization)]
extern crate pyo3;
use pyo3::prelude::*;
// Add bindings to the generated python module
// N.B: names: "librust2py" must be the name of the `.so` or `.pyd` file
/// This module is implemented in Rust.
#[pymodinit]
fn rust2py(py: Python, m: &PyModule) -> PyResult<()> {
#[pyfn(m, "sum_as_string")]
// ``#[pyfn()]` converts the arguments from Python objects to Rust values
// and the Rust return value back into a Python object.
fn sum_as_string_py(a:i64, b:i64) -> PyResult<String> {
let out = sum_as_string(a, b);
Ok(out)
}
Ok(())
}
// The logic can be implemented as a normal rust function
fn sum_as_string(a:i64, b:i64) -> String {
format!("{}", a + b).to_string()
}
```
On windows and linux, you can build normally with `cargo build --release`. On Mac Os, you need to set additional linker arguments. One option is to compile with `cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup`, the other is to create a `.cargo/config` with the following content:
```toml
[target.x86_64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
```
Also on macOS, you will need to rename the output from \*.dylib to \*.so. On Windows, you will need to rename the output from \*.dll to \*.pyd.
[`setuptools-rust`](https://github.com/PyO3/setuptools-rust) can be used to generate a python package and includes the commands above by default. See [examples/word-count](examples/word-count) and the associated setup.py.
* [examples/word-count](examples/word-count) _Counting the occurences of a word in a text file_
* [hyperjson](https://github.com/mre/hyperjson) _A hyper-fast Python module for reading/writing JSON data using Rust's serde-json_
* [rust-numpy](https://github.com/rust-numpy/rust-numpy) _Rust binding of NumPy C-API_
* [pyo3-built](https://github.com/PyO3/pyo3-built) _Simple macro to expose metadata obtained with the [`built`](https://crates.io/crates/built) crate as a [`PyDict`](https://pyo3.github.io/pyo3/pyo3/struct.PyDict.html)_
## License

View File

@ -20,25 +20,6 @@ curl -SsL "https://sh.rustup.rs/" | sh -s -- -y --default-toolchain=$TRAVIS_RUST
export PATH=$PATH:$HOME/.cargo/bin
### Setup sccache ##############################################################
echo -n "Fetching latest available 'sccache' version... "
INSTALLED=$(_installed sccache)
LATEST=$(_latest sccache)
echo "${LATEST} (installed: ${INSTALLED})"
if [ "$INSTALLED" = "$LATEST" ]; then
echo "Using cached 'sccache'"
else
echo "Installing latest 'sccache' from mozilla/sccache"
URL="https://github.com/mozilla/sccache/releases/download/${LATEST}/sccache-${LATEST}-x86_64-unknown-linux-musl.tar.gz"
curl -SsL $URL | tar xzv -C /tmp
mv /tmp/sccache-${LATEST}-x86_64-unknown-linux-musl/sccache $HOME/.cargo/bin/sccache
fi
mkdir -p $SCCACHE_DIR
### Setup kcov #################################################################
if [ ! -f "$HOME/.cargo/bin/kcov" ]; then

View File

@ -1,15 +0,0 @@
[package]
authors = ["Messense Lv <messense@icloud.com>"]
name = "word-count-cls"
version = "0.1.0"
[dependencies]
rayon = "1.0"
[dependencies.pyo3]
path = "../../"
features = ["extension-module"]
[lib]
name = "word_count_cls"
crate-type = ["cdylib"]

View File

@ -1,15 +0,0 @@
# word-count
## Build
```bash
python setup.py install
```
## Usage
```python
from word_count_cls import WordCounter
WordCounter('path/to/file').search('word')
```

View File

@ -1,54 +0,0 @@
import sys
from setuptools import setup
from setuptools.command.test import test as TestCommand
try:
from setuptools_rust import RustExtension
except ImportError:
import subprocess
errno = subprocess.call([sys.executable, '-m', 'pip', 'install', 'setuptools-rust'])
if errno:
print("Please install setuptools-rust package")
raise SystemExit(errno)
else:
from setuptools_rust import RustExtension
class PyTest(TestCommand):
user_options = []
def run(self):
self.run_command("test_rust")
import subprocess
import sys
errno = subprocess.call([sys.executable, '-m', 'pytest', 'tests'])
raise SystemExit(errno)
setup_requires = ['setuptools-rust>=0.10.1']
install_requires = []
tests_require = install_requires + ['pytest', 'pytest-benchmark']
setup(
name='word-count-cls',
version='0.1.0',
classifiers=[
'License :: OSI Approved :: MIT License',
'Development Status :: 3 - Alpha',
'Intended Audience :: Developers',
'Programming Language :: Python',
'Programming Language :: Rust',
'Operating System :: POSIX',
'Operating System :: MacOS :: MacOS X',
],
packages=['word_count_cls'],
rust_extensions=[RustExtension('word_count_cls.word_count_cls', 'Cargo.toml')],
install_requires=install_requires,
tests_require=tests_require,
setup_requires=setup_requires,
include_package_data=True,
zip_safe=False,
cmdclass=dict(test=PyTest)
)

View File

@ -1,86 +0,0 @@
// Source adopted from
// https://github.com/tildeio/helix-website/blob/master/crates/word_count/src/lib.rs
#![feature(use_extern_macros, specialization)]
extern crate pyo3;
extern crate rayon;
use pyo3::prelude::*;
use rayon::prelude::*;
use std::fs::File;
use std::io::prelude::*;
#[pyclass(dict)]
struct WordCounter {
path: String,
token: PyToken,
}
#[pymethods]
impl WordCounter {
#[new]
fn __new__(obj: &PyRawObject, path: String) -> PyResult<()> {
obj.init(|t| WordCounter { path, token: t })
}
fn search(&self, py: Python, search: String) -> PyResult<i32> {
let mut file = File::open(self.path.as_str())?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
let count = py.allow_threads(move || wc_parallel(&contents, &search));
Ok(count)
}
fn search_sequential(&self, search: String) -> PyResult<i32> {
let mut file = File::open(self.path.as_str())?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
Ok(wc_sequential(&contents, &search))
}
}
fn matches(word: &str, search: &str) -> bool {
let mut search = search.chars();
for ch in word.chars().skip_while(|ch| !ch.is_alphabetic()) {
match search.next() {
None => {
return !ch.is_alphabetic();
}
Some(expect) => {
if ch.to_lowercase().next() != Some(expect) {
return false;
}
}
}
}
return search.next().is_none();
}
fn wc_line(line: &str, search: &str) -> i32 {
let mut total = 0;
for word in line.split(' ') {
if matches(word, search) {
total += 1;
}
}
total
}
fn wc_sequential(lines: &str, search: &str) -> i32 {
lines
.lines()
.map(|line| wc_line(line, search))
.fold(0, |sum, line| sum + line)
}
fn wc_parallel(lines: &str, search: &str) -> i32 {
lines.par_lines().map(|line| wc_line(line, search)).sum()
}
#[pymodinit]
fn word_count_cls(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<WordCounter>()?;
Ok(())
}

View File

@ -1,55 +0,0 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import
import os
import pytest
import word_count_cls
current_dir = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(current_dir, 'zen-of-python.txt')
@pytest.fixture(scope='session', autouse=True)
def textfile():
text = '''The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!\n''' * 1000
with open(path, 'w') as f:
f.write(text)
yield
os.remove(path)
def test_word_count_rust_parallel(benchmark):
count = benchmark(word_count_cls.WordCounter(path).search, 'is')
assert count == 10000
def test_word_count_rust_sequential(benchmark):
count = benchmark(word_count_cls.WordCounter(path).search_sequential, 'is')
assert count == 10000
def test_word_count_python_sequential(benchmark):
count = benchmark(word_count_cls.search_py, path, 'is')
assert count == 10000

View File

@ -1,17 +0,0 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import
from .word_count_cls import WordCounter
__all__ = ['WordCounter', 'search_py']
def search_py(path, needle):
total = 0
with open(path, 'r') as f:
for line in f:
words = line.split(' ')
for word in words:
if word == needle:
total += 1
return total

View File

@ -5,9 +5,7 @@ version = "0.1.0"
[dependencies]
rayon = "1.0"
[dependencies.pyo3]
path = "../../"
pyo3 = { path = "../..", features = ["extension-module"] }
[lib]
name = "word_count"

View File

@ -1,15 +1,27 @@
# word-count
Demonstrates searching for a file in plain python, with rust singlethreaded and with rust multithreaded.
## Build
```bash
```shell
python setup.py install
```
## Usage
```python
from word_count import search
from word_count import search_py, WordCounter
search('path/to/file', 'word')
search_py("path/to/file", "word")
WordCounter("path/to/file").search("word")
WordCounter("path/to/file").search_sequential("word")
```
## Benchmark
There is a benchmark in `tests/test_word_count.py`:
```shell
pytest -v tests
```

View File

@ -7,7 +7,8 @@ try:
from setuptools_rust import RustExtension
except ImportError:
import subprocess
errno = subprocess.call([sys.executable, '-m', 'pip', 'install', 'setuptools-rust'])
errno = subprocess.call([sys.executable, "-m", "pip", "install", "setuptools-rust"])
if errno:
print("Please install setuptools-rust package")
raise SystemExit(errno)
@ -23,32 +24,33 @@ class PyTest(TestCommand):
import subprocess
import sys
errno = subprocess.call([sys.executable, '-m', 'pytest', 'tests'])
errno = subprocess.call([sys.executable, "-m", "pytest", "tests"])
raise SystemExit(errno)
setup_requires = ['setuptools-rust>=0.10.1']
setup_requires = ["setuptools-rust>=0.10.1", "wheel"]
install_requires = []
tests_require = install_requires + ['pytest', 'pytest-benchmark']
tests_require = install_requires + ["pytest", "pytest-benchmark"]
setup(
name='word-count',
version='0.1.0',
name="word-count",
version="0.1.0",
classifiers=[
'License :: OSI Approved :: MIT License',
'Development Status :: 3 - Alpha',
'Intended Audience :: Developers',
'Programming Language :: Python',
'Programming Language :: Rust',
'Operating System :: POSIX',
'Operating System :: MacOS :: MacOS X',
"License :: OSI Approved :: MIT License",
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python",
"Programming Language :: Rust",
"Operating System :: POSIX",
"Operating System :: MacOS :: MacOS X",
],
packages=['word_count'],
rust_extensions=[RustExtension('word_count.word_count', 'Cargo.toml')],
packages=["word_count"],
rust_extensions=[RustExtension("word_count.word_count", "Cargo.toml")],
install_requires=install_requires,
tests_require=tests_require,
setup_requires=setup_requires,
include_package_data=True,
zip_safe=False,
cmdclass=dict(test=PyTest)
cmdclass=dict(test=PyTest),
)

View File

@ -1,19 +1,59 @@
// Source adopted from
// https://github.com/tildeio/helix-website/blob/master/crates/word_count/src/lib.rs
#![feature(use_extern_macros, specialization)]
#[macro_use]
extern crate pyo3;
extern crate rayon;
use std::fs::File;
use std::io::prelude::*;
use pyo3::prelude::*;
use rayon::prelude::*;
use std::fs;
use std::path::PathBuf;
fn matches(word: &str, search: &str) -> bool {
let mut search = search.chars();
#[pyclass]
/// Represents a file that can be searched
struct WordCounter {
path: PathBuf,
}
#[pymethods]
impl WordCounter {
#[new]
fn __new__(obj: &PyRawObject, path: String) -> PyResult<()> {
obj.init(|_| WordCounter { path: PathBuf::from(path) })
}
/// Searches for the word, parallelized by rayon
fn search(&self, py: Python, search: String) -> PyResult<usize> {
let contents = fs::read_to_string(&self.path)?;
let count = py.allow_threads(move || {
contents
.par_lines()
.map(|line| count_line(line, &search))
.sum()
});
Ok(count)
}
/// Searches for a word in a classic sequential fashion
fn search_sequential(&self, needle: String) -> PyResult<usize> {
let contents = fs::read_to_string(&self.path)?;
let result = contents
.lines()
.map(|line| count_line(line, &needle))
.sum();
Ok(result)
}
}
fn matches(word: &str, needle: &str) -> bool {
let mut needle = needle.chars();
for ch in word.chars().skip_while(|ch| !ch.is_alphabetic()) {
match search.next() {
match needle.next() {
None => {
return !ch.is_alphabetic();
}
@ -24,49 +64,25 @@ fn matches(word: &str, search: &str) -> bool {
}
}
}
return search.next().is_none();
return needle.next().is_none();
}
fn wc_line(line: &str, search: &str) -> i32 {
#[pyfunction]
/// Count the occurences of needle in line, case insensitive
fn count_line(line: &str, needle: &str) -> usize {
let mut total = 0;
for word in line.split(' ') {
if matches(word, search) {
if matches(word, needle) {
total += 1;
}
}
total
}
fn wc_sequential(lines: &str, search: &str) -> i32 {
lines
.lines()
.map(|line| wc_line(line, search))
.fold(0, |sum, line| sum + line)
}
fn wc_parallel(lines: &str, search: &str) -> i32 {
lines.par_lines().map(|line| wc_line(line, search)).sum()
}
#[pymodinit]
fn word_count(_py: Python, m: &PyModule) -> PyResult<()> {
#[pyfn(m, "search")]
fn search(py: Python, path: String, search: String) -> PyResult<i32> {
let mut file = File::open(path)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
let count = py.allow_threads(move || wc_parallel(&contents, &search));
Ok(count)
}
#[pyfn(m, "search_sequential")]
fn search_sequential(path: String, search: String) -> PyResult<i32> {
let mut file = File::open(path)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
Ok(wc_sequential(&contents, &search))
}
m.add_function(wrap_function!(count_line))?;
m.add_class::<WordCounter>()?;
Ok(())
}

View File

@ -4,16 +4,16 @@ from __future__ import absolute_import
import os
import pytest
import word_count
current_dir = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(current_dir, 'zen-of-python.txt')
path = os.path.join(current_dir, "zen-of-python.txt")
@pytest.fixture(scope='session', autouse=True)
@pytest.fixture(scope="session", autouse=True)
def textfile():
text = '''The Zen of Python, by Tim Peters
text = """
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
@ -33,23 +33,25 @@ Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!\n''' * 1000
with open(path, 'w') as f:
f.write(text)
Namespaces are one honking great idea -- let's do more of those!
"""
with open(path, "w") as f:
f.write(text * 1000)
yield
os.remove(path)
def test_word_count_rust_parallel(benchmark):
count = benchmark(word_count.search, path, 'is')
count = benchmark(word_count.WordCounter(path).search, "is")
assert count == 10000
def test_word_count_rust_sequential(benchmark):
count = benchmark(word_count.search_sequential, path, 'is')
count = benchmark(word_count.WordCounter(path).search_sequential, "is")
assert count == 10000
def test_word_count_python_sequential(benchmark):
count = benchmark(word_count.search_py, path, 'is')
count = benchmark(word_count.search_py, path, "is")
assert count == 10000

View File

@ -1,16 +1,16 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import
from .word_count import search, search_sequential
from .word_count import WordCounter, count_line
__all__ = ['search', 'search_sequential', 'search_py']
__all__ = ["WordCounter", "count_line", "search_py"]
def search_py(path, needle):
total = 0
with open(path, 'r') as f:
with open(path, "r") as f:
for line in f:
words = line.split(' ')
words = line.split(" ")
for word in words:
if word == needle:
total += 1

View File

@ -180,7 +180,7 @@ mod test {
use args::{parse_arguments, Argument};
use syn;
use proc_macro::TokenStream;
use proc_macro2::TokenStream;
fn items(s: TokenStream) -> Vec<syn::NestedMeta> {
let dummy: syn::ItemFn = parse_quote!{#s fn dummy() {}};

View File

@ -139,7 +139,7 @@ impl <T> CallbackConverter<T> for HashConverter
}
#[inline]
#[cfg_attr(feature = "cargo-clippy", allow(needless_pass_by_value))]
pub unsafe fn cb_convert<C, T>(_c: C, py: Python, value: PyResult<T>) -> C::R
where C: CallbackConverter<T>
{

View File

@ -48,7 +48,7 @@ impl<T> PyBufferProtocolImpl for T {
impl<'p, T> PyBufferProtocolImpl for T where T: PyBufferProtocol<'p>
{
#[inline]
#[cfg_attr(feature = "cargo-clippy", allow(needless_update))]
fn tp_as_buffer() -> Option<ffi::PyBufferProcs> {
Some(ffi::PyBufferProcs{
bf_getbuffer: Self::cb_bf_getbuffer(),

View File

@ -23,7 +23,7 @@ impl PyToken {
}
#[inline(always)]
#[cfg_attr(feature = "cargo-clippy", allow(inline_always))]
pub fn py(&self) -> Python {
unsafe { Python::assume_gil_acquired() }
}
@ -45,7 +45,7 @@ pub trait AsPyRef<T>: Sized {
fn as_ref(&self, py: Python) -> &T;
/// Return mutable reference to object.
#[cfg_attr(feature = "cargo-clippy", allow(mut_from_ref))]
fn as_mut(&self, py: Python) -> &mut T;
/// Acquire python gil and call closure with object reference.

View File

@ -16,7 +16,7 @@ use typeob::PyTypeInfo;
/// Python object model helper methods
#[cfg_attr(feature = "cargo-clippy", allow(len_without_is_empty))]
pub trait ObjectProtocol {
/// Determines whether this object has the given attribute.
@ -152,7 +152,7 @@ pub trait ObjectProtocol {
fn get_base(&self) -> &<Self as PyTypeInfo>::BaseType where Self: PyTypeInfo;
/// Gets the Python base object for this object.
#[cfg_attr(feature = "cargo-clippy", allow(mut_from_ref))]
fn get_mut_base(&self) -> &mut <Self as PyTypeInfo>::BaseType where Self: PyTypeInfo;
/// Casts the PyObject to a concrete Python object type.

View File

@ -54,7 +54,7 @@ impl PyByteArray {
}
/// Gets the Python bytearray data as byte slice.
#[cfg_attr(feature = "cargo-clippy", allow(mut_from_ref))]
pub fn data(&self) -> &mut [u8] {
unsafe {
let buffer = ffi::PyByteArray_AsString(self.0.as_ptr()) as *mut u8;

View File

@ -29,7 +29,7 @@ macro_rules! exc_type(
}
}
impl $name {
#[cfg_attr(feature = "cargo-clippy", allow(new_ret_no_self))]
pub fn new<V: ToPyObject + 'static>(args: V) -> PyErr {
PyErr::new::<$name, V>(args)
}
@ -133,7 +133,7 @@ exc_type!(WindowsError, PyExc_WindowsError);
impl UnicodeDecodeError {
#[cfg_attr(feature = "cargo-clippy", allow(needless_pass_by_value))]
pub fn new_err<'p>(py: Python<'p>, encoding: &CStr, input: &[u8],
range: ops::Range<usize>, reason: &CStr) -> PyResult<&'p PyObjectRef> {
unsafe {

View File

@ -57,7 +57,7 @@ macro_rules! import_exception {
}
impl $name {
#[cfg_attr(feature = "cargo-clippy", allow(new_ret_no_self))]
pub fn new<T: $crate::ToPyObject + 'static>(args: T) -> $crate::PyErr
where Self: $crate::typeob::PyTypeObject + Sized
{

View File

@ -51,7 +51,7 @@ impl IntoPyObject for f64 {
pyobject_extract!(obj to f64 => {
let v = unsafe { ffi::PyFloat_AsDouble(obj.as_ptr()) };
#[cfg_attr(feature = "cargo-clippy", allow(float_cmp))]
{
if v == -1.0 && PyErr::occurred(obj.py()) {
Err(PyErr::fetch(obj.py()))
@ -62,13 +62,13 @@ pyobject_extract!(obj to f64 => {
});
impl ToPyObject for f32 {
#[cfg_attr(feature = "cargo-clippy", allow(cast_lossless))]
fn to_object(&self, py: Python) -> PyObject {
PyFloat::new(py, *self as f64).into()
}
}
impl IntoPyObject for f32 {
#[cfg_attr(feature = "cargo-clippy", allow(cast_lossless))]
fn into_object(self, py: Python) -> PyObject {
PyFloat::new(py, self as f64).into()
}

View File

@ -39,7 +39,6 @@ macro_rules! pyobject_downcast(
impl<'a, $($type_param,)*> $crate::FromPyObject<'a> for &'a $name
{
/// Extracts `Self` from the source `PyObject`.
#[cfg_attr(feature = "cargo-clippy", allow(useless_transmute))]
fn extract(ob: &'a $crate::PyObjectRef) -> $crate::PyResult<Self>
{
unsafe {
@ -60,7 +59,6 @@ macro_rules! pyobject_native_type_named(
impl<$($type_param,)*> $crate::PyNativeType for $name {}
impl<$($type_param,)*> ::std::convert::AsRef<$crate::PyObjectRef> for $name {
#[cfg_attr(feature = "cargo-clippy", allow(useless_transmute))]
fn as_ref(&self) -> &$crate::PyObjectRef {
unsafe{&*(self as *const $name as *const $crate::PyObjectRef)}
}
@ -121,7 +119,7 @@ macro_rules! pyobject_native_type_convert(
&mut $typeobject
}
#[cfg_attr(feature = "cargo-clippy", allow(not_unsafe_ptr_arg_deref))]
fn is_instance(ptr: *mut $crate::ffi::PyObject) -> bool {
#[allow(unused_unsafe)]
unsafe { $checkfunction(ptr) > 0 }

View File

@ -29,7 +29,7 @@ pyobject_native_type!(PyLong, ffi::PyLong_Type, ffi::PyLong_Check);
macro_rules! int_fits_c_long(
($rust_type:ty) => (
impl ToPyObject for $rust_type {
#[cfg_attr(feature = "cargo-clippy", allow(cast_lossless))]
fn to_object(&self, py: Python) -> PyObject {
unsafe {
PyObject::from_owned_ptr_or_panic(py, ffi::PyLong_FromLong(*self as c_long))
@ -37,7 +37,7 @@ macro_rules! int_fits_c_long(
}
}
impl IntoPyObject for $rust_type {
#[cfg_attr(feature = "cargo-clippy", allow(cast_lossless))]
fn into_object(self, py: Python) -> PyObject {
unsafe {
PyObject::from_owned_ptr_or_panic(py, ffi::PyLong_FromLong(self as c_long))

View File

@ -4,7 +4,7 @@ use std::os::raw::c_int;
use python::Python;
use err::{PyErr, PyResult};
#[cfg_attr(feature = "cargo-clippy", allow(needless_pass_by_value))]
pub(super) fn err_if_invalid_value<T: PartialEq>
(py: Python, invalid_value: T, actual_value: T) -> PyResult<T>
{

View File

@ -20,7 +20,7 @@ pyobject_native_type_named!(PySequence);
pyobject_downcast!(PySequence, ffi::PySequence_Check);
#[cfg_attr(feature = "cargo-clippy", allow(len_without_is_empty))]
impl PySequence {
/// Returns the number of objects in sequence. This is equivalent to Python `len()`.
#[inline]

View File

@ -30,7 +30,3 @@ pub use pyo3cls::mod3init as pymodinit;
#[cfg(not(Py_3))]
pub use pyo3cls::mod2init as pymodinit;
// Until the extern macro story is fully fleshed out, this will make wrap_function! work
//#[doc(hidden)]
//pub use mashup::*;

View File

@ -275,7 +275,7 @@ impl<'p> Python<'p> {
}
/// Register `ffi::PyObject` pointer in release pool
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_borrowed_ptr_to_obj(self, ptr: *mut ffi::PyObject) -> &'p PyObjectRef
{
if ptr.is_null() {
@ -287,7 +287,7 @@ impl<'p> Python<'p> {
/// Register `ffi::PyObject` pointer in release pool,
/// and do unchecked downcast to specific type.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_owned_ptr<T>(self, ptr: *mut ffi::PyObject) -> &'p T
where T: PyTypeInfo
{
@ -315,7 +315,7 @@ impl<'p> Python<'p> {
/// Register owned `ffi::PyObject` pointer in release pool.
/// Returns `Err(PyErr)` if the pointer is `null`.
/// do unchecked downcast to specific type.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_owned_ptr_or_err<T>(self, ptr: *mut ffi::PyObject) -> PyResult<&'p T>
where T: PyTypeInfo
{
@ -330,7 +330,7 @@ impl<'p> Python<'p> {
/// Register owned `ffi::PyObject` pointer in release pool.
/// Returns `None` if the pointer is `null`.
/// do unchecked downcast to specific type.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_owned_ptr_or_opt<T>(self, ptr: *mut ffi::PyObject) -> Option<&'p T>
where T: PyTypeInfo
{
@ -345,7 +345,7 @@ impl<'p> Python<'p> {
/// Register borrowed `ffi::PyObject` pointer in release pool.
/// Panics if the pointer is `null`.
/// do unchecked downcast to specific type.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_borrowed_ptr<T>(self, ptr: *mut ffi::PyObject) -> &'p T
where T: PyTypeInfo
{
@ -370,7 +370,7 @@ impl<'p> Python<'p> {
/// Register borrowed `ffi::PyObject` pointer in release pool.
/// Returns `Err(PyErr)` if the pointer is `null`.
/// do unchecked downcast to specific type.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_borrowed_ptr_or_err<T>(self, ptr: *mut ffi::PyObject) -> PyResult<&'p T>
where T: PyTypeInfo
{
@ -385,7 +385,7 @@ impl<'p> Python<'p> {
/// Register borrowed `ffi::PyObject` pointer in release pool.
/// Returns `None` if the pointer is `null`.
/// do unchecked downcast to specific `T`.
#[cfg_attr(feature = "cargo-clippy", allow(wrong_self_convention))]
pub unsafe fn from_borrowed_ptr_or_opt<T>(self, ptr: *mut ffi::PyObject) -> Option<&'p T>
where T: PyTypeInfo
{
@ -418,7 +418,7 @@ impl<'p> Python<'p> {
/// Release `ffi::PyObject` pointer.
/// Undefined behavior if the pointer is invalid.
#[inline]
#[cfg_attr(feature = "cargo-clippy", allow(not_unsafe_ptr_arg_deref))]
pub fn xdecref(self, ptr: *mut ffi::PyObject) {
if !ptr.is_null() {
unsafe {ffi::Py_DECREF(ptr)};

View File

@ -42,13 +42,13 @@ pub trait PyTypeInfo {
unsafe fn type_object() -> &'static mut ffi::PyTypeObject;
/// Check if `*mut ffi::PyObject` is instance of this type
#[cfg_attr(feature = "cargo-clippy", allow(not_unsafe_ptr_arg_deref))]
fn is_instance(ptr: *mut ffi::PyObject) -> bool {
unsafe {ffi::PyObject_TypeCheck(ptr, Self::type_object()) != 0}
}
/// Check if `*mut ffi::PyObject` is exact instance of this type
#[cfg_attr(feature = "cargo-clippy", allow(not_unsafe_ptr_arg_deref))]
fn is_exact_instance(ptr: *mut ffi::PyObject) -> bool {
unsafe {
(*ptr).ob_type == Self::type_object()
@ -186,7 +186,7 @@ impl PyRawObject {
}
/// Return reference to object.
#[cfg_attr(feature = "cargo-clippy", allow(should_implement_trait))]
pub fn as_ref<T: PyTypeInfo>(&self) -> &T {
// TODO: check is object initialized
unsafe {
@ -204,7 +204,7 @@ impl IntoPyPointer for PyRawObject {
}
impl PyObjectWithToken for PyRawObject {
#[cfg_attr(feature = "cargo-clippy", allow(inline_always))]
#[inline(always)]
fn py(&self) -> Python {
unsafe { Python::assume_gil_acquired() }
@ -338,7 +338,7 @@ impl<T> PyTypeObject for T where T: PyObjectAlloc<T> + PyTypeInfo {
let py = gil.python();
initialize_type::<T>(py, None).expect(
format!("An error occurred while initializing class {}", T::NAME).as_ref());
&format!("An error occurred while initializing class {}", T::NAME));
}
}
}
@ -528,7 +528,7 @@ fn py_class_flags<T: PyTypeInfo>(type_object: &mut ffi::PyTypeObject) {
}
}
#[cfg_attr(feature = "cargo-clippy", allow(type_complexity))]
fn py_class_method_defs<T>() -> PyResult<(Option<ffi::newfunc>,
Option<ffi::initproc>,
Option<ffi::PyCFunctionWithKeywords>,

View File

@ -13,6 +13,7 @@ fn sum_as_string(a: i64, b: i64) -> String {
}
#[pyfunction]
/// Doubles the given value
fn double(x: usize) -> usize {
x * 2
}
@ -60,7 +61,9 @@ fn test_module_with_functions() {
run("assert module_with_functions.foo == 'bar'");
run("assert module_with_functions.EmptyClass != None");
run("assert module_with_functions.double(3) == 6");
run("assert module_with_functions.double.__doc__ == 'Doubles the given value'");
run("assert module_with_functions.also_double(3) == 6");
run("assert module_with_functions.also_double.__doc__ == 'Doubles the given value'");
}
#[pymodinit(other_name)]