Support compiling portable "abi3" extension modules even when
the build host Python interpreter configuration is not available
or the discovered Python interpreter version is not supported.
Maturin already implements this by building "abi3" extension wheels
with `PYO3_NO_PYTHON` environment veriable set for cargo when
an `abi3-py3*` feature is detected.
Closes#2292
Automatically generate `python3.dll` import libraries for Windows
compile targets in the build script.
Adds a new PyO3 crate feature `generate-abi3-import-lib` enabling
automatic import library generation.
Closes#2231
According to https://bugs.python.org/issue36707, this tag is useless
since version 3.4, but also the default until version 3.8.
For example, Debian 10 ships `libpython3.7m.so`.
This patch folds the `PYO3_NO_PYTHON` + `abi3` special case into
the existing native and cross compilation code paths.
The cross compilation route is now guaranteed to behave the same
whether `PYO3_NO_PYTHON` is set or not (except for sysconfigdata
discovery for the Unix targets).
The native compilation route now stores the hardcoded abi3 interpreter
configuration in place of the discovered configuration blob.
Adds a new cross-compile target interpreter configuration
environment variable.
This feature allows PyO3 to target PyPy on both Windows and Unix
cross compile targets.
Rename `$OUT_DIR/pyo3-cross-compile-config.txt` to
`$OUT_DIR/<triple>/pyo3-build-config.txt` to exclude the possibility
of using stale build configuration data when the build target changes.
Use the presence of the corresponding build configuration file
in the `pyo3-build-config` build script output directory
to detect whether we are cross compiling or not.
This patch enables cross compilation without using
any of `PYO3_CROSS_*` env variables in many cases.
Try to generalize `windows_hardcoded_cross_compile()`
to all supported target platforms (when possible).
Rename it to `default_cross_compile()` and add some unit tests.
Rewrite `load_cross_compile_config()` to fall back to
the default interpreter configuration when no other config
information sources are available.
Change the `CrossCompileConfig` structure definition and make
the public `lib_dir` field optional to support more flexible
cross-compilation configuration in the future.
FIXME: This change breaks the public `pyo3-build-config` crate API.
Update the sysconfigdata extraction functions to fall through
when `lib_dir` field is not set.
WIP: Add `unwrap()` stubs to the main cross compile switch.
Add a new public crate function `cross_compile_from_to()` using
`target_lexicon::Triple` arguments instead of plain strings
used in `cross_compile()`.
Deprecate `pyo3_build_config::cross_compile()` since v0.17.
Attempt to extract common code patterns into methods and standalone
helper functions. Add docstrings to the new private items.
Make some of the new helper functions public within the PyO3 crate
and reuse them in the build scripts.
Add PYO3_CROSS_PYTHON_VERSION parsing unit test.
Add a ChangeLog entry mentioning the new `pyo3-build-config` API.
* Initial refactor - expose cross-compiling functions and add necessary fields to InterpreterConfig
* Refactor cross_compiling to take arch/vendor/os separately.
* Address review comments.
* Update changelog with note about pyo3-build-config APIs.
* Fix panic when parsing ABI tag on Windows.
* Update parse_sysconfigdata test to best-guess values for linux.
* Revert added fields in InterpreterConfig.
* Refactor parse_sysconfigdata to return Sysconfigdata (HashMap). Add InterpreterConfig::from_sysconfigdata.
* Update BuildFlags test to use from_sysconfigdata.
* Add tests for from_sysconfigdata. Refactor Sysconfigdata API to be more open.
* Add basic tests for not cross compiling. Add some error handling.
* Address review comments.
* Update search_lib_dir to recurse into lib and pypy dirs.
* Look even harder for sysconfigdata.
* Add skip-build-config feature.
* Revert skip-build-config feature.
* Suppress cargo:rerun-if-env-changed without resolve-config feature.
PR #1856 was buggy in that the `pyo3-build-config` crate didn't actually
work in library mode because `include_str!()` was attempting to resolve
missing files as part of populating some `const` values.
We could change the logic of these constants to make them lazy if
we wanted to support possibly getting access to the value. But the
simple solution is to conditionalize their presence on the crate
feature.
Test coverage for building and testing the crate in insolation with the
feature disabled has been added.
Various code has been conditionalized to avoid compiler warnings.
Also, it appears `cargo build|test -p pyo3-build-config
--no-default-features` still passes default features. This seems wrong
to me. But it is how my system behaves. Maybe it is an sccache bug?
I coded the new tests to `cd pyo3-build-config` first to work around.
I have a use case in PyOxidizer where I want to use the
pyo3-build-config crate as a library crate so I can access the
`InterpreterConfig` struct so I can read/write config files without
reinventing the wheel.
This is doable before this commit. But it requires that the
build environment have a Python interpreter. This is undesirable
for library usage.
This commit introduces a cargo feature flag to control whether the
build script does anything. The feature flag must be present for
the build script to resolve a config. The feature flag is enabled
by default for backwards compatibility. The pyo3 and pyo3-macros-backend
crates use this feature by default, for backwards compatibility and
because it is the reasonable default.
This is probably room to conditionalize some APIs and other behavior
based on this feature flag. But we stop short of doing that for
the time being.
PyOxidizer requires advanced control over the settings used to link
libpython. We recently implemented support for configuration files
defining explicit lines to emit from build scripts to give callers
control over what lines to emit from build scripts so use cases
like PyOxidizer's are feasible without hacks in PyO3's code base.
However, the default logic in `emit_link_config()` may not be
appropriate in scenarios where link settings are provided via this
"extra lines" mechanism. The default logic may prohibit use of or
interfere with desired settings provided externally.
This commit defines a new field on the interpreter config that
suppresses the emission of the default link control logic from the
`pyo3` build script. It effectively gives advanced consumers like
PyOxidizer full control over link logic while minimally polluting
PyO3's build logic.
I thought about implementing this control as a crate feature. But
given the expected target audience size of ~1, I thought a crate
feature was too visible for a power user feature and decided to
implement it via the configuration file.
PyOxidizer needs to do some... questionable things with regards to
configuring how the Python interpreter is linked. The way I solved this
problem for the `cpython` / `python3-sys` crates was by adding a bunch
of crate features to control what `cargo:` lines were emitted by the
build scripts. This added a lot of complexity to the those crates for
a target audience of ~1.
Now that PyO3 has support for config files to control settings, this
provides a richer mechanism than crate features to influence the build
script.
This commit defines a new field on the `InterpreterConfig` struct to
hold an arbitrary list of strings/lines that should be emitted by
the build script. This field is only every populated when parsing config
files and it is only read by pyo3's build script to `println!()`
additional values.
My intended use case for this is to have PyOxidizer effectively control
the interpreter link settings via the config file (at my own peril)
while having minimal impact on the maintainability of PyO3's code base.
Given the complexity of the link hacks employed, you probably don't want
this polluting pyo3's code base.
PyOxidizer will want to create interpreter config files. Rather
than reinvent the logic for reading/writing these files, I think
it makes sense to consume the `pyo3-build-config` crate so we can
use the `InterpreterConfig` type directly. But the symbol needs
to be public to allow us to do that. And in order to allow direct
construction, we need to make all the types referenced by its fields
public as well.
If we got more then one file, only take those that contain the arch name.
For ubuntu 20.04 with host architecture x86_64 and a foreign architecture of armhf
this reduces the number of candidates to 1:
$ find /usr/lib/python3.8/ -name '_sysconfigdata*.py' -not -lname '*'
/usr/lib/python3.8/_sysconfigdata__x86_64-linux-gnu.py
/usr/lib/python3.8/_sysconfigdata__arm-linux-gnueabihf.py
CHANGELOG.md: add entry for cross-sysconfigdata filter on arch
commit changelog:
1. initial
2. if filtered list is empty, use pre filtered.
3. clippy is_empty and cloned
David Hewitt