Commit Graph

50 Commits

Author SHA1 Message Date
Ashley Anderson c30ca03b60
Expose cross compiling configuration from pyo3-build-config (#1996)
* Initial refactor - expose cross-compiling functions and add necessary fields to InterpreterConfig

* Refactor cross_compiling to take arch/vendor/os separately.

* Address review comments.

* Update changelog with note about pyo3-build-config APIs.

* Fix panic when parsing ABI tag on Windows.

* Update parse_sysconfigdata test to best-guess values for linux.

* Revert added fields in InterpreterConfig.

* Refactor parse_sysconfigdata to return Sysconfigdata (HashMap). Add InterpreterConfig::from_sysconfigdata.

* Update BuildFlags test to use from_sysconfigdata.

* Add tests for from_sysconfigdata. Refactor Sysconfigdata API to be more open.

* Add basic tests for not cross compiling. Add some error handling.

* Address review comments.

* Update search_lib_dir to recurse into lib and pypy dirs.

* Look even harder for sysconfigdata.

* Add skip-build-config feature.

* Revert skip-build-config feature.

* Suppress cargo:rerun-if-env-changed without resolve-config feature.
2021-12-16 00:18:37 +00:00
David Hewitt a5ef9ff64a ci: clippy whole workspace 2021-12-09 08:11:28 +00:00
Georg Brandl 9990bf9d31 Fix some clippy and dead code warnings. 2021-11-24 09:11:39 +01:00
David Hewitt 70030f130d python: drop support for 3.6 2021-11-20 13:02:42 +00:00
David Hewitt b9cc10f3fb pyo3-build-config: don't enable resolve-config by default 2021-11-19 14:19:24 +00:00
David Hewitt eb5059acc7 release: 0.15.1 2021-11-18 07:34:22 +00:00
messense f01595163c Don't link to python3.lib for PyPy on Windows 2021-11-15 15:40:29 +08:00
messense 73c3911748 Don't emit `Py_LIMITED_API` cfg for PyPy 2021-11-15 15:40:27 +08:00
messense 2325c283b0 Fix mingw platform detection 2021-11-15 15:15:16 +08:00
David Hewitt 47747444c7 release: 0.15.0 2021-11-03 23:00:38 +00:00
Giovanni Barillari c06058888f review sysconfigdata filename match rules to support cp37m 2021-10-12 23:41:26 +01:00
David Hewitt a5d0aa777c py310: add abi3-py310 feature 2021-09-26 16:21:35 +01:00
Sergey Kvachonok 2f74ab0d23 build-config: Always pass `abi3` flag to `default_lib_name_windows()`
Hardcoding `false` here leads to an incorrect abi3 DLL name
when cross-compiling to Windows.
2021-09-22 07:30:21 +01:00
David Hewitt 097d7e09cb pyo3-build-config: fix location for conda interpreter on windows 2021-09-16 23:38:46 +01:00
David Hewitt dca02a92ce changelog: updates for 0.14.5 backports 2021-09-05 15:19:36 +01:00
Gregory Szorc 3957afc9c5 pyo3-build-config: conditionalize symbols on resolve-config feature
PR #1856 was buggy in that the `pyo3-build-config` crate didn't actually
work in library mode because `include_str!()` was attempting to resolve
missing files as part of populating some `const` values.

We could change the logic of these constants to make them lazy if
we wanted to support possibly getting access to the value. But the
simple solution is to conditionalize their presence on the crate
feature.

Test coverage for building and testing the crate in insolation with the
feature disabled has been added.

Various code has been conditionalized to avoid compiler warnings.

Also, it appears `cargo build|test -p pyo3-build-config
--no-default-features` still passes default features. This seems wrong
to me. But it is how my system behaves. Maybe it is an sccache bug?
I coded the new tests to `cd pyo3-build-config` first to work around.
2021-09-02 17:18:55 -07:00
Gregory Szorc 1e951d5d8b pyo3-build-config: add a crate feature to control build script
I have a use case in PyOxidizer where I want to use the
pyo3-build-config crate as a library crate so I can access the
`InterpreterConfig` struct so I can read/write config files without
reinventing the wheel.

This is doable before this commit. But it requires that the
build environment have a Python interpreter. This is undesirable
for library usage.

This commit introduces a cargo feature flag to control whether the
build script does anything. The feature flag must be present for
the build script to resolve a config. The feature flag is enabled
by default for backwards compatibility. The pyo3 and pyo3-macros-backend
crates use this feature by default, for backwards compatibility and
because it is the reasonable default.

This is probably room to conditionalize some APIs and other behavior
based on this feature flag. But we stop short of doing that for
the time being.
2021-09-01 19:44:54 -07:00
David Hewitt 7d40461dcf
Merge pull request #1847 from davidhewitt/test-parse-sysconfigdata
pyo3-build-config: add test for parsing sysconfigdata
2021-09-01 08:50:29 +01:00
Gregory Szorc 05815bf861 pyo3-build-config: derive Clone on BuildFlags
This isn't strictly necessary since you can `.0.clone()`. But it is
more ergonomic.
2021-08-31 18:47:58 -07:00
David Hewitt 4a6ea17959 pyo3-build-config: add test for parsing sysconfigdata 2021-08-31 22:29:02 +01:00
David Hewitt 5e8d43c8bd
Merge pull request #1848 from indygreg/public-interpreter-config
pyo3-build-config: make InterpreterConfig public
2021-08-31 22:21:28 +01:00
Gregory Szorc c9c606f7c6 build: enable suppression of `cargo:rustc-link-*` lines
PyOxidizer requires advanced control over the settings used to link
libpython. We recently implemented support for configuration files
defining explicit lines to emit from build scripts to give callers
control over what lines to emit from build scripts so use cases
like PyOxidizer's are feasible without hacks in PyO3's code base.

However, the default logic in `emit_link_config()` may not be
appropriate in scenarios where link settings are provided via this
"extra lines" mechanism. The default logic may prohibit use of or
interfere with desired settings provided externally.

This commit defines a new field on the interpreter config that
suppresses the emission of the default link control logic from the
`pyo3` build script. It effectively gives advanced consumers like
PyOxidizer full control over link logic while minimally polluting
PyO3's build logic.

I thought about implementing this control as a crate feature. But
given the expected target audience size of ~1, I thought a crate
feature was too visible for a power user feature and decided to
implement it via the configuration file.
2021-08-30 10:01:36 -07:00
Gregory Szorc 04c77e35c5 build: support emitting arbitrary lines from pyo3 build script
PyOxidizer needs to do some... questionable things with regards to
configuring how the Python interpreter is linked. The way I solved this
problem for the `cpython` / `python3-sys` crates was by adding a bunch
of crate features to control what `cargo:` lines were emitted by the
build scripts. This added a lot of complexity to the those crates for
a target audience of ~1.

Now that PyO3 has support for config files to control settings, this
provides a richer mechanism than crate features to influence the build
script.

This commit defines a new field on the `InterpreterConfig` struct to
hold an arbitrary list of strings/lines that should be emitted by
the build script. This field is only every populated when parsing config
files and it is only read by pyo3's build script to `println!()`
additional values.

My intended use case for this is to have PyOxidizer effectively control
the interpreter link settings via the config file (at my own peril)
while having minimal impact on the maintainability of PyO3's code base.
Given the complexity of the link hacks employed, you probably don't want
this polluting pyo3's code base.
2021-08-30 09:52:11 -07:00
Gregory Szorc 62e134242d pyo3-build-config: make interpreter config and references types public
PyOxidizer will want to create interpreter config files. Rather
than reinvent the logic for reading/writing these files, I think
it makes sense to consume the `pyo3-build-config` crate so we can
use the `InterpreterConfig` type directly. But the symbol needs
to be public to allow us to do that. And in order to allow direct
construction, we need to make all the types referenced by its fields
public as well.
2021-08-30 09:45:37 -07:00
David Hewitt 5119bad580 changelog: updates for 0.14.4 2021-08-29 08:07:44 +01:00
David Hewitt bc1a983371 changelog: update for 0.14.3 release 2021-08-22 01:25:29 +01:00
Gregory Szorc 736b03b8cc build: document InterpreterConfig fields
I'm building functionality on top of this config and figured I'd
take the time to document the fields to make things easier to
understand.
2021-08-21 12:47:20 -07:00
David Hewitt 1873264b7d release: 0.14.2 2021-08-09 07:58:31 +01:00
Dean Li 1086fdb965
fix compiler warning
Fix issue [link](https://github.com/rust-lang/rust/issues/79813) which
will eventually deny trailing semicolons in expression macro in rust compiler.
2021-08-07 16:31:54 +08:00
David Hewitt 5af835daed pyo3-build-config: fix build for windows gnu targets 2021-08-07 00:25:14 +01:00
David Hewitt 2cf2c2fef9 pyo3-build-config: improve test coverage
[review] birkenfeld

Co-authored-by: Georg Brandl <georg@python.org>
2021-08-06 12:55:03 +01:00
David Hewitt 49387e9a70 pyo3-build-config: many tidy ups 2021-08-05 23:19:43 +01:00
David Hewitt 9507979d93
pyo3-build-config: inline config when not cross compiling 2021-08-04 12:54:45 +01:00
David Hewitt ed994ca44f pyo3-build-config: improve config file format 2021-08-04 09:02:10 +01:00
David Hewitt c2d6a50b35 build: revert usage of cargo:rustc-cdylib-link-arg 2021-08-03 20:04:56 +01:00
David Hewitt 582d9c530f build: move print-config into environment variable 2021-08-01 17:26:27 +01:00
David Hewitt ee9a1678d5 rust: updates for rust & clippy 1.54 2021-07-31 08:19:38 +01:00
David Hewitt 3794aab11f docs: use pyo3.rs/latest instead of pyo3.rs/main 2021-07-24 08:47:02 +01:00
David Hewitt 3ba58fedb1 pyo3-build-config: add print-config helper 2021-07-17 12:44:20 +01:00
David Hewitt 2fbe57d629 pyo3-build-config: improve error messaging 2021-07-14 22:13:58 +01:00
David Hewitt 3f8c4ad38e release: 0.14.1 2021-07-04 16:49:36 +01:00
David Hewitt adf6bdba8e release: 0.14.0 2021-07-03 08:37:00 +01:00
David Hewitt 5cc1e0f6dd pyo3-build-config: increased test coverage 2021-06-26 19:30:26 +01:00
David Hewitt 8fb4ce6fbc pyo3-build-config: fix cross compilation 2021-06-26 14:19:03 +01:00
messense ceb6dbf766 Detect Python implementation from `SOABI` on Unix 2021-06-09 09:03:23 +08:00
Alon Levy e5542f2214 pyo3-build-config: if found more than one candidate, filter on arch
If we got more then one file, only take those that contain the arch name.
For ubuntu 20.04 with host architecture x86_64 and a foreign architecture of armhf
this reduces the number of candidates to 1:

  $ find /usr/lib/python3.8/ -name '_sysconfigdata*.py' -not -lname '*'
  /usr/lib/python3.8/_sysconfigdata__x86_64-linux-gnu.py
  /usr/lib/python3.8/_sysconfigdata__arm-linux-gnueabihf.py

CHANGELOG.md: add entry for cross-sysconfigdata filter on arch

commit changelog:
1. initial
2. if filtered list is empty, use pre filtered.
3. clippy is_empty and cloned
2021-05-30 10:07:03 +03:00
David Hewitt 1e1605f0db pyo3-build-config: finish docs 2021-05-24 08:44:20 +01:00
David Hewitt 6def8fe714 pyo3-build-config: check python version and architecture in PyO3 crate 2021-05-21 08:40:43 +01:00
David Hewitt 825ec08681 pyo3-build-config: docs wip 2021-05-21 08:34:51 +01:00
David Hewitt 79c7149d52 pyo3-build-config: new crate to re-use build.rs across crates 2021-05-20 09:03:33 +01:00