Commit graph

157 commits

Author SHA1 Message Date
David Hewitt 1d50a7cca8 cleanup: always enable #[track_caller] 2022-01-16 19:16:36 +00:00
Gregory Szorc c9c606f7c6 build: enable suppression of cargo:rustc-link-* lines
PyOxidizer requires advanced control over the settings used to link
libpython. We recently implemented support for configuration files
defining explicit lines to emit from build scripts to give callers
control over what lines to emit from build scripts so use cases
like PyOxidizer's are feasible without hacks in PyO3's code base.

However, the default logic in `emit_link_config()` may not be
appropriate in scenarios where link settings are provided via this
"extra lines" mechanism. The default logic may prohibit use of or
interfere with desired settings provided externally.

This commit defines a new field on the interpreter config that
suppresses the emission of the default link control logic from the
`pyo3` build script. It effectively gives advanced consumers like
PyOxidizer full control over link logic while minimally polluting
PyO3's build logic.

I thought about implementing this control as a crate feature. But
given the expected target audience size of ~1, I thought a crate
feature was too visible for a power user feature and decided to
implement it via the configuration file.
2021-08-30 10:01:36 -07:00
Gregory Szorc 04c77e35c5 build: support emitting arbitrary lines from pyo3 build script
PyOxidizer needs to do some... questionable things with regards to
configuring how the Python interpreter is linked. The way I solved this
problem for the `cpython` / `python3-sys` crates was by adding a bunch
of crate features to control what `cargo:` lines were emitted by the
build scripts. This added a lot of complexity to the those crates for
a target audience of ~1.

Now that PyO3 has support for config files to control settings, this
provides a richer mechanism than crate features to influence the build
script.

This commit defines a new field on the `InterpreterConfig` struct to
hold an arbitrary list of strings/lines that should be emitted by
the build script. This field is only every populated when parsing config
files and it is only read by pyo3's build script to `println!()`
additional values.

My intended use case for this is to have PyOxidizer effectively control
the interpreter link settings via the config file (at my own peril)
while having minimal impact on the maintainability of PyO3's code base.
Given the complexity of the link hacks employed, you probably don't want
this polluting pyo3's code base.
2021-08-30 09:52:11 -07:00
David Hewitt 3a90ff77f4 list,tuple,sequence: add slice indexing 2021-08-26 07:57:12 +01:00
David Hewitt 49387e9a70 pyo3-build-config: many tidy ups 2021-08-05 23:19:43 +01:00
David Hewitt 9507979d93
pyo3-build-config: inline config when not cross compiling 2021-08-04 12:54:45 +01:00
Tom Milligan 20b34a5528 create pyo3 config file directory if not exist 2021-08-04 09:02:10 +01:00
David Hewitt ed994ca44f pyo3-build-config: improve config file format 2021-08-04 09:02:10 +01:00
David Hewitt c2d6a50b35 build: revert usage of cargo:rustc-cdylib-link-arg 2021-08-03 20:04:56 +01:00
David Hewitt b5ca39c4f0 pypy: remove workaround for 7.3.4 2021-08-02 23:11:01 +01:00
David Hewitt 582d9c530f build: move print-config into environment variable 2021-08-01 17:26:27 +01:00
David Hewitt 4c5aee92d5 pycell: fix calculation of dictoffset on 32-bit Windows 2021-07-31 15:09:12 +01:00
David Hewitt ee9a1678d5 rust: updates for rust & clippy 1.54 2021-07-31 08:19:38 +01:00
David Hewitt 2fbe57d629 pyo3-build-config: improve error messaging 2021-07-14 22:13:58 +01:00
David Hewitt 8fb4ce6fbc pyo3-build-config: fix cross compilation 2021-06-26 14:19:03 +01:00
David Hewitt 6def8fe714 pyo3-build-config: check python version and architecture in PyO3 crate 2021-05-21 08:40:43 +01:00
David Hewitt 79c7149d52 pyo3-build-config: new crate to re-use build.rs across crates 2021-05-20 09:03:33 +01:00
Simon Rainerson b0f7145f9c Add support for cross compilation for wasm/wasi
When using rust after this commit
1a491e2304
the target family for wasm targets is now set to "wasm". Also fix the
assumption that OsStrExt lives inside std::os::unix for all non-windows
targets.
2021-05-14 14:38:28 +02:00
Caio 9bc5089e4b Add support for arbitrary arrays 2021-04-26 08:22:14 +01:00
David Hewitt fe9ed5c1fd build: refactor for simplicity 2021-04-24 11:19:58 +01:00
David Hewitt 2b9085abdb build: avoid rebuilds when using clippy in a virtualenv 2021-04-19 09:45:01 +01:00
David Hewitt d0f1600571 pypy: support PyPy 7.3.4 2021-04-18 09:20:05 +01:00
David Hewitt 355df5a37f auto-initialize: better error messages and embedding docs 2021-04-12 01:29:03 +01:00
James Hilliard e7b1ea48a8 Rerun if PYO3_CROSS env variable changes. 2021-04-09 02:28:20 -06:00
David Hewitt b213f06df4 macos: automatically provide required linker arguments 2021-04-02 14:18:28 +01:00
David Hewitt bc826d15de
Merge pull request #1530 from davidhewitt/build-error-messages
build: improve error message for multiple sysconfigs found
2021-03-30 08:01:10 +01:00
David Hewitt 2939bf1ed6
Merge pull request #1529 from ravenexp/fix-cargo-target-parsing
Clean up CARGO_CFG_TARGET_FAMILY variable parsing
2021-03-29 22:00:09 +01:00
Sergey Kvachonok ccaabe3ba5 Apply suggestions from code review: env::var() -> env::var_os()
and fall back to load_cross_compile_from_sysconfigdata() when unset.

Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2021-03-29 11:46:17 +03:00
David Hewitt a229ae48de build: rerun on a few more env vars changing 2021-03-29 08:46:09 +01:00
David Hewitt f89756238a build: improve error message for multiple sysconfigs found 2021-03-29 08:42:31 +01:00
Sergey Kvachonok f91f795fc7 Clean up CARGO_CFG_TARGET_FAMILY variable parsing
Match the target OS family strings exactly and provide better error messages
when CARGO_CFG_TARGET_FAMILY is not set (e.g. on wasm32).
2021-03-29 09:00:57 +03:00
Sergey Kvachonok 6d6d030bf7 Remove unused cross-compile environment variables
PYO3_CROSS_VERSION was renamed to PYO3_CROSS_PYTHON_VERSION.

PYO3_CROSS_INCLUDE_DIR was removed in
https://github.com/PyO3/pyo3/pull/1521
2021-03-29 08:48:54 +03:00
David Hewitt 8ad3827341
Merge pull request #1514 from jameshilliard/cross-env-detection
Ensure we are cross compiling when any cross env variables are set.
2021-03-28 12:46:59 +01:00
Sergey Kvachonok 80c747d2c0 Remove pyconfig.h header parsing
The config header parsing code was supposed to be only invoked when
cross-compiling for Windows, but in reality it fails to correctly parse
the config header files shipped with the upstream Python for Windows.

Given that there are now better options for reliable cross-compiling
for Windows such as `PYO3_CROSS_PYTHON_VERSION` or the `abi3-py3*` features,
it should be OK to remove this config for v0.14.

Update the cross-compilation instructions section of the user guide.

Fixes https://github.com/PyO3/pyo3/issues/1337
2021-03-26 09:57:18 +03:00
David Hewitt c4b7a44622
Merge pull request #1518 from ravenexp/main
Always use correct abi3 Python DLL name for MinGW
2021-03-25 18:17:19 +00:00
Sergey Kvachonok a0d1c5509d Always set Py_SHARED when building for Windows
Setting `PYO3_NO_PYTHON` results in an extension module built with
a different config feature set (on Windows).
Specifically, `Py_SHARED` config option gets omitted in this code path.

Maturin always sets `PYO3_NO_PYTHON` when cross-compiling abi3 extensions,
which creates a subtle configuration mismatch between `cargo build`
and `maturin build` artifacts.

Always set `Py_SHARED` when compiling abi3 extensions for Windows.
2021-03-25 11:51:15 +03:00
Sergey Kvachonok 2503e61c65 Always use correct abi3 Python DLL name for MinGW
Compiling an abi3 extension module for `x86_64-pc-windows-gnu` target
links to `python3.dll` import library when `PYO3_NO_PYTHON` is set
and to `python3.Y.dll` import library when `PYO3_NO_PYTHON` is not set.

All abi3 extensions should link to `python3.dll` on Windows,
as required by

https://www.python.org/dev/peps/pep-0384/#linkage

Update the code path for the case when `PYO3_NO_PYTHON` is not set
to yield the same Python DLL import library name.
2021-03-25 08:40:36 +03:00
David Hewitt 38c8355f91
build.rs: adjust pypy warning wording 2021-03-23 21:56:22 +00:00
James Hilliard 2c3e8b1c50 Ensure we are cross compiling when any cross env variables are set. 2021-03-23 05:13:52 -06:00
messense 9199c78e1b Update build.rs
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2021-03-12 16:56:14 +08:00
messense 884cd9d726 Apply suggestion from code review
Co-authored-by: kngwyu <yuji.kngw.80s.revive@gmail.com>
2021-03-12 16:56:14 +08:00
messense b9a2c64157 Use _PYTHON_SYSCONFIGDATA_NAME in find_sysconfigdata when cross-compiling 2021-03-12 16:56:14 +08:00
Yuji Kanagawa 6f8f958f34
Apply suggestions from code review
Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com>
2021-03-06 18:03:17 +09:00
kngwyu 4ec78e6b18 Warning when 'abi3' is used with PyPy 2021-03-06 12:52:32 +09:00
kngwyu 89f555f506 Refactor Python scripts in build.rs 2021-02-20 18:41:04 +09:00
messense ebc52c4aea Don't require PYO3_CROSS_LIB_DIR when compiling for x86_64 from macOS arm64 and reverse 2021-02-14 16:43:50 +08:00
Christoph Reiter e067b32521 Support building with Mingw Python
The DLL of the mingw Python in MSYS2 is named libpython3.8.dll:

$ python3 -m sysconfig | grep LIBPYTHON
        LIBPYTHON = "-lpython3.8"

Add another special case to in get_rustc_link_lib() to handle that case.
Afaik the mingw build doesn't support the limited ABI, so skipt that as well.

This makes all tests pass in an MSYS2 environment and lets us build
python-cryptography.
2021-02-12 12:31:20 +00:00
David Hewitt 59dac57689 build: fix support for non-utf8 systems 2021-01-26 23:16:47 +00:00
konstin c8b290a254 build: make abi3 linking work on windows 2021-01-10 10:13:49 +00:00
David Hewitt b152fd6317
Merge pull request #1359 from dalcde/patch-2
Let pyconfig.h not define Py_ENABLE_SHARED
2021-01-05 22:22:50 +00:00