I have a use case in PyOxidizer where I want to use the
pyo3-build-config crate as a library crate so I can access the
`InterpreterConfig` struct so I can read/write config files without
reinventing the wheel.
This is doable before this commit. But it requires that the
build environment have a Python interpreter. This is undesirable
for library usage.
This commit introduces a cargo feature flag to control whether the
build script does anything. The feature flag must be present for
the build script to resolve a config. The feature flag is enabled
by default for backwards compatibility. The pyo3 and pyo3-macros-backend
crates use this feature by default, for backwards compatibility and
because it is the reasonable default.
This is probably room to conditionalize some APIs and other behavior
based on this feature flag. But we stop short of doing that for
the time being.
PyOxidizer requires advanced control over the settings used to link
libpython. We recently implemented support for configuration files
defining explicit lines to emit from build scripts to give callers
control over what lines to emit from build scripts so use cases
like PyOxidizer's are feasible without hacks in PyO3's code base.
However, the default logic in `emit_link_config()` may not be
appropriate in scenarios where link settings are provided via this
"extra lines" mechanism. The default logic may prohibit use of or
interfere with desired settings provided externally.
This commit defines a new field on the interpreter config that
suppresses the emission of the default link control logic from the
`pyo3` build script. It effectively gives advanced consumers like
PyOxidizer full control over link logic while minimally polluting
PyO3's build logic.
I thought about implementing this control as a crate feature. But
given the expected target audience size of ~1, I thought a crate
feature was too visible for a power user feature and decided to
implement it via the configuration file.
PyOxidizer needs to do some... questionable things with regards to
configuring how the Python interpreter is linked. The way I solved this
problem for the `cpython` / `python3-sys` crates was by adding a bunch
of crate features to control what `cargo:` lines were emitted by the
build scripts. This added a lot of complexity to the those crates for
a target audience of ~1.
Now that PyO3 has support for config files to control settings, this
provides a richer mechanism than crate features to influence the build
script.
This commit defines a new field on the `InterpreterConfig` struct to
hold an arbitrary list of strings/lines that should be emitted by
the build script. This field is only every populated when parsing config
files and it is only read by pyo3's build script to `println!()`
additional values.
My intended use case for this is to have PyOxidizer effectively control
the interpreter link settings via the config file (at my own peril)
while having minimal impact on the maintainability of PyO3's code base.
Given the complexity of the link hacks employed, you probably don't want
this polluting pyo3's code base.
PyOxidizer has crates depending on `pyo3` that would like to access
the `pyo3` crate configuration. (This use case isn't unique to
PyOxidizer.)
Cargo has a facility for enabling the build scripts of dependent
crates to access _exported_ variables via `DEP_` environment
variables. However, this only works if the exporting crate defines a
`links` key in its Cargo manifest. See
https://doc.rust-lang.org/cargo/reference/build-scripts.html#the-links-manifest-key.
While `pyo3`'s build script doesn't yet export the variables that
PyOxidizer will need, a prerequisite to making this work is adding
the `links` key. Since this change could introduce unintended
side-effects, it warrants being made in its own commit, which is
why we're making this change outside of #1793.
I _think_ this change should be mostly safe: the `links` key is
effectively metadata advertising that a crate links against a named
library. The only side-effects setting it has is to enable the
aforementioned `DEP_` environment variables in build scripts and
enforcing a limitation that only a single crate may link against the
same native library. I believe the only potential for this change
to cause problems is if there are multiple crates with `links =
"python"` entries. I'm not aware of any other crates that advertise
`links = "python"`: even `python3-sys` / `cpython` use `links =
"python3"` so this change should not prevent dual use of `pyo3` and
`cpython` in the same build.
PyOxidizer will want to create interpreter config files. Rather
than reinvent the logic for reading/writing these files, I think
it makes sense to consume the `pyo3-build-config` crate so we can
use the `InterpreterConfig` type directly. But the symbol needs
to be public to allow us to do that. And in order to allow direct
construction, we need to make all the types referenced by its fields
public as well.
Move multiple limited api to cpython. Many API in `pythonrun` are
removed in python 3.10.
There is also some function and macro share the same name (documented
below) in cpython, which I choose to skip macro definition.
- PyRun_String
- PyRun_AnyFile
- PyRun_AnyFileEx
- PyRun_AnyFileFlags
David Hewitt