Inline tests compile the same set of source files as the original crate.
The set of aliases required will be the same or occasionally a superset.
We shouldn't make users repeat them across targets.
Previously we were adding these early on in the link args.
This was a bug - if some transitive dep foo has a linkopt of `-lbar`,
the `-lbar` needs to be later in the link args list than `-lfoo`.
Rather than arbitrarily adding them early on and hoping things work,
this adds the args for each library just after we add a dependency on
that library.
As discussed in issue
https://github.com/bazelbuild/rules_rust/issues/2917,
this updated cross compilation example serves as a bridge until the
exact cause of the sys crates not building on Mac issue has been found
and better regression testing has been added.
Also, I took the freedom to trim and streamline the LLVM setup making it
easier to configure while preserving the property that it caches failing
sys crate builds. Updated the Readme accordingly.
---------
Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>
Proc macro resolution wasn't working properly for sparse repos. This
fixes #2938.
---------
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
Co-authored-by: Daniel Wagner-Hall <dwagnerhall@apple.com>
This PR is follow up to #2453 and marks the `crate` extension as
`os_dependent` and `arch_dependent`.
For `crate.from_cargo`, the extension specifies reproducibility via the
`reproducible` attribute of `extension_metadata` which makes Bazel skip
the extension when writing the lockfile.
But in the case of `crate.from_specs`, the current implementation marks
the extension as non reproducible since the crates will not be backed by
a lockfile.
And because the `crate` module extension depends on `rust_host_tools`
(which are os / arch dependent), the entry in the lockfile for the
module extension includes the checksum of the `rustc` and `cargo`
binaries for whatever host it was resolved for.
This makes the bazel lock file platform dependent in that case.
This PR will result in a new os/arch specific entry in the lockfile for
users of `crate.from_specs`.
And will have no impact for users of `crate.from_cargo`.
Thanks to @fmeum for the guidance.
Hi, if I'm not mistaken, in Bazel `deps(a)+deps(b)` is equivalent to
`deps(a+b)`. The latter is more efficient because it doesn't have to
deduplicate common dependencies between a and b. This performance issue
has been triggered in our configuration, as we track roughly 700 targets
and 1600 transitive dependencies in our monorepo.
Running `aquery` with the original implementation takes 15 seconds on my
machine while the implementation proposed in this PR takes 1.5 seconds.
This PR makes two changes:
1. It removes the duplicately generated code in the prost `lib.rs`.
2. It allows opting into transitively including all of the proto deps.
## Duplicately generated code.
Previously there was a bug where we generated the same code at every
proto package level. This created separate Rust types for each module
which made it easy to accidentally import an incompatible type.
Before:
```rust
// @generated
pub mod b_ar {
pub mod b_az {
pub mod qaz {
pub mod qu_x {
// @generated
// This file is @generated by prost-build.
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Bar {
#[prost(string, tag = "1")]
pub name: ::prost::alloc::string::String,
#[prost(message, optional, tag = "2")]
pub foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::Foo,
>,
#[prost(message, optional, tag = "3")]
pub nested_foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::foo::NestedFoo,
>,
}
/// Nested message and enum types in `Bar`.
pub mod bar {
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Baz {
#[prost(string, tag = "4")]
pub name: ::prost::alloc::string::String,
}
}
// @@protoc_insertion_point(module)
}
}
}
}
pub mod b_az {
pub mod qaz {
pub mod qu_x {
// @generated
// This file is @generated by prost-build.
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Bar {
#[prost(string, tag = "1")]
pub name: ::prost::alloc::string::String,
#[prost(message, optional, tag = "2")]
pub foo:
::core::option::Option<::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::Foo>,
#[prost(message, optional, tag = "3")]
pub nested_foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::foo::NestedFoo,
>,
}
/// Nested message and enum types in `Bar`.
pub mod bar {
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Baz {
#[prost(string, tag = "4")]
pub name: ::prost::alloc::string::String,
}
}
// @@protoc_insertion_point(module)
}
}
}
pub mod qaz {
pub mod qu_x {
// @generated
// This file is @generated by prost-build.
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Bar {
#[prost(string, tag = "1")]
pub name: ::prost::alloc::string::String,
#[prost(message, optional, tag = "2")]
pub foo: ::core::option::Option<::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::Foo>,
#[prost(message, optional, tag = "3")]
pub nested_foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::foo::NestedFoo,
>,
}
/// Nested message and enum types in `Bar`.
pub mod bar {
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Baz {
#[prost(string, tag = "4")]
pub name: ::prost::alloc::string::String,
}
}
// @@protoc_insertion_point(module)
}
}
pub mod qu_x {
// @generated
// This file is @generated by prost-build.
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Bar {
#[prost(string, tag = "1")]
pub name: ::prost::alloc::string::String,
#[prost(message, optional, tag = "2")]
pub foo: ::core::option::Option<::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::Foo>,
#[prost(message, optional, tag = "3")]
pub nested_foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::foo::NestedFoo,
>,
}
/// Nested message and enum types in `Bar`.
pub mod bar {
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Baz {
#[prost(string, tag = "4")]
pub name: ::prost::alloc::string::String,
}
}
// @@protoc_insertion_point(module)
}
```
Now:
```rust
// @generated
pub use foo_proto;
pub mod b_ar {
pub mod b_az {
pub mod qaz {
pub mod qu_x {
// @generated
// This file is @generated by prost-build.
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Bar {
#[prost(string, tag = "1")]
pub name: ::prost::alloc::string::String,
#[prost(message, optional, tag = "2")]
pub foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::Foo,
>,
#[prost(message, optional, tag = "3")]
pub nested_foo: ::core::option::Option<
::foo_proto::foo::quu_x::co_rg_e::grault::ga_rply::foo::NestedFoo,
>,
}
/// Nested message and enum types in `Bar`.
pub mod bar {
#[allow(clippy::derive_partial_eq_without_eq)]
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Baz {
#[prost(string, tag = "4")]
pub name: ::prost::alloc::string::String,
}
}
// @@protoc_insertion_point(module)
}
}
}
}
```
## Including transitive depenencies.
Because the proto rules rely on aspects, we are compiling targets which
may not be directly accessible as a label in Bazel. Out of convenience,
the original implementation automatically included all transitive
dependencies to the `rust_library` or `rust_binary` that depended on the
proto crate.
As an example, let's say we have the following hierarchy:
```
my_crate -> a.proto
a.proto -> b.proto
b.proto -> c.proto
```
The previous setup made it possible for `my_crate` to directly import
`a_proto`, `b_proto`, and `c_proto`.
This is problematic though because it leads to significant crate
dependency bloat. `my_crate` may only use `a_proto` directly but it now
also depends on the unused crates `b_proto` and `c_proto`.
This PR makes every generated crate re-export its direct dependencies.
So, for the same example we now can import via those reexports:
```rust
//! my_crate
use a_proto::A;
use a_proto::b_proto::B;
use a_proto::b_proto::c_proto::C;
```
This means that you can also add the rustc flag
`-Dunused_crate_dependencies` to ensure you're not depending on crates
which are unused. In our monorepo we've had to disable this flag for
crates depending on protos.
If you need to depend on an intermediate crate directly, you can just
add a `rust_prost_library` definition and since it's all generated with
aspects, it will be the identical underlying crate.
Finally, if you really want the extra crates included, you can set the
flag `include_transitive_deps` on the prost toolchain and that will
bring back the previous behavior.
#2931 added tests for _pwd_flags, however
the test as-is passes with #2911 and #2922,
which still caused problems and reverted.
Here I introduce a test case for likely
unfixed in reverted #2925.
The test needs strictter check for cflags. As-ls
`flag in cflags` is a just substring search and
very weak test for abs path.
For #2917
This is trivial copy/paste implementation, to
minimize regression probability.
Previously it was combined with refactoring, but
introduced regression and was reverted #2911.
This time I split refactoring into follow up PR.
`js_info` needs to jump through some hoops to support the legacy names.
The new ones are shorter and clearer.
Also, set the toolchain on the action to support Auto Exec Groups while
I'm here.
Co-authored-by: UebelAndre <github@uebelandre.com>
# PR Summary
Commit c080d7bfa1 moved the location of
`BUILD.bazel`. This PR adjusts sources to changes.
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
Revert "Restore lost "is_absolute" (#2922)"
This reverts commit 2a85e13c74.
Revert "Use absolute path with "-fsanitize-ignorelist" (#2911)"
This reverts commit 9594fa70b5.
After https://github.com/bazelbuild/rules_rust/pull/2887 I started
finding similar linker errors in environments that supported runfiles
and symlinks but have yet to see this issue in environments without
them. I believe my understanding of how the
[DefaultInfo.default_runfiles](https://bazel.build/rules/lib/providers/DefaultInfo#default_runfiles)
objects worked and in the previous change had attempted to rely on it
for a runfiles directory. However, there is no guarantee a directory
will be rendered and no way to force it to be rendered for actions
either (https://github.com/bazelbuild/bazel/issues/15486). The
consequence is that creating a custom runfiles directory is no longer a
contingency and explicitly creating it is the correct thing to do to
ensure `CARGO_MANIFEST_DIR` linker inputs are always available to the
consuming actions.
With the propagation of `CARGO_MANIFEST_DIR` (now an action output) into
`Rustc` actions, a new flag is also introduced to prune unnecessary
files out of the directory to reduce bloat and potential invalidation in
downstream actions,
`--@rules_rust//cargo/settings:cargo_manifest_dir_filename_suffixes_to_retain`.
This flag accepts a list of path suffixes used to determine what (if
anything) in `CARGO_MANIFEST_DIR` should be retained and passed to
downstream actions.
Note that the failure mode this addresses is hard to observe as it
requires a crate which defines a linker path to something in
`CARGO_MANIFEST_DIR` and then for a clean build to be done with a change
to a dependent of said crate. If the runfiles directory for the build
script is never rendered then there will be no file to link into the
crate.
---------
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
This fix validation errors for the new `wasm32-wasip1` target. The older
`wasm32-wasi` is already covered. Fixes #2782
Signed-off-by: Chaitanya Munukutla <chaitanya.m61292@gmail.com>
Co-authored-by: UebelAndre <github@uebelandre.com>
This change allows for the definition of platform constraints without
introducing plumbing in repository rules that would attempt to fetch
artifacts that do not exist.
---------
Co-authored-by: Daniel Wagner-Hall <dwagnerhall@apple.com>
This change introduces the `action_args` crate which is something I feel
I keep writing in various repos now. It's original design is to make it
easier to pass args to built binaries. E.g.
```rust
use action_args;
use clap::Parser;
use runfiles::{rlocation, Runfiles};
#[command(version, about, long_about = None)]
struct ClapArgs {}
fn main() {
let args = {
let runfiles = Runfiles::create().unwrap();
let var = std::env::var("ARGS_FILE").unwrap();
let runfile = rlocation!(runfiles, var).unwrap();
let text = std::fs::read_to_string(runfile).unwrap();
let argv = action_args::parse_args(text);
ClapArgs::parse_from(std::env::args().take(1).chain(argv))
};
// ...
// ...
// ...
}
```
This utility will likely be unnecessary should
https://github.com/bazelbuild/bazel/issues/16076 ever be implemented.
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
Bazel 8 uses `+` as the separator for segments of canonical repo names,
whereas Bazel 7 uses `~` or `+` depending on the value of
`--incompatible_use_plus_in_repo_names`.
---------
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
Partially addresses https://github.com/bazelbuild/rules_rust/issues/1156
This pull-request implements an additional change to enable this
behavior which aggregates all transitive `BuildInfo.compile_data` into
`Rustc` actions. While this seems to bloat these actions with
unnecessary data, it addresses a catastrophic flaw in how Windows works
at all.
As of Bazel 7.3.1, Windows does not run any actions in a sandbox
(https://github.com/bazelbuild/bazel/discussions/18401), this means that
references to the current working directory will be consistent since
they always refer to the execroot. On top of this fact,
`cargo_build_script` will assign
[CARGO_MANIFEST_DIR](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
to a [path within the runfiles directory of the build
script](d9ee6172d1/cargo/private/cargo_build_script.bzl (L218)).
The combination of these two facts leads crates like
[windows_x86_64_msvc](https://crates.io/crates/windows_x86_64_msvc),
which assign a linker path using `CARGO_MANIFEST_DIR`
([@windows_x86_64_msvc//build.rs](https://github.com/microsoft/windows-rs/blob/0.59.0/crates/targets/x86_64_msvc/build.rs#L1-L8))
to introduce un-tracked dependencies into dependent `Rustc` actions.
This then leads to build failures if the `windows_x86_64_msvc` crate is
ever a remote cache hit for the dependents as runfiles (or
`.cargo_runfiles` in the case of this PR) are not fetched and will not
exist on the host at link time.
This change addresses this issue of untracked dependencies by ensuring
the runfiles of `cargo_build_script.script` targets are aggregated into
`Rustc` actions.
This change expands the use of `cargo tree` to resolve features for
different combinations of `host -> target` platform triples where before
`cargo-bazel` was only capable of resolving host dependencies for the
current host platform and not a foreign platform.
A lengthy description of how this was done can be found at
`crate_universe/src/metadata/cargo_tree_resolver.rs -
TreeResolver::create_rustc_wrapper`.
closes https://github.com/bazelbuild/rules_rust/issues/2212
---------
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
This PR adds support for `wasm64-unknown-unknown` although there are no
prebuilt `wasm64-unknown-unknown` rustc toolchains. Compiling for
`wasm64` will be left up to anyone using this target triple until we get
T2 support for wasm64.
---------
Co-authored-by: Andre Brisco <andre.brisco@freeform.co>
Sam McCall