Hi, if I'm not mistaken, in Bazel `deps(a)+deps(b)` is equivalent to
`deps(a+b)`. The latter is more efficient because it doesn't have to
deduplicate common dependencies between a and b. This performance issue
has been triggered in our configuration, as we track roughly 700 targets
and 1600 transitive dependencies in our monorepo.
Running `aquery` with the original implementation takes 15 seconds on my
machine while the implementation proposed in this PR takes 1.5 seconds.
This change introduces the `action_args` crate which is something I feel
I keep writing in various repos now. It's original design is to make it
easier to pass args to built binaries. E.g.
```rust
use action_args;
use clap::Parser;
use runfiles::{rlocation, Runfiles};
#[command(version, about, long_about = None)]
struct ClapArgs {}
fn main() {
let args = {
let runfiles = Runfiles::create().unwrap();
let var = std::env::var("ARGS_FILE").unwrap();
let runfile = rlocation!(runfiles, var).unwrap();
let text = std::fs::read_to_string(runfile).unwrap();
let argv = action_args::parse_args(text);
ClapArgs::parse_from(std::env::args().take(1).chain(argv))
};
// ...
// ...
// ...
}
```
This utility will likely be unnecessary should
https://github.com/bazelbuild/bazel/issues/16076 ever be implemented.
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
Partially addresses https://github.com/bazelbuild/rules_rust/issues/1156
This pull-request implements an additional change to enable this
behavior which aggregates all transitive `BuildInfo.compile_data` into
`Rustc` actions. While this seems to bloat these actions with
unnecessary data, it addresses a catastrophic flaw in how Windows works
at all.
As of Bazel 7.3.1, Windows does not run any actions in a sandbox
(https://github.com/bazelbuild/bazel/discussions/18401), this means that
references to the current working directory will be consistent since
they always refer to the execroot. On top of this fact,
`cargo_build_script` will assign
[CARGO_MANIFEST_DIR](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
to a [path within the runfiles directory of the build
script](d9ee6172d1/cargo/private/cargo_build_script.bzl (L218)).
The combination of these two facts leads crates like
[windows_x86_64_msvc](https://crates.io/crates/windows_x86_64_msvc),
which assign a linker path using `CARGO_MANIFEST_DIR`
([@windows_x86_64_msvc//build.rs](https://github.com/microsoft/windows-rs/blob/0.59.0/crates/targets/x86_64_msvc/build.rs#L1-L8))
to introduce un-tracked dependencies into dependent `Rustc` actions.
This then leads to build failures if the `windows_x86_64_msvc` crate is
ever a remote cache hit for the dependents as runfiles (or
`.cargo_runfiles` in the case of this PR) are not fetched and will not
exist on the host at link time.
This change addresses this issue of untracked dependencies by ensuring
the runfiles of `cargo_build_script.script` targets are aggregated into
`Rustc` actions.
https://github.com/bazelbuild/rules_rust/pull/2826 ended up introducing
a regression for crates with build scripts that require extra data at
compile time. This change adds a default glob for all
`cargo_build_script` targets generated by `crate_universe` and also
introduces an `annotation.build_script_compile_data` attribute to add
custom targets to generated outputs.
This change updates the `rust_analyzer_aspect` to attempt to match
generated crate root sources to `srcs` inputs and use the the path of
the source file in place of the generated path. The generated path is
then added to `sources.include_dirs` for compatibility. The impact of
this can be seen in the following diff of the
`@rules_rust//:rust-project.json` file:
```diff
--- rust-project.old.json 2024-06-28 09:00:51
+++ rust-project.json 2024-06-28 09:04:43
@@ -821,10 +821,16 @@
},
{
"display_name": "libgensrc_with_crate_root",
- "root_module": "/private/var/tmp/_bazel_user/76282c66b0dfe3c5cb9a230bdc913a52/execroot/rules_rust/bazel-out/darwin_arm64-fastbuild/bin/test/generated_inputs/lib.rs",
+ "root_module": "test/generated_inputs/lib.rs",
"edition": "2018",
"deps": [],
"is_workspace_member": true,
+ "source": {
+ "include_dirs": [
+ "/private/var/tmp/_bazel_user/76282c66b0dfe3c5cb9a230bdc913a52/execroot/rules_rust/bazel-out/darwin_arm64-fastbuild/bin/test/generated_inputs"
+ ],
+ "exclude_dirs": []
+ },
"cfg": [
"test",
"debug_assertions"
@@ -850,10 +856,16 @@
...
...
...
```
closes https://github.com/bazelbuild/rules_rust/issues/2716
Before this, if we re-generate all of the BUILD.bazel files for our
vendored deps, they are invalid because they only have BUILD.bazel files
and the globs present in them expect srcs.
First two commits are real changes, third commit is just a repin.
The toolchain_files() and current_rust_toolchain() rules return files
for the current 'exec' platform. This is perfecly reasonable if you want
to use these tools as part of ctx.actions.run(). But if you want to use
these as part of 'data = []' (runfiles), they must be built for the
target.
Fixes: https://github.com/bazelbuild/rules_rust/issues/2684
At the moment, rust_analyzer's rust-project.json will list the root of a
proto crate as, for example:
`$HOME/.cache/bazel/_bazel_msta/60654fdd7bbd86377eab8595eb8577e8/execroot/_main/bazel-out/k8-dbg/bin/examples/prost/proto/example_proto.lib.rs`.
However, this file does not exist, since it wasn't actually declared as
the output of any rule.
With this change, tools/rust-analyzer will actually generate the
generated sources.
Currently, if the repo mapping maps "foo" to "real_foo", then:
* `rlocation!(r, "foo/bar/baz")` will return "real_foo/bar/baz"
* `rlocation!(r, "foo")` will return `foo`
This PR handles the case where the user attempts to do something like
`rlocation!("rules_rust")` (which is valid for directory based
runfiles).
The Cargo [Feature Resolver version
2](https://doc.rust-lang.org/cargo/reference/resolver.html#feature-resolver-version-2)
behavior is currently not supported by the `cargo metadata` sub command
(https://github.com/rust-lang/cargo/issues/9863) which `crate_universe`
uses to determine the dependencies of a target, leading to inaccuracies
when dependencies are introduced via feature resolution for a particular
configuration.
In https://github.com/bazelbuild/rules_rust/pull/1710 functionality was
added to use `cargo tree` to perform feature resolution for each
supported platform. This change expands on this trick to collect
dependency information at the same time and use that to determine
whether or not to include optional dependencies located in standard
`cargo metadata` output in the rendered Bazel targets. Non optional or
`target.cfg` (conditional) dependencies behave as they did before this
change.
Implementation details:
- `FeatureGenerator` was replaced by `TreeResolver`
- Optional dependencies are now rendered as selects on explicit
platforms. This will expand the size of `cargo-bazel-lock.json` files
but is expected to be more correct.
This implements the repo_mapping capabilities and provides a new
runfiles API. The new API can be accessed explicitly as
`runfiles::Runfiles::rlocation_from` or with the `runfiles::rlocation!`
macro, which adds compile-time support for correctly embedding the
external repo. This is a purely new API, existing usage continues to
work, although we mark it deprecated because it's not fully correct. We
can remove it at some point in the future.
This PR also transitions in-repo examples/tests to using it, in case
anyone copies them.
---------
Co-authored-by: scentini <rosica@google.com>
On windows (and some other platforms), the file extension of cargo,
rustc, etc have an extension. The module extension for loading crates
did not take this into account, causing it to error on windows.
Additionally, when using bzlmod to build vendored crates, the runfiles
tree would exceed the 260 char windows path limit. To mitigate this, I
have shortened the internal_deps module extension to just `i` and
changed the build script suffix to `_bs` from `_build_script`. This
makes the path names below the 260 char limit.
This makes the bzlmod CI run on windows, to avoid regressing this.
Currently gen_rust_project does not run on windows for other reasons,
although we do build this.
This change cleans up the rust-analyzer aspect to support external rules
providing their own crate specs. For now only prost implements behavior
for this and the rust-analyzer interface is still private. In the future
if this proves to be performant and a consistent interface then there
should be no issue making this a public part of the `//rust` package.
This change incorporates
https://github.com/bazelbuild/rules_rust/pull/1875 (special thanks to
@snowp!) and addresses performance issues in the generator tool by
allowing users of `bazelisk` to ensure their `tools/bazel` scripts run
should one be provided and to disable running validation actions when
building crate specs.
Currently when running `gen_rust_project`, the aspect given to bazel has
the canonical repo name but only a single `@`. This does not work with
bzlmod, since it tries to resolve this using the repository mappings,
and complains that the repository does not exist:
```
ERROR: Unable to find package for @@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl: The repository '@@[unknown repo 'rules_rust~0.38.0' requested from @@]' could not be resolved: No repository visible as '@rules_rust~0.38.0' from main repository.
ERROR: Analysis of aspects '[@@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl%rust_analyzer_aspect] with parameters {} on //prost:prost_toolchain_impl' failed; build aborted: Unable to find package for @@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl: The repository '@@[unknown repo 'rules_rust~0.38.0' requested from @@]' could not be resolved: No repository visible as '@rules_rust~0.38.0' from main repository.
```
This adds an extra `@`, so that the repository in the label is of
canonical form.
Fixes #2449
Currently when trying to generate a `rust-project.json`, if there aren't
actually any Rust targets defined, the script mysteriously fails.
This adds a better error message.
Explicitly set --include_artifacts to override users' .bazelrc file
settings.
Evidently I had set `aquery --include_artifacts=false` in ~/.bazelrc,
which was very difficult to track down from the error when generating
the project file:
```
Error: missing field `outputIds` at line 9 column 3
```
Co-authored-by: UebelAndre <github@uebelandre.com>
See #2246 for how we declare dependencies automatically for third party
crates. I'm using the same mechanism for all other repositories.
Note that the MODULE.bazel changes in use_repo are autogenerated, and if
you change the third party crates you depend on, you should get an error
message saying something like "run this command to fix it" on your
bzlmod presubmit environment.
Currently, when you run `bazel build --enable_bzlmod
//crate_universe:bin`, you get the error `undefined repo @cui`.
When you add a dependency on CUI, you get errors like `undefined repo
@cui__<crate>-<version>` (the repo for the third party crate).
This change will allow us to automatically declare dependencies on our
third party crates from bzlmod (see
https://docs.google.com/document/d/1dj8SN5L6nwhNOufNqjBhYkk5f-BJI_FPYWKxlB3GAmA/edit#heading=h.5mcn15i0e1ch).
---------
Co-authored-by: UebelAndre <github@uebelandre.com>
The generated content uses ${pwd} and in the rust-analyzer server output
I see error logs saying can't find dir with ${pwd} references
Though I'm not sure if this actually affect anything as it seems work
fine most of the time, but I think it maybe good to not have
rust-analyzer errors
While running from CARGO_MANIFEST_DIR is the simplest thing for
compatibility, there are cases where a cargo build script may be easier
to run from the exec root as most bazel actions do (e.g. where a C++
toolchain specifies in-repo include paths).
This mirrors the `rundir` attribute of `go_test`, which has similar
concerns:
https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/rules.md#go_test-rundir
Changes:
- Updates various dependencies in `//crate_universe:Cargo.toml`
- Update the Rust edition for crate_universe to 2021
- Update crate_unverse version to `0.9.0`
* Exclude .tmp_git_root from globs
This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by https://github.com/bazelbuild/bazel/pull/18271. Even after this fix, excluding this directory avoids depending on unnecessary files.
Fixes #1927.
* Regenerate vendored crates
* Update clippy and rustfmt aspects to require CrateInfo providers
* Updated rustfmt rules to work with `rust_shared/static_library`
* Regenerate documentation
Lucas Pluvinage