Commit Graph

112 Commits

Author SHA1 Message Date
Lucas Pluvinage 2e061509ba
gen_rust_project: optimize aquery with many targets (#2941)
Hi, if I'm not mistaken, in Bazel `deps(a)+deps(b)` is equivalent to
`deps(a+b)`. The latter is more efficient because it doesn't have to
deduplicate common dependencies between a and b. This performance issue
has been triggered in our configuration, as we track roughly 700 targets
and 1600 transitive dependencies in our monorepo.
Running `aquery` with the original implementation takes 15 seconds on my
machine while the implementation proposed in this PR takes 1.5 seconds.
2024-10-18 17:45:08 +00:00
UebelAndre fa943c7f16
Added utility for parsing action Args param files (#2897)
This change introduces the `action_args` crate which is something I feel
I keep writing in various repos now. It's original design is to make it
easier to pass args to built binaries. E.g.

```rust
use action_args;
use clap::Parser;
use runfiles::{rlocation, Runfiles};

#[command(version, about, long_about = None)]
struct ClapArgs {}

fn main() {
    let args = {
        let runfiles = Runfiles::create().unwrap();

        let var = std::env::var("ARGS_FILE").unwrap();
        let runfile = rlocation!(runfiles, var).unwrap();
        let text = std::fs::read_to_string(runfile).unwrap();
        let argv = action_args::parse_args(text);
        ClapArgs::parse_from(std::env::args().take(1).chain(argv))
    };

    // ...
    // ...
    // ...
}
```

This utility will likely be unnecessary should
https://github.com/bazelbuild/bazel/issues/16076 ever be implemented.

Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
2024-10-01 18:42:22 +00:00
UebelAndre 59464fe1d9
Update `cargo_build_script` to work without runfiles support. (#2887)
Partially addresses https://github.com/bazelbuild/rules_rust/issues/1156

This pull-request implements an additional change to enable this
behavior which aggregates all transitive `BuildInfo.compile_data` into
`Rustc` actions. While this seems to bloat these actions with
unnecessary data, it addresses a catastrophic flaw in how Windows works
at all.

As of Bazel 7.3.1, Windows does not run any actions in a sandbox
(https://github.com/bazelbuild/bazel/discussions/18401), this means that
references to the current working directory will be consistent since
they always refer to the execroot. On top of this fact,
`cargo_build_script` will assign
[CARGO_MANIFEST_DIR](https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
to a [path within the runfiles directory of the build
script](d9ee6172d1/cargo/private/cargo_build_script.bzl (L218)).
The combination of these two facts leads crates like
[windows_x86_64_msvc](https://crates.io/crates/windows_x86_64_msvc),
which assign a linker path using `CARGO_MANIFEST_DIR`
([@windows_x86_64_msvc//build.rs](https://github.com/microsoft/windows-rs/blob/0.59.0/crates/targets/x86_64_msvc/build.rs#L1-L8))
to introduce un-tracked dependencies into dependent `Rustc` actions.
This then leads to build failures if the `windows_x86_64_msvc` crate is
ever a remote cache hit for the dependents as runfiles (or
`.cargo_runfiles` in the case of this PR) are not fetched and will not
exist on the host at link time.

This change addresses this issue of untracked dependencies by ensuring
the runfiles of `cargo_build_script.script` targets are aggregated into
`Rustc` actions.
2024-09-19 19:57:07 +00:00
UebelAndre 3daf96e315
Fix runfiles creation for directory based runfiles. (#2883)
closes https://github.com/bazelbuild/rules_rust/issues/2868
2024-09-18 16:10:01 +00:00
UebelAndre 5b7cff3f80
Move all settings flags into `//rust/settings` (#2870)
Aliases are still in place for backward compatibility and intended for
public use.
2024-09-18 15:54:25 +00:00
UebelAndre 928eb101d1
Added tests for `runfiles::parse_repo_mapping` (#2869) 2024-09-18 15:53:51 +00:00
UebelAndre 3bae148709
Update crate universe to generate build script targets with compile data (#2855)
https://github.com/bazelbuild/rules_rust/pull/2826 ended up introducing
a regression for crates with build scripts that require extra data at
compile time. This change adds a default glob for all
`cargo_build_script` targets generated by `crate_universe` and also
introduces an `annotation.build_script_compile_data` attribute to add
custom targets to generated outputs.
2024-09-11 13:38:53 +00:00
UebelAndre d07ac80ac6
Update `runfiles::rlocation!` to return `Option` instead of panicing (#2847)
closes https://github.com/bazelbuild/rules_rust/issues/2653
2024-09-10 21:30:20 +00:00
UebelAndre 3877078566
Updated rust-analyzer to use repo vs generated files as crate roots (#2717)
This change updates the `rust_analyzer_aspect` to attempt to match
generated crate root sources to `srcs` inputs and use the the path of
the source file in place of the generated path. The generated path is
then added to `sources.include_dirs` for compatibility. The impact of
this can be seen in the following diff of the
`@rules_rust//:rust-project.json` file:
```diff
--- rust-project.old.json	2024-06-28 09:00:51
+++ rust-project.json	2024-06-28 09:04:43
@@ -821,10 +821,16 @@
         },
         {
             "display_name": "libgensrc_with_crate_root",
-            "root_module": "/private/var/tmp/_bazel_user/76282c66b0dfe3c5cb9a230bdc913a52/execroot/rules_rust/bazel-out/darwin_arm64-fastbuild/bin/test/generated_inputs/lib.rs",
+            "root_module": "test/generated_inputs/lib.rs",
             "edition": "2018",
             "deps": [],
             "is_workspace_member": true,
+            "source": {
+                "include_dirs": [
+                    "/private/var/tmp/_bazel_user/76282c66b0dfe3c5cb9a230bdc913a52/execroot/rules_rust/bazel-out/darwin_arm64-fastbuild/bin/test/generated_inputs"
+                ],
+                "exclude_dirs": []
+            },
             "cfg": [
                 "test",
                 "debug_assertions"
@@ -850,10 +856,16 @@
...
...
...
```

closes https://github.com/bazelbuild/rules_rust/issues/2716
2024-07-04 15:35:09 +00:00
Peter Kolloch c1ab10aaad
upstream wrappers for cargo, rustc, cargo-clippy (#2703)
...and a bazel_env example.

As discussed in
https://bazelbuild.slack.com/archives/CSV56UT0F/p1718207171652029?thread_ts=1718182474.631279&cid=CSV56UT0F
2024-06-25 11:34:39 +00:00
Daniel Wagner-Hall 4ea03d6fd0
rust-analyzer: Factor aliases into the project (#2709)
Fixes #2707
2024-06-20 15:47:56 +00:00
Daniel Wagner-Hall 9c672947a1
Remote vendor doesn't require srcs present (#2688)
Before this, if we re-generate all of the BUILD.bazel files for our
vendored deps, they are invalid because they only have BUILD.bazel files
and the globs present in them expect srcs.

First two commits are real changes, third commit is just a repin.
2024-06-13 16:13:39 +00:00
Ed Schouten 837b7f892a
Depend on a copy of rustfmt for the target (#2685)
The toolchain_files() and current_rust_toolchain() rules return files
for the current 'exec' platform. This is perfecly reasonable if you want
to use these tools as part of ctx.actions.run(). But if you want to use
these as part of 'data = []' (runfiles), they must be built for the
target.

Fixes: https://github.com/bazelbuild/rules_rust/issues/2684
2024-06-07 14:54:21 +00:00
David Zbarsky dc08bde7a8
Use repo-mapping-aware runfiles API in rust-analyzer (#2666)
Fixes https://github.com/bazelbuild/rules_rust/issues/2615 and
https://github.com/bazelbuild/rules_rust/issues/2438
2024-05-31 00:23:58 +00:00
Matt 1566bad7a0
Add support for generated sources to rust-analyzer. (#2654)
At the moment, rust_analyzer's rust-project.json will list the root of a
proto crate as, for example:
`$HOME/.cache/bazel/_bazel_msta/60654fdd7bbd86377eab8595eb8577e8/execroot/_main/bazel-out/k8-dbg/bin/examples/prost/proto/example_proto.lib.rs`.
However, this file does not exist, since it wasn't actually declared as
the output of any rule.

With this change, tools/rust-analyzer will actually generate the
generated sources.
2024-05-16 09:45:14 +00:00
Matt a6222cd8e4
Allow runfiles object to be an arbitrary expression. (#2651)
Our use case is `runfiles::rlocation!(cache.runfiles, "foo")` but this
would also allow `runfiles::rlocation!(Runfiles::create()?, "foo")`
2024-05-15 09:18:32 +00:00
Matt d87eadfe68
Allow searching for the rlocation of a module. (#2652)
Currently, if the repo mapping maps "foo" to "real_foo", then:
* `rlocation!(r, "foo/bar/baz")` will return "real_foo/bar/baz"
* `rlocation!(r, "foo")` will return `foo`

This PR handles the case where the user attempts to do something like
`rlocation!("rules_rust")` (which is valid for directory based
runfiles).
2024-05-15 09:17:59 +00:00
Daniel Wagner-Hall 4a3ffcb1e8
Add target for running rustfmt directly (#2648)
Previously you either needed to go via our bazel-aware wrapper, or write
a rule to grab the file from the toolchain.
2024-05-10 16:41:10 +00:00
UebelAndre f22926a2b4
Use `cargo tree` to determine feature dependent optional deps (#2636)
The Cargo [Feature Resolver version
2](https://doc.rust-lang.org/cargo/reference/resolver.html#feature-resolver-version-2)
behavior is currently not supported by the `cargo metadata` sub command
(https://github.com/rust-lang/cargo/issues/9863) which `crate_universe`
uses to determine the dependencies of a target, leading to inaccuracies
when dependencies are introduced via feature resolution for a particular
configuration.

In https://github.com/bazelbuild/rules_rust/pull/1710 functionality was
added to use `cargo tree` to perform feature resolution for each
supported platform. This change expands on this trick to collect
dependency information at the same time and use that to determine
whether or not to include optional dependencies located in standard
`cargo metadata` output in the rendered Bazel targets. Non optional or
`target.cfg` (conditional) dependencies behave as they did before this
change.

Implementation details:
- `FeatureGenerator` was replaced by `TreeResolver`
- Optional dependencies are now rendered as selects on explicit
platforms. This will expand the size of `cargo-bazel-lock.json` files
but is expected to be more correct.
2024-05-08 17:53:05 +00:00
David Zbarsky 8d074a0b9e
Bzlmod-aware runfiles library (#2566)
This implements the repo_mapping capabilities and provides a new
runfiles API. The new API can be accessed explicitly as
`runfiles::Runfiles::rlocation_from` or with the `runfiles::rlocation!`
macro, which adds compile-time support for correctly embedding the
external repo. This is a purely new API, existing usage continues to
work, although we mark it deprecated because it's not fully correct. We
can remove it at some point in the future.

This PR also transitions in-repo examples/tests to using it, in case
anyone copies them.

---------

Co-authored-by: scentini <rosica@google.com>
2024-04-22 08:27:51 +00:00
UebelAndre 1afc1ebf20
Re-vendor crate_universe outputs (#2609) 2024-04-10 14:13:09 +00:00
UebelAndre 4bda11a16d
Added Rust 1.77.0 (#2568)
https://blog.rust-lang.org/2024/03/21/Rust-1.77.0.html
2024-03-21 16:34:39 +00:00
James Leitch f8ffba5ab2
Rust Analyzer added NixOS supported platforms (#2547) 2024-03-19 04:27:15 +00:00
Cameron Martin 33fdddd03c
Fix bzlmod examples on windows (#2454)
On windows (and some other platforms), the file extension of cargo,
rustc, etc have an extension. The module extension for loading crates
did not take this into account, causing it to error on windows.

Additionally, when using bzlmod to build vendored crates, the runfiles
tree would exceed the 260 char windows path limit. To mitigate this, I
have shortened the internal_deps module extension to just `i` and
changed the build script suffix to `_bs` from `_build_script`. This
makes the path names below the 260 char limit.

This makes the bzlmod CI run on windows, to avoid regressing this.
Currently gen_rust_project does not run on windows for other reasons,
although we do build this.
2024-02-23 12:55:58 +00:00
Cameron Martin cf5f0bd888
Re-vendor all crates (#2514)
It looks like the vendored crates got out of sync again. This updates
them all, using the new script `//crate_universe/tools:vendor`.
2024-02-23 10:03:02 +00:00
UebelAndre 377314b83d
Allow rules to provide their own rust-analyzer providers (#2487)
This change cleans up the rust-analyzer aspect to support external rules
providing their own crate specs. For now only prost implements behavior
for this and the rust-analyzer interface is still private. In the future
if this proves to be performant and a consistent interface then there
should be no issue making this a public part of the `//rust` package.

This change incorporates
https://github.com/bazelbuild/rules_rust/pull/1875 (special thanks to
@snowp!) and addresses performance issues in the generator tool by
allowing users of `bazelisk` to ensure their `tools/bazel` scripts run
should one be provided and to disable running validation actions when
building crate specs.
2024-02-20 10:19:55 +00:00
Nick Collier 172b34aabd
Generate package license metadata (#2476)
#2475
2024-02-14 00:37:38 +00:00
UebelAndre 9f68ed24fc
Added Rust 1.76.0 (#2468)
https://blog.rust-lang.org/2024/02/08/Rust-1.76.0.html
2024-02-08 18:22:54 +00:00
Cameron Martin 0f777e1983
Use canonical repo name for aspects (#2450)
Currently when running `gen_rust_project`, the aspect given to bazel has
the canonical repo name but only a single `@`. This does not work with
bzlmod, since it tries to resolve this using the repository mappings,
and complains that the repository does not exist:

```
ERROR: Unable to find package for @@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl: The repository '@@[unknown repo 'rules_rust~0.38.0' requested from @@]' could not be resolved: No repository visible as '@rules_rust~0.38.0' from main repository.
ERROR: Analysis of aspects '[@@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl%rust_analyzer_aspect] with parameters {} on //prost:prost_toolchain_impl' failed; build aborted: Unable to find package for @@[unknown repo 'rules_rust~0.38.0' requested from @@]//rust:defs.bzl: The repository '@@[unknown repo 'rules_rust~0.38.0' requested from @@]' could not be resolved: No repository visible as '@rules_rust~0.38.0' from main repository.
```

This adds an extra `@`, so that the repository in the label is of
canonical form.

Fixes #2449
2024-02-01 17:32:31 +00:00
John Hughes d72f4a8c7b
Provide a better error message when trying to generate rust-project.json (#2196)
Currently when trying to generate a `rust-project.json`, if there aren't
actually any Rust targets defined, the script mysteriously fails.

This adds a better error message.
2024-01-13 05:11:05 -08:00
Neil Isaac 137b5f288c
rust_analyzer: run bazel aquery with --include_artifacts (#2328)
Explicitly set --include_artifacts to override users' .bazelrc file
settings.

Evidently I had set `aquery --include_artifacts=false` in ~/.bazelrc,
which was very difficult to track down from the error when generating
the project file:
```
Error: missing field `outputIds` at line 9 column 3
```

Co-authored-by: UebelAndre <github@uebelandre.com>
2023-12-28 05:49:40 -08:00
Matt 44ffccb91c
Allow additional targets to compile with bzlmod (#2314)
See #2246 for how we declare dependencies automatically for third party
crates. I'm using the same mechanism for all other repositories.

Note that the MODULE.bazel changes in use_repo are autogenerated, and if
you change the third party crates you depend on, you should get an error
message saying something like "run this command to fix it" on your
bzlmod presubmit environment.
2023-12-12 13:52:58 +00:00
Matt 3fd4987269
Declare dependencies in vendored crates (#2246)
Currently, when you run `bazel build --enable_bzlmod
//crate_universe:bin`, you get the error `undefined repo @cui`.

When you add a dependency on CUI, you get errors like `undefined repo
@cui__<crate>-<version>` (the repo for the third party crate).

This change will allow us to automatically declare dependencies on our
third party crates from bzlmod (see
https://docs.google.com/document/d/1dj8SN5L6nwhNOufNqjBhYkk5f-BJI_FPYWKxlB3GAmA/edit#heading=h.5mcn15i0e1ch).

---------

Co-authored-by: UebelAndre <github@uebelandre.com>
2023-12-04 07:50:52 -08:00
Yuwei Ba 364f130477
replace pwd in rust-project.json output #1 (#2272)
The generated content uses ${pwd} and in the rust-analyzer server output
I see error logs saying can't find dir with ${pwd} references

Though I'm not sure if this actually affect anything as it seems work
fine most of the time, but I think it maybe good to not have
rust-analyzer errors
2023-11-21 11:18:41 +00:00
UebelAndre ce1b842c9d
Support rustfmt on crates with generated sources (#2255)
closes https://github.com/bazelbuild/rules_rust/issues/2254
2023-11-16 11:40:34 +01:00
Matt 5d89c9245d
Run "bazel run //*:crates_vendor" (#2245)
This avoids very large diffs from some an unrelated PR I have coming.
2023-11-08 06:18:30 -08:00
Daniel Wagner-Hall 1f9b63a408
cargo_build_scripts can be run from custom directories (#2152)
While running from CARGO_MANIFEST_DIR is the simplest thing for
compatibility, there are cases where a cargo build script may be easier
to run from the exec root as most bazel actions do (e.g. where a C++
toolchain specifies in-repo include paths).

This mirrors the `rundir` attribute of `go_test`, which has similar
concerns:
https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/rules.md#go_test-rundir
2023-09-06 18:38:35 +02:00
UebelAndre 05a3c4470e
Updated crate_universe dependencies (#2063)
Changes:
- Updates various dependencies in `//crate_universe:Cargo.toml`
- Update the Rust edition for crate_universe to 2021
- Update crate_unverse version to `0.9.0`
2023-07-14 17:39:40 +01:00
UebelAndre b327b25422
Update rust_analyzer dependencies (#2065) 2023-07-14 07:03:28 -07:00
Cameron Martin 36f8251f97
Exclude .tmp_git_root from globs (#1948)
* Exclude .tmp_git_root from globs

This directory contains the original git repo when the crate is from git. Including this directory currently makes these rules re-run whenever the repository rule re-runs, although this is fixed by https://github.com/bazelbuild/bazel/pull/18271. Even after this fix, excluding this directory avoids depending on unnecessary files.

Fixes #1927.

* Regenerate vendored crates
2023-06-23 04:27:16 -07:00
David Tolnay 56e7604879
Fix typo in crate_universe-generated defs.bzl comment (#1981) 2023-05-30 10:45:18 +01:00
Joseph Ryan baeb0664d4
Remove ios/android/wasm support for gen_rust_project deps (#1684) 2023-02-10 09:43:18 -08:00
UebelAndre ba0fb5956c
Added support for `--nolegacy_external_runfiles` to `rust_doc_test` (#1790) 2023-01-20 12:08:30 +00:00
UebelAndre 1b1dae1962
Added Rust 1.66.1 (#1767)
* Updated `rust_bindgen` to use `rustfmt_toolchain`

* Regenerate documentation

* Added Rust 1.66.1

* Regenerate documentation

* Updated crate_universe lock files

* Addressed clippy error
2023-01-19 14:22:58 +00:00
UebelAndre 49906eb29e
Update clippy and rustfmt aspects to require CrateInfo providers (#1772)
* Update clippy and rustfmt aspects to require CrateInfo providers

* Updated rustfmt rules to work with `rust_shared/static_library`

* Regenerate documentation
2023-01-13 10:56:03 -08:00
David Tolnay d6e3003594
Regenerate BUILD files using serde_starlark renderer (#1746) 2023-01-04 09:33:08 -08:00
UebelAndre 92977d1bfb
Re-pinned all dependencies managed by crate_universe (#1735) 2023-01-03 18:06:40 +00:00
UebelAndre d5289ad1c9
Added `rustfmt_toolchain` and refactored toolchain repository rules (#1719)
* Added `rustfmt_toolchain` and refactored toolchain repository rules

* Regenerate documentation
2023-01-03 15:44:26 +00:00
Matt d4e5586d0a
Support the RUNFILES_DIR environment variable. (#1732)
* Support the RUNFILES_DIR environment variable.

RUNFILES_DIR is the preferred way to get runfiles directories in bazel.

* Run rustfmt
2022-12-28 18:17:23 -08:00
UebelAndre 1357b85b1b
Addressed clippy warnings from `clippy 0.1.67 (ec56537c 2022-12-15)` (#1717)
* Addressed clippy warnings from `clippy 0.1.67 (ec56537c 2022-12-15)`

* Rust 1.59.0 support
2022-12-22 09:07:01 -08:00