This PR provides documentation of Bazelmod and several code examples
that addresses a number of issues related to Bazelmod.
Preview of the documentation:
https://github.com/marvin-hansen/rules_rust/blob/main/docs/crate_universe_bzlmod.md
First and foremost it paves the way for a meaningful update the Bazelmod
documentation that references these and existing code examples. This
touches at least the following issues:
* #2670
* #2181
The compile_opt example addresses or resolves:
* #515
* #2701
The musl_cross_compilling example addresses or resolves
* #390
* #276
The oci_container does not relate to any open issue,
although the tokio example in it gives a nice end to end example so
this definitely helps those looking for something non-trivial.
The proto example addresses or resolves:
* #2668
* #302
* #2534
* Possibly a few more if I were to search longer
Formalities
* I've signed the CLA
* I've signed all commits
---------
Signed-off-by: Marvin Hansen <marvin.hansen@gmail.com>
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
- Create `bzlmod/hello_world` package based upon
`examples/bzlmod/hello_world`.
- Add `rules_bazel_integration_test` so that `//bzlmod:hello_world_test`
can be executed as a test from the parent workspace.
- Add `filegroup` targets named `all_files` to collect files for all of
the packages that are necessary to use `rules_rust`. These are an input
to the `rules_bazel_integration_test` test.
- Replace `.bcr/presubmit.yml` to test `//bzlmod:hello_world_test`.
- Update CI to test `//bzlmod:hello_world_test`.
Related to
https://github.com/bazelbuild/bazel-central-registry/pull/1199#discussion_r1429129814.
Initial support for `no_std` mode.
This allows us to
1. Don't pass the whole standard library to compile actions that specify `no_std`
2. Conditionally select `crate_features` and `deps` based on whether `no_std` mode is used.
Currently the only supported modes are `off` and `alloc`, with a possibility to expand in the future.
The `no_std` support comes with the following caveats:
1. Targets in `exec` mode are still built with `std`; the logic here being that if a device has enough space to run bazel and rustc, std's presence would not be a problem. This also saves some additional transitions on `proc_macro`s (they need `std`), as they are built in `exec` mode.
2. Tests are still built with `std`, as `libtest` depends on `libstd`
There is quite an ugly hack to make us be able to `select` on the `no_std` flavor taking `exec` into account; I'm looking forward to the day where Bazel will expose better ways to inspect the cfg.
There is also one part I didn't manage to make work - having a `rust_test` that tests the `rust_shared_library` in `cc_common.link` mode; I got a link error for missing `__rg_alloc` & co. symbols, which should be present as we pass `--@rules_rust//rust/settings:experimental_use_global_allocator=True`. Unfortunately I could only spot this error on CI, and could not reproduce locally. I removed the test because the `rust_shared_library` is already tested via a `cc_test`. I will however give another shot at inspecting how my local setup differs from CI.
The `rust_binary` source code in `main.rs` was borrowed from https://github.com/jfarrell468/no_std_examples, big thanks to @jfarrell468 for letting me use it.
Co-authored-by: Krasimir Georgiev <krasimir@google.com>
Co-authored-by: UebelAndre <github@uebelandre.com>
Previously running `bazel build ...` in examples/cargo_manfest_dir/external_crate
would leave the following untracked files:
Untracked files:
(use "git add <file>..." to include in what will be committed)
examples/cargo_manifest_dir/external_crate/bazel-bin
examples/cargo_manifest_dir/external_crate/bazel-external_crate
examples/cargo_manifest_dir/external_crate/bazel-out
examples/cargo_manifest_dir/external_crate/bazel-testlogs
I noticed that of the 5 workspaces found by `find . -name WORKSPACE.bazel`
three of them were ignoring "bazel-*" via the top-level .gitignore while
one had its own .gitignore containing only "bazel-*". I consolidated
that one into the top-level .gitignore and added the 5th.
* Emit rust_analyzer_crate_spec files for all workspace crates
* Regenerated cargo-raze outputs
* Address PR feedback
* Increase Bazel min test version
* Addressed PR feedback
* Make sysroot_src a data dependency
* Fix runfiles path specifier
* Use bazel-like space separated target specification
* Fix out-of-date comment
* Addressed PR feedback
* Added additional testing
* Added test comment
Co-authored-by: David Marcin <david@metawork.com>
* Added bootstrap script for crate_universe and github action for auto-releasing new binaries
* Addressed PR feedback
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
* Updated local dev documentation
* Add announce_rc to crate_universe.bazelrc
* Addressed buildifier defect
* Addressed additional user feedback
* Apply suggestions from code review
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
* Fixed missing extension for windows.
Co-authored-by: Daniel Wagner-Hall <dawagner@gmail.com>
* Added test script for ensuring docs are up to date.
* Fixed bug in doc attribute formatting.
* Regenerate documentation
* `_tidy` now ignores empty first lines
* Updated comment
* Regenerate documentation
This rule dynamically looks up crate dependencies as a repository rule.
See https://github.com/bazelbuild/rules_rust/issues/2 for context.
Note that this rule is currently experimental, and offers absolutely no stability guarantees. Expect significant churn in near future.
This PR achieves the following:
* buildifier bzl-visibility check was not detecting violations as it assumes private package, not just a directory
* adds //rust:common.bzl file where all the publicly available and supported API for writing custom rules interacting with Rust will reside
* the private package now sets the stability expectations, and it forces us to think about API layering and abstractions.
This turned out to be much bigger PR than expected :(
The current documentation was difficult to maintain and not in sync with the inline one.
These changes adjust the setting so we can generate the documentation in markdown format and it will continue to be served on Github pages, this time with an easy way to sync it.
* Also add the dependencies in /examples/WORKSPACE
This allow to runs the tests in examples as a remote repository
and as a local repository.
* Add a dummy Cargo craze to use cargo raze on it.
With this craze, and cargo raze, we can simply use raze generated repositories
to add the necessary dependencies.
* Add generated files for the raze package of protobuf
* Add rust_(proto|grpc)_library
Those library add protobuf / gRPC support to Rust language in Bazel
using the native proto_library. It generates one crate per proto_library
which is efficient for incrementality but can lead to naming collisions.
We expose a bit of rust_library internal to re-use it in rust_proto_library
and create a new provider "rust_libs" that will declare a group of rust libraries.
* Add a gRPC client/server example
And a test with it.
* Add documentation for the rust_(proto|grpc)_library
* Use the new runfiles library in the test rather than a custom algorithm
* Add the missing dependencies to examples/WORKSPACE
This new dependency is quite unfortunate, it make using rules_rust
more complex since it needs to import a bzl file from skylib to be
able to use rules_rust, that adds 10 lines in the WORKSPACE without
much improvement. I guess this cannot be better until recursive
workspace are a thing.
* Generates cargo raze files compatible with macOS and Linux
Also to keep the same lockfile, check in the Cargo.lock and have
a lever to not re-run cargo generate-lockfile.
* Compile all proto stubs in one library
This a workaround a bug in Bazel that prevent the toolchain from being
loaded with the target transition (see
https://github.com/bazelbuild/bazel/pull/5650).
This is not ideal because it recompiles several time the same
proto if you depends on it from several dependency path
and can lead to seeing two compilation of the same proto
as different type. This is also terrible for incremental build
and contrary to the (protobuf blog
entry)[https://blog.bazel.build/2017/02/27/protocol-buffers.html].
* Force set the lazy_static implementation configuration.
The lazy_static crates now set a configuration from the build.rs, ideally we would be
able to handle that but that is not the case with the current cargo raze method.
* Final touch: documenting and simplifying toolchain creation
- Documents how to create your custom toolchain when you need to import
your own crates (or to use a different protobuf/grpc stubs compiler).
- Add extra_aliased_targets to simply the cargo-raze generation
- Use skylark http_archive to import protobuf.
- Regenerate crates with google/cargo-raze#77 so that we don't need to
have custom rewriting policy. Hence, deleted the wrapper around
cargo raze, an update to the build file should simply be
`cargo genrate-lockfiles && cargo raze` now.
* Fix nits suggested by @mfarrugi
* Remove usage of CrateInfos
This was a leftover from compiling the crates in the proto aspect.
* Use python for the optional_output_wrapper
Also move the optional_output_wrapper from the toolchain to the library
itself.
For some reason the runfiles where not shipped to the actions with the
old way which wasn't important for Bash that did not had any runfiles
but it is important for python.
* fix: README format
* Second pass of nits from @mfarrugi
* fix: add back the list of grpc stubs to create if not created
Marvin Hansen