Commit Graph

403 Commits

Author SHA1 Message Date
Malte Poll 1697a3275b
fix: coreutils toolchain: Use statically linked linux amd64 variant (#706)
* coreutils toolchain: Use statically linked linux amd64 variant

Uutils has a musl release artifact for linux amd64.
In the future, it should probably also be possible to add a
aarch64 musl toolchain. At the moment, this is not an upstream release
artifact.

* coreutils toolchain: temporarily add back old darwin variant

On release 0.0.26 of uutils/coreutils, the darwin x86_64 binary is missing.
Also, any releases between 0.0.23 and 0.0.26 are missing binary artifacts.
Downgrade coreutils toolchain on darwin x86_64 for now.

https://github.com/uutils/coreutils/releases/tag/0.0.26
2024-05-08 07:06:12 -07:00
Alex Eagle b15dc31a81
fix(tar): handle spaces in input filenames (#835) 2024-05-07 17:52:35 -07:00
Sahin Yort d1d063f3e5
feat: introduce zstd toolchain (#831) 2024-05-03 16:12:56 -07:00
Alex Eagle 977f27f7a0
feat(tar): add ergonomic way to strip_prefix (#829) 2024-05-01 12:36:39 -07:00
Greg Magolan 41413388da
chore: add bazel test support to javascript --config=debug preset (#825) 2024-04-25 16:33:16 -07:00
Greg Magolan 5cc1a3a1be
chore: bump to bsdtar-prebuilt 3.7.2-1 binaries for linux and darwin (#819) 2024-04-20 15:34:39 -07:00
Thomas Lam faaada2eaa
feat: add platform_transition_test (#770)
* Add platform_transition_test

Signed-off-by: Thomas Lam <thomaslam@canva.com>

* Set target platform constraints for tests

Signed-off-by: Thomas Lam <thomaslam@canva.com>

* Update docs

Signed-off-by: Thomas Lam <thomaslam@canva.com>

---------

Signed-off-by: Thomas Lam <thomaslam@canva.com>
2024-04-03 14:42:07 -07:00
Greg Magolan 7716549ab9
feat: export utils from //lib:utils.bzl to match repo_utils and platform_utils (#810) 2024-04-03 14:00:44 -07:00
Greg Magolan abf56a6454
feat: add get_home_directory to repo_utils (#809) 2024-04-03 13:53:18 -07:00
Greg Magolan 41a9295f07
feat: add //lib:enable_runfiles config_setting (#807) 2024-04-01 07:27:38 -07:00
Alex Eagle f4f588f4ea
Use statically-linked bsdtar on all platforms (#804)
* [tar] Switch to statically linked binaries

* chore: don't use bsdtar for extract yet

* chore: add dzbarsky integrity hashes

* Upgrade to 3.7.2.bcr.2 (#806)

* chore: replace URLs with aspect-build fork

---------

Co-authored-by: David Zbarsky <dzbarsky@gmail.com>
2024-03-29 11:35:56 -07:00
Joy Gao 624e8acac5
clean rm of copied dirs and files (#799) 2024-03-27 10:02:47 -04:00
Alex Eagle fea9515087
feat: don't require 'out' on expand_template (#798)
In a lot of cases the name of the generated file is unimportant.
For example in https://github.com/bazel-contrib/rules_oci/pull/534 I wanted to remove 'out' in a bunch of these calls.
2024-03-26 16:57:16 -07:00
Alex Eagle 0fc838839c
feat: add a helper for rules to work with resource_sets (#792)
* feat: add a helper for rules to work with resource_sets

This API is poorly designed and needs some help to let rule users pick a value in cases that they aren't also the rule author

* chore: add some cpu resource_set values as well
2024-03-18 15:38:24 -07:00
Sahin Yort a29dd93c0b
fix: srcs is not mandatory (#786) 2024-03-08 10:47:38 -08:00
Sahin Yort d93c87fbbd
fix: tar#srcs is optional (#785) 2024-03-07 17:44:35 -08:00
Greg Magolan fdb7bf3a56
docs: add warning about using use_default_shell_env (#779) 2024-03-05 10:33:05 -08:00
Greg Magolan 73e620161a
docs: add use_default_shell_env to docstring of run_binary (#778) 2024-03-05 10:27:02 -08:00
Sahin Yort 197b2da974
feat: support location expansion in tar (#774) 2024-03-01 14:51:47 -08:00
Matt Mackay aa6a58f73c
chore: bump libxml2 deb to USN-6658-1 (#768) 2024-02-27 13:10:08 -05:00
Marc Redemske a3d7efe298
feat(list): add `unique` function (#716)
* feat(list): add `unique` function

* Update lists.bzl

* Update lists.bzl

* Update lists.bzl

* Update lists.bzl

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2024-02-27 08:25:00 -08:00
Mike Lundy dfcffca8f5
jq: add data file and location expansion support (#757)
This teaches the jq rule about a `data` attribute and adds conditional
support to do bazel location expansion. The expansion is disabled by
default because it likely would cause compat concerns (since jq can take
arbitrary text as input args).
2024-02-26 17:43:56 -08:00
Jason Bedard 8e5a86f8b8 docs: add bats docs 2024-02-26 15:55:48 -08:00
Tiago Quelhas e514283dd8 Remove unnecessary ctx.resolve_tools.
`ctx.actions.run(executable = ...)` is sufficient to include the runfiles in the action inputs. (More generally, ctx.resolve_tools is discouraged in favor of `ctx.actions.run(tools = ...)`.)
2024-02-25 17:43:38 -08:00
Alex Eagle 1c98ca9f34
chore: make it possible to override run_binary#use_default_shell_env (#762)
Still not documented as public API. We think this leads to non-hermeticity and could cause cache misses, so it must be used with care.

Part of https://github.com/aspect-build/rules_js/issues/1303
2024-02-21 13:37:47 -08:00
Alex Eagle 6d3195f05c
presets: remove eternal test_timeout_filtering (#758)
In practice we've disabled this setting at some clients, because there's not a surrounding workflow to find these targets are being silently dropped from CI, nor run them on another schedule.

We can re-introduce something better when we add Test Selection to Aspect Workflows.
2024-02-16 13:56:38 -08:00
Alex Eagle 38fecbcbb5
Update tar.bzl (#751)
* Update tar.bzl

Fix header so it's not presented at the same level as the parent (the `tar` macro)

* fix docs
2024-02-08 15:26:05 -08:00
Greg Magolan 71a8052589
fix: merge user tags in stardoc_with_diff_test (#746) 2024-02-05 17:43:58 -08:00
renovate[bot] e19315230c chore(deps): update dependency bazel_skylib to v1.5.0 2024-01-30 00:18:44 -08:00
Matt Mackay 59cc7058de
perf: remove unnecessary calls to 'to_list' (#737) 2024-01-29 12:43:32 -05:00
Alex Eagle f0a19f5a7c
Update paths.bzl (#735)
Add a more helpful error when runfiles.bash can't be found.
2024-01-27 12:26:12 -08:00
Eric Hauser ee0442db9c
FR-723: make WriteSourceFileInfo public (#724)
* FR-723: make WriteSourceFileInfo public

* Update docs

---------

Co-authored-by: Greg Magolan <greg@aspect.dev>
2024-01-22 21:51:05 -08:00
Greg Magolan 1baf14f65c
chore: add example of using write_source_file to merge output tree artifact files with source files that are already in an output directory (#730) 2024-01-22 21:03:31 -08:00
Alex Eagle 58b65c1736
chore: upgrade bash rlocation lib to v3 (#727) 2024-01-18 13:23:14 -08:00
Greg Magolan 5b3b7d7794
feat: allow write_source_file(s) to write source files to bazel packages outside of the target's package (#717) 2024-01-10 15:07:03 -08:00
Greg Magolan 57bcf288bd
chore: fix failing test on arm64 machines (#718) 2024-01-10 11:16:37 -08:00
Greg Magolan c9d08234ce
refactor: remove --enable-runfiles from javascript recommended bazelrc settings (#715) 2024-01-07 21:29:45 -08:00
Greg Magolan 516a5fe725
feat: add is_bazel_7_or_greater utility (#714) 2024-01-07 16:42:57 -08:00
Greg Magolan 5aaa785b96
chore: test against bazel 7.0.0 (#713) 2024-01-07 16:12:05 -08:00
Alex Eagle 1be6994ca6
Fix prerelease (#708)
* fix: use source toolchains for git archive prereleases

* Add comment about "g" prefix
2024-01-04 12:07:19 -08:00
Greg Magolan d313c8bf75
fix: move flags that should only be set with bazel 6 to bazel6.bazelrc recommended settings (#711) 2024-01-04 08:16:59 -08:00
Sahin Yort ac0055e092
test: fix timeout (#707) 2023-12-22 11:44:44 -08:00
Sahin Yort 33bc4443fd
fix: create windows launcher for bats (#704) 2023-12-21 14:27:21 -08:00
Sahin Yort 4f6b4bd5cb
feat: implement bats test runner (#699) 2023-12-20 13:08:47 -08:00
Tobias Schlatter 8d805e16bf
fix #697: use to_repository_relative_path in mtree_spec (#696)
* Reproduction for mtree_spec behaving different in other repo

* Fix the bug (but not the test)

* Fix the test

* Rename mtree_spec e2e dir to tar

* Fix bzlmod tests

* Remove unnecessary skylib dependency from e2e repos

* Add e2e tests to matrix

* Replace e2e test with a normal test

To do this, we somewhat abuse the skylib dependency, but that's
probalby OK.
2023-12-20 11:22:42 -08:00
Alex Eagle 303779e9ef
fix(tar): propagate testonly attr to mtree_spec (#691) 2023-12-13 15:21:02 -08:00
Alex Eagle f65019be4e
chore: improve docs about mtree mutation (#692) 2023-12-13 15:03:32 -08:00
Sahin Yort cef25915d7
feat: introduce extension compression mapping (#683) 2023-12-08 21:47:20 +00:00
Sahin Yort a219f5260d
fix: expose tar_lib as public (#680) 2023-12-08 10:31:01 -08:00
Greg Magolan b1c342a96f
chore: bazel run //:format (#682) 2023-12-08 00:30:39 -08:00
Sahin Yort d1642aa020
feat: expose common tar variables (#679) 2023-12-07 12:29:35 -08:00
Tobias Schlatter 975a933e24
docs: fix load path for copy_file_to_bin_action (#673) 2023-11-30 10:58:48 -08:00
Derek Cormier 2b38ad5d29
feat: expose toolchains used for copy actions (#661) 2023-11-16 07:15:50 -08:00
Derek Cormier 95e7ad5e0b
fix: fix a bug where toolchain repositories were potentially duplicated (#662) 2023-11-16 06:57:29 -08:00
Derek Cormier 5bd6e5fdd4
fix(ci): fix bzlmod issues and enable on ci (#658) 2023-11-15 15:07:03 -08:00
David Zbarsky 01ca8f9432
Use coreutils toolchain for copy_file action (#622)
refactor: use coreutils toolchain for copy_file action
---------

Co-authored-by: Greg Magolan <greg@aspect.dev>
Co-authored-by: Derek Cormier <derek@aspect.dev>
2023-11-14 11:03:51 -08:00
David Zbarsky b74a45456d
Upgrade coreutils and add darwin arm64 binary (#649) 2023-11-14 08:47:21 -08:00
Greg Magolan 602b7b8f80
Revert: feat: expose a config_setting for copy execution_requirements (#606) (#640) 2023-10-31 15:19:38 -07:00
Greg Magolan a47eebfa65
fix: fix execution requirements for 'build without the bytes' (#639) 2023-10-31 13:38:32 -07:00
Alex Eagle 8cf7e6f995
feat: support treeartifacts (#630) (#631)
Co-authored-by: Sahin Yort <thesayyn@gmail.com>
2023-10-19 15:57:48 -07:00
David Zbarsky a4f99946c6 Unconditionally use runfiles.merge_all in write_source_file 2023-10-16 15:45:24 -07:00
Derek Cormier 5077d5ae4a
chore: remove bazel5 presets (#614) 2023-10-16 13:35:41 -07:00
Alex Eagle eda4929c72
chore: add windows binaries (#610)
* chore: add windows binaries

* chore: fix/exclude windows brokenness

* chore: try to see why diff tests fail on windows

* fix: rm bazelisk rc again for windows

* fix: try our own diff_test

* chore: use only our own diff_test
2023-10-10 14:13:17 -07:00
Alex Eagle 177b883991
refactor: use _mtree_line helper (#612)
Also restore the .bazeliskrc file I accidentally removed in 4bfe55711a
2023-10-10 12:33:21 -07:00
Derek Cormier 20f1ac39e0
refactor: rework toolchain registration for WORKSPACE and bzlmod users (#597) 2023-10-10 12:16:13 -07:00
Alex Eagle e1d9d9b58f fix: windows home directory (#608)
HOMEPATH isn't always set. For example https://buildkite.com/bazel/bazel-bazel-examples/builds/2163#018b1a9d-0127-461f-8200-e89894397f35
USERPROFILE seems more widely documented as being the equivalent of HOME
2023-10-10 10:57:18 -07:00
Douglas Parker 323329f119 fix: always include files from the same workspace as the build target in `copy_to_directory()` (#360)
* fix: always include files from the same workspace as the build target in `copy_to_directory`

Fixes #359.

This updates the `copy_to_directory` tool to accept a workspace name representing the workspace of the target it is executing under. Any files in this workspace are automatically included, regardless of the `include_external_repositories` option. This makes it support usage within an external target (such as `@wksp//:dir`).

* test: add e2e test which uses `copy_to_directory` within an external workspace

Refs #359.

This should catch regressions where no files are copied when built within an external workspace and not using `include_external_repositories`.

* ci: fix stray workspace refs

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2023-10-10 10:57:08 -07:00
Alex Eagle 472bf9b122
feat: tar includes runfiles (#595)
* feat: tar includes runfiles

* chore: try to fix red circleci

* fix: tracked down problem

* chore: document tar#srcs supports runfiles

* chore: add comment about logic for trimming manifest suffix

* chore: missed a replacement spot

* chore: give up on the listing test for now
2023-10-09 15:57:52 -07:00
Alex Eagle 4bfe55711a
feat: expose a config_setting for copy execution_requirements (#606)
* feat: expose a config_setting for copy execution_requirements

Fixes #604

* chore: add user docs

* chore: improve docs

* chore: better link to copy_file
2023-10-09 13:57:34 -07:00
Alex Eagle bc97305fcd Revert "refactor: reduce execution requirements for copy" (#594)
* Revert "refactor: reduce execution requirements for copy"

This reverts commit 40c2ddc71a.

* Update ci.bazelrc
2023-10-06 19:19:18 -07:00
Greg Magolan 1cd2eb7e0a
refactor: remove exclude_prefixes from copy_to_directory and copy_to_directory_bin_action (#591) 2023-10-06 10:17:33 -07:00
Greg Magolan cb9b74f41e
refactor: remove to_workspace_path and to_manifest_path (#590) 2023-10-06 08:37:53 -07:00
Greg Magolan 231dad5c92
refactor: remove output_dir from run_binary and expand_variables (#588) 2023-10-06 08:32:55 -07:00
Greg Magolan 3a16b4de26
refactor: remove legacy copy_directory_action helper (#589) 2023-10-06 08:32:25 -07:00
Alex Eagle 894ba445c4
chore: remove is_windows attributes (#587)
Fixes some TODOs for 2.0
2023-10-05 16:03:15 -07:00
Derek Cormier 99f36cc298 feat(presets): always fetch all coverage files on coverage runs (#564) 2023-10-05 15:57:37 -07:00
Derek Cormier dca2b1df0c
chore: remove legacy copy_to_directory_action (#582) 2023-10-05 15:52:37 -07:00
Alex Eagle 6a4381bf07
chore: run gazelle (#584)
Also delete the local_config_platform workaround for bazel 5 which made our docs setup complex
2023-10-05 14:22:38 -07:00
David Zbarsky 8fe4f6f8d5
Avoid expanding mtree spec during analysis phase (#576)
* Avoid expanding mtree spec during analysis phase

* Update tar.bzl

remove comment and unused code

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2023-10-05 12:36:55 -07:00
Derek Cormier 46a16e500c
chore: remove expand_template re-export (#580) 2023-10-05 12:33:13 -07:00
Alex Eagle a283a8216d feat: add a tar toolchain (#468)
* feat: add a BSD tar toolchain

@thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar
including fine-grained file permissions and symlinks:
https://man.freebsd.org/cgi/man.cgi?mtree(8)

* show example of mtree usage

* feat: introduce tar rule

* cleanup and get test passing

* more cleanup

* chore: add support for compress flags

* chore: add docs

* chore: add docs

* feat: implement linux bsdtar toolchain (#566)

* chore: improve target naming

* WIP: args

* feat: generate mtree spec

Also allow arbitrary args

* refactor: mtree is required

* refactor: style nits

* fix: support mix of source and generated artifacts

* feat: demonstrate strip_prefix

* chore: regen docs

* fix: make host toolchain a fallback toolchain

* fix: include libarchive13.so when installing BSD tar

* chore: buildifier

* fix: aarch64 cpu constraint

* fix(ci): include libarchive13.so when running tar

* chore: add libnettle

* refactor: inputs mutated less

* refactor: remove unneeded substitution arg

* refactor: don't advertise unsupported modes

* fix: hack enough to make it run on my machine

* chore: dynamic libraries included in sh_binary under toolchain

* make sh_binary work

* refactor: drop arm64 for now

* fix toolchain

* fix test

* chore: improve test naming scheme

---------

Co-authored-by: Sahin Yort <thesayyn@gmail.com>
2023-10-03 13:50:55 -07:00
Derek Cormier 174bb9425d feat: support jq 1.7 toolchain (#520) 2023-10-03 13:50:55 -07:00
Alex Eagle e00ea2b977
chore: update pre-commit buildifier (#563)
It needs to match the one CI runs in
https://github.com/aspect-build/bazel-lib/actions/runs/6357283303/job/17268197322
2023-09-29 14:42:33 -07:00
Greg Magolan bcb10f8154
feat: add undeclared output mode to expand_template (#561) 2023-09-29 11:30:02 -07:00
Greg Magolan 1d816e011b
feat: add assert_directory_contains test to testing.bzl (#560) 2023-09-29 09:53:00 -07:00
Alex Eagle bc54655e71
ci: add assertion that bcr patches apply (#556) 2023-09-27 10:26:38 -07:00
Aaron Son a4e997de1f
feat: add `propagate_common_{,test_,binary_}rule_attributes` to lib/utils. (#553)
* feat: add `propagate_common_{,test_,binary_}rule_attributes` to lib/utils.

* lib/private/utils: PR feedback: extract COMMON_BINARY_RULE_ATTRIBUTES and COMMON_TEST_RULE_ATTRIBUTES to global constants.

* lib/private/utils: PR feedback: remove default value for attrs param to propagate_common_*rule_attributes

* fix: lib/private/utils: buildifier.
2023-09-26 14:29:42 -07:00
Sahin Yort ef364b54b4
refactor: consume tools from source if unstamped (#543) 2023-09-24 15:06:10 -07:00
Matt Mackay 8fa2127518
fix: allow for BUILD files in outputs of write_source_file (#540) 2023-09-21 19:50:04 -04:00
Sahin Yort 719b8f33b0
refactor: add ability to test against tools from source (#532) 2023-09-21 15:12:46 -07:00
Matt Mackay 62394fbb68
fix: lookup source files from execroot when running diff_test on RBE (#531) 2023-09-21 13:27:08 -07:00
Marc Redemske 45986f000d
feat: add list utils (#512)
* feat: add list utils

* chore: add docs for lists

* fixes accoding to review

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2023-09-21 10:12:47 -07:00
Alex Eagle 882bc95615
feat: expand_template allows inline template content (#533)
Co-authored-by: thesayyn <thesayyn@gmail.com>
2023-09-20 18:32:05 -07:00
Ted Kaplan fc3c3181f3
docs: Add --incompatible_disallow_empty_glob to correctness.bazelrc (#524) 2023-09-18 19:58:27 -07:00
Alex Eagle d2203e9020
chore: pick up new expand_template go binary (#518)
Also fix an error that was getting reported in our release pipeline
2023-09-06 16:47:15 -07:00
peter woodman 3e9577ab16
yq: allow any filename in srcs (#496)
* yq: allow any filename in srcs

* yq: add test cases for free-form file extensions in input

* run buildifier

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2023-08-30 09:18:10 -07:00
Alex Eagle 40c2ddc71a refactor: reduce execution requirements for copy
It causes warnings like 753cd459e4/src/main/java/com/google/devtools/build/lib/sandbox/SandboxModule.java (L514)
and also makes toolchain selection broken for host!=exec

Fixes #466
2023-08-29 14:56:22 -07:00
Greg Magolan 76ef6c8988
chore: add test coverage for run_binary & expand_variables makevars (#504) 2023-08-26 13:59:48 -07:00
Matt Mackay ef4830bf25
docs: add comment regarding linux-arm64 JQ toolchain (#503) 2023-08-24 09:54:44 -07:00
Steven E. Harris 3af9e00308
docs: clarify write_source_files output constraint (#498)
* docs: clarify write_source_files output constraint

For both the "write_source_files" rule's "files" attribute and the
"write_source_file" rule's "out_file" attribute, we ensure that the
destination files and directories to which we'll write sit within the
same containing Bazel package as the Bazel target requesting this
writing. Clarify the documentation for each attribute to make it clear
that we're constraining the destinations and not the sources of the
content to be written.

* Update lib/write_source_files.bzl

---------

Co-authored-by: Derek Cormier <derek@aspect.dev>
2023-08-23 12:32:56 -07:00