2
0
Fork 0
mirror of https://github.com/bazel-contrib/bazel-lib synced 2024-11-30 01:41:21 +00:00
bazel-lib/docs/copy_to_directory.md
Douglas Parker c6b39ab8f6
fix: always include files from the same workspace as the build target in copy_to_directory() (#360)
* fix: always include files from the same workspace as the build target in `copy_to_directory`

Fixes #359.

This updates the `copy_to_directory` tool to accept a workspace name representing the workspace of the target it is executing under. Any files in this workspace are automatically included, regardless of the `include_external_repositories` option. This makes it support usage within an external target (such as `@wksp//:dir`).

* test: add e2e test which uses `copy_to_directory` within an external workspace

Refs #359.

This should catch regressions where no files are copied when built within an external workspace and not using `include_external_repositories`.

* ci: fix stray workspace refs

---------

Co-authored-by: Alex Eagle <alex@aspect.dev>
2023-10-10 07:16:13 -07:00

22 KiB
Generated

Copy files and directories to an output directory.

NB: See notes on copy_file regarding execution_requirements settings for remote execution. These settings apply to the rules below as well.

copy_to_directory

copy_to_directory(name, allow_overwrites, exclude_prefixes, exclude_srcs_packages,
                  exclude_srcs_patterns, hardlink, include_external_repositories,
                  include_srcs_packages, include_srcs_patterns, out, replace_prefixes, root_paths,
                  srcs, verbose)

Copies files and directories to an output directory.

Files and directories can be arranged as needed in the output directory using the root_paths, include_srcs_patterns, exclude_srcs_patterns and replace_prefixes attributes.

Filters and transformations are applied in the following order:

  1. include_external_repositories

  2. include_srcs_packages

  3. exclude_srcs_packages

  4. root_paths

  5. include_srcs_patterns

  6. exclude_srcs_patterns

  7. replace_prefixes

For more information each filters / transformations applied, see the documentation for the specific filter / transformation attribute.

Glob patterns are supported. Standard wildcards (globbing patterns) plus the ** doublestar (aka. super-asterisk) are supported with the underlying globbing library, https://github.com/bmatcuk/doublestar. This is the same globbing library used by gazelle. See https://github.com/bmatcuk/doublestar#patterns for more information on supported globbing patterns.

ATTRIBUTES

Name Description Type Mandatory Default
name A unique name for this target. Name required
allow_overwrites If True, allow files to be overwritten if the same output file is copied to twice.

The order of srcs matters as the last copy of a particular file will win when overwriting. Performance of copy_to_directory will be slightly degraded when allow_overwrites is True since copies cannot be parallelized out as they are calculated. Instead all copy paths must be calculated before any copies can be started.
Boolean optional False
exclude_prefixes List of path prefixes (with glob support) to exclude from output directory.

DEPRECATED: use exclude_srcs_patterns instead

Files in srcs are not copied to the output directory if their output directory path, after applying root_paths, starts with or fully matches one of the patterns specified.

Forward slashes (/) should be used as path separators.

Files that do not have matching output directory paths are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be.

Globs are supported (see rule docstring above).
List of strings optional []
exclude_srcs_packages List of Bazel packages (with glob support) to exclude from output directory.

Files in srcs are not copied to the output directory if the Bazel package of the file matches one of the patterns specified.

Forward slashes (/) should be used as path separators. A first character of "." will be replaced by the target's package path.

Files that have do not have matching Bazel packages are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be.

Globs are supported (see rule docstring above).
List of strings optional []
exclude_srcs_patterns List of paths (with glob support) to exclude from output directory.

Files in srcs are not copied to the output directory if their output directory path, after applying root_paths, matches one of the patterns specified.

Forward slashes (/) should be used as path separators.

Files that do not have matching output directory paths are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be.

Globs are supported (see rule docstring above).
List of strings optional []
hardlink Controls when to use hardlinks to files instead of making copies.

Creating hardlinks is much faster than making copies of files with the caveat that hardlinks share file permissions with their source.

Since Bazel removes write permissions on files in the output tree after an action completes, hardlinks to source files are not recommended since write permissions will be inadvertently removed from sources files.

- auto: hardlinks are used for generated files already in the output tree - off: all files are copied - on: hardlinks are used for all files (not recommended)
String optional "auto"
include_external_repositories List of external repository names (with glob support) to include in the output directory.

Files from external repositories are only copied into the output directory if the external repository they come from matches one of the external repository patterns specified or if they are in the same external repository as this target.

When copied from an external repository, the file path in the output directory defaults to the file's path within the external repository. The external repository name is not included in that path.

For example, the following copies @external_repo//path/to:file to path/to/file within the output directory.

 copy_to_directory(     name = "dir",     include_external_repositories = ["external_*"],     srcs = ["@external_repo//path/to:file"], ) 


Files that come from matching external are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be. The external repository name of the file from an external repository is not included in the output directory path and is considered in subsequent filters and transformations.

Globs are supported (see rule docstring above).
List of strings optional []
include_srcs_packages List of Bazel packages (with glob support) to include in output directory.

Files in srcs are only copied to the output directory if the Bazel package of the file matches one of the patterns specified.

Forward slashes (/) should be used as path separators. A first character of "." will be replaced by the target's package path.

Defaults to ["**"] which includes sources from all packages.

Files that have matching Bazel packages are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be.

Globs are supported (see rule docstring above).
List of strings optional ["**"]
include_srcs_patterns List of paths (with glob support) to include in output directory.

Files in srcs are only copied to the output directory if their output directory path, after applying root_paths, matches one of the patterns specified.

Forward slashes (/) should be used as path separators.

Defaults to ["**"] which includes all sources.

Files that have matching output directory paths are subject to subsequent filters and transformations to determine if they are copied and what their path in the output directory will be.

Globs are supported (see rule docstring above).
List of strings optional ["**"]
out Path of the output directory, relative to this package.

If not set, the name of the target is used.
String optional ""
replace_prefixes Map of paths prefixes (with glob support) to replace in the output directory path when copying files.

If the output directory path for a file starts with or fully matches a a key in the dict then the matching portion of the output directory path is replaced with the dict value for that key. The final path segment matched can be a partial match of that segment and only the matching portion will be replaced. If there are multiple keys that match, the longest match wins.

Forward slashes (/) should be used as path separators.

Replace prefix transformation are the final step in the list of filters and transformations. The final output path of a file being copied into the output directory is determined at this step.

Globs are supported (see rule docstring above).
Dictionary: String -> String optional {}
root_paths List of paths (with glob support) that are roots in the output directory.

If any parent directory of a file being copied matches one of the root paths patterns specified, the output directory path will be the path relative to the root path instead of the path relative to the file's workspace. If there are multiple root paths that match, the longest match wins.

Matching is done on the parent directory of the output file path so a trailing '**' glob patterm will match only up to the last path segment of the dirname and will not include the basename. Only complete path segments are matched. Partial matches on the last segment of the root path are ignored.

Forward slashes (/) should be used as path separators.

A "." value expands to the target's package path (ctx.label.package).

Defaults to ["."] which results in the output directory path of files in the target's package and and sub-packages are relative to the target's package and files outside of that retain their full workspace relative paths.

Globs are supported (see rule docstring above).
List of strings optional ["."]
srcs Files and/or directories or targets that provide DirectoryPathInfo to copy into the output directory. List of labels optional []
verbose If true, prints out verbose logs to stdout Boolean optional False

copy_to_directory_action

copy_to_directory_action(ctx, srcs, dst, additional_files, root_paths,
                         include_external_repositories, include_srcs_packages, exclude_srcs_packages,
                         include_srcs_patterns, exclude_srcs_patterns, exclude_prefixes,
                         replace_prefixes, allow_overwrites, is_windows)

Legacy factory function to copy files to a directory.

This helper calculates copy paths in Starlark during analysis and performs the copies in a bash/bat script. For improved analysis and runtime performance, it is recommended the switch to copy_to_directory_bin_action which calculates copy paths and performs copies with a tool binary, typically the @aspect_bazel_lib//tools/copy_to_directory go_binary either built from source or provided by a toolchain.

This helper is used by copy_to_directory. It is exposed as a public API so it can be used within other rule implementations where additional_files can also be passed in.

PARAMETERS

Name Description Default Value
ctx The rule context. none
srcs Files and/or directories or targets that provide DirectoryPathInfo to copy into the output directory. none
dst The directory to copy to. Must be a TreeArtifact. none
additional_files List or depset of additional files to copy that are not in the DefaultInfo or DirectoryPathInfo of srcs []
root_paths List of paths that are roots in the output directory.

See copy_to_directory rule documentation for more details.
["."]
include_external_repositories List of external repository names to include in the output directory.

See copy_to_directory rule documentation for more details.
[]
include_srcs_packages List of Bazel packages to include in output directory.

See copy_to_directory rule documentation for more details.
["**"]
exclude_srcs_packages List of Bazel packages (with glob support) to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
include_srcs_patterns List of paths (with glob support) to include in output directory.

See copy_to_directory rule documentation for more details.
["**"]
exclude_srcs_patterns List of paths (with glob support) to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
exclude_prefixes List of path prefixes to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
replace_prefixes Map of paths prefixes to replace in the output directory path when copying files.

See copy_to_directory rule documentation for more details.
{}
allow_overwrites If True, allow files to be overwritten if the same output file is copied to twice.

See copy_to_directory rule documentation for more details.
False
is_windows Deprecated and unused None

copy_to_directory_bin_action

copy_to_directory_bin_action(ctx, name, dst, copy_to_directory_bin, files, targets, root_paths,
                             include_external_repositories, include_srcs_packages,
                             exclude_srcs_packages, include_srcs_patterns, exclude_srcs_patterns,
                             exclude_prefixes, replace_prefixes, allow_overwrites, hardlink, verbose,
                             override_execution_requirements)

Factory function to copy files to a directory using a tool binary.

The tool binary will typically be the @aspect_bazel_lib//tools/copy_to_directory go_binary either built from source or provided by a toolchain.

This helper is used by copy_to_directory. It is exposed as a public API so it can be used within other rule implementations where additional_files can also be passed in.

PARAMETERS

Name Description Default Value
ctx The rule context. none
name Name of target creating this action used for config file generation. none
dst The directory to copy to. Must be a TreeArtifact. none
copy_to_directory_bin Copy to directory tool binary. none
files List of files to copy into the output directory. []
targets List of targets that provide DirectoryPathInfo to copy into the output directory. []
root_paths List of paths that are roots in the output directory.

See copy_to_directory rule documentation for more details.
["."]
include_external_repositories List of external repository names to include in the output directory.

See copy_to_directory rule documentation for more details.
[]
include_srcs_packages List of Bazel packages to include in output directory.

See copy_to_directory rule documentation for more details.
["**"]
exclude_srcs_packages List of Bazel packages (with glob support) to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
include_srcs_patterns List of paths (with glob support) to include in output directory.

See copy_to_directory rule documentation for more details.
["**"]
exclude_srcs_patterns List of paths (with glob support) to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
exclude_prefixes List of path prefixes to exclude from output directory.

See copy_to_directory rule documentation for more details.
[]
replace_prefixes Map of paths prefixes to replace in the output directory path when copying files.

See copy_to_directory rule documentation for more details.
{}
allow_overwrites If True, allow files to be overwritten if the same output file is copied to twice.

See copy_to_directory rule documentation for more details.
False
hardlink Controls when to use hardlinks to files instead of making copies.

See copy_to_directory rule documentation for more details.
"auto"
verbose If true, prints out verbose logs to stdout False
override_execution_requirements specify execution_requirements for this action None

copy_to_directory_lib.impl

copy_to_directory_lib.impl(ctx)

PARAMETERS

Name Description Default Value
ctx

-

none