* Update to new stardoc so we can work with bzlmod.
* Requires update to merge.py to account for stardoc changes.
* put integration test back on bazel at head.
Fixes #808
* Fill out the CI matrix to cover the bazels that people use.
- bazel at head, LTS (7.x) and LTS-1
- with and without bzlmod enabled.
* add names to replicate the task names - CI should do this rather than making them
Get bzlmod CI working
* fix a lot of python which needs tochange
* make extension for rpmbuild
* restore macos CI to rolling
* better documentation on how to use rpmbuild.
* no docbuild with bzlmod -> stardoc problems
* disable a tar test that can not work with bzlmod
Next: Split rpmbuild to a separately distributable artifact
* try useing rules_cc to clean up CI runs which are failing on gcc"
* use their toolchains
* add rules_cc so the tests run
* do not do distro tests on rolling macos
* only tests tests/... for rolling_macos
* Correct for case where tar has prefix_dir and files/symlinks, but the
user has already mapped the prefix_dir into those paths. Under the old
behavior, prefix_dir was (incorrectly) not added to symlinks, so some
users made the desired prefix part of the link. Protecting them
against the double inclusion of the prefix is probably the least
surprising behavior.
Add tests for this, which required improving verify_archive_test.
* make buildifier happy
* do not run the symlink test on windows
* huh? why are symlinks in a tree failing on CI?
- Update to a newer rules_license
- update ci to prove that we can work with --incompatible_use_platforms_repo_for_constraints
- do not test with no_host_transition yet
* Create basic bzlmod setup for rules_pkg.
- Shows it working for one example
- Has only runtime deps
- rpm and git toolchains not done yet
- Still not sure how to get the external repo test working
- Make platforms and stardoc dependency deps.
* Restore RPM tests in CI; factor rpm test skipping int terms of `*skip_rpm`
During this month's public meeting, we discovered that none of the RPM packaging tests were not being run in CI. This looks like it was overlooked as of 85eabf2.
This change fixes this by leaving rpm tests off by default and adding `default_tests_with_rpm` for the platforms which can run rpmbuild.
* Improved doc generation: Produce a single reference doc.
- moves doc generation out of distro into a separate doc_build tree
- creates a single reference.md from the common attributes section
plus the individual page.
Future plans
- post transform the .html to add links between sections
- post transform the .html to add external links to Bazel docs.
- add procedures to cut doc releases alongside distribution releases
* Reorganize the source tree
- WORKSPACE moves up one level
- this makes //:pkg.bzl effectively //pkg:pkg.bzl
- put forwarders in place at top level to not break anyone
- tests moves up one level
- toolchains moves up one level
- examples/**/BUILD do not change to the new rule locations
- this is explicitly to provide some proof that the forwarders work
- new examples should use the proper paths to .bzl files
- near the 1.x release we can clean the current ones up
Fixes: #111
* Start testing for UTF-8 handling.
- create some files with UTF-8 encoded names
- add a simple test to see if we process them correctly
- templatize the manifest golden tests.
Future plans:
- Integration tests with pkg_tar/zip/...
- pkg_mkdir of UTF-8 string
- Fix bazel to support UTF-8 better, and re-enable windows tests
- Decide if we should test 4-byte (emoticon) files?
- 4-byte chars in file names does not work on Windows 10
This change provides the `pkg_install` macro, which allows users to create
runnable installation scripts directly from Bazel BUILD files.
Installation scripts are generated from a template that consumes the manifest
files currently known to `pkg_tar`. They are intended to be run by `bazel run`,
although they can be run within Bazel as tools if needed. See the documentation
of `pkg_install` in `pkg/install.bzl` for usage details.
`pkg_install` is currently incomplete. In particular, the implementation so far
only supports installing files and directories on UNIXy platforms. The
following are not yet supported:
- Changing of Windows permissions outside of the context of UNIX permissions
- Installation of symbolic links and TreeArtifacts
- Control of default permissions/attributes
The following are not adequately tested due to other complications:
- Changing of file ownership (UNIX group/user)
Full support for Windows is likely to occur separately.
Other changes include:
- Support for emitting "short" paths as manifest destinations (needed to support
`bazel run`)
Advances #308.
* Fix mapping tests to work on windows.
The problem is the sh_binary emits an extra "name.exe".
The tests were failing because the .exe was not expected.
Instead we strip it from the content matcher.
It's a hack, but that is fine for a test.
* Fix detection of compression from tar file extension
- so that a '.' is not required in the extension itself.
- add some tests to see that we really do compress the right way.
- fixes bug where user could specify `extension=".tar.gz"` and get the wrong thing
The `pkg_filegroup` framework has evolved considerably since its creation, and
the `rpm` packaging rule that uses it (in `pkg/experimental/rpm.bzl`) has as
well. We believe it is time to move it out of `experimental` and into general
use.
This change moves `pkg/experimental/rpm.bzl` to `pkg/rpm_pfg.bzl` for direct
access, and the compatibility macro in `pkg/rpm.bzl` modified to use it. All
tests have been updated and moved to subdirectories of `pkg/tests/rpm`.
`pkg/legacy/rpm.bzl` was also updated to indicate its deprecated status. Distro
packaging and CI configuration were also updated.
`@rules_pkg` has been opportunistically removed from `load`s in test BUILD files
as observed. There's likely a few other places where it would need to be
cleaned up.
Resolves #307.
The `pkg_filegroup` framework has evolved considerably since its creation, and
the `rpm` packaging rule that uses it (in `pkg/experimental/rpm.bzl`) has as
well. We believe it is time to move it out of `experimental` and into general
use.
This change prepares for this by:
- Moving the existing `pkg/rpm.bzl` module and its associated tests to new
package in `pkg/legacy`.
- Replacing the contents `pkg/rpm.bzl` with a compatibility macro that allows
for easily switching between the two (the interfaces are significantly
different). The macro switches between the two based on mandatory attributes
that are available in one instance of `pkg_rpm` but not the other.
CI, packaging, and test targets were also updated as needed.
Part of #307.
This change modifies the `pkg_rpm` rule in `pkg/experimental/rpm.bzl` to use the
rpmbuild toolchain currently in use by the existing RPM packaging rule in
`pkg/rpm.bzl`.
Additionally, the RPM tests were removed from the disabled test set for CentOS
hosts. Enablement on systems that do not have a working `rpmbuild` binary
requires additional work that was attempted and found to be non-trivial.
Documentation was also updated to reflect bazel version compatibility, along
with other minor tweaks. Most notably, the use of `pkg_filegroup` and related
rules requires Bazel 4.0.0.
Fixes #299, partially resolves #183.
* Make helpers_test work on Windows
We can't run the test from within a NamedTemporaryFile context because
the file is locked while being written so we can not read it. So use a
NamedDirectory, create the file, close the file, and then check the content.
* try to make archive test work on windows
* handle / in expected file paths
* is TarFileWriter really the problem?
* same patch to add_tar
* do a test the right way
* cleanups
* fix impact of normpath
* got it!
* try another
* Move examples tree and create one for naming packages.
TIL that having examples with their own WORKSPACE inside the scope of the
main WORKSPACE leads to strange behavior. Doing
```
bazel test ...
```
from the top would try to build the examples, which we don't want.
* add an example of package naming
* Replace build_deb_test.sh with pkg_deb_test.py
* enable deb test on macos and windows
* remove third_party bashunit
* test needs hack on windows for #213 and #214
Admin override because this is not code. This is adding more tests to the CI.
* add windows to CI. Not everything, but enough to get started
* fix a missing dependency
* remove all but one test because path.py is broken for windows.
* Provide tests for experimental/pkg_rpm.bzl
This change provides a basic test system for testing changes to the experimental
`pkg_filegroup`-based RPM generator. The test framework itself is a python script
that makes several calls out to `rpm(8)`'s query command on a generated RPM, and
comparing output information to expected values.
This could also be theoretically applied to the existing RPM generator, if so
desired.
There is more that could be tested here, but this is adequate in the meantime to
demonstrate how this might be applied to other RPM features.
* Switch up python string concat
* Improve test manifest formatting, test completeness
- Fix file size calculations. Turns out the issue was related to following
symlinks.
- Take @aiuto's suggestion for formatting the RPM CSV manifest writing for
improved maintainability
* Allow experimental RPM builder tests to pass on CentOS 7 and CentOS 8
Tests on CentOS 8 with the RPM generator test code fail due to:
- RPM file changing magic
- Directory size calculation changes
(2cf7096ba5)
This change accounts for these issues by ignoring RPM spec file "magic" and
throwing out the whole concept of "size" to begin with (hashes are good enough),
respectively.
Quoting was also opportunistically adjusted in the manifest CSV `genrule`.
* Add centos 7 test target; tag rpm tests as not runnable on macOS or Windows
* Disable the experimental RPM packaging test until CI has rpmbuild(8)
See #183.
* Whoops, forgot macOS.
Add some more explanatory comments while I'm at it.
* add macos to CI, fix most tests, skip some others
* forgot a :
* still no sign of land
* attempt to fix working_dir for macos
* ... but that fumblefingered linux, so put that back
aiuto