Commit Graph

382 Commits

Author SHA1 Message Date
Ezekiel Warren f14a8a5de8
Treat Pop!_OS as Ubuntu (#148)
Pop!_OS support was broken for me as I am currently running 22.04.

#62 seemed to mix Pop!_OS with Linux Mint and specific version checks, but AFAIK Pop!_OS uses the same version as Ubuntu. This change simply treats Pop!_OS as Ubuntu.
2022-06-02 16:06:03 -07:00
Siddhartha Bagaria c63521c22a Update README for 0.7.2 2022-05-23 23:00:00 +00:00
Siddhartha Bagaria 83e69ba9e4 Fix archlinux container test
It is not easy to run LLVM 14 on the latest archlinux container which
come with a gcc distribution with compressed sections for its object
files, and LLVM 14 does not have zlib support in ld.lld. LLVM 13 does,
so let's run the tests for archlinux with LLVM 13.
2022-05-23 22:42:03 +00:00
Siddhartha Bagaria d69557a501 LLVM 13.0.1 checksums
Also, sort the list of checksums for all other releases.
2022-05-23 22:37:56 +00:00
Zhanyong Wan 3c5a553a31
Reference latest LLVM version in README (#147) 2022-05-23 13:17:45 -07:00
Siddhartha Bagaria 15549d770e Run tests for minimum supported bazel version 2022-05-23 11:47:47 -07:00
Zhanyong Wan 3cf6c59742 Avoid nested skylark functions as they don't work with bazel 4.* 2022-05-20 16:17:11 -07:00
Siddhartha Bagaria 9e71d56202 Update README for 0.7.1 2022-04-17 20:47:45 +00:00
Siddhartha Bagaria e8b2091df1 Disable SUSE tests
See #143.
2022-04-17 20:41:04 +00:00
Siddhartha Bagaria 4e6d9960a2 Add test for absolute paths feature 2022-04-17 20:36:00 +00:00
Siddhartha Bagaria e0d608fffb Move location of symlinked tools 2022-04-17 20:31:28 +00:00
Siddhartha Bagaria 14e7622e48 Minor whitespace fix in README 2022-04-17 19:39:46 +00:00
Siddhartha Bagaria de8386b818 Replace symlinked directory with individual symlinks
We needed the symlink hack only for the tools explicitly defined by
bazel. Replacing the directory with individually symlinked files is a
better design now that we have not discovered any other uses of the hack
in some time.

It also helps avoid warnings like in
https://github.com/grailbio/bazel-toolchain/issues/125#issuecomment-1100226305
2022-04-17 03:16:32 +00:00
Siddhartha Bagaria d2269a47d8 Update release_name logic for LLVM 14
LLVM 14 does not have a Ubuntu 20.04 release, but it does have Ubuntu
18.04. Currently, Suse tests are failing which needs more work to
resolve.
2022-04-13 21:15:46 +00:00
Siddhartha Bagaria a9a973d1b7 Fix file_dependency_test on Ubuntu
The file libc++.a can now be inside a target architecture specific
directory.
2022-04-13 19:51:49 +00:00
Siddhartha Bagaria 095f07aae3 Add tools/__pycache__ dir to gitignore 2022-04-13 19:26:49 +00:00
Siddhartha Bagaria 3c735acfd9 Remove toolchain library and include dirs from cmd
CFE is smart enough to automatically include them if the files are
present in the sandbox.
2022-04-13 19:24:06 +00:00
Thulio Ferraz Assis 0fae876df7 fix: missing include and lib search paths
Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
2022-04-13 12:19:23 -07:00
Siddhartha Bagaria 869212179a Support LLVM 14
Also update LLVm version in tests to be 14. Some tests are currently
failing, which will be fixed in subsequent changes.
2022-04-13 18:41:52 +00:00
Thulio Ferraz Assis 32d7596dd9
chore: glob's allow_empty defaults to True (#141)
https://docs.bazel.build/versions/5.1.0/be/functions.html#glob
2022-04-06 13:32:51 -07:00
Siddhartha Bagaria bb6202519d Fix external tests on linux
Some rules_rust tests needed more fixes in the patch (and the PR).

Also reorganized how we are listing the tests to run to be more clear
and be able to skip tests incompatible with the platform.
2022-03-24 23:04:29 +00:00
Siddhartha Bagaria 489dbca480 Separate tests workspace
Also update rules_go and rules_rust dependencies. Tests for
compatibility with rules_rust have been simplified.

This change will also help avoid pull in unnecessary dependencies when
using bazel modules.
2022-03-24 15:25:20 -07:00
Tim Brown f841a13309
Use ubuntu20.04 on suse for llvm >= 13 (#139)
Starting with version 13 llvm no longer provides suse specific
binaries. Therefore just use the ubuntu20.04 binaries which seem to run
without issues.
2022-03-20 17:48:25 -07:00
Siddhartha Bagaria 5628a6c07c Re-enable migration test
As of bazel 5.0, --all_incompatible_changes flag is a no-op. So we
should switch to using bazelisk for checking any migration issues.

https://github.com/bazelbuild/bazel/issues/13892

Because bazelisk does not allow specifying flag overrides on the command
line, we can not override the known failing incompatible flags, and so
our migration test will now fail. This is not ideal, but is the only
option going forward.
2022-03-08 01:42:22 -08:00
Siddhartha Bagaria 7c9e0d5b64 Update README with 0.7 release 2022-03-08 00:58:27 -08:00
Siddhartha Bagaria 58af2170da README section on supporting new targets 2022-03-08 00:35:01 -08:00
Siddhartha Bagaria b84d6720b0 Use stdc++ as default when cross-compiling
This is also consistent with the README.
2022-03-07 23:58:43 -08:00
Siddhartha Bagaria 7580bbe44e Remove lpthread and ldl from link_libs
The original use case of #103 is better satisfied through
the users specifying `link_libs = {"": ["-lpthread", "-ldl"]}` in the
repository rule if they need it globally. Else they can specify it on a
per-target basis as `linkopt`.

We have started supporting the use of stdlib implementations that are
not libc++ and therefore do not require lpthread and ldl. I think the
historic availability of these libraries was by coincidence and we can
expect users to manually specify them if they need. This is also the
default with the LLVM toolchain outside of bazel.
2022-03-08 07:36:47 +00:00
Siddhartha Bagaria bbecbf1e6f Comment out the lines used for manual testing 2022-03-08 07:23:52 +00:00
Siddhartha Bagaria c0e0937e63
User overrides for all flags (#137)
Most users require some configurability of the toolchain through being
able to override the default:
1. stdlib
2. c++ standard
3. linker

The first two have already been solved through attributes to the
repository rule. Instead of continuing to add specialized repository
rules, we should pass through all flag groups in bazel's
unix_cc_toolchain_config.bzl as repository rule attributes. The existing
defaults continue to exist, but users now have the option to override
these defaults for each target os-arch pair.

These flag overrides can also contain a placeholder to the root
directory for the LLVM distribution.

An example solution to satisfy #132 (using mold linker) could be one of:
- Pick up mold linker and libstdc++ from the host.
```
llvm_toolchain(
    name = "llvm_toolchain",
    link_flags = {"": [
        "-lm",
        "-no-canonical-prefixes",
        "-fuse-ld=mold",
        "-Wl,--build-id=md5",
        "-Wl,--hash-style=gnu",
        "-Wl,-z,relro,-z,now",
        "-l:libstdc++.a",
    ]},
    stdlib = {"": "stdc++"},
    llvm_version = "12.0.0",
)
```
- Pick up mold linker from the host but continue to link bundled libc++.
```
llvm_toolchain(
    name = "llvm_toolchain",
    link_flags = {"": [
        "-lm",
        "-no-canonical-prefixes",
        "-fuse-ld=mold",
        "-Wl,--build-id=md5",
        "-Wl,--hash-style=gnu",
        "-Wl,-z,relro,-z,now",
        "-L{toolchain_path_prefix}lib",
        "-l:libc++.a",
        "-l:libc++abi.a",
        "-l:libunwind.a",
        "-rtlib=compiler-rt",
    ]},
    llvm_version = "12.0.0",
)
```
2022-03-07 23:23:37 -08:00
Siddhartha Bagaria c728048463 urls and sha256 attributes for llvm archive download
Current strategy for detecting OS name and matching it to an LLVM
distribution archive has many corner cases. Moreover, people might have
their own URLs where they host these archives.

Having an explicit dict of URLs and sha256, keyed by OS name, version
and arch, will help people get over these corner cases, and be able to
try new LLVM releases without waiting for an update to this repository.
Note that the keys here are different than the `toolchain_roots`
attribute.

While this method does have an extra setup step for each new OS type
that the user's workspace needs to support, this approach is more
flexible.

If we notice that people are using this feature more than the
auto inferred URLs, or that the llvm_release_names.py script is out of
date, we may just retain this feature, and delete the other way of
getting archives.

Additionally, fixes #125. People with that use case can now use the
`urls` attribute, or use the new convenience aliases.
2022-03-07 22:01:16 +00:00
Artem V. Navrotskiy dd9e6a64fc
Update logic for Ubuntu LLVM version detection (#124)
Changes:
 - fallback on Ubuntu to latest supported version (for example use `linux-gnu-ubuntu-20.10` for 11.1.0 on Ubuntu 21.10);
 - don't try to non-exists `linux-gnu-ubuntu-18.04` for clang 11.1.0 and 11.0.1.
2022-03-06 13:53:43 -08:00
Liu Jiaming 7e2edd4af4
Convenience target for bundled libomp (#136)
Users wishing to use openmp through `-fopenmp` currently can not do so,
because libomp is available only as a shared library.

Shared libraries are not exposed by this toolchain because there is limited
utility in doing so (only the sandboxed compile step will succeed, and because
the lib is missing from runfiles, the executable won't run).

The proper way to depend on openmp will be the same as depending on other
shared libraries, i.e. either depend on the host/sysroot to provide it, or provide
it through `deps` using perhaps a `cc_import`.

While it was technically possible to create a `cc_import` target for libomp, it
was a little awkward because of the different extensions between darwin and
linux. So we now provide a convenience target `@llvm_toolchain//:omp` and
include a unit test as a demo.

Co-authored-by: Siddhartha Bagaria <sbagaria@grailbio.com>
2022-03-06 13:43:52 -08:00
Kaiming Yang 5105e05f4a
Allow user specification of c++ standard (#129)
The standard is currently fixed for all target platforms.

This can change soon as configurability is designed more properly.
2022-03-06 11:22:22 -08:00
James Sharpe 38ab9f6f49
Allow selection of stdlib to use (#112)
Allow selection of stdc++ in llvm_toolchain repository rule. Fixes #63

Co-authored-by: Siddhartha Bagaria <sbagaria@grailbio.com>
2022-03-06 11:01:39 -08:00
Laurenz 560cf0f5b7
Fix dead link to Chromium sysroot doc (#126) 2021-12-29 11:05:31 -08:00
Lachlan Sneff a060e1323a
Fix to support pop os (and pop os 21.x) (#127) 2021-12-17 22:31:30 +00:00
Shuai Zhang a912bb381b
Feature: add llvm_mirror_patterns for llvm_toolchain() (#120)
* Improvement: make llvm_mirror option doc more clear.

* Feature: add llvm_mirror_patterns for llvm_toolchain()
2021-10-29 09:19:28 -05:00
Rahul Butani 66f0ab4ea0
platforms: fix the constraints for `//platforms:linux-aarch64` (#118)
FIxes #117.
2021-10-28 11:33:18 -05:00
James Sharpe 357e2fc439
Use exec to call the compiler (#116) 2021-10-15 11:35:40 -05:00
Rahul Butani 25f6fb9582
release: update the README instructions for 0.6.3 2021-10-14 10:37:20 -05:00
Rahul Butani 4ed2de142f
Support older `libtool` versions (#115)
Fixes #114.

* toolchain: update the tool names

* toolchain: add machinery to expose information about host tools to `cc_toolchain_config`

`cc_toolchain_config` is not a repository rule and cannot poke at the host tools itself

* toolchain: add logic to detect if a tool supports arg files

* toolchain: generalize the `cc_wrapper` plumbing to support "wrapper files"

* toolchain: move the `host_tools` utils to a struct

* toolchain: add a libtool wrapper that flattens arg files

* toolchain: use the libtool wrapper when needed

this commit also changes `cc_toolchain_config.bzl` to check that `strip` and `ld`
exist on the host machine when we use them

we use tags here instead of the `/usr/bin/` paths so that we can easily transition
to using `rctx.which` to find where the tools live instead if we want to do so in
the future

* misc: fmt
2021-10-13 21:45:24 -05:00
Yasushi Saito 03f380c8f9
Support Clang 13 (#113)
Closes #110.

* Support Clang 13

* misc: update the README

* distributions: add LLVM 12.0.1

* distributions: add a missing armv7a-hf entry for LLVM 10

Co-authored-by: Rahul Butani <rr.butani@gmail.com>
2021-10-13 21:43:37 -05:00
Siddhartha Bagaria e961023917 Remove Makevars support
There is not enough customization to warrant keeping it in this repo
where it is untested, does not fit well with multiple target toolchains,
and does not have a clear use case.
2021-10-01 00:41:20 -07:00
James Fish 806346a3a8
Include the symlinked dwp/objcopy/strip files in the toolchain (#108)
* Fix including symlink extra files in dwp/objcopy

* Add strip filegroups to support strip inside sandbox

* tests: add a `.stripped` test

* tests: test that `.dwp` targets can be built

This has some things we'll eventually want to remove; see #109 for some context.

* tests: have `dwp_file` transition to `-c dbg` so .dwo files are generated with clang-12+

Issue #109 and [this comment](https://github.com/grailbio/bazel-toolchain/pull/108#issuecomment-928839768)
have some context; the short version is that `-gsplit-dwarf` no longer implies `-g2` which means under
`-c fastbuild` (implies `-g0`) no `.dwo` files are created.

There's a patch to `unix_cc_toolchain_config.bzl` but for now we'll just use `-c dbg` for this one
target using a transition.

* tests: migration fixes

Co-authored-by: Rahul Butani <rr.butani@gmail.com>
2021-09-28 02:38:31 -05:00
Dudko Alexey 4760f4ac4e
Make platforms targets publicly visible (#107) 2021-09-27 11:54:53 -05:00
Siddhartha Bagaria ddcdd61f64 Update latest tag in README to 0.6.2 2021-09-26 13:18:46 -07:00
Siddhartha Bagaria 70d91f14d9
Ensure /usr/bin is in PATH for macOS (#106)
This is an alternative fix to what was included in #105.
2021-09-26 07:40:39 -07:00
Dorian Peake 8c24bfdac8
Symlink '/usr/bin/ld' to llvm toolchain directory on Mac (#105)
* Symlink '/usr/bin/ld' to llvm toolchain directory on Mac

Mac systems still use the local `ld` executable since `lld.ld` is still
experimental. Some applications, e.g. rustc when compiling rust, expect
to find `ld` within the same directory as the compiler and so when
compiling rust we run into an issue where the linker is not found.

The first immediate solution is to use the rust rule's
`extra_rustc_flags` to specify the linker explicitly (via
`-Clinker=/usr/bin/ld`. This works for some crates but fails for crates
using a build script, since the extra flags are not propagated to rustc
invocations within the exec configuration.

Another solution attempt was to link `ld` into the toolchain
configuration repository instead of directly in the llvm repository,
such that `ld` becomes a sibling file to `cc_wrapper.sh`. Unfortunately
rustc was still unable to find the linker with this configuration.

The final fully working solution is to link `ld` into the bin folder of
the llvm repository. This should be fine as LLVM's own linker exists
under `ld.lld` and symlinking the local linker will not clobber any
files.

* tests: add a `rules_rust` test

I picked `git2-rs` since it links against a C library and has tests but isn't _massive_.

There are a couple of hacks/patches here to workaround some bugs in `rules_rust`'s `crate-universe`
but on the whole it's not too messy.

* Add --incompatible_no_rule_outputs_param=false to run_external_tests.sh

The rules_rust repository is incompatible with this flag enabled. Have
disabled it for now.

* Add --incompatible_no_rule_outputs_param=false to run_tests.sh

Flag already added to 'run_external_tests.sh' but also needed here.

* Move 'git2-rs-cargo-toml.patch' to 'tests/rust'

* tests: move some shared test args for bazel into `bazel.sh`

Note that this also drops the check for `$TEST_MIGRATION` in run_external_tests.sh; we don't run that script in migration.yml

* tests: add a note about using `git2-rs` instead of the `@rules_rust` tests

Co-authored-by: Rahul Butani <rr.butani@gmail.com>
2021-09-25 20:26:10 -07:00
Dorian Peake 5ec548797f
Add missing closing parenthesis on cc_toolchain (#104)
Closing parenthesis was missing on cc_toolchain definition when
`absolute_paths` is True.
2021-09-25 11:14:00 -07:00