Add custom C++ toolchain example.

Inspired by https://github.com/bazelbuild/bazel/issues/12942.
This commit is contained in:
Greg Estren 2021-02-11 15:53:44 -05:00
parent 40548a2974
commit 541927eaa0
6 changed files with 320 additions and 0 deletions

View File

@ -0,0 +1,116 @@
# Copyright 2021 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Proof-of-concept example showing how to write a custom C++ toolchain.
#
# Important documentation:
#
# - https://docs.bazel.build/versions/master/platforms-intro.html#c
# - https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html
# - https://docs.bazel.build/versions/master/be/c-cpp.html#cc_toolchain
#
# There are two ways to select C++ toolchains:
#
# - NEW (USE IF POSSIBLE): with the --platforms flag
# - LEGACY: with the --crosstool_top and --cpu flags
#
# See https://docs.bazel.build/versions/master/platforms-intro.html#c for details.
#
# This example demonstrates both approaches.
# Load the Starlark logic defining the toolchain's behavior. For example: what
# program runs to compile a source file and how its command line is
# constructed. See toolchain_config.bzl for details.
load(":toolchain_config.bzl", "cc_toolchain_config")
# The library we want to build. Building this calls two C++ actions: compile (.cc ->
# .o) and archive (.o -> .a).
cc_library(
name = "buildme",
srcs = ["buildme.cc"],
)
# This example intentionally makes the cc_toolchain_config definition
# simple. You could alternative add attributes to support multiple
# cc_toolchain_config targets with finer customization.
cc_toolchain_config(
name = "toolchain_semantics",
)
# Register the toolchain with Bazel. Most of these attribute just tell Bazel
# where to find the files needed to run C++ commands. The toolchain_config
# attribute registers the behavior specification declared above.
cc_toolchain(
name = "my_custom_toolchain",
all_files = ":toolchain_files",
ar_files = ":toolchain_files",
compiler_files = ":toolchain_files",
dwp_files = ":toolchain_files",
linker_files = ":toolchain_files",
objcopy_files = ":toolchain_files",
strip_files = ":toolchain_files",
toolchain_config = ":toolchain_semantics",
)
filegroup(
name = "toolchain_files",
srcs = [
"sample_compiler",
"sample_linker",
],
)
# Implements legacy toolchain selection.
#
# Setting --crosstool_top here registers the set of available
# toolchains. Setting --cpu to one of the toolchain attribute's keys selects a
#toolchain.
cc_toolchain_suite(
name = "legacy_selector",
toolchains = {
"x86": ":my_custom_toolchain",
},
)
# Implements platform-based (recommended) toolchain selection.
#
# See https://docs.bazel.build/versions/master/platforms-intro.html. The main
# differences are:
#
# 1. --cpu / --crosstool_top are replaced by a platform() definition with
# much more customizable properties. For example, a platform can specify
# OS, device type (server, phone, tablet) or custom hardware extensions.
# 2. All languages can support platform-based toolchains. A single --platforms
# value can choose C++, Python, Scala, and all other toolchains in your
# build. This is especially useful for multi-language builds.
# 3. Platforms support features like incompatible target skipping:
# https://docs.bazel.build/versions/master/platforms.html#skipping-incompatible-targets.
toolchain(
name = "platform_based_toolchain",
# Trigger this toolchain for x86-compatible platforms.
# See https://github.com/bazelbuild/platforms.
target_compatible_with = ["@platforms//cpu:x86_64"],
# Register this toolchain with platforms.
toolchain = ":my_custom_toolchain",
# The public interface for all C++ toolchains. Starlark rules that use C++
# access the toolchain through this interface.
toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
)
# Define a platform matching any x86-compatible toolchain. See
# https://docs.bazel.build/versions/master/platforms.html.
platform(
name = "x86_platform",
constraint_values = ["@platforms//cpu:x86_64"],
)

View File

@ -0,0 +1,78 @@
# Writing a custom C++ toolchain
This example shows how to define and use a simple custom C++ toolchain.
Output is non-functional: simple scripts replace compilation and linking
with "I compiled!" and "I linked!" messages.
[BUILD](BUILD) provides detailed implementation walkthrough. The fundamental
sequence is:
1. Define the toolchain
1. Define how to invoke the toolchain.
`1` is C++-specific: the logic and structure depends specifically on C++'s
language model. Other languages have their own models.
`2` supports two variations. `--crosstool_top` / `--cput`, the legacy variation,
is C++-specific. `--platforms`, the modern variation, is much more generic and
supports all languages and features like [incompatible target
skipping](https://docs.bazel.build/versions/master/platforms.html#skipping-incompatible-targets). See
[Building with
Platforms](https://docs.bazel.build/versions/master/platforms-intro.html) and
its [C++
annotations](https://docs.bazel.build/versions/master/platforms-intro.html#c) for
full review.
## Building with the default toolchain
```
$ bazel clean
$ bazel build //examples/custom_toolchain:buildme
$ file bazel-bin/examples/custom_toolchain/libbuildme.a
bazel-bin/examples/custom_toolchain/libbuildme.a: current ar archive
```
## Building with platforms
This mode requires `--incompatible_enable_cc_toolchain_resolution`. Without this
flag, `--platforms` and `--extra_toolchains` are ignored and the default
toolchain triggers.
```
$ bazel clean
$ bazel build //examples/custom_toolchain:buildme --platforms=//examples/custom_toolchain:x86_platform --extra_toolchains=//examples/custom_toolchain:platform_based_toolchain --incompatible_enable_cc_toolchain_resolution
DEBUG: /usr/local/google/home/gregce/bazel/rules_cc/examples/custom_toolchain/toolchain_config.bzl:17:10: Invoking my custom toolchain!
INFO: From Compiling examples/custom_toolchain/buildme.cc:
examples/custom_toolchain/sample_compiler: running sample cc_library compiler (produces .o output).
INFO: From Linking examples/custom_toolchain/libbuildme.a:
examples/custom_toolchain/sample_linker: running sample cc_library linker (produces .a output).
$ cat bazel-bin/examples/custom_toolchain/libbuildme.a
examples/custom_toolchain/sample_linker: sample output
```
This example uses a long command line for demonstration purposes. A real project
would [register toolchains](https://docs.bazel.build/versions/master/toolchains.html#registering-and-building-with-toolchains)
in `WORKSPACE` and auto-set
`--incompatible_enable_cc_toolchain_resolution`. That reduces the command to:
```
$ bazel build //examples/custom_toolchain:buildme --platforms=//examples/custom_toolchain:x86_platform
```
## Building with legacy toolchain selection:
```
$ bazel clean
$ bazel build //examples/custom_toolchain:buildme --crosstool_top=//examples/custom_toolchain:legacy_selector --cpu=x86
DEBUG: /usr/local/google/home/gregce/bazel/rules_cc/examples/custom_toolchain/toolchain_config.bzl:17:10: Invoking my custom toolchain!
INFO: From Compiling examples/custom_toolchain/buildme.cc:
examples/custom_toolchain/sample_compiler: running sample cc_library compiler (produces .o output).
INFO: From Linking examples/custom_toolchain/libbuildme.a:
examples/custom_toolchain/sample_linker: running sample cc_library linker (produces .a output).
$ cat bazel-bin/examples/custom_toolchain/libbuildme.a
examples/custom_toolchain/sample_linker: sample output
```

View File

@ -0,0 +1,4 @@
int some_function() {
return 0;
}

View File

@ -0,0 +1,21 @@
#!/bin/bash
#
# Sample script demonstrating custom C++ toolchain selection: handles
# the command that translates a cc_library's .cc (source file) into .o (object
# file).
echo "$0: running sample cc_library compiler (produces .o output)."
# https://docs.bazel.build/versions/master/cc-toolchain-config-reference.html
# defines fancier ways to generate custom command lines. This script just shows
# the default, which looks like:
#
# examples/custom_toolchain/sample_compiler <various compiler flags> -o bazel-out/x86-fastbuild/bin/examples/custom_toolchain/_objs/buildme/buildme.o.
# The .o is the last parameter.
OBJECT_FILE=${@: -1}
# Swap out .o for .d to get expected .d (source dependency output).
DOTD_FILE=${OBJECT_FILE%?}d
echo "$0: sample .o output" > $OBJECT_FILE
echo "sample .d output ($0)" > $DOTD_FILE

View File

@ -0,0 +1,23 @@
#!/bin/bash
#
# Sample script demonstrating custom C++ toolchain selection: handles
# the command that translates a cc_library's .o (object file) into
# .a (archive).
echo "$0: running sample cc_library linker (produces .a output)."
# https://docs.bazel.build/versions/master/cc-toolchain-config-reference.html
# defines fancier ways to generate custom command lines. This script just shows
# the default, which looks like:
#
# examples/custom_toolchain/sample_linker @bazel-out/x86-fastbuild/bin/examples/custom_toolchain/libbuildme.a-2.params.
# Get "@bazel-out/.../libbuildme.a-2.params".
PARAMS_FILE=${@: -1}
# Remove the "@" prefix.
OUTFILE=${PARAMS_FILE#?}
# Replace "libbuildme.a-2.params" with "libbuildme.a".
OUTFILE=${OUTFILE%-*}
echo "$0: sample output" > $OUTFILE

View File

@ -0,0 +1,78 @@
# Sample Starlark definition defining a C++ toolchain's behavior.
#
# When you build a cc_* rule, this logic defines what programs run for what
# build steps (e.g. compile / link / archive) and how their command lines are
# structured.
#
# This is a proof-of-concept simple implementation. It doesn't construct fancy
# command lines and uses mock shell scripts to compile and link
# ("sample_compiler" and "sample_linker"). See
# https://docs.bazel.build/versions/master/cc-toolchain-config-reference.html and
# https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html for
# advanced usage.
load("@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl", "tool_path")
def _impl(ctx):
print("Invoking my custom toolchain!")
tool_paths = [
tool_path(
name = "ar",
path = "sample_linker",
),
tool_path(
name = "cpp",
path = "not_used_in_this_example",
),
tool_path(
name = "gcc",
path = "sample_compiler",
),
tool_path(
name = "gcov",
path = "not_used_in_this_example",
),
tool_path(
name = "ld",
path = "sample_linker",
),
tool_path(
name = "nm",
path = "not_used_in_this_example",
),
tool_path(
name = "objdump",
path = "not_used_in_this_example",
),
tool_path(
name = "strip",
path = "not_used_in_this_example",
),
]
# Documented at
# https://docs.bazel.build/versions/master/skylark/lib/cc_common.html#create_cc_toolchain_config_info.
#
# create_cc_toolchain_config_info is the public interface for registering
# C++ toolchain behavior.
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
toolchain_identifier = "custom-toolchain-identifier",
host_system_name = "local",
target_system_name = "local",
target_cpu = "sample_cpu",
target_libc = "unknown",
compiler = "gcc",
abi_version = "unknown",
abi_libc_version = "unknown",
tool_paths = tool_paths,
)
cc_toolchain_config = rule(
implementation = _impl,
# You can alternatively define attributes here that make it possible to
# instantiate different cc_toolchain_config targets with different behavior.
attrs = {},
provides = [CcToolchainConfigInfo],
)