diff --git a/.gitignore b/.gitignore index 8a35a81b..faa973e3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,8 @@ __pycache__ +/venv + bazel-* + node_modules .idea available_tests.txt @@ -8,3 +11,5 @@ csharp/nuget/nuget2config.json ruby/.bundle js/requirements/yarn-error.log .vscode + +/docs/build diff --git a/Makefile b/Makefile index f4f17664..f4849af4 100644 --- a/Makefile +++ b/Makefile @@ -3,10 +3,17 @@ rulegen: bazel query '//example/routeguide/... - attr(tags, manual, //example/routeguide/...)' > available_tests.txt; \ bazel run --run_under="cd $$PWD && " //tools/rulegen -- --ref=$$(git describe --abbrev=0 --tags); \ - rm available_tests.txt; \ + rm available_tests.txt; + + +# Build docs locally +.PHONY: docs +docs: + python3 -m sphinx -c docs -a -E -T -W --keep-going docs/src docs/build # Apply buildifier +.PHONY: buildifier buildifier: bazel run //tools:buildifier diff --git a/README.rst b/README.rst index 6d26d8fd..41f87304 100644 --- a/README.rst +++ b/README.rst @@ -11,6 +11,8 @@ Protobuf and gRPC rules for `Bazel `_ and `gRPC `_ code and libraries from `proto_library `_ targets +`Docs `__ | `GitHub `__ + .. image:: https://img.shields.io/github/v/tag/rules-proto-grpc/rules_proto_grpc?label=release&sort=semver&color=38a3a5 :alt: Latest Release :target: https://github.com/rules-proto-grpc/rules_proto_grpc/releases @@ -56,24 +58,24 @@ Contents - `Overview`_ - `Installation`_ - `Rules`_ - - `Android `_ - - `Buf `_ - - `C `_ - - `C++ `_ - - `C# `_ - - `D `_ - - `Documentation `_ - - `Go `_ - - `grpc-gateway `_ - - `Java `_ - - `JavaScript `_ - - `Objective-C `_ - - `PHP `_ - - `Python `_ - - `Ruby `_ - - `Rust `_ - - `Scala `_ - - `Swift `_ + - `Android `_ + - `Buf `_ + - `C `_ + - `C++ `_ + - `C# `_ + - `D `_ + - `Documentation `_ + - `Go `_ + - `grpc-gateway `_ + - `Java `_ + - `JavaScript `_ + - `Objective-C `_ + - `PHP `_ + - `Python `_ + - `Ruby `_ + - `Rust `_ + - `Scala `_ + - `Swift `_ - `Example Usage`_ - `Developers`_ - `Code Layout`_ @@ -115,7 +117,7 @@ Installation Add ``rules_proto_grpc`` to your ``WORKSPACE`` file and then look at the language specific examples linked below: -.. code-block:: starlark +.. code-block:: python load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") @@ -152,428 +154,207 @@ Rules * - Language - Rule - Description - * - `Android `_ - - `android_proto_compile `_ - - Generates an Android protobuf ``.jar`` file (`example `_) - * - `Android `_ - - `android_grpc_compile `_ - - Generates Android protobuf and gRPC ``.jar`` files (`example `_) - * - `Android `_ - - `android_proto_library `_ - - Generates an Android protobuf library using ``android_library`` from ``rules_android`` (`example `_) - * - `Android `_ - - `android_grpc_library `_ - - Generates Android protobuf and gRPC library using ``android_library`` from ``rules_android`` (`example `_) - * - `Buf `_ - - `buf_proto_breaking_test `_ - - Checks .proto files for breaking changes (`example `_) - * - `Buf `_ - - `buf_proto_lint_test `_ - - Lints .proto files (`example `_) - * - `C `_ - - `c_proto_compile `_ - - Generates C protobuf ``.h`` & ``.c`` files (`example `_) - * - `C `_ - - `c_proto_library `_ - - Generates a C protobuf library using ``cc_library``, with dependencies linked (`example `_) - * - `C++ `_ - - `cpp_proto_compile `_ - - Generates C++ protobuf ``.h`` & ``.cc`` files (`example `_) - * - `C++ `_ - - `cpp_grpc_compile `_ - - Generates C++ protobuf and gRPC ``.h`` & ``.cc`` files (`example `_) - * - `C++ `_ - - `cpp_proto_library `_ - - Generates a C++ protobuf library using ``cc_library``, with dependencies linked (`example `_) - * - `C++ `_ - - `cpp_grpc_library `_ - - Generates a C++ protobuf and gRPC library using ``cc_library``, with dependencies linked (`example `_) - * - `C# `_ - - `csharp_proto_compile `_ - - Generates C# protobuf ``.cs`` files (`example `_) - * - `C# `_ - - `csharp_grpc_compile `_ - - Generates C# protobuf and gRPC ``.cs`` files (`example `_) - * - `C# `_ - - `csharp_proto_library `_ - - Generates a C# protobuf library using ``csharp_library`` from ``rules_dotnet``. Note that the library name must end in ``.dll`` (`example `_) - * - `C# `_ - - `csharp_grpc_library `_ - - Generates a C# protobuf and gRPC library using ``csharp_library`` from ``rules_dotnet``. Note that the library name must end in ``.dll`` (`example `_) - * - `D `_ - - `d_proto_compile `_ - - Generates D protobuf ``.d`` files (`example `_) - * - `D `_ - - `d_proto_library `_ - - Generates a D protobuf library using ``d_library`` from ``rules_d`` (`example `_) - * - `Documentation `_ - - `doc_docbook_compile `_ - - Generates DocBook ``.xml`` documentation file (`example `_) - * - `Documentation `_ - - `doc_html_compile `_ - - Generates ``.html`` documentation file (`example `_) - * - `Documentation `_ - - `doc_json_compile `_ - - Generates ``.json`` documentation file (`example `_) - * - `Documentation `_ - - `doc_markdown_compile `_ - - Generates Markdown ``.md`` documentation file (`example `_) - * - `Go `_ - - `go_proto_compile `_ - - Generates Go protobuf ``.go`` files (`example `_) - * - `Go `_ - - `go_grpc_compile `_ - - Generates Go protobuf and gRPC ``.go`` files (`example `_) - * - `Go `_ - - `go_proto_library `_ - - Generates a Go protobuf library using ``go_library`` from ``rules_go`` (`example `_) - * - `Go `_ - - `go_grpc_library `_ - - Generates a Go protobuf and gRPC library using ``go_library`` from ``rules_go`` (`example `_) - * - `grpc-gateway `_ - - `gateway_grpc_compile `_ - - Generates grpc-gateway ``.go`` files (`example `_) - * - `grpc-gateway `_ - - `gateway_openapiv2_compile `_ - - Generates grpc-gateway OpenAPI v2 ``.json`` files (`example `_) - * - `grpc-gateway `_ - - `gateway_grpc_library `_ - - Generates grpc-gateway library files (`example `_) - * - `Java `_ - - `java_proto_compile `_ - - Generates a Java protobuf srcjar file (`example `_) - * - `Java `_ - - `java_grpc_compile `_ - - Generates a Java protobuf and gRPC srcjar file (`example `_) - * - `Java `_ - - `java_proto_library `_ - - Generates a Java protobuf library using ``java_library`` (`example `_) - * - `Java `_ - - `java_grpc_library `_ - - Generates a Java protobuf and gRPC library using ``java_library`` (`example `_) - * - `JavaScript `_ - - `js_proto_compile `_ - - Generates JavaScript protobuf ``.js`` and ``.d.ts`` files (`example `_) - * - `JavaScript `_ - - `js_grpc_node_compile `_ - - Generates JavaScript protobuf and gRPC-node ``.js`` and ``.d.ts`` files (`example `_) - * - `JavaScript `_ - - `js_grpc_web_compile `_ - - Generates JavaScript protobuf and gRPC-Web ``.js`` and ``.d.ts`` files (`example `_) - * - `JavaScript `_ - - `js_proto_library `_ - - Generates a JavaScript protobuf library using ``js_library`` from ``rules_nodejs`` (`example `_) - * - `JavaScript `_ - - `js_grpc_node_library `_ - - Generates a Node.js protobuf + gRPC-node library using ``js_library`` from ``rules_nodejs`` (`example `_) - * - `JavaScript `_ - - `js_grpc_web_library `_ - - Generates a JavaScript protobuf + gRPC-Web library using ``js_library`` from ``rules_nodejs`` (`example `_) - * - `Objective-C `_ - - `objc_proto_compile `_ - - Generates Objective-C protobuf ``.m`` & ``.h`` files (`example `_) - * - `Objective-C `_ - - `objc_grpc_compile `_ - - Generates Objective-C protobuf and gRPC ``.m`` & ``.h`` files (`example `_) - * - `Objective-C `_ - - `objc_proto_library `_ - - Generates an Objective-C protobuf library using ``objc_library`` (`example `_) - * - `Objective-C `_ - - `objc_grpc_library `_ - - Generates an Objective-C protobuf and gRPC library using ``objc_library`` (`example `_) - * - `PHP `_ - - `php_proto_compile `_ - - Generates PHP protobuf ``.php`` files (`example `_) - * - `PHP `_ - - `php_grpc_compile `_ - - Generates PHP protobuf and gRPC ``.php`` files (`example `_) - * - `Python `_ - - `python_proto_compile `_ - - Generates Python protobuf ``.py`` files (`example `_) - * - `Python `_ - - `python_grpc_compile `_ - - Generates Python protobuf and gRPC ``.py`` files (`example `_) - * - `Python `_ - - `python_grpclib_compile `_ - - Generates Python protobuf and grpclib ``.py`` files (supports Python 3 only) (`example `_) - * - `Python `_ - - `python_proto_library `_ - - Generates a Python protobuf library using ``py_library`` from ``rules_python`` (`example `_) - * - `Python `_ - - `python_grpc_library `_ - - Generates a Python protobuf and gRPC library using ``py_library`` from ``rules_python`` (`example `_) - * - `Python `_ - - `python_grpclib_library `_ - - Generates a Python protobuf and grpclib library using ``py_library`` from ``rules_python`` (supports Python 3 only) (`example `_) - * - `Ruby `_ - - `ruby_proto_compile `_ - - Generates Ruby protobuf ``.rb`` files (`example `_) - * - `Ruby `_ - - `ruby_grpc_compile `_ - - Generates Ruby protobuf and gRPC ``.rb`` files (`example `_) - * - `Ruby `_ - - `ruby_proto_library `_ - - Generates a Ruby protobuf library using ``ruby_library`` from ``rules_ruby`` (`example `_) - * - `Ruby `_ - - `ruby_grpc_library `_ - - Generates a Ruby protobuf and gRPC library using ``ruby_library`` from ``rules_ruby`` (`example `_) - * - `Rust `_ - - `rust_proto_compile `_ - - Generates Rust protobuf ``.rs`` files (`example `_) - * - `Rust `_ - - `rust_grpc_compile `_ - - Generates Rust protobuf and gRPC ``.rs`` files (`example `_) - * - `Rust `_ - - `rust_proto_library `_ - - Generates a Rust protobuf library using ``rust_library`` from ``rules_rust`` (`example `_) - * - `Rust `_ - - `rust_grpc_library `_ - - Generates a Rust protobuf and gRPC library using ``rust_library`` from ``rules_rust`` (`example `_) - * - `Scala `_ - - `scala_proto_compile `_ - - Generates a Scala protobuf ``.jar`` file (`example `_) - * - `Scala `_ - - `scala_grpc_compile `_ - - Generates Scala protobuf and gRPC ``.jar`` file (`example `_) - * - `Scala `_ - - `scala_proto_library `_ - - Generates a Scala protobuf library using ``scala_library`` from ``rules_scala`` (`example `_) - * - `Scala `_ - - `scala_grpc_library `_ - - Generates a Scala protobuf and gRPC library using ``scala_library`` from ``rules_scala`` (`example `_) - * - `Swift `_ - - `swift_proto_compile `_ - - Generates Swift protobuf ``.swift`` files (`example `_) - * - `Swift `_ - - `swift_grpc_compile `_ - - Generates Swift protobuf and gRPC ``.swift`` files (`example `_) - * - `Swift `_ - - `swift_proto_library `_ - - Generates a Swift protobuf library using ``swift_library`` from ``rules_swift`` (`example `_) - * - `Swift `_ - - `swift_grpc_library `_ - - Generates a Swift protobuf and gRPC library using ``swift_library`` from ``rules_swift`` (`example `_) - -Example Usage -------------- - -These steps walk through the actions required to go from a raw ``.proto`` file to a C++ library. Other languages will have -a similar high-level layout. - -**Step 1**: Write a Protocol Buffer .proto file (example: ``thing.proto``): - -.. code-block:: proto - - syntax = "proto3"; - - package example; - - import "google/protobuf/any.proto"; - - message Thing { - string name = 1; - google.protobuf.Any payload = 2; - } - -**Step 2**: Write a ``BAZEL.build`` file with a -`proto_library `_ target: - -.. code-block:: starlark - - proto_library( - name = "thing_proto", - srcs = ["thing.proto"], - deps = ["@com_google_protobuf//:any_proto"], - ) - -In this example we have a dependency on a well-known type ``any.proto``, hence the ``proto_library`` to ``proto_library`` -dependency (``"@com_google_protobuf//:any_proto"``) - -**Step 3**: Add a ``cpp_proto_compile`` target - -**Note**: In this example ``thing.proto`` does not include service definitions (gRPC). For protos with services, use the -``cpp_grpc_compile`` rule instead. - -.. code-block:: starlark - - # BUILD.bazel - load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_compile") - - cpp_proto_compile( - name = "cpp_thing_proto", - protos = [":thing_proto"], - ) - -But wait, before we can build this, we need to load the dependencies necessary for this rule -(see `cpp `_): - -**Step 4**: Load the workspace macro corresponding to the build rule. - -.. code-block:: starlark - - # WORKSPACE - load("@rules_proto_grpc//cpp:repositories.bzl", "cpp_repos") - - cpp_repos() - -We're now ready to build the target. - -**Step 5**: Build it! - -.. code-block:: bash - - $ bazel build //example/proto:cpp_thing_proto - Target //example/proto:cpp_thing_proto up-to-date: - bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h - bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc - -If we were only interested in the generated files, the ``cpp_grpc_compile`` rule would be fine. However, for -convenience we'd rather have the outputs compiled into a C++ library. To do that, let's change the rule from -``cpp_proto_compile`` to ``cpp_proto_library``: - -.. code-block:: starlark - - # BUILD.bazel - load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_library") - - cpp_proto_library( - name = "cpp_thing_proto", - protos = [":thing_proto"], - ) - - -.. code-block:: bash - - $ bazel build //example/proto:cpp_thing_proto - Target //example/proto:cpp_thing_proto up-to-date: - bazel-bin/example/proto/libcpp_thing_proto.a - bazel-bin/example/proto/libcpp_thing_proto.so bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h - bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc - -This way, we can use ``//example/proto:cpp_thing_proto`` as a dependency of any other ``cc_library`` or ``cc_binary`` target -as per normal. - -**Note**: The ``cpp_proto_library`` target implicitly calls ``cpp_proto_compile``, and we can access that rule's by adding -``_pb`` at the end of the target name, like ``bazel build //example/proto:cpp_thing_proto_pb`` - - -Developers ----------- - -Code Layout -*********** - -Each language ``{lang}`` has a top-level subdirectory that contains: - -1. ``{lang}/README.rst``: Generated documentation for the language rules - -1. ``{lang}/repositories.bzl``: Macro functions that declare repository rule dependencies for that language - -2. ``{lang}/{rule}.bzl``: Rule implementations of the form ``{lang}_{kind}_{type}``, where ``kind`` is one of ``proto|grpc`` and - ``type`` is one of ``compile|library`` - -3. ``{lang}/BUILD.bazel``: ``proto_plugin()`` declarations for the available plugins for the language - -4. ``example/{lang}/{rule}/``: Generated ``WORKSPACE`` and ``BUILD.bazel`` demonstrating standalone usage of the rules - -5. ``{lang}/example/routeguide/``: Example routeguide example implementation, if possible - - -Rule Generation -*************** - -To help maintain consistency of the rule implementations and documentation, all of the rule implementations are -generated by the tool ``//tools/rulegen``. Changes in the main ``README.rst`` should be placed in -``tools/rulegen/README.header.rst`` or ``tools/rulegen/README.footer.rst```. Changes to generated rules should be put in the -source files (example: ``tools/rulegen/java.go``). - - -Developing Custom Plugins -************************* - -Generally, follow the pattern seen in the multiple language examples in this -repository. The basic idea is: - -1. Load the plugin rule: ``load("@rules_proto_grpc//:defs.bzl", "proto_plugin")`` -2. Define the rule, giving it a ``name``, ``options`` (not mandatory), ``tool`` and ``outputs``. ``tool`` is a label that refers - to the binary executable for the plugin itself -3. Choose your output type (pick one!): - - ``outputs``: A list of strings patterns that predicts the pattern of files generated by the plugin. For plugins that - produce one output file per input proto file - - ``out``: The name of a single output file generated by the plugin - - ``output_directory``: Set to true if your plugin generates files in a non-predictable way. e.g. if the output paths - depend on the service names within the files -4. Create a compilation rule and aspect using the following template: - -.. code-block:: starlark - - load("@rules_proto//proto:defs.bzl", "ProtoInfo") - load( - "@rules_proto_grpc//:defs.bzl", - "ProtoLibraryAspectNodeInfo", - "ProtoPluginInfo", - "proto_compile_aspect_attrs", - "proto_compile_aspect_impl", - "proto_compile_attrs", - "proto_compile_impl", - ) - - # Create aspect - example_aspect = aspect( - implementation = proto_compile_aspect_impl, - provides = [ProtoLibraryAspectNodeInfo], - attr_aspects = ["deps"], - attrs = dict( - proto_compile_aspect_attrs, - _plugins = attr.label_list( - doc = "List of protoc plugins to apply", - providers = [ProtoPluginInfo], - default = [ - Label("//