luxolus/open-consul
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
open-consul/agent
History
Dan Upton 1d95609fb7
grpc: protoc plugin for generating gRPC rate limit specifications (#15564) 
Adds automation for generating the map of `gRPC Method Name → Rate Limit Type`
used by the middleware introduced in #15550, and will ensure we don't forget
to add new endpoints.

Engineers must annotate their RPCs in the proto file like so:

```
rpc Foo(FooRequest) returns (FooResponse) {
  option (consul.internal.ratelimit.spec) = {
    operation_type: READ,
  };
}
```

When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will
be installed that writes rate-limit specs as a JSON array to a file called
`.ratelimit.tmp` (one per protobuf package/directory).

After running Buf, `make proto` will execute a post-process script that will
ingest all of the `.ratelimit.tmp` files and generate a Go file containing the
mappings in the `agent/grpc-middleware` package. In the enterprise repository,
it will write an additional file with the enterprise-only endpoints.

If an engineer forgets to add the annotation to a new RPC, the plugin will
return an error like so:

```
RPC Foo is missing rate-limit specification, fix it with:

	import "proto-public/annotations/ratelimit/ratelimit.proto";

	service Bar {
	  rpc Foo(...) returns (...) {
	    option (hashicorp.consul.internal.ratelimit.spec) = {
	      operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT,
	    };
	  }
	}
```

In the future, this annotation can be extended to support rate-limit
category (e.g. KV vs Catalog) and to determine the retry policy.
2023-01-04 16:07:02 +00:00
..
ae sdk: add TestLogLevel for setting log level in tests 2022-02-03 13:42:28 -05:00
auto-config Deprecate -join and -join-wan (#15598) 2022-12-14 20:28:25 +00:00
cache Fix agent cache incorrectly notifying unchanged protobufs. (#15866) 2023-01-03 10:11:56 -06:00
cache-types Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
checks Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
config Deprecate -join and -join-wan (#15598) 2022-12-14 20:28:25 +00:00
configentry [OSS] feat: access logs for listeners and listener filters (#15864) 2022-12-22 15:18:15 -05:00
connect Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
consul Wire in rate limiter to handle internal and external gRPC calls (#15857) 2022-12-23 13:42:16 -06:00
debug bulk rewrite using this script 2022-01-20 10:46:23 -06:00
dns test: fix incorrect use of t instead of r in retry test (#13146) 2022-05-19 14:00:07 -05:00
exec
grpc-external grpc: protoc plugin for generating gRPC rate limit specifications (#15564) 2023-01-04 16:07:02 +00:00
grpc-internal grpc: protoc plugin for generating gRPC rate limit specifications (#15564) 2023-01-04 16:07:02 +00:00
grpc-middleware grpc: protoc plugin for generating gRPC rate limit specifications (#15564) 2023-01-04 16:07:02 +00:00
hcp removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
local Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
log-drop add log-drop package (#15670) 2022-12-15 12:52:48 -05:00
metadata Change serf-tag references to field references. 2022-08-31 16:38:42 -05:00
mock
pool Use rpcHoldTimeout to calculate blocking timeout (#15541) 2022-11-24 10:13:02 -05:00
proxycfg Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865) 2023-01-03 10:44:08 -06:00
proxycfg-glue Fixup authz for data imported from peers (#15347) 2022-11-14 11:36:27 -07:00
proxycfg-sources chore: remove unused argument from MergeNodeServiceWithCentralConfig (#15024) 2022-11-09 14:54:57 +00:00
router Regenerate files according to 1.19.2 formatter 2022-10-24 16:12:08 -04:00
routine-leak-checker removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
rpc Wire in rate limiter to handle internal and external gRPC calls (#15857) 2022-12-23 13:42:16 -06:00
rpcclient/health Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
structs [OSS] feat: access logs for listeners and listener filters (#15864) 2022-12-22 15:18:15 -05:00
submatview Update hcp-scada-provider to fix diamond dependency problem with go-msgpack (#15185) 2022-11-07 11:34:30 -05:00
systemd
token removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
uiserver removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
xds Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865) 2023-01-03 10:44:08 -06:00
acl.go Unify various status errors into one HTTP error type. (#12594) 2022-04-29 13:42:49 -04:00
acl_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
acl_endpoint_legacy.go
acl_endpoint_legacy_test.go
acl_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
acl_oss.go agent: support X-Consul-Results-Filtered-By-ACLs header in agent-local endpoints (#11610) 2021-12-03 20:36:28 +00:00
acl_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
agent.go Wire in rate limiter to handle internal and external gRPC calls (#15857) 2022-12-23 13:42:16 -06:00
agent_endpoint.go Merge branch 'main' of github.com:hashicorp/consul into derekm/split-grpc-ports 2022-09-08 14:53:08 -05:00
agent_endpoint_oss.go Fixup acl.EnterpriseMeta 2022-04-05 15:11:49 -07:00
agent_endpoint_oss_test.go Add oss test 2022-05-09 10:07:19 -07:00
agent_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
agent_oss.go proxycfg: replace direct agent cache usage with interfaces (#13320) 2022-06-01 16:18:06 +01:00
agent_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
apiserver.go
apiserver_test.go
catalog_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
catalog_endpoint_oss.go
catalog_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
check.go Fixup acl.EnterpriseMeta 2022-04-05 15:11:49 -07:00
config_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
config_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
connect_auth.go Unify various status errors into one HTTP error type. (#12594) 2022-04-29 13:42:49 -04:00
connect_ca_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
connect_ca_endpoint_test.go removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
coordinate_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
coordinate_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
delegate_mock_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
denylist.go
denylist_test.go
discovery_chain_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
discovery_chain_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
dns.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
dns_oss.go Add peering .service and .node DNS lookups. (#15596) 2022-11-29 12:23:18 -06:00
dns_oss_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
dns_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
enterprise_delegate_oss.go
event_endpoint.go Revert getPathSuffixUnescaped (#13256) 2022-06-01 13:17:14 -04:00
event_endpoint_test.go Unify various status errors into one HTTP error type. (#12594) 2022-04-29 13:42:49 -04:00
federation_state_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
health_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
health_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
http.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
http_decode_test.go Regenerate files according to 1.19.2 formatter 2022-10-24 16:12:08 -04:00
http_oss.go Unify various status errors into one HTTP error type. (#12594) 2022-04-29 13:42:49 -04:00
http_oss_test.go Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
http_register.go Leadership transfer cmd (#14132) 2022-11-14 15:35:12 -05:00
http_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
intentions_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
intentions_endpoint_oss_test.go
intentions_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
keyring.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
keyring_test.go removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
kvs_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
kvs_endpoint_test.go
metrics.go
metrics_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
nodeid.go removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
nodeid_test.go removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
notify.go
notify_test.go
operator_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
operator_endpoint_oss.go
operator_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
peering_endpoint.go Support Stale Queries for Trust Bundle Lookups (#14724) 2022-09-28 09:56:59 -07:00
peering_endpoint_oss_test.go [sync oss] api: add peering api module (#12911) 2022-05-02 11:49:05 -07:00
peering_endpoint_test.go Bring back parameter ServerExternalAddresses in GenerateToken endpoint (#15267) 2022-11-08 14:55:18 -06:00
prepared_query_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
prepared_query_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
reload.go
remote_exec.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
remote_exec_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
retry_join.go Deprecate -join and -join-wan (#15598) 2022-12-14 20:28:25 +00:00
retry_join_test.go add HCP integration component (#14723) 2022-09-26 14:58:15 -04:00
service_checks_test.go Service http checks data source for agentless proxies (#14924) 2022-10-12 07:49:56 -07:00
service_manager.go Merge central config for GetEnvoyBootstrapParams (#14869) 2022-10-10 12:40:27 -05:00
service_manager_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
session_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
session_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
setup.go Use rpcHoldTimeout to calculate blocking timeout (#15541) 2022-11-24 10:13:02 -05:00
setup_oss.go
sidecar_service.go PR #14057 follow up fix: service id parsing from sidecar id (#14541) 2022-09-09 09:47:10 -05:00
sidecar_service_test.go Backport test from ENT: "Fix missing test fields" (#15258) 2022-11-04 09:29:16 -05:00
signal_unix.go
signal_windows.go
snapshot_endpoint.go
snapshot_endpoint_test.go
status_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
status_endpoint_test.go
streaming_test.go
testagent.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
testagent_test.go
translate_addr.go
txn_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
txn_endpoint_test.go increase the size of txn to support vault (#14599) 2022-09-19 09:07:19 -07:00
ui_endpoint.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
ui_endpoint_oss_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
ui_endpoint_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
user_event.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
user_event_test.go Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
util.go
util_test.go
watch_handler.go
watch_handler_test.go removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00