* Update go-bexpr to v0.1.1
This brings in:
• `in`/`not in` operators to do substring matching
• `matches` / `not matches` operators to perform regex string matching.
* Add the capability to auto-generate the filtering selector ops tables for our docs
* Support for maximum size for Output of checks
This PR allows users to limit the size of output produced by checks at the agent
and check level.
When set at the agent level, it will limit the output for all checks monitored
by the agent.
When set at the check level, it can override the agent max for a specific check but
only if it is lower than the agent max.
Default value is 4k, and input must be at least 1.
This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.
* clarify possibilities for centralized proxy configuration
* add line breaks to config entries file
* add info about centralized config to built in proxy doc
* mondify connect landing page to help with navigation
* move internals details to its own page
* link fixes and shortening text on main page
* put built-in proxy options on its own page
* add configuration details for connect
* clarify security title and add observability page
* reorganize menu
* remove observability from configuration section
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* rename connect section to include service mesh
* reorganize sections per suggestions from paul
* add configuration edits from paul
* add internals edits from paul
* add observability edits from paul
* reorganize pages and menu
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* menu corrections and edits
* incorporate some of pauls comments
* incorporate more of pauls comments
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* incorporate kaitlin and pavanni feedback
* add redirect
* fix conflicts in index file
* Resolve conflicts in index file
* correct links for new organization
* Update website/source/docs/connect/proxies.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* add title to service registration page
* Add some config entry docs
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Get rid of double negative
* Some incremental updates
* Update the config list docs to not point to service-default related things.
* A few more doc updates to get rid of some service-defaults specific linking info in the cli docs
* In progress update
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Reword bootstrap section
* Update example proxy-defaults config
* Finish up the examples section for managing config entries with the CLI
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Use $ for shell command start
* Make it very clear that the normal way to manage things is via the API/CLI
* Update website/source/docs/agent/options.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Add api docs for the config entry endpoints
* Add enable_central_service_config field to agent docs
* Add docs for config entry CLI operations
* Fix wording and links in config entry docs
* Add links to the central service config option
* Update the central service config setting description.
* starting broken link fixes
* Updating the other links for ACLs
* Updating the rest of the links
* fixing acl required links.
* update a bunch of other links
* updated a couple more broken links based on Alvins checker
* removed the extra s
Fixes: #4222
# Data Filtering
This PR will implement filtering for the following endpoints:
## Supported HTTP Endpoints
- `/agent/checks`
- `/agent/services`
- `/catalog/nodes`
- `/catalog/service/:service`
- `/catalog/connect/:service`
- `/catalog/node/:node`
- `/health/node/:node`
- `/health/checks/:service`
- `/health/service/:service`
- `/health/connect/:service`
- `/health/state/:state`
- `/internal/ui/nodes`
- `/internal/ui/services`
More can be added going forward and any endpoint which is used to list some data is a good candidate.
## Usage
When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of:
```
<selector> == <value>
<selector> != <value>
<value> in <selector>
<value> not in <selector>
<selector> contains <value>
<selector> not contains <value>
<selector> is empty
<selector> is not empty
not <other expression>
<expression 1> and <expression 2>
<expression 1> or <expression 2>
```
Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library
## Other changes
Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.
* website: specify value of acquire/release params for kv
* website: clarify leader election usage in TTL docs
* website: document minimal value of lockdelay
I believe it uses the default when parsing 0 as it
views that as an empty parameter in this case.
This PR adds two features which will be useful for operators when ACLs are in use.
1. Tokens set in configuration files are now reloadable.
2. If `acl.enable_token_persistence` is set to `true` in the configuration, tokens set via the `v1/agent/token` endpoint are now persisted to disk and loaded when the agent starts (or during configuration reload)
Note that token persistence is opt-in so our users who do not want tokens on the local disk will see no change.
Some other secondary changes:
* Refactored a bunch of places where the replication token is retrieved from the token store. This token isn't just for replicating ACLs and now it is named accordingly.
* Allowed better paths in the `v1/agent/token/` API. Instead of paths like: `v1/agent/token/acl_replication_token` the path can now be just `v1/agent/token/replication`. The old paths remain to be valid.
* Added a couple new API functions to set tokens via the new paths. Deprecated the old ones and pointed to the new names. The names are also generally better and don't imply that what you are setting is for ACLs but rather are setting ACL tokens. There is a minor semantic difference there especially for the replication token as again, its no longer used only for ACL token/policy replication. The new functions will detect 404s and fallback to using the older token paths when talking to pre-1.4.3 agents.
* Docs updated to reflect the API additions and to show using the new endpoints.
* Updated the ACL CLI set-agent-tokens command to use the non-deprecated APIs.
* Add common blocking implementation details to docs
These come up over and over again with blocking query loops in our own code and third-party's. #5333 is possibly a case (unconfirmed) where "badly behaved" blocking clients cause issues, however since we've never explicitly documented these things it's not reasonable for third-party clients to have guessed that they are needed!
This hopefully gives us something to point to for the future.
It's a little wordy - happy to consider breaking some of the blocking stuff out of this page if we think it's appropriate but just wanted to quickly plaster over this gap in our docs for now.
* Update index.html.md
* Apply suggestions from code review
Co-Authored-By: banks <banks@banksco.de>
* Update index.html.md
* Update index.html.md
* Clearified monotonically
* Fixing formating
Given a query like:
```
{
"Name": "tagged-connect-query",
"Service": {
"Service": "foo",
"Tags": ["tag"],
"Connect": true
}
}
```
And a Consul configuration like:
```
{
"services": [
"name": "foo",
"port": 8080,
"connect": { "sidecar_service": {} },
"tags": ["tag"]
]
}
```
If you executed the query it would always turn up with 0 results. This was because the sidecar service was being created without any tags. You could instead make your config look like:
```
{
"services": [
"name": "foo",
"port": 8080,
"connect": { "sidecar_service": {
"tags": ["tag"]
} },
"tags": ["tag"]
]
}
```
However that is a bit redundant for most cases. This PR ensures that the tags and service meta of the parent service get copied to the sidecar service. If there are any tags or service meta set in the sidecar service definition then this copying does not take place. After the changes, the query will now return the expected results.
A second change was made to prepared queries in this PR which is to allow filtering on ServiceMeta just like we allow for filtering on NodeMeta.
This endpoint aggregates all checks related to <service id> on the agent
and return an appropriate http code + the string describing the worst
check.
This allows to cleanly expose service status to other component, hiding
complexity of multiple checks.
This is especially useful to use consul to feed a load balancer which
would delegate health checking to consul agent.
Exposing this endpoint on the agent is necessary to avoid a hit on
consul servers and avoid decreasing resiliency (this endpoint will work
even if there is no consul leader in the cluster).
Mike Morris